Add GLM Support (#763)

* initial commit * method for step halfing * cleanups and replace Stamann eta update; works now without fixed effects * Logit without fixed effects works * add probit class * delete Fepois * type hints * type hints * rename to get_fit * add some tests * add gaussian, fix crit bug * no coefficient crit * more tests * test run N = 100 * inherit from Feols, add predict method * vcov reorg * add depvar checks * update weights * gaussian gives identical results * gaussian works except iid * fix irls_weights, add to tests * update tests * test cleanup * iid hetero match * work with scores vor crv1 * crv1 inference passes for glms * error when glms with fixed effects * dones test scores for gaussian vs fixest * mention GLMs in quickstart * mention GLMs in quickstart * add bug in bread for IV * custom iid method for poisson * compare deviance in tests * fix probit deviance -> no more convergence problems * avg_slopes to docs * pyarrow to docs dependencies * fix error in docs * update changelog * another docs bug * adress Juan's comments * udate readmes * update lock file * align pyproject toml and lock with main * update lock file * small cleanups * merge with master * enable context for glms * enforce formulaic < 1.1.0 * add pyarrow to docs deps, required by marginaleffects
py-econometrics · Jan 4, 2025 · 83a6f27 · 83a6f27
1 parent a492a29
commit 83a6f27
Show file tree

Hide file tree

Showing 22 changed files with 3,665 additions and 2,049 deletions.
diff --git a/README.md b/README.md
@@ -25,9 +25,10 @@ For questions on `PyFixest`, head on over to our [PyFixest Discourse forum](http
 
 ## Features
 
--   **OLS**, **WLS** and **IV** Regression
+-   **OLS**, **WLS** and **IV** Regression with Fixed-Effects Demeaning via [Frisch-Waugh-Lovell](https://bookdown.org/ts_robinson1994/10EconometricTheorems/frisch.html)
 -   **Poisson Regression** following the [pplmhdfe algorithm](https://journals.sagepub.com/doi/full/10.1177/1536867X20909691)
 -   Multiple Estimation Syntax
+-   Probit, Logit and Gaussian Family GLMs (currently without fixed effects demeaning, this is WIP)
 -   Several **Robust** and **Cluster Robust Variance-Covariance** Estimators
 -   **Wild Cluster Bootstrap** Inference (via
     [wildboottest](https://github.com/py-econometrics/wildboottest))

diff --git a/docs/changelog.qmd b/docs/changelog.qmd
@@ -2,6 +2,7 @@
 
 ## PyFixest 0.28.0 (In Development, can be installed from github)
 
+- Adds a `pf.feglm()` function that supports GLMs with normal and binomial families (gaussian, logit, probit) without fixed effects. Fixed effects support is work in progress.
 - Fix a bug that caused reindexing of `LPDID._coeftable` when calling `LPDID.iplot()`. As a result, a second call of `LPDID.iplot()` would fail.
 
 ## PyFixest 0.27.0

diff --git a/docs/pyfixest.md b/docs/pyfixest.md
@@ -24,9 +24,10 @@ For questions on `PyFixest`, head on over to our [PyFixest Discourse forum](http
 
 ## Features
 
--   **OLS**, **WLS** and **IV** Regression
+-   **OLS**, **WLS** and **IV** Regression with Fixed-Effects Demeaning via [Frisch-Waugh-Lovell](https://bookdown.org/ts_robinson1994/10EconometricTheorems/frisch.html)
 -   **Poisson Regression** following the [pplmhdfe algorithm](https://journals.sagepub.com/doi/full/10.1177/1536867X20909691)
 -   Multiple Estimation Syntax
+-   Probit, Logit and Gaussian Family GLMs (currently without fixed effects demeaning, this is WIP)
 -   Several **Robust** and **Cluster Robust Variance-Covariance** Estimators
 -   **Wild Cluster Bootstrap** Inference (via
     [wildboottest](https://github.com/py-econometrics/wildboottest))

diff --git a/docs/quickstart.qmd b/docs/quickstart.qmd
@@ -34,6 +34,7 @@ import matplotlib.pyplot as plt
 import numpy as np
 import pandas as pd
 from lets_plot import LetsPlot
+from marginaleffects import slopes, avg_slopes
 
 import pyfixest as pf
 
@@ -408,6 +409,40 @@ q1 = np.array([2.0, 1.0])
 fit.wald_test(R=R1, q=q1)
 ```
 
+# Other GLMs (without fixed effects)
+
+`PyFixest` supports a range of other GLMs without fixed effects (adding fixed effect support is WIP) via the `pf.feglm()` function.
+
+```{python}
+data_glm = pf.get_data(N=100, seed = 170)
+data_glm["Y"] = np.where(data_glm["Y"] > 0, 1, 0)
+
+fit_gaussian = pf.feglm(fml = "Y~X1", data = data_glm, family = "gaussian")
+fit_logit = pf.feglm(fml = "Y~X1", data = data_glm, family = "logit")
+fit_probit = pf.feglm(fml = "Y~X1", data = data_glm, family = "probit")
+
+pf.etable([
+    fit_gaussian,
+    fit_logit,
+    fit_probit,
+])
+```
+
+You can make predictions on the `response` and `link` scale via the `predict()` method:
+
+```{python}
+fit_logit.predict(type = "response")[0:5]
+fit_logit.predict(type = "link")[0:5]
+```
+
+You can compute the **average marginal effect** via the [marginaleffects package](https://github.com/vincentarelbundock/pymarginaleffects):
+
+```{python}
+avg_slopes(fit_logit, variables = "X1")
+```
+
+Please take a look at the [marginaleffects book](https://marginaleffects.com/) to learn about other transformations that the `marginaleffects` package supports.
+
 # Multiple Estimation
 
 `PyFixest` supports a range of multiple estimation functionality: `sw`, `sw0`, `csw`, `csw0`, and multiple dependent variables. The meaning of these options is explained in the [Multiple Estimations](https://lrberge.github.io/fixest/articles/multiple_estimations.html) vignette of the `fixest` package:

diff --git a/pixi.lock b/pixi.lock
diff --git a/pyfixest/__init__.py b/pyfixest/__init__.py
@@ -11,6 +11,7 @@
 # Import frequently used functions and classes
 from pyfixest.estimation import (
     bonferroni,
+    feglm,
     feols,
     fepois,
     rwolf,
@@ -33,6 +34,7 @@
     "estimation",
     "etable",
     "event_study",
+    "feglm",
     "feols",
     "fepois",
     "get_data",

diff --git a/pyfixest/estimation/FixestMulti_.py b/pyfixest/estimation/FixestMulti_.py
@@ -5,10 +5,13 @@
 
 import pandas as pd
 
+from pyfixest.estimation.fegaussian_ import Fegaussian
 from pyfixest.estimation.feiv_ import Feiv
+from pyfixest.estimation.felogit_ import Felogit
 from pyfixest.estimation.feols_ import Feols, _check_vcov_input, _deparse_vcov_input
 from pyfixest.estimation.feols_compressed_ import FeolsCompressed
 from pyfixest.estimation.fepois_ import Fepois
+from pyfixest.estimation.feprobit_ import Feprobit
 from pyfixest.estimation.FormulaParser import FixestFormulaParser
 from pyfixest.utils.dev_utils import DataFrameType, _narwhals_to_pandas
 from pyfixest.utils.utils import capture_context
@@ -320,6 +323,7 @@ def _estimate_all_models(
                             lean=_lean,
                             sample_split_value=sample_split_value,
                             sample_split_var=_splitvar,
+                            context=_context,
                         )
                         FIT.prepare_model_matrix()
                         FIT.demean()
@@ -348,12 +352,100 @@ def _estimate_all_models(
                             sample_split_value=sample_split_value,
                             sample_split_var=_splitvar,
                             separation_check=separation_check,
+                            context=_context,
                             # solver=_solver
                         )
                         FIT.prepare_model_matrix()
                         FIT.to_array()
                         FIT.drop_multicol_vars()
 
+                    elif _method == "feglm-logit":
+                        FIT = Felogit(
+                            FixestFormula=FixestFormula,
+                            data=_data,
+                            ssc_dict=_ssc_dict,
+                            drop_singletons=_drop_singletons,
+                            drop_intercept=_drop_intercept,
+                            weights=_weights,
+                            weights_type=_weights_type,
+                            solver=solver,
+                            collin_tol=collin_tol,
+                            fixef_tol=_fixef_tol,
+                            lookup_demeaned_data=lookup_demeaned_data,
+                            tol=iwls_tol,
+                            maxiter=iwls_maxiter,
+                            store_data=_store_data,
+                            copy_data=_copy_data,
+                            lean=_lean,
+                            sample_split_value=sample_split_value,
+                            sample_split_var=_splitvar,
+                            separation_check=separation_check,
+                            context=_context,
+                        )
+
+                        FIT.prepare_model_matrix()
+                        FIT.to_array()
+                        FIT._check_dependent_variable()
+                        FIT.drop_multicol_vars()
+
+                    elif _method == "feglm-probit":
+                        FIT = Feprobit(
+                            FixestFormula=FixestFormula,
+                            data=_data,
+                            ssc_dict=_ssc_dict,
+                            drop_singletons=_drop_singletons,
+                            drop_intercept=_drop_intercept,
+                            weights=_weights,
+                            weights_type=_weights_type,
+                            solver=solver,
+                            collin_tol=collin_tol,
+                            fixef_tol=_fixef_tol,
+                            lookup_demeaned_data=lookup_demeaned_data,
+                            tol=iwls_tol,
+                            maxiter=iwls_maxiter,
+                            store_data=_store_data,
+                            copy_data=_copy_data,
+                            lean=_lean,
+                            sample_split_value=sample_split_value,
+                            sample_split_var=_splitvar,
+                            separation_check=separation_check,
+                            context=_context,
+                        )
+
+                        FIT.prepare_model_matrix()
+                        FIT.to_array()
+                        FIT._check_dependent_variable()
+                        FIT.drop_multicol_vars()
+
+                    elif _method == "feglm-gaussian":
+                        FIT = Fegaussian(
+                            FixestFormula=FixestFormula,
+                            data=_data,
+                            ssc_dict=_ssc_dict,
+                            drop_singletons=_drop_singletons,
+                            drop_intercept=_drop_intercept,
+                            weights=_weights,
+                            weights_type=_weights_type,
+                            solver=solver,
+                            collin_tol=collin_tol,
+                            fixef_tol=_fixef_tol,
+                            lookup_demeaned_data=lookup_demeaned_data,
+                            tol=iwls_tol,
+                            maxiter=iwls_maxiter,
+                            store_data=_store_data,
+                            copy_data=_copy_data,
+                            lean=_lean,
+                            sample_split_value=sample_split_value,
+                            sample_split_var=_splitvar,
+                            separation_check=separation_check,
+                            context=_context,
+                        )
+
+                        FIT.prepare_model_matrix()
+                        FIT.to_array()
+                        FIT._check_dependent_variable()
+                        FIT.drop_multicol_vars()
+
                     elif _method == "compression":
                         FIT = FeolsCompressed(
                             FixestFormula=FixestFormula,

diff --git a/pyfixest/estimation/__init__.py b/pyfixest/estimation/__init__.py
@@ -6,18 +6,22 @@
     detect_singletons,
 )
 from pyfixest.estimation.estimation import (
+    feglm,
     feols,
     fepois,
 )
+from pyfixest.estimation.fegaussian_ import Fegaussian
 from pyfixest.estimation.feiv_ import (
     Feiv,
 )
+from pyfixest.estimation.felogit_ import Felogit
 from pyfixest.estimation.feols_ import (
     Feols,
 )
 from pyfixest.estimation.fepois_ import (
     Fepois,
 )
+from pyfixest.estimation.feprobit_ import Feprobit
 from pyfixest.estimation.FixestMulti_ import (
     FixestMulti,
 )
@@ -31,13 +35,17 @@
 )
 
 __all__ = [
+    "Fegaussian",
     "Feiv",
+    "Felogit",
     "Feols",
     "Fepois",
+    "Feprobit",
     "FixestMulti",
     "bonferroni",
     "demean",
     "detect_singletons",
+    "feglm",
     "feols",
     "fepois",
     "literals",