Add GLM Support

[s3alfisc]

Introduces pf.feglm(), which allows to fit GLMs with Gaussian & Binomial families (Logit and Probit).

In Python:

%load_ext autoreload
%autoreload 2

import pyfixest as pf
import numpy as np

data = pf.get_data()
data["Y"] = np.where(data["Y"] > 0, 1, 0)

fit_ols = pf.feols("Y ~ X1", data = data)
fit_gaussian = pf.feglm("Y ~ X1", data = data, family = "gaussian")
fit_probit = pf.feglm("Y ~ X1", data = data, family = "probit")
fit_logit = pf.feglm("Y ~ X1", data = data, family = "logit")

pf.etable([fit_ols, fit_gaussian, fit_probit, fit_logit],digits = 6)

In R:

library(reticulate)
library(fixest)
data = py$data
fml = Y ~ X1

fit_ols = feols(Y~X1, data = data)
fit_gaussian =feglm(Y ~ X1, data = data, family = "gaussian")
fit_probit = feglm(Y ~ X1, data = data, family = binomial(link = "probit"))
fit_logit = feglm(Y ~ X1, data = data, family = binomial(link = "logit"))

etable(fit_ols, fit_gaussian, fit_probit, fit_logit)

To do:

• point estimates match
• inference matches
• prediction matches

Only apparent issues / error (?):

• IWLS sometimes fails, even with step halfing. Due to separation? Seems to be mostly probit, so likely implementation error. -> fixed, incorrect deviance implementation
• - scores for Gaussian match OLS scores in pyfixest but not fixest (Gaussian scores in fixest <> OLS scores, not sure why).

[juanitorduz]

Here are some minor code-style comments. I am unfamiliar with the numerical estimation details, so someone else can help review them.

[juanitorduz]

Why these commented lines ? :)

[s3alfisc]

It's a relict from an earlier implementation in which the Feglm class inherited from Fepois, which has these arguments. Now it inherits from Feols, which does not have them, so both arguments are set in Feglm.init. I've cleaned this up, coming with the next commit.

[juanitorduz]

commented line?

[juanitorduz]

do you a case where the user would need to modify the step_halfing_tolerance parameter?

[s3alfisc]

I checked the glmnet API, and they do not seem to support setting the step halfing tolerance: link. For now I would maybe not support it, but add it in case users request it?

[juanitorduz]

why 0.1? It clearly seems to be numerical stability, but could it be smaller (just curious)?

[s3alfisc]

I actually don't know - I am using this as it is mimic fixest, which itself follows R's glm implementation, which is using this convergence criterion: link

[juanitorduz]

I had some issues locking the file in #767 (initial commits). I had to make sure to update the environments that I really needed. It seems a total update is breaking something (in case it helps)

[s3alfisc]

Tests fail due to an incorrect assumption on formulaic behavior, which seems to have changed with version 1.1.0. Setting formulaic requirements to <1.1.0 for now. Details here: #777