blog/2024/03/21/demystifying-ate-att-atu/

[giscus[bot]]

blog/2024/03/21/demystifying-ate-att-atu/

Explore why we care about the ATE, ATT, and ATU and figure out how to calculate them with observational data

https://www.andrewheiss.com/blog/2024/03/21/demystifying-ate-att-atu/

[Lulliter]

Great article! I think the worked examples are especially effective to understand the different estimands. Thanks for sharing.

[ASKurz]

Great topic. One small quibble--your statement "Beyond introducing the idea that we can’t find individual-level causal effects without a time machine" is overly strong. We can compute individual-level causal effects with within-person study designs. One example of this is the use of cross-over designs in medical trials and sometimes used by exercise scientists. Another example is the single-case methods used by behavior analysts and education researchers. You can also compute them from the kinds of multi-trial reaction-time-based computer tasks psychologists sometimes use. All of these designs still come with various kinds of assumptions, but their common feature is each participant receives two or more levels of the independent variable.

[rduesing]

Dear Andrew!

Many thanks for your wonderful blog posts and this really comprehesible examples. I was working through your examples but I am puzzled about two reslults which do not match, but in my opinion should. Maybe you can help me out?

Weights for the ATE:

I calculated the weights in a first step

weights_ate <- weightit(
net ~ income + health,
data = nets,
method = "glm",
estimand = "ATE"
)

nets$wts_ate <- weights_ate$weights

Estimate ATE with lm():

model_outcome_ate <- lm(
malaria_risk ~ net,
data = nets,
weights = wts_ate
)

avg_comparisons(model_outcome_ate,
variables = "net")

The result was the same as given by you:
-14.7 SE(0.559)

Estimate ATE with lm_weightit()

I tried to use the lm_weightit function from the weightit package, to save one step

model_outcome_wate <- lm_weightit(
malaria_risk ~ net,
data = nets,
weightit = weights_ate
)

avg_comparisons(model_outcome_wate,
variables = "net")

The estimate is the same but the standard error (and hence CI, z-score etc) differ:
-14.7 SE(0.72)

Do you have any idea why, I expcted it to be the same.
Many thanks for any help!!

All the best wishes from Germany!
Rainer

[rduesing]

P.S.: if I include health and income as covariables in the lm_weightit() command, it gives small, but still different SEs, same when I include them into the lm() command. They do not become the same.

[brshallo]

Really good piece! Just watched Malcolm Barrett's 'whole game' intro here: https://www.youtube.com/watch?v=FasUOajUG64 and reading through this helped to fill in a number of details. Seems like a ton of overlap between your post and the beats of what he goes through. At the end he talks about one of the values of causal inference being the ability to control for the estimand.
One thing, in comparing your post to his intro, where I was a little confused... is in his approach, during the propensity modeling step, he uses propensity::wt_ate(). After this, when building the model with the resulting weights inputted, he points to just the coefficient as now representing the "average treatment effect". In your post however you use avg_comparisons() to create the two separate treatments "net" or "no net" and apply them across all observations and then use the average score across observations as your "average treatment effect". Are the results just going to be the same? But then does this work for ATT or do you need to use your approach (that filters to just net == 1)? Or would have been using propensity::wt_att() during the propensity modeling step and then looking at the coefficient from the resulting linear model have been sufficient and shown the same result...?