Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conformal bootstrap model #76

Merged
merged 110 commits into from
Oct 12, 2023
Merged
Changes from 1 commit
Commits
Show all changes
110 commits
Select commit Hold shift + click to select a range
d165986
started working on bootstrap model
lennybronner Jun 16, 2023
d191ad3
works again
lennybronner Jun 21, 2023
ab3a692
back to bivariate model, but univariate model works now and in comments
lennybronner Jun 21, 2023
6b01ef2
Merge branch 'develop' into conformal-bootstrap-model
lennybronner Jun 22, 2023
feb2a13
removed model settings from estimand
lennybronner Jun 22, 2023
ee6b8c3
now doing percentage margin
lennybronner Jun 22, 2023
b6cbef6
added some clipping
lennybronner Jun 22, 2023
47acd2b
added comments for todo
lennybronner Jun 27, 2023
8e41b7d
implemented random effects. doesn't work quite well enough yet
lennybronner Jun 27, 2023
b1ec7fa
Merge branch 'develop' into conformal-bootstrap-model
lennybronner Jul 2, 2023
c503292
epsilon sampling now done better. intervals look good. TODO: robust m…
lennybronner Jul 2, 2023
8731d74
removed units that are clear outliers
lennybronner Jul 6, 2023
05274b5
made progress on narrowing intervals
lennybronner Jul 6, 2023
9700055
still at the same stage. tried parametric bootstrap
lennybronner Jul 8, 2023
a6c4e54
started implementing regularization with OLS
lennybronner Jul 8, 2023
eb03e86
weights issue
lennybronner Jul 10, 2023
f595db1
fixed small bug
lennybronner Jul 11, 2023
af9df02
implemented aggregate model
lennybronner Jul 11, 2023
2fe896c
decomposed code
lennybronner Jul 11, 2023
823ba5f
fixed issues with strata
lennybronner Jul 13, 2023
8740be5
made progress on using reporting percentage
lennybronner Jul 14, 2023
5afb5bb
updated comments
lennybronner Jul 14, 2023
962f6d9
partial reporting integrated
lennybronner Jul 18, 2023
3fa88c0
changed some of the comments/todos
lennybronner Jul 18, 2023
f3a4542
first attempt at including partial reporting for prediction
lennybronner Jul 20, 2023
f4abf5e
forgot weights_test
lennybronner Jul 20, 2023
77d76bb
small change to preprocess data
lennybronner Jul 21, 2023
748dd2b
filter out bad partial reporting bounds
jjcherian Jul 22, 2023
0376aac
first attempt at improving random effect estimate stability
jjcherian Jul 25, 2023
aa23f05
run with cv
lennybronner Aug 14, 2023
23e67f6
small changes on generating estimands
lennybronner Aug 23, 2023
9d0d112
first pass at allowing arbitrary aggregation
lennybronner Aug 23, 2023
08950e7
fixed arbitrary aggregates
lennybronner Aug 24, 2023
738e1b1
finished up national summary statistics: called races etc.
lennybronner Aug 24, 2023
f7deedb
made some progress to run old models
lennybronner Aug 25, 2023
fab0a4a
normal model works again
lennybronner Aug 25, 2023
4ccf60f
old models now run
lennybronner Aug 25, 2023
abffed5
Merge branch 'develop' into conformal-bootstrap-model
lennybronner Aug 25, 2023
300bc2b
working through unit tests
lennybronner Aug 25, 2023
12068a4
update tests
lennybronner Aug 25, 2023
e8b3854
dropping units with zero reporting and also added alpha to national a…
lennybronner Aug 28, 2023
605c5f8
added throwing error if baseline_margin not included as feature
lennybronner Aug 29, 2023
6c2fa9b
changed baseline_margin to baseline_normalized_margin
lennybronner Aug 29, 2023
061038e
drop now works for handle unreporting
lennybronner Aug 30, 2023
d7daf30
dealing with uncontested contests
lennybronner Aug 30, 2023
8f48695
fixed small uncertainty bug in district case
lennybronner Aug 31, 2023
303ca5a
fixed merge conflict
lennybronner Aug 31, 2023
180fb6e
can now run with arbitrary parameters
lennybronner Aug 31, 2023
458d932
started commenting
lennybronner Sep 4, 2023
7c06b1e
added more comments
lennybronner Sep 4, 2023
07ce5d0
fixed merge conflicts
lennybronner Sep 5, 2023
e2cdac1
dealing with last parameters to pass into bootstrap model
lennybronner Sep 5, 2023
322311f
updated reaadme for variables
lennybronner Sep 5, 2023
43a36ff
small fixed
lennybronner Sep 5, 2023
9c85aba
added some comments
lennybronner Sep 5, 2023
64a4b78
added more comments
lennybronner Sep 7, 2023
6277a4c
added comments
lennybronner Sep 7, 2023
f12e3c6
added more comments
lennybronner Sep 7, 2023
2d7e23a
added more comments
lennybronner Sep 7, 2023
0b6f6c2
ran linter
lennybronner Sep 8, 2023
7b02970
ran some linting
lennybronner Sep 8, 2023
21540a7
finalized comments
lennybronner Sep 8, 2023
25f7eda
fixed merge conflicts
lennybronner Sep 13, 2023
5621cea
fixed merge conflicts
lennybronner Sep 15, 2023
26e77df
started working on unit tests
lennybronner Sep 16, 2023
85c1f9a
added comment
lennybronner Sep 18, 2023
ed73a71
updated bootstrapmodel with new elex-solver
lennybronner Sep 19, 2023
615a09d
updated conformal models to use new elex-solver
lennybronner Sep 19, 2023
184f7fe
ran pre-commits, also removed parametric bootstrap model
lennybronner Sep 19, 2023
19b9297
fixed merge conflicts
lennybronner Sep 21, 2023
33e04f7
made progress on getting estimandizer to work with margin model
lennybronner Sep 21, 2023
364c945
looks like estimandizer now works with bootstrap model. still need to…
lennybronner Sep 21, 2023
cc6a5f0
works for old models now
lennybronner Sep 21, 2023
57bf7ad
removed unncessary numpy
lennybronner Sep 21, 2023
4d4c71d
removed some spacing
lennybronner Sep 21, 2023
795d0cd
ran pre-commit
lennybronner Sep 21, 2023
b014498
small change
lennybronner Sep 21, 2023
3274948
fixed featurizer unit tests
lennybronner Sep 21, 2023
c4c8c78
added linter
lennybronner Sep 21, 2023
9b1220a
removed duplicate code
lennybronner Sep 21, 2023
2f64aa5
moved where weights are added
lennybronner Sep 24, 2023
33e8bf6
added more tests, fixed small bug
lennybronner Sep 25, 2023
7260d0f
started working on unit tests
lennybronner Sep 25, 2023
422c997
fixed merge conflict
lennybronner Sep 25, 2023
275f4cd
unit tests pass again
lennybronner Sep 25, 2023
0d52fce
linter
lennybronner Sep 25, 2023
862c3b3
removed breakpoint
lennybronner Sep 25, 2023
4ccbef3
added some more tests
lennybronner Sep 25, 2023
e94bf6a
temporary commit for treating uncontested races 2019 VA
lennybronner Sep 26, 2023
e62e508
removed uncontested. also added dropping wehre baseline turnout is zero
lennybronner Sep 26, 2023
8565c4d
finalized model
lennybronner Sep 26, 2023
fd34b26
updated unit tests
lennybronner Sep 27, 2023
c835dd5
fixed integration tests and added one more
lennybronner Sep 27, 2023
31ec12e
updated estimandizser unit test
lennybronner Sep 27, 2023
d913bbc
linger
lennybronner Sep 27, 2023
5b745b1
added another unit test
lennybronner Sep 28, 2023
d30a701
added another test
lennybronner Sep 28, 2023
69ec8c8
removed breakpoint
lennybronner Sep 29, 2023
ec12384
added more unit tests
lennybronner Sep 30, 2023
7739b7c
added more unit tests
lennybronner Oct 1, 2023
749d0f9
fixed small bug, working on new unit test
lennybronner Oct 2, 2023
024f890
Correcting some spelling mistakes in comments in BootstrapElectionMod…
dmnapolitano Oct 2, 2023
109f3d4
finished first pass at tests
lennybronner Oct 2, 2023
169b192
Merge branch 'conformal-bootstrap-model' of https://github.com/washin…
lennybronner Oct 3, 2023
39e62b8
removed pdb
lennybronner Oct 3, 2023
fb3d220
removed unncessary line in test
lennybronner Oct 4, 2023
8a5b000
added print statement
lennybronner Oct 6, 2023
7bc3e01
unit tests pass again
lennybronner Oct 9, 2023
9d5041f
test epsilons now use sample variance. turnout factor cutoffs increase
lennybronner Oct 12, 2023
e273c40
fixed sitatuon where there is only one contest, including unit tests
lennybronner Oct 12, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion src/elexmodel/models/BootstrapElectionModel.py
Original file line number Diff line number Diff line change
Expand Up @@ -888,6 +888,9 @@ def compute_bootstrap_errors(
n_feat_ignore_reg=1,
)

print("ols coefficients, normalized margin: ", ols_y.coefficients.flatten())
print("bootstraped ols coefficients, normalized margin: ", ols_y_B.coefficients.mean(axis=-1))

dmnapolitano marked this conversation as resolved.
Show resolved Hide resolved
# we cannot just apply ols_y_B/old_z_B to the test units because that would be missing our contest level random effect
# so we need to compute an bootstrapped estimate of the contest level random effect (epsilon)
# to do that we first compute the bootstrapped leave-one-out training residuals and then use that to
Expand Down Expand Up @@ -969,7 +972,7 @@ def compute_bootstrap_errors(
# and turn into turnout estimate
self.weighted_z_test_pred = z_test_pred * weights_test
self.ran_bootstrap = True

def get_unit_predictions(
self, reporting_units: pd.DataFrame, nonreporting_units: pd.DataFrame, estimand: str, **kwargs
) -> np.ndarray:
Expand Down Expand Up @@ -1227,6 +1230,7 @@ def get_aggregate_prediction_intervals(
).T
interval_upper = interval_upper.reshape(-1, 1)
interval_lower = interval_lower.reshape(-1, 1)

return PredictionIntervals(interval_lower, interval_upper)

def get_national_summary_estimates(
Expand Down
Loading