-
Notifications
You must be signed in to change notification settings - Fork 30
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Skip Stage 2 years #390
Skip Stage 2 years #390
Conversation
@andersonfrailey, What does this PR add that cannot be done with the Makefile? |
@martinholmer, this allows us to skip individual years in the stage two process. To the best of my knowledge, a makefile only allows us to either run stage 2 or skip it entirely. With this PR, we won't create new weights for years in which the CBO projections or SOI estimates do not change. For example, if we only update the CBO projections or SOI estimates, the 2013 weights will not change (assuming no other aspect of the file creation process has changed) since the targets for that year are no longer being updated. But right now we still run the full solver to find the 2013 weights anyway. With this PR, we would just default to using the weights that have already been calculated. If any of the other files related to creating the PUF/CPS or the stage two logic change, then the solver will still run for every year. |
@andersonfrailey, Thanks for clarifying the rationale for pull request #390. |
Latest commit re-runs everything so that the weights and PUF ratios are no longer being changed. This PR is good to go! I'll leave it open for another couple of days for any review and in case I find anything else that needs updating. |
This PR implements the suggestion in issue #386. It's still a work in progress, but here's a high level summary.
I created hashes for all of the stage 2 scripts and datasets and saved them in
datahashes.json
. The stage 2 scripts then compare these hashes, the targets, and the factors before running the LP model. If all of these are equal for a given year, the solver is skipped and the weights we already have are used again. I believe that these checks will be sufficient to catch any changes that would effect the final weights. All of the initial hash values are created byinithash.py
. If the input files are changed, they'll need to be updated after creating the new weights.For some reason, the PUF weights are being changed right now even after skipping the solver. I need to get to the bottom of that bug before this PR is ready to merge. I also think it should be merged after we switch solvers and after #387.