Add archiver for NREL Standard Scenarios #563
Conversation
| "keywords": sorted( | ||
| { | ||
| "nrel", | ||
| "standard scenarios", |
There was a problem hiding this comment.
Other keywords that could go in here, cribbed from pudl/metadata/sources.py@nrelatb:
+ KEYWORDS["us_govt"]
+ KEYWORDS["electricity"]
| "standard scenarios", | ||
| } | ||
| ), | ||
| "license_raw": LICENSES["cc-by-4.0"], |
There was a problem hiding this comment.
They have a weirdo disclaimer that says (approximately) "you have to cite us but you can't make it look like we endorse you" which seems close enough to cc-by?
There was a problem hiding this comment.
I would ask our resident license scrutinizer @zaneselvans on this one!
| "major cost declines for electricity generation technologies (e.g., using cost" | ||
| "inputs from the Annual Technology Baseline)." | ||
| "For select scenarios, the models are run using the PLEXOS software and the" | ||
| "Cambium tool that assembles structured data sets of hourly cost, emissions, and" |
There was a problem hiding this comment.
We could consider pulling in the cambium results as well (as a second partition) but A) they only go back to 2020, and B) they're like 6GB for each year
There was a problem hiding this comment.
If its simple to also add in the camdium results i'd say add em & add a second partition of project or scenario_type or something! but this seems like a lower priority than grabbing just the standard scenarios.
There was a problem hiding this comment.
are the camdium results 6 GB zipped? if so w/ the standard scenarios (assuming they are a similar size) its pushing up against the 50 GB archive limit
There was a problem hiding this comment.
Yeah zipped. The standard scenarios are like two orders of magnitude smaller since they don't include hourly data though, so it's less "will cambium push this archiver over the limit" and more "can we archive cambium on zenodo at all"
I'll go with "not right now" and write up cambium as a separate issue
| text: text containing HTML. | ||
| filter_pattern: If present, only return links that contain pattern. | ||
| """ | ||
| parser = _HyperlinkExtractor() |
There was a problem hiding this comment.
I believe I have carefully sliced this to not conflict with Marianne's get_hyperlink changes but I'll handle any massaging necessary if not
krivard
changed the title
krivard
force-pushed
the
561-nrel-standard-scenarios
branch
from
811331d to
49a2974
Compare
krivard
force-pushed
the
561-nrel-standard-scenarios
branch
from
a967aeb to
7e7b211
Compare
krivard
force-pushed
the
561-nrel-standard-scenarios
branch
from
6df426b to
3b9e2d2
Compare
For more information, see https://pre-commit.ci
Overview
Closes #561.
What problem does this address?
What did you change in this PR?
Testing
How did you make sure this worked? How can a reviewer verify this?
To-do list
Tasks