Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mdims: COVID cleaning #3974

Merged
merged 116 commits into from
Feb 14, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
116 commits
Select commit Hold shift + click to select a range
1ecb8c6
📊 covid: mdims
lucasrodes Nov 4, 2024
cca85b2
snapshot
lucasrodes Nov 5, 2024
afd3eb3
Merge branch 'master' into covid-mdims-b5
lucasrodes Dec 11, 2024
1f5eaa5
clean
lucasrodes Dec 11, 2024
42cead4
add schema
lucasrodes Dec 12, 2024
07979df
wip
lucasrodes Dec 12, 2024
c81ed61
Merge branch 'master' into covid-mdims-b5
lucasrodes Dec 12, 2024
f6d05cb
improve description
lucasrodes Dec 12, 2024
ab9038c
Merge branch 'master' into covid-mdims-b5
lucasrodes Dec 16, 2024
2f58a77
Merge branch 'master' into covid-mdims-b5
lucasrodes Jan 9, 2025
d243849
✨ covid mdims maintenance
lucasrodes Jan 9, 2025
2f6da68
Merge branch 'covid-mdims-b5' into enhance-covid-mdims-maintenance-c6…
lucasrodes Jan 9, 2025
862202b
remove titleVariant
lucasrodes Jan 9, 2025
e0511c9
📊 Update origin URL for COVID cases data
lucasrodes Jan 9, 2025
4b12152
Test update of covid.cases.yml
rakyi Jan 9, 2025
bccba6e
Update multidim-schema.json
rakyi Feb 4, 2025
a84b7bc
Restore definitions
rakyi Feb 7, 2025
7f60c1c
Merge branch 'master' into enhance-covid-mdims-maintenance-c6c2c9
lucasrodes Feb 7, 2025
3bbbcb2
📊 work on covid mdims
lucasrodes Feb 7, 2025
c3bc53f
Merge branch 'enhance-covid-mdims-maintenance-c6c2c9' into data-covid…
lucasrodes Feb 7, 2025
ddb05ca
Merge branch 'mdim-schema-display' into data-covid-mdims
lucasrodes Feb 7, 2025
d5edfa3
wip
lucasrodes Feb 7, 2025
e8dedd1
wip
lucasrodes Feb 7, 2025
311ac59
support display setting in upsert check, add other dimensions than y
lucasrodes Feb 7, 2025
76d23b9
Merge branch 'mdim-schema-display' into data-covid-mdims
lucasrodes Feb 7, 2025
f3d33da
format
lucasrodes Feb 7, 2025
d530073
Merge branch 'mdim-schema-display' into data-covid-mdims
lucasrodes Feb 7, 2025
a851627
add descriptions
lucasrodes Feb 7, 2025
f32e16a
rollback
lucasrodes Feb 7, 2025
5c77915
interval -> period
lucasrodes Feb 7, 2025
454538b
fix publication date
lucasrodes Feb 7, 2025
5eb9c68
fix publication date
lucasrodes Feb 7, 2025
dde1481
interval -> period
lucasrodes Feb 7, 2025
32436cc
fix publication date
lucasrodes Feb 10, 2025
12b48fb
fix year type
lucasrodes Feb 10, 2025
70cfe5f
undo debug change in topic url
lucasrodes Feb 10, 2025
e3ada92
integrate changes proposed in #3945
lucasrodes Feb 10, 2025
1e9e997
integrate changes proposed in #3945 (pt 2)
lucasrodes Feb 10, 2025
778951c
Merge branch 'master' into mdim-schema-display
lucasrodes Feb 10, 2025
f9c8bcc
format
lucasrodes Feb 10, 2025
e5d469f
wip
lucasrodes Feb 10, 2025
b00bbe8
add comments, remove deprecate code
lucasrodes Feb 10, 2025
415325c
Merge branch 'mdim-schema-display' into data-covid-mdims
lucasrodes Feb 10, 2025
020ebd6
fix
lucasrodes Feb 10, 2025
271903c
wip
lucasrodes Feb 10, 2025
d0ab36a
roll-back
lucasrodes Feb 10, 2025
4477d5d
Merge branch 'master' into data-covid-mdims
lucasrodes Feb 10, 2025
5a2fb0b
add missing descriptions
lucasrodes Feb 10, 2025
7d3f9f3
🎉 Let export steps execute implicit grapher://grapher steps
pabloarosado Feb 10, 2025
543d700
wip
lucasrodes Feb 10, 2025
460639a
Let export steps execute implicit grapher://grapher steps
pabloarosado Feb 10, 2025
08e8ea6
Hide causes of death mdim, which is causing issues
pabloarosado Feb 10, 2025
e33fed1
Improve format
pabloarosado Feb 10, 2025
29ca6d2
Merge branch 'master' into data-covid-mdims
lucasrodes Feb 11, 2025
74c2c0c
Merge branch 'feature-export-steps-grapher' into data-covid-mdims
lucasrodes Feb 11, 2025
7312176
suggestions by Ed
lucasrodes Feb 11, 2025
4b773f5
fix missing description
lucasrodes Feb 11, 2025
6119586
cases/deaths metadata
lucasrodes Feb 11, 2025
5054a50
use data steps
lucasrodes Feb 11, 2025
a2c5b3b
hospital mdim
lucasrodes Feb 11, 2025
7dbd67c
fix date publication
lucasrodes Feb 11, 2025
bbf104a
custom changes to boosters MDIM
lucasrodes Feb 11, 2025
f67fc12
Reference only a subset of the grapher schema
rakyi Feb 11, 2025
6ce9b65
adapt vaccinations
lucasrodes Feb 11, 2025
9c33c2a
wip
lucasrodes Feb 11, 2025
3af3f9d
Merge branch 'fix-mdim-grapher-schema' into data-covid-mdims
lucasrodes Feb 11, 2025
9d2ba22
xm
lucasrodes Feb 11, 2025
9eeaf2b
fix _expand for new API
lucasrodes Feb 11, 2025
acf695b
wip models
lucasrodes Feb 11, 2025
57df9fe
support expand path for dict-defined indicators
lucasrodes Feb 11, 2025
d4b9daf
adding support for all dimensions
lucasrodes Feb 11, 2025
8445d30
infections model
lucasrodes Feb 11, 2025
9f406b0
format
lucasrodes Feb 11, 2025
594ee18
format
lucasrodes Feb 11, 2025
7e8ce6e
covax
lucasrodes Feb 11, 2025
e3e5eb6
🔨 Refactor indicator path handling in multidimensional config
lucasrodes Feb 12, 2025
87a6755
📊 Update COVAX configuration with sorting options and improved titles
lucasrodes Feb 12, 2025
96b1955
style
lucasrodes Feb 12, 2025
2c9fd0f
covax mdim
lucasrodes Feb 12, 2025
950b194
set value in df with .loc
lucasrodes Feb 12, 2025
8542029
update xm URLs
lucasrodes Feb 12, 2025
045cc19
wip: xm models mdim
lucasrodes Feb 12, 2025
e9b370d
Merge branch 'master' into data-covid-mdims
lucasrodes Feb 12, 2025
4bc5982
look for table-to-uri mapping also in private steps
lucasrodes Feb 12, 2025
b261d43
add help text
lucasrodes Feb 12, 2025
6860704
📊 Remove xm_who data reference from COVID DAG
lucasrodes Feb 12, 2025
3a31cec
undo commented dependencies
lucasrodes Feb 12, 2025
e27cd58
fix minor metadata typo
lucasrodes Feb 12, 2025
678af89
add more help texts to MDIMs dropdown options
lucasrodes Feb 12, 2025
7d39900
typo
lucasrodes Feb 12, 2025
f29872f
add description
lucasrodes Feb 12, 2025
521a85b
cases_tests
lucasrodes Feb 12, 2025
449dae1
add missing dependency regions
lucasrodes Feb 12, 2025
2f0080f
fix map brackets
lucasrodes Feb 12, 2025
893b5c6
add description key in vax by manuf
lucasrodes Feb 12, 2025
4bbc5bc
add description key to testing
lucasrodes Feb 12, 2025
c552b22
fix map bracket sin xm
lucasrodes Feb 12, 2025
93cfa96
remove unnecessary schema reference
lucasrodes Feb 12, 2025
99403bd
wip vax breakdown
lucasrodes Feb 12, 2025
f42f8d6
wip
lucasrodes Feb 12, 2025
4dd871d
Merge branch 'master' into data-covid-mdims
lucasrodes Feb 12, 2025
089220d
wip
lucasrodes Feb 12, 2025
0ea0668
wip
lucasrodes Feb 13, 2025
0ad7abf
change function, deprecate it
lucasrodes Feb 13, 2025
b820a9c
wip: expand_views
lucasrodes Feb 13, 2025
24faea3
wip: automate view generation
lucasrodes Feb 13, 2025
a48da02
wip
lucasrodes Feb 14, 2025
b76a752
Merge branch 'master' into data-covid-mdims
lucasrodes Feb 14, 2025
94f4795
MDIM config auto-generate improvements
lucasrodes Feb 14, 2025
d48eac3
automate mobility covid MDIM
lucasrodes Feb 14, 2025
073bd34
MDIM on vax breakdown can't really be used atm
lucasrodes Feb 14, 2025
df75f99
minor cleaning
lucasrodes Feb 14, 2025
aef89a1
Merge branch 'master' into data-covid-mdims
lucasrodes Feb 14, 2025
4e17a48
minor fix
lucasrodes Feb 14, 2025
283cb1a
minor fix
lucasrodes Feb 14, 2025
058bab3
remove unnecessary dependency
lucasrodes Feb 14, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion dag/covid.yml
Original file line number Diff line number Diff line change
Expand Up @@ -328,4 +328,3 @@ steps:
- data://grapher/covid/latest/infections_model
- data://grapher/covid/latest/google_mobility
- data://grapher/regions/2023-01-01/regions
- data://grapher/un/2024-07-12/un_wpp
6 changes: 3 additions & 3 deletions etl/steps/export/multidim/covid/latest/covid.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,8 @@ def run(dest_dir: str) -> None:
"covid.covax.yml",
"covid.models.yml",
"covid.xm_models.yml",
# "covid.vax_breakdowns.yml",
"covid.cases_tests.yml",
# "covid.vax_breakdowns.yml",
]
for fname in filenames:
paths.log.info(fname)
Expand All @@ -56,12 +56,12 @@ def run(dest_dir: str) -> None:
)

# Combine dimension info from YAML + programmatically obtained
config_new["dimensions"] = multidim.combine_config_dimensions(
config["dimensions"] = multidim.combine_config_dimensions(
config_dimensions=config_new["dimensions"],
config_dimensions_yaml=config["dimensions"],
)

# # Combine views info from YAML + programmatically obtained
# Combine views info from YAML + programmatically obtained
config["views"] = config["views"] + config_new["views"]

# Upsert to DB
Expand Down