🎉 Let export steps execute implicit grapher://grapher steps

[pabloarosado]

Let the --export flag execute the corresponding grapher://grapher steps to all data://grapher dependencies. This allow us to remove grapher://grapher dependencies, and instead have the corresponding data://grapher ones.

NOTE: I'm not totally sure if this will work with mdims that depend on private steps (like export://multidim/health/latest/causes_of_death). For now, I omitted it from the dag.

[owidbot]

Quick links (staging server):

Site Dev	Site Preview	Admin	Wizard	Docs

Login: ssh owid@staging-site-feature-export-steps-grapher

chart-diff: ✅

No charts for review.

data-diff: ❌ Found differences

= Dataset garden/wpf/2025-01-17/famines_by_place
  = Table famines_by_place
    ~ Column decadal_famine_deaths (changed metadata)
-       -   Famines were included in the catalogue based on the total number of excess deaths (magnitude), not the percentage of the population affected. A minimum threshold of 100,000 deaths was set because there is limited research on how famines impacted death rates relative to population size. This means famines with at least 100,000 deaths are included, regardless of the total population of the affected area.
+       +   Famines are assessed based on severity, magnitude, and duration. Magnitude, measured as the total number of excess deaths, was used to determine inclusion in the catalogue. A threshold of 100,000 deaths was applied due to limited demographic research on proportional death rate increases.
-       -   - |-
-       -     WPF coded the most credible estimate of the number of deaths across sources. If there were several equally credible estimates, WPF used their median.
-       -   - |-
-       -     The 1910–1919 famine in British Somaliland and the African Red Sea Region (Sudan, Northern Ethiopia, Eritrea, and Djibouti) is treated as a single event because the 100,000+ mortality estimate applies to the entire region, not individual areas.
-       -   - |-
-       -     For the Ottoman Empire (1894–1896), East Africa (1896–1900), and the combined Somaliland–African Red Sea Region famine (1910–1919), the 100,000 death estimate is a **minimum**, meaning the actual death toll was likely higher.
+       + description_processing: |-
+       +   The deaths were assumed to be evenly distributed over the duration of each famine, except for the famine in China between 1958 and 1962, where the source provides a year-by-year breakdown of mortality.
    ~ Column wpf_authoritative_mortality_estimate (changed metadata)
-       -   Famines were included in the catalogue based on the total number of excess deaths (magnitude), not the percentage of the population affected. A minimum threshold of 100,000 deaths was set because there is limited research on how famines impacted death rates relative to population size. This means famines with at least 100,000 deaths are included, regardless of the total population of the affected area.
+       +   Famines are assessed based on severity, magnitude, and duration. Magnitude, measured as the total number of excess deaths, was used to determine inclusion in the catalogue. A threshold of 100,000 deaths was applied due to limited demographic research on proportional death rate increases.
-       -   - |-
-       -     WPF coded the most credible estimate of the number of deaths across sources. If there were several equally credible estimates, WPF used their median.
-       -   - |-
-       -     The 1910–1919 famine in British Somaliland and the African Red Sea Region (Sudan, Northern Ethiopia, Eritrea, and Djibouti) is treated as a single event because the 100,000+ mortality estimate applies to the entire region, not individual areas.
-       -   - |-
-       -     For the Ottoman Empire (1894–1896), East Africa (1896–1900), and the combined Somaliland–African Red Sea Region famine (1910–1919), the 100,000 death estimate is a **minimum**, meaning the actual death toll was likely higher.
+       + description_processing: |-
+       +   The deaths were assumed to be evenly distributed over the duration of each famine, except for the famine in China between 1958 and 1962, where the source provides a year-by-year breakdown of mortality.
= Dataset garden/wpf/2025-01-17/famines_by_trigger
  = Table famines_by_trigger
    ~ Column decadal_famine_deaths (changed metadata)
-       -   - |-
-       -     WPF coded the most credible estimate of the number of deaths across sources. If there were several equally credible estimates, WPF used their median.
-       -   - |-
-       -     The 1910–1919 famine in British Somaliland and the African Red Sea Region (Sudan, Northern Ethiopia, Eritrea, and Djibouti) is treated as a single event because the 100,000+ mortality estimate applies to the entire region, not individual areas.
-       -   - |-
-       -     For the Ottoman Empire (1894–1896), East Africa (1896–1900), and the combined Somaliland–African Red Sea Region famine (1910–1919), the 100,000 death estimate is a **minimum**, meaning the actual death toll was likely higher.
    ~ Column famine_deaths (changed metadata)
-       -   - |-
-       -     WPF coded the most credible estimate of the number of deaths across sources. If there were several equally credible estimates, WPF used their median.
-       -   - |-
-       -     The 1910–1919 famine in British Somaliland and the African Red Sea Region (Sudan, Northern Ethiopia, Eritrea, and Djibouti) is treated as a single event because the 100,000+ mortality estimate applies to the entire region, not individual areas.
-       -   - |-
-       -     For the Ottoman Empire (1894–1896), East Africa (1896–1900), and the combined Somaliland–African Red Sea Region famine (1910–1919), the 100,000 death estimate is a **minimum**, meaning the actual death toll was likely higher.
= Dataset garden/wpf/2025-01-17/total_famines_by_year_decade
  = Table total_famines_by_year_decade
    ~ Column decadal_famine_deaths (changed metadata)
-       -   - |-
-       -     WPF coded the most credible estimate of the number of deaths across sources. If there were several equally credible estimates, WPF used their median.
-       -   - |-
-       -     The 1910–1919 famine in British Somaliland and the African Red Sea Region (Sudan, Northern Ethiopia, Eritrea, and Djibouti) is treated as a single event because the 100,000+ mortality estimate applies to the entire region, not individual areas.
-       -   - |-
-       -     For the Ottoman Empire (1894–1896), East Africa (1896–1900), and the combined Somaliland–African Red Sea Region famine (1910–1919), the 100,000 death estimate is a **minimum**, meaning the actual death toll was likely higher.
    ~ Column decadal_famine_deaths_rate (changed metadata)
-       -   - |-
-       -     WPF coded the most credible estimate of the number of deaths across sources. If there were several equally credible estimates, WPF used their median.
-       -   - |-
-       -     The 1910–1919 famine in British Somaliland and the African Red Sea Region (Sudan, Northern Ethiopia, Eritrea, and Djibouti) is treated as a single event because the 100,000+ mortality estimate applies to the entire region, not individual areas.
-       -   - |-
-       -     For the Ottoman Empire (1894–1896), East Africa (1896–1900), and the combined Somaliland–African Red Sea Region famine (1910–1919), the 100,000 death estimate is a **minimum**, meaning the actual death toll was likely higher.
    ~ Column famine_deaths (changed metadata)
-       -   - |-
-       -     WPF coded the most credible estimate of the number of deaths across sources. If there were several equally credible estimates, WPF used their median.
-       -   - |-
-       -     The 1910–1919 famine in British Somaliland and the African Red Sea Region (Sudan, Northern Ethiopia, Eritrea, and Djibouti) is treated as a single event because the 100,000+ mortality estimate applies to the entire region, not individual areas.
-       -   - |-
-       -     For the Ottoman Empire (1894–1896), East Africa (1896–1900), and the combined Somaliland–African Red Sea Region famine (1910–1919), the 100,000 death estimate is a **minimum**, meaning the actual death toll was likely higher.
    ~ Column famine_deaths_per_rate (changed metadata)
-       -   - |-
-       -     WPF coded the most credible estimate of the number of deaths across sources. If there were several equally credible estimates, WPF used their median.
-       -   - |-
-       -     The 1910–1919 famine in British Somaliland and the African Red Sea Region (Sudan, Northern Ethiopia, Eritrea, and Djibouti) is treated as a single event because the 100,000+ mortality estimate applies to the entire region, not individual areas.
-       -   - |-
-       -     For the Ottoman Empire (1894–1896), East Africa (1896–1900), and the combined Somaliland–African Red Sea Region famine (1910–1919), the 100,000 death estimate is a **minimum**, meaning the actual death toll was likely higher.


Legend: +New  ~Modified  -Removed  =Identical  Details
Hint: Run this locally with etl diff REMOTE data/ --include yourdataset --verbose --snippet

Automatically updated datasets matching weekly_wildfires|excess_mortality|covid|fluid|flunet|country_profile|garden/ihme_gbd/2019/gbd_risk are not included

Edited: 2025-02-10 18:47:30 UTC
Execution time: 19.13 seconds

[lucasrodes]

LGTM! thanks for the implementation. Tested on COVID, and seems to work just fine

[pabloarosado]

Running, e.g. etlr energy_prices --export raises an error related to the gbd_cause mdim page. This may be related to having private dependencies, so we may need to look into that in the future. But for now, we can keep that problematic step commented out in the DAG. I'll merge this PR!