Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update topic_tags dataset-schema.json enum with current values #3961

Merged
merged 8 commits into from
Feb 12, 2025

Conversation

ikesau
Copy link
Member

@ikesau ikesau commented Feb 11, 2025

Changes:

  • Updates the enum with latest values (alphabetizes the existing list first, so it's easier to compare the diff)
  • Updates the datasette query
  • Updates all instances of Antibiotics to Antibiotics & Antibiotic Resistance

Hopefully this will catch any other invalid tags in CI? Not sure if that's how this repo is set up though 🙂

@ikesau ikesau requested a review from lucasrodes February 11, 2025 20:39
@ikesau ikesau self-assigned this Feb 11, 2025
@owidbot
Copy link
Contributor

owidbot commented Feb 11, 2025

Quick links (staging server):

Site Dev Site Preview Admin Wizard Docs

Login: ssh owid@staging-site-fix-invalid-tags

chart-diff: ✅ No charts for review.
data-diff: ❌ Found differences
= Dataset garden/antibiotics/2024-10-09/gram
  = Table gram
    ~ Column antibiotic_consumption__ddd_1_000_day (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column lower_uncertainty_interval (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column upper_uncertainty_interval (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
= Dataset garden/antibiotics/2024-10-09/gram_children
  = Table gram_children
    ~ Column antibiotic_usage__pct (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column lower_uncertainty_interval__pct (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column upper_uncertainty_interval__pct (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
= Dataset garden/antibiotics/2024-10-09/gram_level
  = Table gram_level
    ~ Column antibiotic_consumption__ddd_1_000_day (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
= Dataset garden/antibiotics/2024-10-18/who_glass_by_antibiotic
  = Table bci_table
    ~ Column bcis_per_million (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column total_bcis (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
  = Table antibiotic_table
    ~ Column bcis_with_ast_per_million (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column share_bcis_with_ast (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column total_bcis_with_ast (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
= Dataset garden/antibiotics/2024-10-23/animuse_year
  = Table animuse_year
    ~ Column mg_per_kg_adjusted (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column mg_per_kg_non_adjusted (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column participants (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
= Dataset garden/antibiotics/2024-10-23/tracss
  = Table tracss
    ~ Column amr_surveillance_human_3_3 (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column amu_data_submission_whoah_4_5_b (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column law_antimicrobials_aquatic_2_8_3 (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column law_antimicrobials_terrestrial_growth_promotion_2_8_5 (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column laws_antimicrobials_terrestrial_2_8_2 (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column monitoring_amr_water_6_3 (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column monitoring_consumption_human_3_2 (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column monitoring_sales_use_animals_4_5_a (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column surveillance_amr_aquatic_4_8 (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column surveillance_amr_food_5_3 (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column surveillance_amr_terrestrial_4_7 (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column woah_reporting_options_4_6 (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
= Dataset garden/antibiotics/2024-10-25/esvac_sales
  = Table esvac_sales
    ~ Column all_other_forms_share_of_sales (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column all_other_forms_tonnes (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column tablets_share_of_sales (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column tablets_tonnes (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column total_share_of_sales (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column total_tonnes (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
= Dataset garden/antibiotics/2024-10-25/esvac_sales_corrected
  = Table esvac_sales_corrected
    ~ Column mg_per_pcu (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column pcu (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column percentage_change_mg_per_pcu (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column percentage_change_pcu (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column sales_for_food_producing_animals (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
= Dataset garden/antibiotics/2024-11-15/testing_coverage
  = Table testing_coverage
    ~ Column ctas_with_reported_bcis_bloodstream (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column ctas_with_reported_bcis_gonorrhea (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column ctas_with_reported_bcis_stool (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column ctas_with_reported_bcis_urinary_tract (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column ctas_with_reported_bcis_with_ast__gt__80_bcis_bloodstream (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column ctas_with_reported_bcis_with_ast__gt__80_bcis_gonorrhea (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column ctas_with_reported_bcis_with_ast__gt__80_bcis_stool (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column ctas_with_reported_bcis_with_ast__gt__80_bcis_urinary_tract (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column share_ctas_with_reported_bcis_bloodstream (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column share_ctas_with_reported_bcis_gonorrhea (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column share_ctas_with_reported_bcis_stool (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column share_ctas_with_reported_bcis_urinary_tract (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column share_ctas_with_reported_bcis_with_ast__gt__80_bcis_bloodstream (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column share_ctas_with_reported_bcis_with_ast__gt__80_bcis_gonorrhea (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column share_ctas_with_reported_bcis_with_ast__gt__80_bcis_stool (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column share_ctas_with_reported_bcis_with_ast__gt__80_bcis_urinary_tract (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column total_bcis_bloodstream (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column total_bcis_gonorrhea (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column total_bcis_stool (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column total_bcis_urinary_tract (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column total_bcis_with_ast_bloodstream (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column total_bcis_with_ast_gonorrhea (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column total_bcis_with_ast_stool (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column total_bcis_with_ast_urinary_tract (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
  = Table specimen
    ~ Column infections_not_tested_for_susceptibility (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
    ~ Column total_bcis_with_ast (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
= Dataset garden/antibiotics/2024-12-03/glass_enrolment
  = Table glass_enrolment
    ~ Column enrolment (changed metadata)
-       -     - Antibiotics
+       +     - Antibiotics & Antibiotic Resistance
= Dataset garden/energy/2025-02-03/energy_prices
  = Table energy_prices_monthly
  = Table energy_prices_annual
    ~ Dim country
+       + New values: 4 / 341 (1.17%)
           year             country
           2020             Albania
           2021             Albania
           2017 European Union (27)
           2018 European Union (27)
    ~ Dim year
+       + New values: 4 / 341 (1.17%)
                      country  year
                      Albania  2020
                      Albania  2021
          European Union (27)  2017
          European Union (27)  2018
    ~ Column annual_electricity_all_wholesale_euro (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_all_wholesale_euro
                      Albania  2020                                   <NA>
                      Albania  2021                                   <NA>
          European Union (27)  2017                                   <NA>
          European Union (27)  2018                                   <NA>
    ~ Column annual_electricity_household_capacity_taxes_allowances_euro (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_household_capacity_taxes_allowances_euro
                      Albania  2020                                                         <NA>
                      Albania  2021                                                         <NA>
          European Union (27)  2017                                                         <NA>
          European Union (27)  2018                                                         <NA>
    ~ Column annual_electricity_household_capacity_taxes_allowances_pps (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_household_capacity_taxes_allowances_pps
                      Albania  2020                                                        <NA>
                      Albania  2021                                                        <NA>
          European Union (27)  2017                                                        <NA>
          European Union (27)  2018                                                        <NA>
    ~ Column annual_electricity_household_capacity_taxes_euro (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_household_capacity_taxes_euro
                      Albania  2020                                               0.0
                      Albania  2021                                              <NA>
          European Union (27)  2017                                              <NA>
          European Union (27)  2018                                              <NA>
    ~ Column annual_electricity_household_capacity_taxes_pps (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_household_capacity_taxes_pps
                      Albania  2020                                              0.0
                      Albania  2021                                             <NA>
          European Union (27)  2017                                             <NA>
          European Union (27)  2018                                             <NA>
    ~ Column annual_electricity_household_energy_and_supply_euro (new data, changed data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_household_energy_and_supply_euro
                      Albania  2020                                                 <NA>
                      Albania  2021                                                 <NA>
          European Union (27)  2017                                            64.029808
          European Union (27)  2018                                            70.124504
        ~ Changed values: 2 / 341 (0.59%)
          country  year  annual_electricity_household_energy_and_supply_euro -  annual_electricity_household_energy_and_supply_euro +
          Germany  2017                                                   <NA>                                              76.834816
          Germany  2018                                                   <NA>                                              70.323074
    ~ Column annual_electricity_household_energy_and_supply_pps (new data, changed data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_household_energy_and_supply_pps
                      Albania  2020                                                <NA>
                      Albania  2021                                                <NA>
          European Union (27)  2017                                           65.728775
          European Union (27)  2018                                            71.48716
        ~ Changed values: 2 / 341 (0.59%)
          country  year  annual_electricity_household_energy_and_supply_pps -  annual_electricity_household_energy_and_supply_pps +
          Germany  2017                                                  <NA>                                             70.875328
          Germany  2018                                                  <NA>                                             64.768494
    ~ Column annual_electricity_household_environmental_taxes_allowance_euro (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_household_environmental_taxes_allowance_euro
                      Albania  2020                                                             <NA>
                      Albania  2021                                                             <NA>
          European Union (27)  2017                                                             <NA>
          European Union (27)  2018                                                             <NA>
    ~ Column annual_electricity_household_environmental_taxes_allowance_pps (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_household_environmental_taxes_allowance_pps
                      Albania  2020                                                            <NA>
                      Albania  2021                                                            <NA>
          European Union (27)  2017                                                            <NA>
          European Union (27)  2018                                                            <NA>
    ~ Column annual_electricity_household_environmental_taxes_euro (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_household_environmental_taxes_euro
                      Albania  2020                                                    0.0
                      Albania  2021                                                   <NA>
          European Union (27)  2017                                                   <NA>
          European Union (27)  2018                                                   <NA>
    ~ Column annual_electricity_household_environmental_taxes_pps (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_household_environmental_taxes_pps
                      Albania  2020                                                   0.0
                      Albania  2021                                                  <NA>
          European Union (27)  2017                                                  <NA>
          European Union (27)  2018                                                  <NA>
    ~ Column annual_electricity_household_network_costs_euro (new data, changed data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_household_network_costs_euro
                      Albania  2020                                        45.707066
                      Albania  2021                                             45.5
          European Union (27)  2017                                         60.31332
          European Union (27)  2018                                        60.271431
        ~ Changed values: 2 / 341 (0.59%)
          country  year  annual_electricity_household_network_costs_euro -  annual_electricity_household_network_costs_euro +
          Germany  2017                                               <NA>                                          79.601723
          Germany  2018                                               <NA>                                          81.222626
    ~ Column annual_electricity_household_network_costs_pps (new data, changed data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_household_network_costs_pps
                      Albania  2020                                       89.062317
                      Albania  2021                                            86.0
          European Union (27)  2017                                       60.419506
          European Union (27)  2018                                       60.166611
        ~ Changed values: 2 / 341 (0.59%)
          country  year  annual_electricity_household_network_costs_pps -  annual_electricity_household_network_costs_pps +
          Germany  2017                                              <NA>                                         73.429398
          Germany  2018                                              <NA>                                         74.724815
    ~ Column annual_electricity_household_nuclear_taxes_allowance_euro (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_household_nuclear_taxes_allowance_euro
                      Albania  2020                                                       <NA>
                      Albania  2021                                                       <NA>
          European Union (27)  2017                                                       <NA>
          European Union (27)  2018                                                       <NA>
    ~ Column annual_electricity_household_nuclear_taxes_allowance_pps (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_household_nuclear_taxes_allowance_pps
                      Albania  2020                                                      <NA>
                      Albania  2021                                                      <NA>
          European Union (27)  2017                                                      <NA>
          European Union (27)  2018                                                      <NA>
    ~ Column annual_electricity_household_nuclear_taxes_euro (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_household_nuclear_taxes_euro
                      Albania  2020                                              0.0
                      Albania  2021                                             <NA>
          European Union (27)  2017                                             <NA>
          European Union (27)  2018                                             <NA>
    ~ Column annual_electricity_household_nuclear_taxes_pps (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_household_nuclear_taxes_pps
                      Albania  2020                                             0.0
                      Albania  2021                                            <NA>
          European Union (27)  2017                                            <NA>
          European Union (27)  2018                                            <NA>
    ~ Column annual_electricity_household_other_allowance_euro (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_household_other_allowance_euro
                      Albania  2020                                               <NA>
                      Albania  2021                                               <NA>
          European Union (27)  2017                                               <NA>
          European Union (27)  2018                                               <NA>
    ~ Column annual_electricity_household_other_allowance_pps (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_household_other_allowance_pps
                      Albania  2020                                              <NA>
                      Albania  2021                                              <NA>
          European Union (27)  2017                                              <NA>
          European Union (27)  2018                                              <NA>
    ~ Column annual_electricity_household_other_euro (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_household_other_euro
                      Albania  2020                                      0.0
                      Albania  2021                                     <NA>
          European Union (27)  2017                                     <NA>
          European Union (27)  2018                                     <NA>
    ~ Column annual_electricity_household_other_pps (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_household_other_pps
                      Albania  2020                                     0.0
                      Albania  2021                                    <NA>
          European Union (27)  2017                                    <NA>
          European Union (27)  2018                                    <NA>
    ~ Column annual_electricity_household_renewable_taxes_allowance_euro (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_household_renewable_taxes_allowance_euro
                      Albania  2020                                                         <NA>
                      Albania  2021                                                         <NA>
          European Union (27)  2017                                                         <NA>
          European Union (27)  2018                                                         <NA>
    ~ Column annual_electricity_household_renewable_taxes_allowance_pps (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_household_renewable_taxes_allowance_pps
                      Albania  2020                                                        <NA>
                      Albania  2021                                                        <NA>
          European Union (27)  2017                                                        <NA>
          European Union (27)  2018                                                        <NA>
    ~ Column annual_electricity_household_renewable_taxes_euro (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_household_renewable_taxes_euro
                      Albania  2020                                                0.0
                      Albania  2021                                               <NA>
          European Union (27)  2017                                               <NA>
          European Union (27)  2018                                               <NA>
    ~ Column annual_electricity_household_renewable_taxes_pps (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_household_renewable_taxes_pps
                      Albania  2020                                               0.0
                      Albania  2021                                              <NA>
          European Union (27)  2017                                              <NA>
          European Union (27)  2018                                              <NA>
    ~ Column annual_electricity_household_taxes_fees_levies_and_charges_allowance_euro (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_household_taxes_fees_levies_and_charges_allowance_euro
                      Albania  2020                                                                       <NA>
                      Albania  2021                                                                       <NA>
          European Union (27)  2017                                                                       <NA>
          European Union (27)  2018                                                                       <NA>
    ~ Column annual_electricity_household_taxes_fees_levies_and_charges_allowance_pps (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_household_taxes_fees_levies_and_charges_allowance_pps
                      Albania  2020                                                                      <NA>
                      Albania  2021                                                                      <NA>
          European Union (27)  2017                                                                      <NA>
          European Union (27)  2018                                                                      <NA>
    ~ Column annual_electricity_household_taxes_fees_levies_and_charges_euro (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_household_taxes_fees_levies_and_charges_euro
                      Albania  2020                                                          15.6447
                      Albania  2021                                                             15.6
          European Union (27)  2017                                                             <NA>
          European Union (27)  2018                                                             <NA>
    ~ Column annual_electricity_household_taxes_fees_levies_and_charges_pps (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_household_taxes_fees_levies_and_charges_pps
                      Albania  2020                                                       30.573629
                      Albania  2021                                                            29.5
          European Union (27)  2017                                                            <NA>
          European Union (27)  2018                                                            <NA>
    ~ Column annual_electricity_household_total_price_including_taxes_euro (new data, changed data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_household_total_price_including_taxes_euro
                      Albania  2020                                                      61.351765
                      Albania  2021                                                      61.099998
          European Union (27)  2017                                                     124.343132
          European Union (27)  2018                                                     130.395935
        ~ Changed values: 2 / 341 (0.59%)
          country  year  annual_electricity_household_total_price_including_taxes_euro -  annual_electricity_household_total_price_including_taxes_euro +
          Germany  2017                                                             <NA>                                                       156.436539
          Germany  2018                                                             <NA>                                                         151.5457
    ~ Column annual_electricity_household_total_price_including_taxes_pps (new data, changed data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_household_total_price_including_taxes_pps
                      Albania  2020                                                    119.635941
                      Albania  2021                                                         115.5
          European Union (27)  2017                                                    126.148285
          European Union (27)  2018                                                    131.653778
        ~ Changed values: 2 / 341 (0.59%)
          country  year  annual_electricity_household_total_price_including_taxes_pps -  annual_electricity_household_total_price_including_taxes_pps +
          Germany  2017                                                            <NA>                                                      144.304718
          Germany  2018                                                            <NA>                                                      139.493317
    ~ Column annual_electricity_household_value_added_tax_vat_euro (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_household_value_added_tax_vat_euro
                      Albania  2020                                                15.6447
                      Albania  2021                                                   15.6
          European Union (27)  2017                                                   <NA>
          European Union (27)  2018                                                   <NA>
    ~ Column annual_electricity_household_value_added_tax_vat_pps (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_household_value_added_tax_vat_pps
                      Albania  2020                                             30.573629
                      Albania  2021                                                  29.5
          European Union (27)  2017                                                  <NA>
          European Union (27)  2018                                                  <NA>
    ~ Column annual_electricity_non_household_capacity_taxes_euro (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_non_household_capacity_taxes_euro
                      Albania  2020                                                   0.0
                      Albania  2021                                                   0.0
          European Union (27)  2017                                                  <NA>
          European Union (27)  2018                                                  <NA>
    ~ Column annual_electricity_non_household_capacity_taxes_pps (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_non_household_capacity_taxes_pps
                      Albania  2020                                                  0.0
                      Albania  2021                                                  0.0
          European Union (27)  2017                                                 <NA>
          European Union (27)  2018                                                 <NA>
    ~ Column annual_electricity_non_household_energy_and_supply_euro (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_non_household_energy_and_supply_euro
                      Albania  2020                                                     <NA>
                      Albania  2021                                                     <NA>
          European Union (27)  2017                                                     <NA>
          European Union (27)  2018                                                     <NA>
    ~ Column annual_electricity_non_household_energy_and_supply_pps (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_non_household_energy_and_supply_pps
                      Albania  2020                                                    <NA>
                      Albania  2021                                                    <NA>
          European Union (27)  2017                                                    <NA>
          European Union (27)  2018                                                    <NA>
    ~ Column annual_electricity_non_household_environmental_taxes_euro (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_non_household_environmental_taxes_euro
                      Albania  2020                                                        0.0
                      Albania  2021                                                        0.0
          European Union (27)  2017                                                       <NA>
          European Union (27)  2018                                                       <NA>
    ~ Column annual_electricity_non_household_environmental_taxes_pps (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_non_household_environmental_taxes_pps
                      Albania  2020                                                       0.0
                      Albania  2021                                                       0.0
          European Union (27)  2017                                                      <NA>
          European Union (27)  2018                                                      <NA>
    ~ Column annual_electricity_non_household_network_costs_euro (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_non_household_network_costs_euro
                      Albania  2020                                            45.707066
                      Albania  2021                                                 45.5
          European Union (27)  2017                                                 <NA>
          European Union (27)  2018                                                 <NA>
    ~ Column annual_electricity_non_household_network_costs_pps (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_non_household_network_costs_pps
                      Albania  2020                                           89.062317
                      Albania  2021                                                86.0
          European Union (27)  2017                                                <NA>
          European Union (27)  2018                                                <NA>
    ~ Column annual_electricity_non_household_nuclear_taxes_euro (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_non_household_nuclear_taxes_euro
                      Albania  2020                                                  0.0
                      Albania  2021                                                  0.0
          European Union (27)  2017                                                 <NA>
          European Union (27)  2018                                                 <NA>
    ~ Column annual_electricity_non_household_nuclear_taxes_pps (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_non_household_nuclear_taxes_pps
                      Albania  2020                                                 0.0
                      Albania  2021                                                 0.0
          European Union (27)  2017                                                <NA>
          European Union (27)  2018                                                <NA>
    ~ Column annual_electricity_non_household_other_euro (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_non_household_other_euro
                      Albania  2020                                          0.0
                      Albania  2021                                          0.0
          European Union (27)  2017                                         <NA>
          European Union (27)  2018                                         <NA>
    ~ Column annual_electricity_non_household_other_pps (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_non_household_other_pps
                      Albania  2020                                         0.0
                      Albania  2021                                         0.0
          European Union (27)  2017                                        <NA>
          European Union (27)  2018                                        <NA>
    ~ Column annual_electricity_non_household_renewable_taxes_euro (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_non_household_renewable_taxes_euro
                      Albania  2020                                                    0.0
                      Albania  2021                                                    0.0
          European Union (27)  2017                                                   <NA>
          European Union (27)  2018                                                   <NA>
    ~ Column annual_electricity_non_household_renewable_taxes_pps (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_non_household_renewable_taxes_pps
                      Albania  2020                                                   0.0
                      Albania  2021                                                   0.0
          European Union (27)  2017                                                  <NA>
          European Union (27)  2018                                                  <NA>
    ~ Column annual_electricity_non_household_taxes_fees_levies_and_charges_euro (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_non_household_taxes_fees_levies_and_charges_euro
                      Albania  2020                                                            21.166359
                      Albania  2021                                                                 21.1
          European Union (27)  2017                                                                 <NA>
          European Union (27)  2018                                                                 <NA>
    ~ Column annual_electricity_non_household_taxes_fees_levies_and_charges_pps (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_non_household_taxes_fees_levies_and_charges_pps
                      Albania  2020                                                           41.207935
                      Albania  2021                                                           39.799999
          European Union (27)  2017                                                                <NA>
          European Union (27)  2018                                                                <NA>
    ~ Column annual_electricity_non_household_total_price_including_taxes_euro (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_non_household_total_price_including_taxes_euro
                      Albania  2020                                                          66.873428
                      Albania  2021                                                          66.599998
          European Union (27)  2017                                                               <NA>
          European Union (27)  2018                                                               <NA>
    ~ Column annual_electricity_non_household_total_price_including_taxes_pps (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_non_household_total_price_including_taxes_pps
                      Albania  2020                                                        130.270248
                      Albania  2021                                                        125.800003
          European Union (27)  2017                                                              <NA>
          European Union (27)  2018                                                              <NA>
    ~ Column annual_electricity_non_household_value_added_tax_vat_euro (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_non_household_value_added_tax_vat_euro
                      Albania  2020                                                  21.166359
                      Albania  2021                                                       21.1
          European Union (27)  2017                                                       <NA>
          European Union (27)  2018                                                       <NA>
    ~ Column annual_electricity_non_household_value_added_tax_vat_pps (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_electricity_non_household_value_added_tax_vat_pps
                      Albania  2020                                                 41.207935
                      Albania  2021                                                 39.799999
          European Union (27)  2017                                                      <NA>
          European Union (27)  2018                                                      <NA>
    ~ Column annual_gas_household_capacity_taxes_euro (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_gas_household_capacity_taxes_euro
                      Albania  2020                                      <NA>
                      Albania  2021                                      <NA>
          European Union (27)  2017                                      <NA>
          European Union (27)  2018                                      <NA>
    ~ Column annual_gas_household_capacity_taxes_pps (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_gas_household_capacity_taxes_pps
                      Albania  2020                                     <NA>
                      Albania  2021                                     <NA>
          European Union (27)  2017                                     <NA>
          European Union (27)  2018                                     <NA>
    ~ Column annual_gas_household_energy_and_supply_euro (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_gas_household_energy_and_supply_euro
                      Albania  2020                                         <NA>
                      Albania  2021                                         <NA>
          European Union (27)  2017                                    29.200993
          European Union (27)  2018                                    29.768847
    ~ Column annual_gas_household_energy_and_supply_pps (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_gas_household_energy_and_supply_pps
                      Albania  2020                                        <NA>
                      Albania  2021                                        <NA>
          European Union (27)  2017                                   32.386555
          European Union (27)  2018                                   33.227901
    ~ Column annual_gas_household_environmental_taxes_euro (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_gas_household_environmental_taxes_euro
                      Albania  2020                                           <NA>
                      Albania  2021                                           <NA>
          European Union (27)  2017                                           <NA>
          European Union (27)  2018                                           <NA>
    ~ Column annual_gas_household_environmental_taxes_pps (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_gas_household_environmental_taxes_pps
                      Albania  2020                                          <NA>
                      Albania  2021                                          <NA>
          European Union (27)  2017                                          <NA>
          European Union (27)  2018                                          <NA>
    ~ Column annual_gas_household_network_costs_euro (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_gas_household_network_costs_euro
                      Albania  2020                                     <NA>
                      Albania  2021                                     <NA>
          European Union (27)  2017                                 16.24637
          European Union (27)  2018                                16.561541
    ~ Column annual_gas_household_network_costs_pps (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_gas_household_network_costs_pps
                      Albania  2020                                    <NA>
                      Albania  2021                                    <NA>
          European Union (27)  2017                               17.839151
          European Union (27)  2018                                18.02902
    ~ Column annual_gas_household_other_euro (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_gas_household_other_euro
                      Albania  2020                             <NA>
                      Albania  2021                             <NA>
          European Union (27)  2017                             <NA>
          European Union (27)  2018                             <NA>
    ~ Column annual_gas_household_other_pps (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_gas_household_other_pps
                      Albania  2020                            <NA>
                      Albania  2021                            <NA>
          European Union (27)  2017                            <NA>
          European Union (27)  2018                            <NA>
    ~ Column annual_gas_household_renewable_taxes_euro (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_gas_household_renewable_taxes_euro
                      Albania  2020                                       <NA>
                      Albania  2021                                       <NA>
          European Union (27)  2017                                       <NA>
          European Union (27)  2018                                       <NA>
    ~ Column annual_gas_household_renewable_taxes_pps (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_gas_household_renewable_taxes_pps
                      Albania  2020                                      <NA>
                      Albania  2021                                      <NA>
          European Union (27)  2017                                      <NA>
          European Union (27)  2018                                      <NA>
    ~ Column annual_gas_household_taxes_fees_levies_and_charges_euro (new data, changed data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_gas_household_taxes_fees_levies_and_charges_euro
                      Albania  2020                                                     <NA>
                      Albania  2021                                                     <NA>
          European Union (27)  2017                                                     <NA>
          European Union (27)  2018                                                     <NA>
        ~ Changed values: 1 / 341 (0.29%)
          country  year  annual_gas_household_taxes_fees_levies_and_charges_euro -  annual_gas_household_taxes_fees_levies_and_charges_euro +
            Italy  2018                                                       <NA>                                                  16.642376
    ~ Column annual_gas_household_taxes_fees_levies_and_charges_pps (new data, changed data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_gas_household_taxes_fees_levies_and_charges_pps
                      Albania  2020                                                    <NA>
                      Albania  2021                                                    <NA>
          European Union (27)  2017                                                    <NA>
          European Union (27)  2018                                                    <NA>
        ~ Changed values: 1 / 341 (0.29%)
          country  year  annual_gas_household_taxes_fees_levies_and_charges_pps -  annual_gas_household_taxes_fees_levies_and_charges_pps +
            Italy  2018                                                      <NA>                                                 16.642376
    ~ Column annual_gas_household_total_price_including_taxes_euro (new data, changed data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_gas_household_total_price_including_taxes_euro
                      Albania  2020                                                   <NA>
                      Albania  2021                                                   <NA>
          European Union (27)  2017                                              45.447361
          European Union (27)  2018                                              46.330387
        ~ Changed values: 1 / 341 (0.29%)
          country  year  annual_gas_household_total_price_including_taxes_euro -  annual_gas_household_total_price_including_taxes_euro +
            Italy  2018                                                     <NA>                                                16.642376
    ~ Column annual_gas_household_total_price_including_taxes_pps (new data, changed data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_gas_household_total_price_including_taxes_pps
                      Albania  2020                                                  <NA>
                      Albania  2021                                                  <NA>
          European Union (27)  2017                                             50.225704
          European Union (27)  2018                                             51.256924
        ~ Changed values: 1 / 341 (0.29%)
          country  year  annual_gas_household_total_price_including_taxes_pps -  annual_gas_household_total_price_including_taxes_pps +
            Italy  2018                                                    <NA>                                               16.642376
    ~ Column annual_gas_household_value_added_tax_vat_euro (new data, changed data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_gas_household_value_added_tax_vat_euro
                      Albania  2020                                           <NA>
                      Albania  2021                                           <NA>
          European Union (27)  2017                                           <NA>
          European Union (27)  2018                                           <NA>
        ~ Changed values: 1 / 341 (0.29%)
          country  year  annual_gas_household_value_added_tax_vat_euro -  annual_gas_household_value_added_tax_vat_euro +
            Italy  2018                                             <NA>                                        16.642376
    ~ Column annual_gas_household_value_added_tax_vat_pps (new data, changed data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_gas_household_value_added_tax_vat_pps
                      Albania  2020                                          <NA>
                      Albania  2021                                          <NA>
          European Union (27)  2017                                          <NA>
          European Union (27)  2018                                          <NA>
        ~ Changed values: 1 / 341 (0.29%)
          country  year  annual_gas_household_value_added_tax_vat_pps -  annual_gas_household_value_added_tax_vat_pps +
            Italy  2018                                            <NA>                                       16.642376
    ~ Column annual_gas_non_household_capacity_taxes_euro (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_gas_non_household_capacity_taxes_euro
                      Albania  2020                                          <NA>
                      Albania  2021                                          <NA>
          European Union (27)  2017                                          <NA>
          European Union (27)  2018                                          <NA>
    ~ Column annual_gas_non_household_capacity_taxes_pps (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_gas_non_household_capacity_taxes_pps
                      Albania  2020                                         <NA>
                      Albania  2021                                         <NA>
          European Union (27)  2017                                         <NA>
          European Union (27)  2018                                         <NA>
    ~ Column annual_gas_non_household_energy_and_supply_euro (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_gas_non_household_energy_and_supply_euro
                      Albania  2020                                             <NA>
                      Albania  2021                                             <NA>
          European Union (27)  2017                                        23.360794
          European Union (27)  2018                                        25.156771
    ~ Column annual_gas_non_household_energy_and_supply_pps (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_gas_non_household_energy_and_supply_pps
                      Albania  2020                                            <NA>
                      Albania  2021                                            <NA>
          European Union (27)  2017                                       21.343271
          European Union (27)  2018                                       22.955553
    ~ Column annual_gas_non_household_environmental_taxes_euro (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_gas_non_household_environmental_taxes_euro
                      Albania  2020                                               <NA>
                      Albania  2021                                               <NA>
          European Union (27)  2017                                               <NA>
          European Union (27)  2018                                               <NA>
    ~ Column annual_gas_non_household_environmental_taxes_pps (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_gas_non_household_environmental_taxes_pps
                      Albania  2020                                              <NA>
                      Albania  2021                                              <NA>
          European Union (27)  2017                                              <NA>
          European Union (27)  2018                                              <NA>
    ~ Column annual_gas_non_household_network_costs_euro (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_gas_non_household_network_costs_euro
                      Albania  2020                                         <NA>
                      Albania  2021                                         <NA>
          European Union (27)  2017                                     7.857721
          European Union (27)  2018                                     7.022932
    ~ Column annual_gas_non_household_network_costs_pps (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_gas_non_household_network_costs_pps
                      Albania  2020                                        <NA>
                      Albania  2021                                        <NA>
          European Union (27)  2017                                    7.114423
          European Union (27)  2018                                    6.394013
    ~ Column annual_gas_non_household_other_euro (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_gas_non_household_other_euro
                      Albania  2020                                 <NA>
                      Albania  2021                                 <NA>
          European Union (27)  2017                                 <NA>
          European Union (27)  2018                                 <NA>
    ~ Column annual_gas_non_household_other_pps (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_gas_non_household_other_pps
                      Albania  2020                                <NA>
                      Albania  2021                                <NA>
          European Union (27)  2017                                <NA>
          European Union (27)  2018                                <NA>
    ~ Column annual_gas_non_household_renewable_taxes_euro (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_gas_non_household_renewable_taxes_euro
                      Albania  2020                                           <NA>
                      Albania  2021                                           <NA>
          European Union (27)  2017                                           <NA>
          European Union (27)  2018                                           <NA>
    ~ Column annual_gas_non_household_renewable_taxes_pps (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_gas_non_household_renewable_taxes_pps
                      Albania  2020                                          <NA>
                      Albania  2021                                          <NA>
          European Union (27)  2017                                          <NA>
          European Union (27)  2018                                          <NA>
    ~ Column annual_gas_non_household_taxes_fees_levies_and_charges_euro (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_gas_non_household_taxes_fees_levies_and_charges_euro
                      Albania  2020                                                         <NA>
                      Albania  2021                                                         <NA>
          European Union (27)  2017                                                         <NA>
          European Union (27)  2018                                                         <NA>
    ~ Column annual_gas_non_household_taxes_fees_levies_and_charges_pps (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_gas_non_household_taxes_fees_levies_and_charges_pps
                      Albania  2020                                                        <NA>
                      Albania  2021                                                        <NA>
          European Union (27)  2017                                                        <NA>
          European Union (27)  2018                                                        <NA>
    ~ Column annual_gas_non_household_total_price_including_taxes_euro (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_gas_non_household_total_price_including_taxes_euro
                      Albania  2020                                                       <NA>
                      Albania  2021                                                       <NA>
          European Union (27)  2017                                                  31.218515
          European Union (27)  2018                                                  32.179703
    ~ Column annual_gas_non_household_total_price_including_taxes_pps (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_gas_non_household_total_price_including_taxes_pps
                      Albania  2020                                                      <NA>
                      Albania  2021                                                      <NA>
          European Union (27)  2017                                                 28.457693
          European Union (27)  2018                                                 29.349567
    ~ Column annual_gas_non_household_value_added_tax_vat_euro (new data)
+       + New values: 4 / 341 (1.17%)
                      country  year  annual_gas_non_household_value_added_tax_vat_euro
                      Albania  2020                             

...diff too long, truncated...

Automatically updated datasets matching weekly_wildfires|excess_mortality|covid|fluid|flunet|country_profile|garden/ihme_gbd/2019/gbd_risk are not included

Edited: 2025-02-12 17:13:58 UTC
Execution time: 124.25 seconds

Copy link
Member

@lucasrodes lucasrodes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks and keeping this up-to-date, Ike! LGTM, Feel free to merge once it passes CI/CD!

I've added some changes to apps/wizard/etl_steps/utils.py, which reduces one bit that was hardcoded.

More details:
PAST: If couldn't access DB, we would use a hardcoded list of tags from the above python script.
NOW: If can't access the DB, we will use the list defined in the enum field from the schema file that you updated. If for some reason we couldn't access nor the DB neither the field in the schema, then we'll use a fallback (and hardcoded) list in the python script.

@ikesau
Copy link
Member Author

ikesau commented Feb 12, 2025

@lucasrodes presumably this is an expected CI failure and we're safe to merge?

@lucasrodes
Copy link
Member

Good catch, just a minor styling thing. Fixed!

@lucasrodes lucasrodes merged commit ceadb7a into master Feb 12, 2025
9 checks passed
@lucasrodes lucasrodes deleted the fix-invalid-tags branch February 12, 2025 17:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants