-
Notifications
You must be signed in to change notification settings - Fork 27
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #946 from PolicyEngine/data-migrate
Migrate to policyengine-uk-data
- Loading branch information
Showing
97 changed files
with
488 additions
and
10,491 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -50,7 +50,4 @@ policyengine_uk/calibration/*.h5 | |
| **/*.csv | ||
| **/*.pkl | ||
| **/*.log | ||
| **/ukmod.json | ||
| *.ipynb | ||
|
|
||
| !test9.ipynb | ||
| **/ukmod.json | ||
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,9 +1,2 @@ | ||
| @import url('https://fonts.googleapis.com/css2?family=Roboto+Serif:[email protected]&family=Roboto:wght@300&display=swap'); | ||
|
|
||
| h1, h2, h3, h4, h5, h6 { | ||
| font-family: "Roboto"; | ||
| } | ||
|
|
||
| body { | ||
| font-family: "Roboto Serif"; | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
File renamed without changes.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,245 @@ | ||
| { | ||
| "cells": [ | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "# Baseline forecast\n", | ||
| "\n", | ||
| "To anchor PolicyEngine to official forecasts, this page presents both the Office for Budget Responsibility (OBR)'s forecast for public sector net debt (PSND) and that implied by PolicyEngine.\n", | ||
| "\n", | ||
| "PSND is generally used as the key fiscal metric in the UK, and all talk of 'headroom' and 'cost' of fiscal measures is in relation to this metric. In calculating PolicyEngine's projection for PSND, we use the OBR's forecast and subtract the cost and revenue projections for benefits modelled in PolicyEngine:\n", | ||
| "\n", | ||
| "* Income Tax\n", | ||
| "* National Insurance (all classes)\n", | ||
| "* VAT\n", | ||
| "* Stamp Duty Land Tax\n", | ||
| "* Fuel duties\n", | ||
| "* Universal Credit\n", | ||
| "* Child Benefit\n", | ||
| "* Working Tax Credit\n", | ||
| "* Child Tax Credit\n", | ||
| "* Housing Benefit\n", | ||
| "* Personal Independence Payment\n", | ||
| "* Disability Living Allowance\n", | ||
| "* TV Licence" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": 38, | ||
| "metadata": { | ||
| "tags": [ | ||
| "hide-input" | ||
| ] | ||
| }, | ||
| "outputs": [], | ||
| "source": [ | ||
| "OBR_PSND = [2_540, 2_691, 2_793, 2_820, 2_903, 2_995, 3_078]\n", | ||
| "from policyengine_uk_data.storage import STORAGE_FOLDER\n", | ||
| "import pandas as pd\n", | ||
| "tax_ben = pd.read_csv(STORAGE_FOLDER / \"tax_benefit.csv\")\n", | ||
| "REVENUES = [\n", | ||
| " \"income_tax\",\n", | ||
| " \"ni_employee\",\n", | ||
| " \"ni_employer\",\n", | ||
| " \"vat\",\n", | ||
| " \"fuel_duties\",\n", | ||
| " \"tv_licence_fee\",\n", | ||
| "]\n", | ||
| "SPENDING = [\n", | ||
| " \"universal_credit\",\n", | ||
| " \"tax_credits\",\n", | ||
| " \"housing_benefit\",\n", | ||
| " \"pip\",\n", | ||
| " \"dla\",\n", | ||
| "]\n", | ||
| "tax_ben_revenues = tax_ben[tax_ben.name.isin(REVENUES)].set_index(\"name\").loc[REVENUES]\n", | ||
| "tax_ben_spending = tax_ben[tax_ben.name.isin(SPENDING)].set_index(\"name\").loc[SPENDING]\n", | ||
| "from policyengine_uk import Microsimulation\n", | ||
| "PE_REVENUES = [\n", | ||
| " \"income_tax\",\n", | ||
| " \"ni_employee\",\n", | ||
| " \"ni_employer\",\n", | ||
| " \"vat\",\n", | ||
| " \"fuel_duty\",\n", | ||
| " \"tv_licence\",\n", | ||
| "]\n", | ||
| "PE_SPENDING = [\n", | ||
| " \"universal_credit\",\n", | ||
| " \"tax_credits\",\n", | ||
| " \"housing_benefit\",\n", | ||
| " \"pip\",\n", | ||
| " \"dla\",\n", | ||
| "]\n", | ||
| "baseline = Microsimulation()\n", | ||
| "\n", | ||
| "pe_psnd = []\n", | ||
| "\n", | ||
| "for year in range(2022, 2029):\n", | ||
| " obr_psnd = OBR_PSND[year - 2022]\n", | ||
| " obr_totals = tax_ben_spending[str(year)].sum() - tax_ben_revenues[str(year)].sum()\n", | ||
| " pe_revenues = sum(baseline.calculate(variable, map_to=\"household\", period=year).sum()/1e9 for variable in PE_REVENUES)\n", | ||
| " pe_spending = sum(baseline.calculate(variable, map_to=\"household\", period=year).sum()/1e9 for variable in PE_SPENDING)\n", | ||
| " pe_totals = pe_spending - pe_revenues\n", | ||
| " pe_psnd.append(obr_psnd + pe_totals - obr_totals)\n", | ||
| "\n", | ||
| "df = pd.DataFrame({\n", | ||
| " \"year\": range(2022, 2029),\n", | ||
| " \"OBR\": OBR_PSND,\n", | ||
| " \"PE\": pe_psnd,\n", | ||
| "})" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": 47, | ||
| "metadata": { | ||
| "tags": [ | ||
| "hide-input" | ||
| ] | ||
| }, | ||
| "outputs": [ | ||
| { | ||
| "data": { | ||
| "text/html": [ | ||
| "<div>\n", | ||
| "<style scoped>\n", | ||
| " .dataframe tbody tr th:only-of-type {\n", | ||
| " vertical-align: middle;\n", | ||
| " }\n", | ||
| "\n", | ||
| " .dataframe tbody tr th {\n", | ||
| " vertical-align: top;\n", | ||
| " }\n", | ||
| "\n", | ||
| " .dataframe thead th {\n", | ||
| " text-align: right;\n", | ||
| " }\n", | ||
| "</style>\n", | ||
| "<table border=\"1\" class=\"dataframe\">\n", | ||
| " <thead>\n", | ||
| " <tr style=\"text-align: right;\">\n", | ||
| " <th></th>\n", | ||
| " <th>year</th>\n", | ||
| " <th>OBR</th>\n", | ||
| " <th>PE</th>\n", | ||
| " <th>diff</th>\n", | ||
| " <th>diff_pct</th>\n", | ||
| " </tr>\n", | ||
| " </thead>\n", | ||
| " <tbody>\n", | ||
| " <tr>\n", | ||
| " <th>0</th>\n", | ||
| " <td>2022</td>\n", | ||
| " <td>2540</td>\n", | ||
| " <td>2538.0</td>\n", | ||
| " <td>-1.5</td>\n", | ||
| " <td>-0.001</td>\n", | ||
| " </tr>\n", | ||
| " <tr>\n", | ||
| " <th>1</th>\n", | ||
| " <td>2023</td>\n", | ||
| " <td>2691</td>\n", | ||
| " <td>2693.0</td>\n", | ||
| " <td>2.2</td>\n", | ||
| " <td>0.001</td>\n", | ||
| " </tr>\n", | ||
| " <tr>\n", | ||
| " <th>2</th>\n", | ||
| " <td>2024</td>\n", | ||
| " <td>2793</td>\n", | ||
| " <td>2800.0</td>\n", | ||
| " <td>7.5</td>\n", | ||
| " <td>0.003</td>\n", | ||
| " </tr>\n", | ||
| " <tr>\n", | ||
| " <th>3</th>\n", | ||
| " <td>2025</td>\n", | ||
| " <td>2820</td>\n", | ||
| " <td>2831.0</td>\n", | ||
| " <td>10.5</td>\n", | ||
| " <td>0.004</td>\n", | ||
| " </tr>\n", | ||
| " <tr>\n", | ||
| " <th>4</th>\n", | ||
| " <td>2026</td>\n", | ||
| " <td>2903</td>\n", | ||
| " <td>2915.0</td>\n", | ||
| " <td>12.2</td>\n", | ||
| " <td>0.004</td>\n", | ||
| " </tr>\n", | ||
| " <tr>\n", | ||
| " <th>5</th>\n", | ||
| " <td>2027</td>\n", | ||
| " <td>2995</td>\n", | ||
| " <td>3013.0</td>\n", | ||
| " <td>18.1</td>\n", | ||
| " <td>0.006</td>\n", | ||
| " </tr>\n", | ||
| " <tr>\n", | ||
| " <th>6</th>\n", | ||
| " <td>2028</td>\n", | ||
| " <td>3078</td>\n", | ||
| " <td>3106.0</td>\n", | ||
| " <td>27.8</td>\n", | ||
| " <td>0.009</td>\n", | ||
| " </tr>\n", | ||
| " </tbody>\n", | ||
| "</table>\n", | ||
| "</div>" | ||
| ], | ||
| "text/plain": [ | ||
| " year OBR PE diff diff_pct\n", | ||
| "0 2022 2540 2538.0 -1.5 -0.001\n", | ||
| "1 2023 2691 2693.0 2.2 0.001\n", | ||
| "2 2024 2793 2800.0 7.5 0.003\n", | ||
| "3 2025 2820 2831.0 10.5 0.004\n", | ||
| "4 2026 2903 2915.0 12.2 0.004\n", | ||
| "5 2027 2995 3013.0 18.1 0.006\n", | ||
| "6 2028 3078 3106.0 27.8 0.009" | ||
| ] | ||
| }, | ||
| "execution_count": 47, | ||
| "metadata": {}, | ||
| "output_type": "execute_result" | ||
| } | ||
| ], | ||
| "source": [ | ||
| "df[\"diff\"] = df[\"PE\"] - df[\"OBR\"]\n", | ||
| "df[\"diff_pct\"] = (df[\"diff\"] / df[\"OBR\"]).round(3)\n", | ||
| "df[\"PE\"] = df[\"PE\"].round(0)\n", | ||
| "df[\"diff\"] = df[\"diff\"].round(1)\n", | ||
| "df" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "PolicyEngine appears to predict a higher PSND trend line than the OBR, increasing linearly through to 2028-29. This is over 90% explained by PolicyEngine projecting lower income tax receipts than the OBR. It's unclear why this is the case, and it might not actually be a bug: PolicyEngine optimises household weights to fit targets *including* the income tax targets, as well as income statistics. This could be the model saying that income tax receipts being lower is more consistent with other projections for the UK household sector." | ||
| ] | ||
| } | ||
| ], | ||
| "metadata": { | ||
| "kernelspec": { | ||
| "display_name": "base", | ||
| "language": "python", | ||
| "name": "python3" | ||
| }, | ||
| "language_info": { | ||
| "codemirror_mode": { | ||
| "name": "ipython", | ||
| "version": 3 | ||
| }, | ||
| "file_extension": ".py", | ||
| "mimetype": "text/x-python", | ||
| "name": "python", | ||
| "nbconvert_exporter": "python", | ||
| "pygments_lexer": "ipython3", | ||
| "version": "3.12.5" | ||
| } | ||
| }, | ||
| "nbformat": 4, | ||
| "nbformat_minor": 2 | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.