SABS-R3-Epidemiology · anitaapplegarth · Jan 31, 2025 · Jan 23, 2025 · Jan 23, 2025 · Jan 24, 2025
diff --git a/.gitignore b/.gitignore
@@ -6,24 +6,21 @@
 .RData
 .RDataTmp
 
+# IDE specific files
+.DS_Store
+
 # User-specific files
 .Ruserdata
 
 # Example code in package build process
 *-Ex.R
 
-# Input parameters to simulation
-*.json
-
 # Output files from R CMD build
 /*.tar.gz
 
 # Output files from R CMD check
 /*.Rcheck/
 
-# Output from simulation
-*.csv
-
 # RStudio files
 .Rproj.user/
 

diff --git a/R/simulation_flow_check.R b/R/simulation_flow_check.R
@@ -21,7 +21,7 @@ pe <- import("pyEpiabm", delay_load = TRUE)
 base_dir <- here()
 
 # Set config file for Parameters
-pe$Parameters$set_file(here("data", "simple_parameters.json"))
+pe$Parameters$set_file(here("data/Andorra", "andorra_parameters.json"))
 
 # Method to set the seed at the start of the simulation, for reproducibility
 pe$routine$Simulation$set_random_seed(seed = as.integer(42))

diff --git a/README.md b/README.md
diff --git a/data/.DS_Store b/data/.DS_Store
diff --git a/data/simple_parameters.json → data/Andorra/andorra_parameters.json b/data/simple_parameters.json → data/Andorra/andorra_parameters.json
diff --git a/data/Andorra/inputs/Andorra_microcells.csv b/data/Andorra/inputs/Andorra_microcells.csv
diff --git a/images/Developer.jpg b/images/Developer.jpg
diff --git a/images/Settings.jpg b/images/Settings.jpg
diff --git a/images/Token.jpg b/images/Token.jpg
diff --git a/images/andorra_file_structure.png b/images/andorra_file_structure.png
diff --git a/images/download.png b/images/download.png
diff --git a/images/epigeopop_country.png b/images/epigeopop_country.png
diff --git a/images/epigeopop_folder_structure.png b/images/epigeopop_folder_structure.png
diff --git a/images/fine-grained.jpg b/images/fine-grained.jpg
diff --git a/images/prep_sh.png b/images/prep_sh.png
diff --git a/images/program_workflow.png b/images/program_workflow.png
diff --git a/images/your_country.png b/images/your_country.png
diff --git a/walk_through/EpigeoPop_example.ipynb b/walk_through/EpigeoPop_example.ipynb
@@ -0,0 +1,331 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Simulation using Epigeopop\n",
+    "\n",
+    "This walk-through follows the basic instructions for a simulation using Epigeopop in README, but includes further details, especially for generating a population using the EpiGeoPop repository.\n",
+    "\n",
+    "Please use the other example notebook, `toypop_example.ipynb`, for a walk-through of a simulation using a toy population.\n",
+    "\n",
+    "## Step 1: Set up rEpiabm\n",
+    "\n",
+    "### Prerequisites\n",
+    "\n",
+    "Before beginning, ensure you have:\n",
+    "- Git installed and configured\n",
+    "- RStudio installed\n",
+    "- Access to a terminal\n",
+    "- GitHub account with permissions\n",
+    "\n",
+    "### Installation Steps\n",
+    "\n",
+    "**1. Clone the Github rEpiabm repository:**\n",
+    "\n",
+    "```bash\n",
+    "git clone [email protected]:SABS-R3-Epidemiology/rEpiabm.git\n",
+    "cd rEpiabm\n",
+    "```\n",
+    "\n",
+    "**2. Create a GitHub Personal Access Token:**\n",
+    "\n",
+    "- Navigate to GitHub Settings → Developer Settings\n",
+    "- Select \"Personal access tokens (fine-grained)\"\n",
+    "- Create a new token\n",
+    "\n",
+    "<div style=\"margin: 1em 0; padding: 1em; border-left: 4px solid #f0ad4e; background-color: #fcf8f2;\">\n",
+    "<strong>Important:</strong> Make sure to copy your token immediately after creation - you won't be able to see it again!\n",
+    "</div>\n",
+    "\n",
+    "**3. Configure RStudio with your token:**\n",
+    "- In the console:\n",
+    "\n",
+    "```bash\n",
+    "file.edit(\"~/.Renviron\")\n",
+    "```\n",
+    "\n",
+    "- Add this line to the .Renviron file:\n",
+    "\n",
+    "```bash\n",
+    "GITHUB_PAT=<your_personal_access_token>\n",
+    "```\n",
+    "\n",
+    "<div style=\"margin: 1em 0; padding: 1em; border-left: 4px solid #5bc0de; background-color: #f4f8fa;\">\n",
+    "<strong>Note:</strong> Replace <code>&lt;your_personal_access_token&gt;</code> with your actual token from GitHub\n",
+    "</div>\n",
+    "\n",
+    "- Save file and restart RStudio\n",
+    "\n",
+    "**4. Install required R packages:**\n",
+    "- In the console:\n",
+    "   \n",
+    "```bash\n",
+    "install.packages(\"devtools\")\n",
+    "devtools::install_github(\"SABS-R3-Epidemiology/rEpiabm\")\n",
+    "```\n",
+    "\n",
+    "**5. Set up your country's data structure**\n",
+    "- Navigate to the `data` folder\n",
+    "- Copy the `Andorra` folder structure\n",
+    "- Rename the copy with your country's name (capitalize the initial letter)\n",
+    "\n",
+    "Your folder structure should look like this:\n",
+    "\n",
+    "<img src=\"../images/your_country.png\" width=\"40%\"> <br>\n",
+    "\n",
+    "\n",
+    "You are now ready to generate your population."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Step 2. Generate your Github repository [EpiGeoPop](https://github.com/SABS-R3-Epidemiology/EpiGeoPop)\n",
+    "\n",
+    "The following walk-through was completed on a mac. Please amend commands as needed for your operating system.\n",
+    "\n",
+    "### Environment Setup\n",
+    "\n",
+    "**1. Clone the [EpiGeoPop](https://github.com/SABS-R3-Epidemiology/EpiGeoPop) repository:**\n",
+    "\n",
+    "```bash\n",
+    "git clone [email protected]:SABS-R3-Epidemiology/EpiGeoPop.git\n",
+    "cd EpiGeoPop\n",
+    "```\n",
+    "\n",
+    "**2. Create and activate a Python 3.11 environment:**\n",
+    "\n",
+    "```bash\n",
+    "/usr/local/opt/[email protected]/bin/python3.11 -m venv .venv\n",
+    "source .venv/bin/activate\n",
+    "```\n",
+    "\n",
+    "<div style=\"margin: 1em 0; padding: 1em; border-left: 4px solid #5bc0de; background-color: #f4f8fa;\">\n",
+    "<strong>Note:</strong> It will not run on Python 3.12.\n",
+    "</div>\n",
+    "\n",
+    "**3. Install dependencies:**\n",
+    "\n",
+    "```bash\n",
+    "pip install -r requirements.txt\n",
+    "```\n",
+    "\n",
+    " <div style=\"margin: 1em 0; padding: 1em; border-left: 4px solid #5bc0de; background-color: #f4f8fa;\">\n",
+    "<strong>Note:</strong> If you encounter GDAL-related errors, install GDAL first:<code>\n",
+    "brew install gdal\n",
+    "export GDAL_CONFIG=/usr/local/bin/gdal-config\n",
+    "export GDAL_VERSION=$(gdal-config --version)</code></div><br>\n",
+    "\n",
+    "At this point, you should have your local repository, dependencies and environment set up and activated."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Configuration steps\n",
+    "\n",
+    "**1. Configure <your_country> parameters:**\n",
+    "\n",
+    "- Locate the parameter files in the `configs/countries` directory\n",
+    "- Edit or copy an existing parameter file\n",
+    "\n",
+    "- Update the `country` and `household_size_distribution` data parameters:\n",
+    "\n",
+    "```bash\n",
+    "{\n",
+    "\"household_size_distribution\": [\n",
+    "    0.228400, 0.252600, 0.252600, 0.110500,\n",
+    "    0.110400, 0.020000, 0.015000, 0.005500,\n",
+    "    0.002500, 0.002500\n",
+    "],\n",
+    "\"country\": <your_country>\n",
+    "}\n",
+    "```\n",
+    "- Save file with <your_country> at start of filename\n",
+    "\n",
+    "**2. Update the data preparation script:**\n",
+    "- Open `prep.sh`\n",
+    "- Replace the population data URL:\n",
+    "\n",
+    "```bash\n",
+    "# Comment out old URL\n",
+    "curl -O https://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL/GHS_POP_MT_GLOBE_R2019A/GHS_POP_E2015_GLOBE_R2019A_4326_30ss/V1-0/GHS_POP_E2015_GLOBE_R2019A_4326_30ss_V1_0.zip\n",
+    "\n",
+    "# Add new URL\n",
+    "curl -O https://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL/GHS_POP_GLOBE_R2023A/GHS_POP_E2025_GLOBE_R2023A_4326_30ss/V1-0/GHS_POP_E2025_GLOBE_R2023A_4326_30ss_V1_0.zip\n",
+    "```\n",
+    "\n",
+    "- Unzip correct file:\n",
+    "\n",
+    "```bash\n",
+    "# change filename after the unzip command\n",
+    "unzip GHS_POP_E2025_GLOBE_R2023A_4326_30ss_V1_0.zip\n",
+    "```\n",
+    "\n",
+    "- Remove unwanted code:\n",
+    "Comment out the remaining lines of code.\n",
+    "\n",
+    "```bash\n",
+    "curl -LO \"https://www.naturalearthdata.com/http//www.naturalearthdata.com/download/10m/cultural/ne_10m_admin_0_countries_lakes.zip\"\n",
+    "curl -LO \"https://www.naturalearthdata.com/http//www.naturalearthdata.com/download/10m/cultural/ne_10m_admin_1_states_provinces.zip\"\n",
+    "curl -LO \"https://www.naturalearthdata.com/http//www.naturalearthdata.com/download/10m/cultural/ne_10m_urban_areas_landscan.zip\"\n",
+    "echo \"Downloading population age file...\"\n",
+    "# This server uses an outdated SSL protocol so we need to enable legacy renegotiation\n",
+    "OPENSSL_CONF=../../openssl.cnf curl -O \"https://population.un.org/wpp/Download/Files/1_Indicators%20(Standard)/CSV_FILES/WPP2022_PopulationByAge5GroupSex_Medium.zip\"\n",
+    "unzip WPP2022_PopulationByAge5GroupSex_Medium.zip\n",
+    "```\n",
+    "\n",
+    "<div style=\"margin: 1em 0; padding: 1em; border-left: 4px solid #5bc0de; background-color: #f4f8fa;\">\n",
+    "<strong>Note:</strong> We will need to download the files from the website directly.\n",
+    "</div>\n",
+    "\n",
+    "- Save the file and run:\n",
+    "\n",
+    "```bash \n",
+    "bash prep.sh\n",
+    "```\n",
+    "\n",
+    "**3. Download additional required data manually:**\n",
+    "\n",
+    "- Visit [Natural Earth](https://www.naturalearthdata.com/downloads/10m-cultural-vectors/)\n",
+    "- Under Admin 0 - Countries, click on link `Download without boundary lakes`\n",
+    "\n",
+    "<img src=\"../images/download.png\" width=\"50%\"> <br>\n",
+    "    \n",
+    "- Save `ne_10m_admin_0_countries_lakes.zip` to `data/raw/` in local repository."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Generate Population Data\n",
+    "\n",
+    "**1. Update the Snakefile configuration:**\n",
+    "\n",
+    "```bash\n",
+    "# Replace country name\n",
+    "\"data/processed/countries/<your_country>_microcells.csv\",\n",
+    "\n",
+    "# Update .tif file references (3 lines within the file)\n",
+    "\"GHS_POP_E2025_GLOBE_R2023A_4326_30ss_V1_0.tif\"\n",
+    "\n",
+    "# comment out row 9:\n",
+    "\"data/processed/countries/Luxembourg_pop_dist.json\",\n",
+    "    \n",
+    "# comment out row 19:\n",
+    "\"outputs/dag.pdf\"\n",
+    "\n",
+    "# comment out the first rule:\n",
+    "rule render_dag:\n",
+    "input:\n",
+    "    \"Snakefile\"\n",
+    "output:\n",
+    "    \"outputs/dag.pdf\"\n",
+    "shell:\n",
+    "    \"snakemake --dag | dot -Tpdf > outputs/dag.pdf\"\n",
+    "\n",
+    "# comment out the last rule:\n",
+    "rule make_pop_dist:\n",
+    "input:\n",
+    "    \"data/raw/WPP2022_PopulationByAge5GroupSex_Medium.csv\",\n",
+    "    \"configs/{region}/{place}_parameters.json\"\n",
+    "output:\n",
+    "    \"data/processed/{region}/{place}_pop_dist.json\"\n",
+    "script:\n",
+    "    \"scripts/get_pop_dist.py\"\n",
+    "```\n",
+    "\n",
+    "**2. Generate the population data:**\n",
+    "\n",
+    "```bash\n",
+    "snakemake --cores 1\n",
+    "```\n",
+    "\n",
+    "<div style=\"margin: 1em 0; padding: 1em; border-left: 4px solid #5cb85c; background-color: #f3f8f3;\">\n",
+    "<strong>Tip:</strong> You can speed up the process by using multiple cores.\n",
+    "</div>\n",
+    "\n",
+    "**3. Copy the generated files:**\n",
+    "\n",
+    "- Locate `<your_country_microcells>.csv` in `data/raw/processed/countries/`\n",
+    "- Copy it to your rEpiabm repository: `data/<your_country>/inputs/`\n",
+    "\n",
+    "\n",
+    "You're now ready to run simulations with your country's population data."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Step 3. Configure the simulation\n",
+    "\n",
+    "**1. Update country's household parameters:**\n",
+    "- Edit or copy your version of Andorra_parameters.json\n",
+    "- Amend the parameter array household_size_distribution to have your countries' distribution (starting line 29).\n",
+    "```bash\n",
+    "        \"household_size_distribution\": [0.283708848, 0.345103011, 0.150677793,\n",
+    "            0.12764915, 0.057777709, 0.022624223,\n",
+    "            0.007151747, 0.002988208, 0.001179132,\n",
+    "            0.001140178],\n",
+    "```\n",
+    "\n",
+    "<div style=\"margin: 1em 0; padding: 1em; border-left: 4px solid #5bc0de; background-color: #f4f8fa;\">\n",
+    "<strong>Note:</strong> This should match those used in Step 2 when extracting the data from Epigeopop.\n",
+    "</div>\n",
+    "\n",
+    "- Save with <your_country>'s name.\n",
+    "\n",
+    "**2. Update simulation parameters and locations:**\n",
+    "- Open `simulation_epigeopop.R`\n",
+    "- Set simulation parameters:\n",
+    "\n",
+    "```bash\n",
+    "# Amend simulation_duration and initial_infected to your values\n",
+    "run_complete_simulation <- function(output_dir=\"data/simulation_outputs\",\n",
+    "                                    output_file = \"output.csv\",\n",
+    "                                    sir_plot_file = \"SIR_plot.png\",\n",
+    "                                    rt_plot_file = \"Rt_plot.png\",\n",
+    "                                    si_plot_file = \"SerialInterval_plot.png\",\n",
+    "                                    use_toy_example = TRUE,\n",
+    "                                    simulation_duration = 60,\n",
+    "                                    initial_infected = 10) {\n",
+    "```\n",
+    "\n",
+    "<div style=\"margin: 1em 0; padding: 1em; border-left: 4px solid #5bc0de; background-color: #f4f8fa;\">\n",
+    "<strong>Note:</strong> Simulation_duration unit is days and initial_infected unit is individuals.\n",
+    "</div>\n",
+    "\n",
+    "- Amend location and filename:\n",
+    "```bash\n",
+    "# User-defined variables - change Andorra to your country\n",
+    "input_dir <- \"Andorra\"\n",
+    "output_dir=\"data/Andorra/simulation_outputs\"\n",
+    "config_parameters <- \"data/Andorra/andorra_parameters.json\"\n",
+    "seed <- 42\n",
+    "```\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Step 4. Run the simulation\n",
+    "\n",
+    "After saving the configured file `simulation_epigeopop.R`, run this code!"
+   ]
+  }
+ ],
+ "metadata": {
+  "language_info": {
+   "name": "python"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/walk_through/detailed_example.ipynb b/walk_through/detailed_example.ipynb