Infrastructure

Here are Zoltar's major underlying components:

Sites

The zoltar "ecosystem" has these sites:

• zoltar app
• user documentation
• zoltar source code
• zoltpy source
• zoltr source, zoltr pkgdown site
• user mailing list group

Software

• Language: The app is written in Python, currently 3.6.
• Web app: Zoltar runs on Django.
• Data storage: We use Django's ORM to represent all project-oriented information (users, projects, models, forecasts, etc.) We also use it to store the forecast data itself, along with the corresponding scores.
• REST API: We use Django REST framework.
• Asynchronous message queue: We use RQ (Redis Queue), which is backed by Redis. This doc page describes the user-facing representation of asynchronous jobs.
• Libraries: As listed in the Pipfile.

Zoltar user libraries

We've written two libraries to simplify using the API: zoltpy (Python) and zoltr (R).

Hosting

We are hosting web app all components on Heroku, including:

• Production app Heroku dashboard.
• Staging app Heroku dashboard.
• SQL database: Heroku Postgres: Plan: Standard-2 (dashboard).
• Redis database: Redis Cloud. Plan: Redis Cloud 100MB. (dashboard).
• Scheduler: Heroku Scheduler. (dashboard).
• Logging: Papertrail. Plan: Ludvig. (dashboard).
• Dynos: From the Procfile (Procfile docs): One Web dyno (Hobby) running Gunicorn and seven worker dynos (Hobby) running rq. Currently the workers are always on, i.e., they are not dynamically spun up as needed (a post-consulting todo). The issue for it: Research adding Heroku dyno auto-scaling #256. (dashboard).

Documentation generation

Here are the auto-built documentation sites:

Zoltar developer wiki

• Language: Markdown
• Static site generator: GitHub wiki.
• Source: https://github.com/reichlab/forecast-repository.wiki.git.
• Served at: https://github.com/reichlab/forecast-repository/wiki.
• Build: Triggered by commit.

Zoltar user documentation

• Language: Markdown
• Source: https://github.com/reichlab/docs.zoltardata.
• Static site generator: MkDocs.
• Served at: https://docs.zoltardata.com/ (and http://reichlab.io/docs.zoltardata/).
• Build:
  • http://reichlab.io/docs.zoltardata/: Triggered by commit via via GitHub Pages.
  • https://docs.zoltardata.com/: Currently triggered manually via gh-deploy.sh. See the docs.zoltardata readme.md for details.
• Domain configuration: See CustomDomain for details (Heroku, Namecheap, and DNSimple).

zoltr user documentation

• Language: Rmd
• Static site generator: pkgdown.
• Source: Generated from README.md, https://github.com/reichlab/zoltr/tree/master/vignettes, and R comments in https://github.com/reichlab/zoltr/tree/master/R.
• Served at: zoltr pkgdown site.
• Build: Triggered by commit via via GitHub Pages. See zoltr.R for a reminder of the build steps, e.g., pkgdown::build_site().

zoltpy user documentation

TBD, say via a Jupyter Notebook.