reichlab-python-template

[REPLACE WITH A DESCRIPTION OF YOUR PROJECT]

A Python template for Reich Lab projects.

This repo contains a Python package with minimal functionality. It serves as a starting point for new projects (it can be selected as the template when creating a new repo in the Reich Lab org).

There as some opinionated choices here (explained below) which people should override as needed. The main goal is to have a consistent starting point to get up and running with a new Python code base.

Getting started

[REMOVE THIS SECTION AFTER FOLLOWING THE INSTRUCTIONS BELOW]

If you're using this repo as a template for a new project, make the following changes:

1. Rename the reichlab_python_template directory (under src) to the name of your package (no hyphens!).

2. Replace all instances of reichlab-python-template with the name of your repo/project.

3. Replace all instances of reichlab_python_template with the name of your package (remember that Python module names cannot contain hyphens).

4. Update pyproject.toml. This file is required and will describe several aspects of your project. pyproject.toml replaces setup.py and is described in detail on Python's packaging website.

5. Follow the Setup for local development instructions below to ensure that everything works as expected.

Installing and running the package (no development)

To install this package via pip:

pip install git+[GITHUB LINK TO YOUR REPO]

To run it:

reichlab_python_template

Setup for local development

The steps below are for setting up a local development environment. This process entails more than just installing the package, because we need to ensure that all developers have a consistent, reproducible environment.

Assumptions

Developers will be using a Python virtual environment that:

• is based on the Python version specified in .python-version.
• contains the dependency versions specified in the "lockfile" (in this case requirements/requirements-dev.txt).
• contains the package installed in "editable" mode.

Setup steps

1. Clone this repository

2. Change to the repo's root directory:

   cd reichlab-python-template

3. Make sure the correct version of Python is currently active, and create a Python virtual environment:

   python -m venv .venv

4. Activate the virtual environment:

   # MacOs/Linux
   source .venv/bin/activate
   
   # Windows
   .venv\Scripts\activate

5. Install the package dependencies and install the package in editable mode:

   python -m pip install -r requirements/requirements-dev.txt && python -m pip install -e .

6. Optional: if you use pre-commit in your workflow to automate code formatting and other tasks, install it. Otherwise, delete .pre-commit-config.yaml.

7. Run the test suite to confirm that everything is working:

   python -m pytest

Development workflow

Because the package is installed in "editable" mode, you can run the code as though it were a normal Python package, while also being able to make changes and see them immediately.

Updating dependencies

Prerequisites:

• uv

Note: using pipx (instead of pip) to install uv is a handy way to ensure that uv is available for all of the Python environments on your machine.

The "lockfile" for this project is simply an annotated requirements.txt that is generated by uv (uv is a replacement for pip-compile, which could also be used). There's also a requirements-dev.txt file that contains dependencies needed for development (e.g., pytest).

While it's possible to use pip freeze to generate a detailed lockfile without a third-party tool like uv, the output of pip freeze doesn't distinguish between direct and indirect dependencies. This distinction probably doesn't matter for a small project, but on a large project, understanding the dependency graph is critical for resolving conflicts.

Additionally, uv (and pip-compile) are able to use the list of high-level dependencies in pyproject.toml to generate a detailed requirements.txt file, which is a good workflow for keeping everything in sync.

To add or remove a project dependency:

1. Add or remove the dependency in the [dependencies] section of pyproject.toml (or in the dev section of [project.optional-dependencies], if it's a development dependency). Don't pin a specific version, since that will make it harder for users to install the package.

2. Generate updated requirements files:

   uv pip compile pyproject.toml -o requirements/requirements.txt && uv pip compile pyproject.toml --extra dev -o requirements/requirements-dev.txt

3. Update project dependencies:

   Note: This package was originally developed on MacOS. If you have trouble installing the dependencies. uv pip sync has a --python-platform flag that can be used to specify the platform.

   # note: requirements-dev.txt contains the base requirements AND the dev requirements
   #
   # using pip
   python -m pip install -r requirements/requirements-dev.txt
   #
   # alternately, you can use uv to install the dependencies: it is faster and has a
   # a handy sync option that will cleanup unused dependencies
   uv pip sync requirements/requirements-dev.txt && python -m pip install -e .

Opinionated notes on Python tooling

[REMOVE THIS SECTION]

The Python ecosystem is overwhelming! Current opinionated preferences, subject to change:

• To install and manage various versions of Python: pyenv + a local .python-version file
• To install Python packages that are available from anywhere on the machine, regardless of which Python environment is activated: pipx
• To create and manage Python virtual environments: venv.
  • It's handy having the environment packages right there in the project directory
  • Most third-party tools for managing virtual environments (e.g., poetry, PDM, pipenv) do too much and get in the way
• To generate requirements files from pyproject.toml: 'uv'. It's new, but it's orders of magnitude faster than pip-compile.
• To install dependencies: uv again (again, mostly due to speed; good old pip is another fine option)
• Logging: structlog. Python's built-in logging module is tedious.
• Linting and formatting: ruff because it does both and is fast.
• Pre-commit hooks: pre-commit.