Polaris SMS API

The SMS API is part of the Polaris platform (formerly DHOS). This service sends SMS messages via Twilio.

SMS requests are created and immediately sent to Twilio. When a message is sent, assuming it has the appropriate data it will be immediately queued for sending by Twilio. A call back request is generated and Twilio should update the status of the message as appropriate.

The possible statuses are:

• queued
• sent
• error
• undelivered
• delivered

If the call back fails the status of the SMS as inidicated by the service may not be accurate. There is a status update endpoint which goes through all of the queued messages and checks with Twilio if their status has changed and updates the records as appropriate.

Maintainers

The Polaris platform was created by Sensyne Health Ltd., and has now been made open-source. As a result, some of the instructions, setup and configuration will no longer be relevant to third party contributors. For example, some of the libraries used may not be publicly available, or docker images may not be accessible externally. In addition, CICD pipelines may no longer function.

For now, Sensyne Health Ltd. and its employees are the maintainers of this repository.

Setup

These setup instructions assume you are using out-of-the-box installations of:

• pre-commit (https://pre-commit.com/)
• pyenv (https://github.com/pyenv/pyenv)
• poetry (https://python-poetry.org/)

You can run the following commands locally:

make install  # Creates a virtual environment using pyenv and installs the dependencies using poetry
make lint  # Runs linting/quality tools including black, isort and mypy
make test  # Runs unit tests

You can also run the service locally using the script run_local.sh, or in dockerized form by running:

docker build . -t <tag>
docker run <tag>

Documentation

Endpoint	Method	Auth?	Description
/running	GET	No	Verifies that the service is running. Used for monitoring in kubernetes.
/version	GET	No	Get the version number, circleci build number, and git hash.
/dhos/v1/sms	POST	No	Create and send an SMS message with the details provided in the request body
/dhos/v1/sms	GET	No	Get all SMS messages including details of when they were sent and their status.
/dhos/v1/sms/{message_id}	GET	No	Get the SMS message with the UUID provided in the request
/dhos/v1/sms/{message_id}	DELETE	No	Delete the message with the provided UUID
/dhos/v1/sms_status_counts	GET	No	Get a summary of the SMS messages sent between two dates. The results are reported per day, and include the SMS message statuses.
/dhos/v1/sms/callback	POST	No	Update the status of an SMS message. This is the callback endpoint which Twilio is asked to hit when the status of a message in Twilio is updated. Note the Twilio authentication via header.
/dhos/v1/sms/bulk_update	GET	No	Update the status of all known incomplete SMS messages using the Twilio API. Note: only updates messages sent in the last 7 days.

Requirements

At a minimum you require a system with Python 3.9. Tox 3.20 is required to run the unit tests, docker with docker-compose are required to run integration tests. See Development environment setup for a more detailed list of tools that should be installed.

Deployment

All development is done on a branch tagged with the relevant ticket identifier. Code may not be merged into develop unless it passes all CircleCI tests. :partly_sunny: After merging to develop tests will run again and if successful the code is built in a docker container and uploaded to our Azure container registry. It is then deployed to test environments controlled by Kubernetes.

Testing

Unit tests

🔬 Either use make or run tox directly.

tox : Running make test or tox with no arguments runs tox -e lint,default

make clean : Remove tox and pyenv virtual environments.

tox -e debug : Runs last failed unit tests only with debugger invoked on failure. Additional py.test command line arguments may given preceded by --, e.g. tox -e debug -- -k sometestname -vv

make default (or tox -e default) : Installs all dependencies, verifies that lint tools would not change the code, runs security check programs then runs unit tests with coverage. Running tox -e py39 does the same but without starting a database container.

tox -e flask : Runs flask within the tox environment. Pass arguments after --. e.g. tox -e flask -- --help for a list of commands. Use this to create database migrations.

make help : Show this help.

make lint (or tox -e lint) : Run black, isort, and mypy to clean up source files.

make openapi (or tox -e openapi) : Recreate API specification (openapi.yaml) from Flask blueprint

make pyenv : Create pyenv and install required packages (optional).

make readme (or tox -e readme) : Updates the README file with database diagram and commands. (Requires graphviz dot is installed)

make test : Test using tox

make update (or tox -e update) : Updates the poetry.lock file from pyproject.toml

Integration tests

🔩 Integration tests are located in the integration-tests sub-directory. After changing into this directory you can run the following commands:

Database migrations

Any changes affecting the database schema should be reflected in a database migration. Simple migrations may be created automatically:

$ tox -e flask -- db migrate -m "some description"

More complex migration may be handled by creating a migration file as above and editing it by hand. Don't forget to include the reverse migration to downgrade a database.

Configuration

• DATABASE_USER, DATABASE_PASSWORD, DATABASE_NAME, DATABASE_HOST, DATABASE_PORT configure the database connection.
• LOG_LEVEL=ERROR|WARN|INFO|DEBUG sets the log level
• LOG_FORMAT=colour|plain|json configure logging format. JSON is used for the running system but the others may be more useful during development.

Database

SMS message details are stored in a Postgres database.