Skip to content

Commit

Permalink
NPA-3332 Update Development Guide based on feedback
Browse files Browse the repository at this point in the history
  • Loading branch information
@JackPlowman
JackPlowman committed Oct 24, 2024
1 parent 92d928d commit 9f93a4c
Showing 1 changed file with 34 additions and 45 deletions.
79 changes: 34 additions & 45 deletions DEVELOPMENT_GUIDE.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@

This documentation is intended for developers to develop the schema, sandbox and proxies. It may be used by developers working within the NHS Digital organisation and outside contributors.

> [!WARNING]
> Some of this documentation is specific to the maintainers of this repository and is only available to NHS Digital staff.
## Table of Contents

- [Development Guide](#development-guide)
Expand All @@ -10,23 +13,21 @@ This documentation is intended for developers to develop the schema, sandbox and
- [Requirements](#requirements)
- [Make commands](#make-commands)
- [Testing](#testing)
- [Caveats](#caveats)
- [Swagger UI](#swagger-ui)
- [Apigee Portal](#apigee-portal)
- [Platform setup](#platform-setup)
- [Detailed folder walk through](#detailed-folder-walk-through)
- [`/.github`:](#github)
- [`/azure`:](#azure)
- [`/proxies`:](#proxies)
- [`/sandbox`:](#sandbox)
- [Development](#development-1)
- [`/scripts`:](#scripts)
- [`/specification`:](#specification)
- [`/tests`:](#tests)
- [`Makefile`:](#makefile)
- [`ecs-proxies-containers.yml ` and `ecs-proxies-deploy.yml`:](#ecs-proxies-containersyml--and-ecs-proxies-deployyml)
- [`manifest_template.yml`:](#manifest_templateyml)
- [Releasing a new schema version](#releasing-a-new-schema-version)
- [Caveats](#caveats)
- [Swagger UI](#swagger-ui)
- [Apigee Portal](#apigee-portal)

## Development

Expand Down Expand Up @@ -59,56 +60,42 @@ A minimal Pytest test suite is available under the [tests](./tests) folder. Thes
A majority of the testing is co-located with the application in the `Proxy Validated Relationship Service` repository that is not currently open source.

The test dependencies can be installed by running `make install` from the root directory of this project.

### Caveats

#### Swagger UI

Swagger UI unfortunately doesn't correctly render `$ref`s in examples, so when publishing the schema to the API catalogue it uses `make publish` which under the hood uses `speccy serve` instead.

#### Apigee Portal

The Apigee portal will not automatically pull examples from schemas, you must specify them manually.
To run tests run `make test` from the project root. Or for Smoke tests run `make smoketest`

### Platform setup

As currently defined in your `proxies` folder, your proxies do pretty much nothing.
Telling Apigee how to connect to your backend requires a _Target Server_, which you should call named `validated-relationships-service-api-target`.
Our _Target Servers_ defined in the [api-management-infrastructure](https://github.com/NHSDigital/api-management-infrastructure) repository.
Apigee is setup for all live environments (dev, qa, int, prod) proxy the request to a target server. The target server on Apigee should be named `validated-relationships-service-api-target`.
Target Servers defined in the [api-management-infrastructure](https://github.com/NHSDigital/api-management-infrastructure/blob/master/ansible/roles/apigee-keystores-refs-targetservers/vars/main/target-servers.yml) repository.

:bulb: For Sandbox-running environments (`test`) these need to be present for successful deployment but can be set to empty/dummy values.
Sandbox environments use Apigee to proxy the request to an AWS ECS container.

### Detailed folder walk through

To get started developing your API use this template repo alongside guidance provided by the [API Producer Zone confluence](https://nhsd-confluence.digital.nhs.uk/display/APM/Deliver+your+API)
For further information about Apigee and APIM see [API Producer Zone confluence](https://nhsd-confluence.digital.nhs.uk/display/APM/Deliver+your+API)

#### `/.github`:

This folder contains templates that can be customised for items such as opening pull requests or issues within the repo
`/.github/workflows`: This folder contains GitHub workflows, these workflow are mainly used to check pull requests and publish releases.

`/.github/workflows`: This folder contains templates for GitHub action workflows such as:

- `pr-lint.yaml`: This workflow template shows how to link Pull Request's to Jira tickets and runs when a pull request is opened.
- `continuous-integration.yml`: This workflow template shows how to publish a Github release when pushing to master.
- `pr-lint.yaml`: This workflow links Pull Request's to Jira tickets and runs when a pull request is opened/updated.
- `continuous-integration.yml`: This workflow publishes a Github release when pushing to master.
- `sandbox-checks.yaml`: This workflow checks the sandbox meets the formatting and linting rules (Black + Flake8). Also it runs the sandbox unit tests (Pytest)
- `dependency-review.yml`: This workflow checks for any vulnerabilities in dependencies to be added to the project.
- `codeql-analysis.yml`: This workflow checks for any code vulnerabilities in the project.

#### `/azure`:

Contains templates defining Azure Devops pipelines. By default the following pipelines are available:

- `azure-build-pipeline.yml`: Assembles the contents of your repository into a single file ("artifact") on Azure Devops and pushes any containers to our Docker registry. By default this pipeline is enabled for all branches.
- `azure-pr-pipeline.yml`: Deploys ephemeral versions of your proxy/spec to Apigee (and docker containers on AWS) to internal environments. You can run automated and manual tests against these while you develop. By default this pipeline will deploy to internal-dev, but the template can be amended to add other environments as required.
- `azure-release-pipeline.yml`: Deploys the long-lived version of your pipeline to internal and external environments, typically when you merge to master.
Contains Azure Devops pipelines for building and deploying to Apigee:

The `project.yml` file needs to be populated with your service names to make them available to the pipelines
- `azure-build-pipeline.yml`: Assembles the contents of your repository into a single file ("artifact") on Azure Devops and pushes any containers to our Docker registry. This pipeline is enabled for all branches.
- `azure-pr-pipeline.yml`: Deploys ephemeral versions of your proxy/spec to Apigee (and docker containers on AWS) to internal environments. This will deploy a internal-dev environment and internal-dev-sandbox for each pull request.
- `azure-release-pipeline.yml`: Deploys the long-lived version of your pipeline to internal and external environments, when you merge to master.

`/azure/templates`: Here you can define reusable actions, such as running tests, and call these actions during Azure Devops pipelines.

#### `/proxies`:

This folder contains files relating to your Apigee API proxy.
This folder contains files relating to the Apigee API proxy.

There are 2 folders `/live` and `/sandbox` allowing you to define a different proxy for sandbox use. By default, this sandbox proxy is implemented to route to the sandbox target server (code for this sandbox is found under /sandbox of this template repo)

Expand All @@ -123,35 +110,27 @@ See the APM confluence for more information on how the [\_ping](https://nhsd-con

`/targets`: The XMLs within these folders set up target definitions which allow connections to external target servers. The sandbox target definition is implemented to route to the sandbox target server (code for this sandbox is found under /sandbox of this template repo). For more info on setting up a target server see the [API Producer Zone confluence](https://nhsd-confluence.digital.nhs.uk/display/APM/Setting+up+a+target+server)

#### `/sandbox`:

Sandbox provides a API for exploring the Proxy API. It uses pre-canned responses to most common scenarios.

##### Development

For information on how to develop the sandbox API see [README.md](sandbox/README.md)

#### `/scripts`:

Contains useful scripts that are used throughout the project, for example in Makefile and Github workflows
Contains useful scripts that are used throughout the project. Mainly Python scripts used in the release process.

#### `/specification`:

Create an OpenAPI Specification to document your API. For more information about developing specifications see the [API Producer Zone confluence](https://nhsd-confluence.digital.nhs.uk/display/APM/Documenting+your+API).
Contains the OpenAPI specification - [validated-relationships-service-api.yaml](./specification/validated-relationships-service-api.yaml)

#### `/tests`:

End to End tests. These tests are written in Python and use the PyTest test runner. Before running these tests you will need to set environment variables. The `test_endpoint.py` file provides a template of how to set up tests which test your api endpoints. For more information about testing your API see the [API Producer Zone confluence](https://nhsd-confluence.digital.nhs.uk/display/APM/Testing+your+API).

#### `Makefile`:

Useful make targets to get started including: installing dependencies and running smoke tests.
Contains make targets to run parts of the project such as installing dependencies and running smoke tests.

#### `ecs-proxies-containers.yml ` and `ecs-proxies-deploy.yml`:

These files are required to deploy containers alongside your Apigee proxy during the Azure Devops `azure-build-pipeline`. In this template repo we are deploying our sandbox container which is used as the target server for the sandbox proxy.
These files are required to deploy containers alongside your Apigee proxy during the Azure Devops `azure-build-pipeline`.

`ecs-proxies-containers.yml`: The path to a container's Dockerfile is defined here. This path needs to be defined to allow containers to be pushed to our repository during the `azure-build-pipeline`.
`ecs-proxies-containers.yml`: The path to a container's Dockerfile is defined here. This path needs to be defined to allow containers to be pushed to our repository during the `azure-build-pipeline`. This is used for the sandbox proxy docker container.

`ecs-proxies-deploy.yml` : Here you can define config for your container deployment.

Expand All @@ -173,3 +152,13 @@ To release a new version of the schema, follow the steps below:
5. Once the pull request has been approved, merge it into the `master` branch.
6. Request a schema changes are published to the [Validated Relationship Service API](https://digital.nhs.uk/developer/api-catalogue/validated-relationship-service) page.
1. Documentation on how to update the published specification page can be found [here](https://nhsd-confluence.digital.nhs.uk/display/NPA/Deploy+to+Public+Validated+Relationship+Service+API+Page)

### Caveats

#### Swagger UI

Swagger UI unfortunately doesn't correctly render `$ref`s in examples, so when publishing the schema to the API catalogue it uses `make publish` which under the hood uses `speccy serve` instead.

#### Apigee Portal

The Apigee portal will not automatically pull examples from schemas, you must specify them manually.

0 comments on commit 9f93a4c

Please sign in to comment.