Skip to content

Commit

Permalink
Merge pull request #197 from umccr/implement-stateful-token-service
Browse files Browse the repository at this point in the history
Implemented TokenService stack as stateful deployable artefact
  • Loading branch information
victorskl authored Apr 9, 2024
2 parents 890af0f + b2f483e commit 4f46ca9
Show file tree
Hide file tree
Showing 15 changed files with 1,444 additions and 11 deletions.
1 change: 1 addition & 0 deletions .prettierignore
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ skel/
docs/
openapi/
shared/
venv/

# TODO still early days let ignore prettier them (microservice apps) for now
lib/workload/stateless/filemanager/
Expand Down
27 changes: 27 additions & 0 deletions config/constants.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,20 @@ import {
FilemanagerConfig,
} from '../lib/workload/stateless/filemanager/deploy/lib/filemanager';

// upstream infra: vpc
const vpcName = 'main-vpc';
const vpcStackName = 'networking';
const vpcProps = {
vpcName: vpcName,
tags: {
Stack: vpcStackName,
},
};

// upstream infra: cognito
const cognitoUserPoolIdParameterName = '/data_portal/client/cog_user_pool_id';
const cognitoPortalAppClientIdParameterName = '/data_portal/client/data2/cog_app_client_id_stage';

const regName = 'OrcaBusSchemaRegistry';
const eventBusName = 'OrcaBusMain';
const lambdaSecurityGroupName = 'OrcaBusLambdaSecurityGroup';
Expand All @@ -26,6 +40,9 @@ const prodBucket = 'org.umccr.data.oncoanalyser';
const rdsMasterSecretName = 'orcabus/master-rds'; // pragma: allowlist secret
const databasePort = 5432;

const serviceUserSecretName = 'orcabus/token-service-user'; // pragma: allowlist secret
const jwtSecretName = 'orcabus/token-service-jwt'; // pragma: allowlist secret

const orcaBusStatefulConfig = {
schemaRegistryProps: {
registryName: regName,
Expand Down Expand Up @@ -55,6 +72,13 @@ const orcaBusStatefulConfig = {
securityGroupName: lambdaSecurityGroupName,
securityGroupDescription: 'allow within same SecurityGroup and rds SG',
},
tokenServiceProps: {
serviceUserSecretName: serviceUserSecretName,
jwtSecretName: jwtSecretName,
vpcProps: vpcProps,
cognitoUserPoolIdParameterName: cognitoUserPoolIdParameterName,
cognitoPortalAppClientIdParameterName: cognitoPortalAppClientIdParameterName,
},
};

const orcaBusStatelessConfig = {
Expand Down Expand Up @@ -169,6 +193,7 @@ export const getEnvironmentConfig = (
...orcaBusStatefulConfig.securityGroupProps,
},
eventSourceProps: eventSourceConfig(devBucket),
tokenServiceProps: { ...orcaBusStatefulConfig.tokenServiceProps },
},
orcaBusStatelessConfig: {
...orcaBusStatelessConfig,
Expand Down Expand Up @@ -203,6 +228,7 @@ export const getEnvironmentConfig = (
...orcaBusStatefulConfig.securityGroupProps,
},
eventSourceProps: eventSourceConfig(stgBucket),
tokenServiceProps: { ...orcaBusStatefulConfig.tokenServiceProps },
},
orcaBusStatelessConfig: {
...orcaBusStatelessConfig,
Expand Down Expand Up @@ -235,6 +261,7 @@ export const getEnvironmentConfig = (
...orcaBusStatefulConfig.securityGroupProps,
},
eventSourceProps: eventSourceConfig(prodBucket),
tokenServiceProps: { ...orcaBusStatefulConfig.tokenServiceProps },
},
orcaBusStatelessConfig: {
...orcaBusStatelessConfig,
Expand Down
15 changes: 15 additions & 0 deletions lib/workload/orcabus-stateful-stack.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,15 @@ import { ConfigurableDatabaseProps, Database } from './stateful/database/compone
import { SecurityGroupConstruct, SecurityGroupProps } from './stateful/securitygroup/component';
import { SchemaRegistryConstruct, SchemaRegistryProps } from './stateful/schemaregistry/component';
import { EventSource, EventSourceProps } from './stateful/event_source/component';
import { TokenServiceProps, TokenServiceStack } from './stateful/token_service/deploy/stack';

export interface OrcaBusStatefulConfig {
schemaRegistryProps: SchemaRegistryProps;
eventBusProps: EventBusProps;
databaseProps: ConfigurableDatabaseProps;
securityGroupProps: SecurityGroupProps;
eventSourceProps?: EventSourceProps;
tokenServiceProps: TokenServiceProps;
}

export class OrcaBusStatefulStack extends cdk.Stack {
Expand All @@ -22,6 +24,9 @@ export class OrcaBusStatefulStack extends cdk.Stack {
readonly schemaRegistry: SchemaRegistryConstruct;
readonly eventSource?: EventSource;

// stateful stacks
statefulStackArray: cdk.Stack[] = [];

constructor(scope: Construct, id: string, props: cdk.StackProps & OrcaBusStatefulConfig) {
super(scope, id, props);

Expand Down Expand Up @@ -55,5 +60,15 @@ export class OrcaBusStatefulStack extends cdk.Stack {
if (props.eventSourceProps) {
this.eventSource = new EventSource(this, 'EventSourceConstruct', props.eventSourceProps);
}

this.statefulStackArray.push(this.createTokenServiceStack(props));
}

private createTokenServiceStack(props: cdk.StackProps & OrcaBusStatefulConfig) {
return new TokenServiceStack(this, 'TokenServiceStack', {
// reduce the props to the stack needs
env: props.env,
...props.tokenServiceProps,
});
}
}
5 changes: 5 additions & 0 deletions lib/workload/stateful/token_service/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
install:
@pip install -r deps/requirements-test.txt

test:
@python -m unittest token_service/cognitor/tests.py
131 changes: 131 additions & 0 deletions lib/workload/stateful/token_service/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
# Token Service

This service provides the JWT token for API authentication and authorization (AAI) purpose. We use the Cognito as AAI service broker and, it is set up at our infrastructure repo. This service maintains 2 secrets with rotation enabled.

## JWT

For most cases, you would want to lookup JWT token from the secret manager at the following coordinate.
```
orcabus/token-service-jwt
```

An example Python boto3 snippet code as follows.

```python
import json
import boto3

sm_client = boto3.client('secretsmanager')

resp = sm_client().get_secret_value(SecretId='orcabus/token-service-jwt')

jwt_json = resp['SecretString']
jwt_dict = json.loads(jwt_json)

print(jwt_dict['id_token']) # this is your JWT token to use for calling API endpoint
```

Alternatively, try [libumccr/aws/libsm](https://github.com/umccr/libumccr/blob/main/libumccr/aws/libsm.py) module, like so.

```python
import json
from libumccr.aws import libsm

tok = json.loads(libsm.get_secret('orcabus/token-service-jwt'))['id_token']
```

## Service User

As an admin, you must register the service user. This has to be done at Cognito AAI terraform stack. Please follow `AdminCreateUser` [flow noted](https://github.com/umccr/infrastructure/pull/412/files) in `users.tf` at upstream infrastructure repo.

After Token Service stack has been deployed, you should then make JSON payload to [put-secret-value](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/secretsmanager/put-secret-value.html) at the following coordinate.

```
orcabus/token-service-user
```

e.g.

```
export AWS_PROFILE=umccr-dev-admin
aws secretsmanager put-secret-value \
--secret-id "orcabus/token-service-user" \
--secret-string "{\"username\": \"<snip>\", \"password\": \"<snip>\", \"email\": \"<snip>\"}"
```

After then, the scheduled secret rotation should carry on rotating password, every set days.

---

## Stack

### Rotation Lambda
The stack contains 2 Lambda Python code that do secret rotation. This code is derived from AWS secret manager rotation code template for PostgreSQL. See details in each Python module file docstring.

### Cognitor
And, there is the thin service layer package called `cognitor` for interfacing with AWS Cognito through boto3 - in fact it just [a façade](https://www.google.com/search?q=fa%C3%A7ade+pattern) of boto3 for Cognito. See its test cases for how to use and operate it.

### Local Dev

#### App

No major dependencies except boto3 which is already avail in the Lambda Python runtime. So, we do not need to package it. For local dev, just create Python venv and, have it boto3 in.

Do like so:
```
cd lib/workload/stateful/token_service
python -m venv venv
source venv/bin/activate
make install
make test
deactivate
```

#### CDK

The deploy directory contains the CDK code for `TokenServiceStack`. It deploys the rotation Lambdas and, corresponding application artifact using `PythonFunction` (L4?) construct. It runs in the `main-vpc`. And, the secret permissions are bound to the allow resources strictly i.e. see those `grant(...)` flags. It has 1 unit test file and, cdk-nag test through CodePipeline.

Do like so from the repo root:
```
cd ../../../../
```

```
yarn test --- test/stateful/tokenServiceConstruct.test.ts
yarn test --- test/stateful/stateful-deployment.test.ts
yarn test --- test/stateful/
```

```
export AWS_PROFILE=umccr-dev-admin
yarn cdk-stateful ls
yarn cdk-stateful synth -e OrcaBusStatefulPipeline/BetaDeployment/OrcaBusStatefulStack/TokenServiceStack
```

Perhaps copy to clipboard and, paste it into VSCode new file buffer and, observe the CloudFormation template being generated.
```
yarn cdk-stateful synth -e OrcaBusStatefulPipeline/BetaDeployment/OrcaBusStatefulStack/TokenServiceStack | pbcopy
```

Or

```
yarn cdk-stateful synth -e OrcaBusStatefulPipeline/BetaDeployment/OrcaBusStatefulStack/TokenServiceStack > template.yml && code template.yml
```

If that all good, then you may diff e & push straight to dev for giving it the WIP a try...

```
export AWS_PROFILE=umccr-dev-admin
yarn cdk-stateful diff -e OrcaBusStatefulPipeline/BetaDeployment/OrcaBusStatefulStack/TokenServiceStack
yarn cdk-stateful deploy -e OrcaBusStatefulPipeline/BetaDeployment/OrcaBusStatefulStack/TokenServiceStack
yarn cdk-stateful destroy -e OrcaBusStatefulPipeline/BetaDeployment/OrcaBusStatefulStack/TokenServiceStack
```

Run it in dev, check cloudwatch logs to debug, tear it down; rinse & spin.!

When ready, PR and merge it into the `main` branch to let CodePipeline CI/CD takes care of shipping it towards the `prod`.
Loading

0 comments on commit 4f46ca9

Please sign in to comment.