Skip to content

Commit

Permalink
Merge pull request #2 from unity-sds/403-ssm-httpd-config
Browse files Browse the repository at this point in the history
Moving HTTPD configuration to SSM parameters
  • Loading branch information
galenatjpl authored May 30, 2024
2 parents a516b27 + a1770a7 commit 7ee5ec2
Show file tree
Hide file tree
Showing 12 changed files with 229 additions and 325 deletions.
6 changes: 3 additions & 3 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
FROM ubuntu/apache2
LABEL authors="barber"

RUN apt update && apt install -y libapache2-mod-auth-openidc ca-certificates && a2enmod auth_openidc proxy proxy_http proxy_wstunnel rewrite headers && \
RUN apt update && apt install -y libapache2-mod-auth-openidc ca-certificates python3-boto3 && a2enmod auth_openidc proxy proxy_http proxy_html proxy_wstunnel substitute rewrite headers && \
sed -i 's/Listen 80/Listen 8080/' /etc/apache2/ports.conf

COPY write_site.sh /usr/local/bin/
COPY write_site.py /usr/local/bin/

CMD ["/bin/bash", "-c", "/usr/local/bin/write_site.sh && apache2-foreground"]
CMD ["/bin/bash", "-c", "python3 /usr/local/bin/write_site.py && apache2-foreground"]
98 changes: 68 additions & 30 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,61 +6,99 @@ To enable/disable modules, update and change the HTTPD installation then there i

The webservers default port is also 8080 to let it traverse the MCP NACL.

When deployed this terraform code creates an ECS cluster, with an EFS backend that then allows us to store apache configs
in a filesystem that wont vanish when it restarts. As such the EFS filesystem also needs a way to create new files, this is
done via a lambda function that writes valid apache config files to the EFS mount.
When deployed this terraform code creates an ECS cluster, with a baseline set of SSM parameters that other services can then extend with their own Apache HTTPD configurations. The configurations are pulled down and collated by the container on restart, so reloading of the configuration after changes is handled by triggering a lambda function.

A sample trigger:
A sample configuration snippet and trigger:
```
variable "template" {
default = <<EOT
RewriteEngine on
ProxyPass /sample http://test-demo-alb-616613476.us-west-2.elb.amazonaws.com:8888/sample/hello.jsp
ProxyPassReverse /sample http://test-demo-alb-616613476.us-west-2.elb.amazonaws.com:8888/sample/hello.jsp
resource "aws_ssm_parameter" "managementproxy_config" {
depends_on = [aws_ssm_parameter.managementproxy_closevirtualhost]
name = "/unity/${var.project}/${var.venue}/cs/management/proxy/configurations/010-management"
type = "String"
value = <<-EOT
RewriteEngine on
RewriteCond %%{HTTP:Upgrade} websocket [NC]
RewriteCond %%{HTTP:Connection} upgrade [NC]
RewriteRule /management/(.*) ws://${var.mgmt_dns}/$1 [P,L]
<Location "/management/">
ProxyPass http://${var.mgmt_dns}/
ProxyPassReverse http://${var.mgmt_dns}/
ProxyPreserveHost On
FallbackResource /management/index.html
</Location>
EOT
}
resource "aws_lambda_invocation" "demoinvocation2" {
function_name = "ZwUycV-unity-proxy-httpdproxymanagement"
input = jsonencode({
filename = "example_filename1",
template = var.template
})
function_name = "${var.project}-${var.venue}-httpdproxymanagement"
}
```
The config files are written as flat configs. They are then used inside a main apache2 config like this:


The configuration is collated from SSM parameters residing under `/unity/${var.project}/${var.venue}/cs/management/proxy/configurations/`, and assembled like so:
```
<VirtualHost *:8080>
Include /etc/apache2/sites-enabled/mgmt.conf
### ADD MORE HOSTS BELOW THIS LINE
RewriteEngine on
RewriteCond %{HTTP:Upgrade} websocket [NC]
RewriteCond %{HTTP:Connection} upgrade [NC]
RewriteRule /management/(.*) ws://internal-unity-mc-alb-hzs9j-1269535099.us-west-2.elb.amazonaws.com:8080/$1 [P,L]
<Location "/management/">
ProxyPass http://internal-unity-mc-alb-hzs9j-1269535099.us-west-2.elb.amazonaws.com:8080/
ProxyPassReverse http://internal-unity-mc-alb-hzs9j-1269535099.us-west-2.elb.amazonaws.com:8080/
ProxyPreserveHost On
FallbackResource /management/index.html
</Location>
</VirtualHost>
```

They will be added as additional config files below the comment line. The httpd task is then restarted to allow the
config to then take effect.
Live checking of the "current" configuration may be accomplished with `write_site.py` in a local environment:
```
% DEBUG=yes UNITY_PROJECT=btlunsfo UNITY_VENUE=dev11 python write_site.py
<VirtualHost *:8080>
There is currently no way to remove files or fix a broken config other than mounting the EFS mount into an EC2 server and making changes.
To do this you will need to edit the security group to allow access to the EC2 box and then install the EFS utils.
RewriteEngine on
RewriteCond %{HTTP:Upgrade} websocket [NC]
RewriteCond %{HTTP:Connection} upgrade [NC]
RewriteRule /management/(.*) ws://internal-unity-mc-alb-hzs9j-1269535099.us-west-2.elb.amazonaws.com:8080/$1 [P,L]
<Location "/management/">
ProxyPass http://internal-unity-mc-alb-hzs9j-1269535099.us-west-2.elb.amazonaws.com:8080/
ProxyPassReverse http://internal-unity-mc-alb-hzs9j-1269535099.us-west-2.elb.amazonaws.com:8080/
ProxyPreserveHost On
FallbackResource /management/index.html
</Location>
## Manually adding a file/template
</VirtualHost>
One can execute the httpdmanager lambda function directly with the following json syntax:
```

This repository configures only one virtualhost (both open and close directives), but others may be added. This can be accomplished by simply adding more SSM parameters:
```
{
"filename": "example-extension",
"template": "SSLProxyEngine On\nProxyPreserveHost On\n\nProxyPass \/hub https:\/\/jupyter.us-west-2.elb.amazonaws.com:443\/hub\/\nProxyPassReverse \/hub https:\/\/jupyter.us-west-2.elb.amazonaws.com:443\/hub\/"
resource "aws_ssm_parameter" "managementproxy_openvirtualhost" {
name = "/unity/${var.project}/${var.venue}/cs/management/proxy/configurations/001-openvhost8080"
type = "String"
value = <<-EOT
<VirtualHost *:8080>
EOT
}
resource "aws_ssm_parameter" "managementproxy_closevirtualhost" {
depends_on = [aws_ssm_parameter.managementproxy_openvirtualhost]
name = "/unity/${var.project}/${var.venue}/cs/management/proxy/configurations/100-closevhost8080"
type = "String"
value = <<-EOT
</VirtualHost>
EOT
}
```
NOTE the names of each of these SSM parameters:
- 001-openvhost8080
- 010-management
- 100-closevhost8080

The template must be json encoded. I've used https://nddapp.com/json-encoder.html successfully.

For additional virtualhosts, please pick an ordinal number range that is *greater* than 100 (e.g. 101-openTestHost, 120-closeTestHost).

## How do I know what to add in the 'template' file above?
We are not perfect human beings. In order to iterate quickly on the above templat contents, we have created a development proxy environment that can be tested mostly locally. Check out the `develop` directory for instructions.
55 changes: 11 additions & 44 deletions lambda/lambda.py
Original file line number Diff line number Diff line change
@@ -1,67 +1,34 @@
import boto3
import os

def insert_new_host_line(file_path, new_line):
# Marker to find the position where the new line will be inserted
marker = "### ADD MORE HOSTS BELOW THIS LINE"

# Read the original file content
with open(file_path, 'r') as file:
lines = file.readlines()

# Find the marker and insert the new line after it
for i, line in enumerate(lines):
if marker in line:
# Insert new line after the marker line
lines.insert(i + 1, new_line + "\n")
break # Exit the loop once the marker is found and the line is inserted

# Write the modified content back to the file
with open(file_path, 'w') as file:
file.writelines(lines)
import boto3


def lambda_handler(event, context):

filename = event.get('filename')
text_blob = event.get('template')

efs_mount_path = '/mnt/efs'
file_path = os.path.join(efs_mount_path, filename+".conf")
with open(file_path, 'w') as file:
file.write(text_blob)


# Update main file
file_path = "/mnt/efs/main.conf"
new_line = " Include /etc/apache2/sites-enabled/"+filename+".conf"
insert_new_host_line(file_path, new_line)

# Restart an ECS task
ecs_client = boto3.client('ecs')
service_name = os.environ.get('SERVICE_NAME')
cluster_name = os.environ.get('CLUSTER_NAME')
ecs_client = boto3.client("ecs")
service_name = os.environ.get("SERVICE_NAME")
cluster_name = os.environ.get("CLUSTER_NAME")

# List the tasks for a given cluster and service
tasks_response = ecs_client.list_tasks(
cluster=cluster_name,
serviceName=service_name, # Use this if tasks are part of a service
desiredStatus='RUNNING' # Optional: Adjust this based on the task status you're interested in
desiredStatus="RUNNING", # Optional: Adjust this based on the task status you're interested in
)

task_arns = tasks_response.get('taskArns')
task_arns = tasks_response.get("taskArns")
if task_arns:
# Assuming you want to restart the first task in the list
task_id = task_arns[0]

# Stop the task (it should be restarted automatically if it's part of a service)
ecs_client.stop_task(cluster=cluster_name, task=task_id)

return_message = 'File written and ECS task restarted'
return_message = "ECS task restarted"
else:
return_message = 'No running tasks found for the specified service in the cluster'
return_message = (
"No running tasks found for the specified service in the cluster"
)

return {
'statusCode': 200,
'body': return_message
}
return {"statusCode": 200, "body": return_message}
72 changes: 18 additions & 54 deletions terraform-unity/ecs.tf
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
resource "aws_ecs_cluster" "httpd_cluster" {
name = "${var.deployment_name}-httpd-cluster"
name = "${var.project}-${var.venue}-httpd-cluster"
tags = {
Service = "U-CS"
}
Expand All @@ -9,34 +9,8 @@ data "aws_iam_policy" "mcp_operator_policy" {
name = "mcp-tenantOperator-AMI-APIG"
}

resource "aws_iam_policy" "efs_access" {
name = "${var.deployment_name}-EFSAccessPolicy"
description = "Policy for ECS tasks to access EFS"

policy = jsonencode({
Version = "2012-10-17",
Statement = [
{
Effect = "Allow",
Action = [
"elasticfilesystem:ClientMount",
"elasticfilesystem:ClientWrite",
"elasticfilesystem:ClientRootAccess",
"elasticfilesystem:DescribeMountTargets",
],
Resource = "*",
},
],
})
}

resource "aws_iam_role_policy_attachment" "efs_access_attachment" {
role = aws_iam_role.ecs_task_role.name
policy_arn = aws_iam_policy.efs_access.arn
}

resource "aws_iam_role" "ecs_task_role" {
name = "${var.deployment_name}-ecs_task_role"
name = "${var.project}-${var.venue}-ecs_task_role"

assume_role_policy = jsonencode({
Version = "2012-10-17",
Expand All @@ -54,8 +28,13 @@ resource "aws_iam_role" "ecs_task_role" {

}

resource "aws_iam_role_policy_attachment" "ecs_ssm_role_policy" {
role = aws_iam_role.ecs_task_role.name
policy_arn = "arn:aws:iam::aws:policy/AmazonSSMReadOnlyAccess"
}

resource "aws_iam_role" "ecs_execution_role" {
name = "${var.deployment_name}ecs_execution_role"
name = "${var.project}-${var.venue}ecs_execution_role"

assume_role_policy = jsonencode({
Version = "2012-10-17",
Expand All @@ -78,9 +57,8 @@ resource "aws_iam_role_policy_attachment" "ecs_execution_role_policy" {
policy_arn = "arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy"
}


resource "aws_cloudwatch_log_group" "proxyloggroup" {
name = "/ecs/${var.deployment_name}-managementproxy"
name = "/ecs/${var.project}-${var.venue}-managementproxy"
}

resource "aws_ecs_task_definition" "httpd" {
Expand All @@ -91,28 +69,19 @@ resource "aws_ecs_task_definition" "httpd" {
execution_role_arn = aws_iam_role.ecs_execution_role.arn
memory = "512"
cpu = "256"
volume {
name = "httpd-config"

efs_volume_configuration {
file_system_id = aws_efs_file_system.httpd_config_efs.id
transit_encryption = "ENABLED"
transit_encryption_port = 2049
authorization_config {
access_point_id = aws_efs_access_point.httpd_config_ap.id
iam = "ENABLED"
}
}
}


container_definitions = jsonencode([{
name = "httpd"
image = "ghcr.io/unity-sds/unity-proxy/httpd-proxy:0.13.0"
image = "ghcr.io/unity-sds/unity-proxy/httpd-proxy:${var.httpd_proxy_version}"
environment = [
{
name = "ELB_DNS_NAME",
value = var.mgmt_dns
name = "UNITY_PROJECT",
value = var.project
},
{
name = "UNITY_VENUE",
value = var.venue
}
]
logConfiguration = {
Expand All @@ -129,20 +98,14 @@ resource "aws_ecs_task_definition" "httpd" {
hostPort = 8080
}
]
mountPoints = [
{
containerPath = "/etc/apache2/sites-enabled/"
sourceVolume = "httpd-config"
}
]
}])
tags = {
Service = "U-CS"
}
}

resource "aws_security_group" "ecs_sg" {
name = "${var.deployment_name}-ecs_service_sg"
name = "${var.project}-${var.venue}-ecs_service_sg"
description = "Security group for ECS service"
vpc_id = data.aws_ssm_parameter.vpc_id.value

Expand Down Expand Up @@ -194,5 +157,6 @@ resource "aws_ecs_service" "httpd_service" {
}
depends_on = [
aws_lb_listener.httpd_listener,
aws_ssm_parameter.managementproxy_config
]
}
Loading

0 comments on commit 7ee5ec2

Please sign in to comment.