Improvements to deployments (#615)

Improve deployments and CI/CD workflows Summary: Introduced new GitHub Actions workflows for Docker image building and deployments. Enhanced Control Plane CLI and environment setup configurations. Streamlined concurrency management and deployment status tracking. Added new scripts and environment variables for commit tracking and deployment actions. Added documentation for CI automation and Review Apps.
shakacode · Jan 26, 2025 · 5a01718 · 5a01718
1 parent 005eab7
commit 5a01718
Show file tree

Hide file tree

Showing 20 changed files with 1,010 additions and 138 deletions.
diff --git a/.controlplane/Dockerfile b/.controlplane/Dockerfile
@@ -2,6 +2,10 @@
 ARG RUBY_VERSION=3.3.4
 FROM registry.docker.com/library/ruby:$RUBY_VERSION-slim as base
 
+# Current commit hash environment variable
+ARG GIT_COMMIT
+ENV GIT_COMMIT_SHA=${GIT_COMMIT}
+
 # Install packages needed to build gems and node modules
 RUN apt-get update -qq && \
     apt-get install --no-install-recommends -y build-essential curl git libpq-dev libvips node-gyp pkg-config python-is-python3
@@ -76,7 +80,3 @@ ENTRYPOINT ["./.controlplane/entrypoint.sh"]
 # Default args to pass to the entry point that can be overridden
 # For Kubernetes and ControlPlane, these are the "workload args"
 CMD ["./bin/rails", "server"]
-
-# Current commit hash environment variable
-ARG GIT_COMMIT_SHA
-ENV GIT_COMMIT_SHA=${GIT_COMMIT_SHA}
diff --git a/.controlplane/readme.md b/.controlplane/readme.md
@@ -123,3 +123,32 @@ cpflow build-image -a $APP_NAME --commit ABCD
 ### `entrypoint.sh`
 - waits for Postgres and Redis to be available
 - runs `rails db:prepare` to create/seed or migrate the database
+
+## CI Automation, Review Apps and Staging
+
+_Note, some of the URL references are internal for the ShakaCode team._
+
+ Review Apps (deployment of apps based on a PR) are done via Github Actions.
+
+The review apps work by creating isolated deployments for each branch through this automated process. When a branch is pushed, the action:
+
+1. Sets up the necessary environment and tools
+2. Creates a unique deployment for that branch if it doesn't exist
+3. Builds a Docker image tagged with the branch's commit SHA
+4. Deploys this image to Control Plane with its own isolated environment
+
+This allows teams to:
+- Preview changes in a production-like environment
+- Test features independently
+- Share working versions with stakeholders
+- Validate changes before merging to main branches
+
+The system uses Control Plane's infrastructure to manage these deployments, with each branch getting its own resources as defined in the controlplane.yml configuration.
+
+
+### Workflow for Developing Github Actions for Review Apps
+
+1. Create a PR with changes to the Github Actions workflow
+2. Make edits to file such as `.github/actions/deploy-to-control-plane/action.yml`
+3. Run a script like `ga .github && gc -m fixes && gp` to commit and push changes (ga = git add, gc = git commit, gp = git push)
+4. Check the Github Actions tab in the PR to see the status of the workflow
diff --git a/.controlplane/shakacode-team.md b/.controlplane/shakacode-team.md
@@ -0,0 +1,6 @@
+# Internal Notes to the Shakacode Team
+
+## Links
+
+- [Control Plane Org for Staging and Review Apps](https://console.cpln.io/console/org/shakacode-open-source-examples-staging/-info)
+- [Control Plane Org for Deployed App](https://console.cpln.io/console/org/shakacode-open-source-examples/-info)
diff --git a/.dockerignore b/.dockerignore
@@ -19,7 +19,10 @@ dump.rdb
 .DS_Store
 
 # Ignore bundle dependencies
-vendor/ruby
+vendor/bundle
+
+# Ignore GitHub Actions and workflows
+.github/
 
 # RVM gemset
 .ruby-gemset
@@ -45,6 +48,5 @@ yarn-debug.log*
 ###################################################
 # Specific to .dockerignore
 .git/
-.github/
 spec/
 scripts/
diff --git a/.github/actions/build-docker-image/action.yml b/.github/actions/build-docker-image/action.yml
@@ -0,0 +1,32 @@
+name: Build Docker Image
+description: 'Builds a Docker image for the application'
+
+inputs:
+  app_name:
+    description: 'Name of the application'
+    required: true
+  org:
+    description: 'Organization name'
+    required: true
+  commit:
+    description: 'Commit SHA to tag the image with'
+    required: true
+  PR_NUMBER:
+    description: 'PR number'
+    required: true
+
+runs:
+  using: "composite"
+  steps:
+    - name: Build Docker Image
+      id: build
+      shell: bash
+      run: |
+        echo "🏗️ Building Docker image for PR #${PR_NUMBER} (commit ${{ inputs.commit }})..."
+        
+        if cpflow build-image -a "${{ inputs.app_name }}" --commit="${{ inputs.commit }}" --org="${{ inputs.org }}"; then
+          echo "✅ Docker image build successful for PR #${PR_NUMBER} (commit ${{ inputs.commit }})"
+        else
+          echo "❌ Docker image build failed for PR #${PR_NUMBER} (commit ${{ inputs.commit }})"
+          exit 1
+        fi
diff --git a/.github/actions/delete-control-plane-app/action.yml b/.github/actions/delete-control-plane-app/action.yml
@@ -0,0 +1,20 @@
+name: Delete Control Plane App
+description: 'Deletes a Control Plane application and all its resources'
+
+inputs:
+  app_name:
+    description: 'Name of the application to delete'
+    required: true
+  org:
+    description: 'Organization name'
+    required: true
+
+runs:
+  using: "composite"
+  steps:
+    - name: Delete Application
+      shell: bash
+      run: ${{ github.action_path }}/../deploy-to-control-plane/scripts/delete-app.sh
+      env:
+        APP_NAME: ${{ inputs.app_name }}
+        CPLN_ORG: ${{ inputs.org }}
diff --git a/.github/actions/deploy-to-control-plane/action.yml b/.github/actions/deploy-to-control-plane/action.yml
@@ -1,56 +1,91 @@
 # Control Plane GitHub Action
 
-name: Deploy-To-Control-Plane
-description: 'Deploys both to staging and to review apps'
+name: Deploy to Control Plane
+description: 'Deploys an application to Control Plane'
 
 inputs:
   app_name:
-    description: 'The name of the app to deploy'
+    description: 'Name of the application'
     required: true
-    default:
   org:
-    description: 'The org of the app to deploy'
+    description: 'Organization name'
     required: true
-    default:
+  github_token:
+    description: 'GitHub token'
+    required: true
+  wait_timeout:
+    description: 'Timeout in seconds for waiting for workloads to be ready'
+    required: false
+    default: '900'
+
+outputs:
+  review_app_url:
+    description: 'URL of the deployed application'
+    value: ${{ steps.deploy.outputs.review_app_url }}
 
 runs:
-  using: 'composite'
+  using: "composite"
   steps:
     - name: Setup Environment
       uses: ./.github/actions/setup-environment
 
-    - name: Set Short SHA
-      id: vars
+    - name: Get Commit SHA
+      id: get_sha
       shell: bash
-      run: echo "::set-output name=sha_short::$(git rev-parse --short HEAD)"
-
-    # Caching step
-    - uses: actions/cache@v2
-      with:
-        path: /tmp/.docker-cache
-        key: ${{ runner.os }}-docker-${{ hashFiles('**/Dockerfile', '**/package.json', '**/yarn.lock') }}-${{ github.sha   }}
-        restore-keys: |
-          ${{ runner.os }}-docker-${{ hashFiles('**/Dockerfile', '**/package.json', '**/yarn.lock') }}
-          ${{ runner.os }}-docker-
+      run: ${{ github.action_path }}/scripts/get-commit-sha.sh
+      env:
+        GITHUB_TOKEN: ${{ inputs.github_token }}
+        PR_NUMBER: ${{ env.PR_NUMBER }}
 
-    - name: cpflow setup-app
-      shell: bash
-      run: |
-        if ! cpflow exists -a ${{ inputs.app_name }} ; then
-          cpflow setup-app -a ${{ inputs.app_name }}
-        fi
-    # Provision all infrastructure on Control Plane.
-    # app react-webpack-rails-tutorial will be created per definition in .controlplane/controlplane.yml
-    - name: cpflow build-image
-      shell: bash
-      run: |
-        cpln image docker-login
-        # Use BUILDKIT_PROGRESS=plain to get more verbose logging of the build
-        # BUILDKIT_PROGRESS=plain cpflow build-image -a ${{ inputs.app_name }} --commit ${{steps.vars.outputs.sha_short}} --org ${{inputs.org}}
-        cpflow build-image -a ${{ inputs.app_name }} --commit ${{steps.vars.outputs.sha_short}} --org ${{inputs.org}}
-        # --cache /tmp/.docker-cache
     - name: Deploy to Control Plane
+      id: deploy
       shell: bash
       run: |
-        echo "Deploying to Control Plane"
-        cpflow deploy-image -a ${{ inputs.app_name }} --run-release-phase --org ${{inputs.org}}  --verbose
+        echo "🚀 Deploying app for PR #${PR_NUMBER}..."
+        
+        # Create temp file for output
+        TEMP_OUTPUT=$(mktemp)
+        trap 'rm -f "${TEMP_OUTPUT}"' EXIT
+        
+        # Deploy the application and show output in real-time while capturing it
+        if ! cpflow deploy-image -a "${{ inputs.app_name }}" --run-release-phase --org "${{ inputs.org }}" 2>&1 | tee "${TEMP_OUTPUT}"; then
+          echo "❌ Deployment failed for PR #${PR_NUMBER}"
+          echo "Error output:"
+          cat "${TEMP_OUTPUT}"
+          exit 1
+        fi
+        
+        # Extract app URL from captured output
+        REVIEW_APP_URL=$(grep -oP 'https://rails-[^[:space:]]*\.cpln\.app(?=\s|$)' "${TEMP_OUTPUT}" | head -n1)
+        if [ -z "${REVIEW_APP_URL}" ]; then
+          echo "❌ Failed to get app URL from deployment output"
+          echo "Deployment output:"
+          cat "${TEMP_OUTPUT}"
+          exit 1
+        fi
+        
+        # Wait for all workloads to be ready
+        WAIT_TIMEOUT=${WAIT_TIMEOUT:-${{ inputs.wait_timeout }}}
+        if ! [[ "${WAIT_TIMEOUT}" =~ ^[0-9]+$ ]]; then
+          echo "❌ Invalid timeout value: ${WAIT_TIMEOUT}"
+          exit 1
+        fi
+        echo "⏳ Waiting for all workloads to be ready (timeout: ${WAIT_TIMEOUT}s)"
+        
+        # Use timeout command with ps:wait and show output in real-time
+        if ! timeout "${WAIT_TIMEOUT}" bash -c "cpflow ps:wait -a \"${{ inputs.app_name }}\"" 2>&1 | tee -a "${TEMP_OUTPUT}"; then
+          TIMEOUT_EXIT=$?
+          if [ ${TIMEOUT_EXIT} -eq 124 ]; then
+            echo "❌ Timed out waiting for workloads after ${WAIT_TIMEOUT} seconds"
+          else
+            echo "❌ Workloads did not become ready for PR #${PR_NUMBER} (exit code: ${TIMEOUT_EXIT})"
+          fi
+          echo "Full output:"
+          cat "${TEMP_OUTPUT}"
+          exit 1
+        fi
+        
+        echo "✅ Deployment successful for PR #${PR_NUMBER}"
+        echo "🌐 App URL: ${REVIEW_APP_URL}"
+        echo "review_app_url=${REVIEW_APP_URL}" >> $GITHUB_OUTPUT
+        echo "REVIEW_APP_URL=${REVIEW_APP_URL}" >> $GITHUB_ENV
diff --git a/.github/actions/deploy-to-control-plane/scripts/delete-app.sh b/.github/actions/deploy-to-control-plane/scripts/delete-app.sh
@@ -0,0 +1,36 @@
+#!/bin/bash
+
+# Script to delete a Control Plane application
+# Required environment variables:
+# - APP_NAME: Name of the application to delete
+# - CPLN_ORG: Organization name
+
+set -e
+
+# Validate required environment variables
+: "${APP_NAME:?APP_NAME environment variable is required}"
+: "${CPLN_ORG:?CPLN_ORG environment variable is required}"
+
+# Safety check: prevent deletion of production or staging apps
+if echo "$APP_NAME" | grep -iqE '(production|staging)'; then
+    echo "❌ ERROR: Cannot delete apps containing 'production' or 'staging' in their name" >&2
+    echo "🛑 This is a safety measure to prevent accidental deletion of production or staging environments" >&2
+    echo "   App name: $APP_NAME" >&2
+    exit 1
+fi
+
+# Check if app exists before attempting to delete
+echo "🔍 Checking if application exists: $APP_NAME"
+if ! cpflow exists -a "$APP_NAME"; then
+    echo "⚠️ Application does not exist: $APP_NAME"
+    exit 0
+fi
+
+# Delete the application
+echo "🗑️ Deleting application: $APP_NAME"
+if ! cpflow delete -a "$APP_NAME" --force; then
+    echo "❌ Failed to delete application: $APP_NAME" >&2
+    exit 1
+fi
+
+echo "✅ Successfully deleted application: $APP_NAME"
diff --git a/.github/actions/deploy-to-control-plane/scripts/deploy.sh b/.github/actions/deploy-to-control-plane/scripts/deploy.sh
@@ -0,0 +1,51 @@
+#!/bin/bash
+
+# This script handles the deployment to Control Plane and extracts the Rails URL
+# 
+# Required environment variables:
+# - APP_NAME: Name of the application to deploy
+# - CPLN_ORG: Control Plane organization
+#
+# Optional environment variables:
+# - WAIT_TIMEOUT: Timeout in seconds for deployment (default: 900)
+#                 Must be a positive integer
+#
+# Outputs:
+# - rails_url: URL of the deployed Rails application
+
+set -e
+
+# Validate required environment variables
+: "${APP_NAME:?APP_NAME environment variable is required}"
+: "${CPLN_ORG:?CPLN_ORG environment variable is required}"
+
+# Set and validate deployment timeout
+WAIT_TIMEOUT=${WAIT_TIMEOUT:-900}
+if ! [[ "${WAIT_TIMEOUT}" =~ ^[0-9]+$ ]]; then
+  echo "❌ Invalid timeout value: ${WAIT_TIMEOUT}"
+  exit 1
+fi
+
+TEMP_OUTPUT=$(mktemp)
+trap 'rm -f "$TEMP_OUTPUT"' EXIT
+
+# Deploy the application
+echo "🚀 Deploying to Control Plane (timeout: ${WAIT_TIMEOUT}s)"
+if timeout "$WAIT_TIMEOUT" cpflow deploy-image -a "$APP_NAME" --run-release-phase --org "$CPLN_ORG" --verbose | tee "$TEMP_OUTPUT"; then
+    # Extract Rails URL from deployment output
+    RAILS_URL=$(grep -oP 'https://rails-[^[:space:]]*\.cpln\.app(?=\s|$)' "$TEMP_OUTPUT" | head -n1)
+    if [ -n "$RAILS_URL" ]; then
+        echo "rails_url=$RAILS_URL" >> "$GITHUB_OUTPUT"
+        echo "✅ Deployment successful"
+        echo "🚀 Rails URL: $RAILS_URL"
+    else
+        echo "❌ Failed to extract Rails URL from deployment output"
+        exit 1
+    fi
+elif [ $? -eq 124 ]; then
+    echo "❌ Deployment timed out after $WAIT_TIMEOUT seconds"
+    exit 1
+else
+    echo "❌ Deployment to Control Plane failed"
+    exit 1
+fi
diff --git a/.github/actions/deploy-to-control-plane/scripts/get-commit-sha.sh b/.github/actions/deploy-to-control-plane/scripts/get-commit-sha.sh
@@ -0,0 +1,34 @@
+#!/bin/bash
+
+# This script retrieves the commit SHA for deployment
+# It handles both PR and direct branch deployments
+# 
+# Required environment variables:
+# - PR_NUMBER: Pull request number (optional)
+# - GITHUB_TOKEN: GitHub token for API access
+#
+# Outputs:
+# - sha: Full commit SHA
+# - sha_short: Short (7 char) commit SHA
+
+set -e
+
+if [ -n "${PR_NUMBER}" ]; then
+    # If PR_NUMBER is set, get the PR's head SHA
+    if ! PR_SHA=$(gh pr view "${PR_NUMBER}" --json headRefOid --jq '.headRefOid'); then
+        echo "Failed to get PR head SHA" >&2
+        exit 1
+    fi
+    echo "sha=${PR_SHA}" >> "$GITHUB_OUTPUT"
+    echo "sha_short=${PR_SHA:0:7}" >> "$GITHUB_OUTPUT"
+    echo "Using PR head commit SHA: ${PR_SHA:0:7}"
+else
+    # For direct branch deployments, use the current commit SHA
+    if ! CURRENT_SHA=$(git rev-parse HEAD); then
+        echo "Failed to get current SHA" >&2
+        exit 1
+    fi
+    echo "sha=${CURRENT_SHA}" >> "$GITHUB_OUTPUT"
+    echo "sha_short=${CURRENT_SHA:0:7}" >> "$GITHUB_OUTPUT"
+    echo "Using branch commit SHA: ${CURRENT_SHA:0:7}"
+fi