Protein DevOps Engineer Bootcamp

Final Project

Built with;

📓 About

This repository contains the final project developed under the Patika.dev & Protein DevOps Engineer Bootcamp. While some directories are isolated developments, the overall purpose of the project is deploying a containerized application on optimized AWS infrastructure with the ability of Terraform using Gitlab CI/CD pipelines.

If you are looking for something specific, check out the directory content first!

📖 Table of Contents

1. Assignment
   • Essential Tasks
   • Optional Tasks
2. Overview
3. Featured
4. Project Board
5. Project Workflow
6. AWS Infrastructure
   • Infrastructure Visualization
7. Directory Content

#️⃣ Assignment 🔝

Fully cover the DevOps cycle and create a full-on CI/CD and cloud infrastructure.

☑️ Essential Tasks

1. Create a simple React application.
2. Containerize the image with the smallest possible size.
3. Design a Gitlab CI/CD pipeline to automate all processes.
4. Create the ideal VPC and implement security measures.
5. Create an Application Load Balancer in front of the service.
6. Deploy Docker image on ECS Fargate.
7. Deploy the application on Kubernetes.
8. Describe strategies and choices on related files.

*️⃣ Optional Tasks

1. Configure an auto-scaling on the ECS service.
2. Create a Cloudwatch dashboard with proper metrics.
3. Draw the AWS infrastructure and describe the relations between services.
4. Configure a self-hosted runner on Gitlab and share its configuration with the implementation steps.
5. Create a script that sends an email if the disk usage of the system exceeds 90% in the OS.

📔 Overview 🔝

The project presents an example of a fully automated DevOps CI/CD cycle with the ability to deploy on AWS infrastructure. Check out the featured section for a quick preview of the project features.

✨ Featured 🔝

• Simple React application dockerized with the smallest image size. (~ 24 MB)
• Terraform stack that is built with module composition to fully utilize AWS capabilities.
• AWS infrastructure that implements most of the security measures and best practices.
• An automated main CI/CD pipeline on Gitlab.
• Simple workflow on Github to mirror this repository to Gitlab. (Also produces another Docker Image to Docker Hub for the demonstrative purposes)
• Both pipelines are working on self-hosted runners.
• Example project board to demonstrate Scrum framework by using Github Projects & Issues.
• YAML templates for issues and pull requests for making data inputs easier.
• Both pipelines are working on self-hosted runners.
• Auxilary shell scripts and k8s deployment manifests.
• Vagrantfile to easier build a VM test environment for shell scripts.

📅 Project Board 🔝

To demonstrate the Agile framework, a scaled-down Scrum methodology is used to utilize total project time with the most efficiency. Originally, I intended to use Jira Software but due to the timeframe of the project, I decided to use Github Projects and Issues are utilized as a backlog of the board and the board cards are populated from here. The image below is an example taken in the middle of the project.

🔀 Project Workflow 🔝

A simple React project is created and used as a sample application. Github is used as an auxiliary repository to dockerize the application's development environment and pushes the Docker image to Docker Hub along with mirroring the entire repository to Gitlab, which is the main repository. After the mirroring to Gitlab, the main pipeline begins building the project as a production environment and containerizing the application with Docker. This image is pushed to Gitlab's and AWS' (ECR) container registries. A self-hosted runner is hosted on both Github's and Gitlab's pipelines to run these pipelines.

Upon finishing the main pipeline, the child pipeline built for Terraform starts initializing process and plans the requested resources from AWS. Developing a reusable code (IaC) to utilize AWS was critical since the project requires many of the AWS main and sub-services. The execution step(Terraform apply) must be triggered manually since automatic execution may cause unwanted outcomes. The project also contains other deployment methods such as Kubernetes and an additional shell script as a part of the assignment.

Gitlab Pipeline;

🔌 AWS Infrastructure 🔝

Used Services;

Icon	Service Name	Purpose
	AWS Identity and Access Management	Required roles of task definition and ECS service.
	AWS Budgets	Calculating the average cost of the project and creating alarms.
	AWS Simple Storage Service	Terraform state remote storage.
	Amazon DynamoDB	Terraform state locking mechanism for the remote backend.
	Amazon Virtual Private Cloud	The isolated virtual network infrastructure of the project.
	AWS Elastic Load Balancing	Traffic distribution of the containers.
	Amazon Elastic Container Registry	Container registry for the Docker Image of the application.
	Amazon Elastic Container Service	Deployment service of the project.
	AWS Fargate	Capacity provider of the ECS cluster.
	AWS Application Auto Scaling	High availability ability of the system.
	Amazon CloudWatch	Monitoring the network traffic and resources.

Infrastructure Visualization

📂 Directory Content 🔝

/           : Gitlab CI/CD parent pipeline manifest.
.github/    : Github Actions workflow manifests, issue and pull request templates.
.gitlab/    : Terraform deployment manifest, self-hosted runner configuration.
ansible/    : Simple shell script for checking the disk usage and sending email alerts.
assets/     : AWS infrastucture diagram draw.io export.
docker/     : React application, Dockerfiles for production and development environments.
k8s/        : Kubernetes deployment manifests.
terraform/  : Modular Terraform Stack featuring most AWS services.