- This is a multi-tier bank an application written in Java (Springboot).
-
Login to AWS Console: Open the AWS Management Console.
-
Navigate to IAM: Go to the Identity and Access Management (IAM) service.
-
Create User:
-
Click on Users > Add Users.
-
Enter a username, e.g.,
mega-project-user. -
Select Programmatic access to generate an access key.
-
-
Attach Permissions: Attach the policy
AdministratorAccess. -
Generate Access Key:
-
In the Security tab, create an access key.
-
Save the Access Key ID and Secret Access Key securely.
-
If you are on Windows, refer to this guide to integrate an Ubuntu terminal in VSCode for seamless project execution.
-
Fork the Repository:
-
Open the repository DevOps Mega Project on GitHub.
-
Click Fork to create a copy in your GitHub account.
-
-
Clone the Repository:
-
Open the terminal in VSCode.
-
Clone the repository:
git clone https://github.com/<your-username>/DevOps-mega-project.git
-
Switch to the project branch:
git checkout project
-
-
Install AWS CLI:
sudo apt update sudo apt install awscli -y
-
Configure AWS CLI:
aws configure
-
Enter the Access Key ID and Secret Access Key.
-
Specify your preferred AWS region (e.g.,
eu-west-1).
-
-
Build the Docker Image:
docker build -t <dockerhub-username>/bankapp:latest .
-
Login to DockerHub:
docker login
-
Push the Image to DockerHub:
docker push <dockerhub-username>/bankapp:latest
-
Update Deployment File:
- Update the
bankapp-deployment.ymlfile to use your Docker image.
- Update the
-
Generate SSH Key:
ssh-keygen
-
Enter a name as
mega-project-key. -
Update the
variable.tffile with the key name , if you have entered another key name.
-
-
Initialize Terraform: Run the initialization command to download provider plugins and prepare your working directory:
terraform init
-
Plan Terraform Execution: Preview the resources Terraform will create:
terraform plan
-
Apply Terraform Configuration: Deploy the infrastructure using:
terraform apply --auto-approve
-
Connect to EC2 Instance: Once the infrastructure is created, connect to your EC2 instance:
ssh -i mega-project-key.pem ubuntu@<instance-public-ip>
Follow this guide to install:
-
AWS CLI
-
kubectl
-
eksctl
ARCH=amd64
PLATFORM=$(uname -s)_$ARCH
curl -sLO "https://github.com/eksctl-io/eksctl/releases/latest/download/eksctl_$PLATFORM.tar.gz"
tar -xzf eksctl_$PLATFORM.tar.gz -C /tmp && rm eksctl_$PLATFORM.tar.gz
sudo mv /tmp/eksctl /usr/local/bin-
Create EKS Cluster:
eksctl create cluster --name bankapp-cluster --region eu-west-1 --without-nodegroup
-
Verify Cluster Creation:
eksctl get clusters
-
Associate IAM OIDC Provider:
eksctl utils associate-iam-oidc-provider --region=eu-west-1 --cluster=bankapp-cluster --approve
-
Create Node Group:
eksctl create nodegroup \ --cluster=bankapp-cluster \ --region=eu-west-1 \ --name=bankapp-ng \ --node-type=t2.medium \ --nodes=2 \ --nodes-min=2 \ --nodes-max=2 \ --node-volume-size=15 \ --ssh-access \ --ssh-public-key=mega-project-key
To ensure ArgoCD has its own isolated environment within your Kubernetes cluster, create a dedicated namespace.
kubectl create ns argocdUse the official installation manifest from ArgoCD’s GitHub repository to deploy it to your cluster.
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yamlThis command installs all required ArgoCD components in the argocd namespace.
To interact with the ArgoCD server from your local machine or a terminal, install the ArgoCD command-line interface (CLI).
curl -sSL -o argocd-linux-amd64 https://github.com/argoproj/argo-cd/releases/latest/download/argocd-linux-amd64
sudo install -m 555 argocd-linux-amd64 /usr/local/bin/argocd
rm argocd-linux-amd64Once installed, verify the installation using:
argocd versionTo confirm that ArgoCD services are running:
kubectl get svc -n argocdThis lists all services in the argocd namespace. Take note of the argocd-server service, as it will be exposed in the next step.
By default, the argocd-server service is of type ClusterIP, which makes it accessible only within the cluster. Change it to NodePort to expose it externally.
kubectl patch svc argocd-server -n argocd -p '{"spec":{"type": "NodePort"}}'Retrieve the updated service information to identify the assigned NodePort:
kubectl get svc -n argocdNote the port in the PORT(S) column (e.g., 30529).
If your Kubernetes cluster is hosted on AWS, ensure that the assigned NodePort is accessible by adding an inbound rule to your security group. Allow traffic on this port from the internet to the worker node(s).
With the NodePort and the worker node’s public IP, access the ArgoCD web UI:
http://<worker-node-public-ip>:<node-port>
For the initial login:
-
Username:
admin -
Password: Retrieve using the following command:
kubectl get secret argocd-initial-admin-secret -n argocd -o jsonpath="{.data.password}" | base64 -dChange the password after logging in by navigating to the user info section in the ArgoCD UI.
To log in from the CLI, use the public IP and NodePort:
argocd login <worker-node-public-ip>:<node-port> --username adminFor example:
argocd login 54.154.41.147:30529 --username adminTo view the cluster configurations managed by ArgoCD:
argocd cluster listIf your cluster is not already added, first identify its context:
kubectl config get-contextsThen, add the desired cluster to ArgoCD. Replace the placeholders with your actual cluster context and name:
argocd cluster add <kube-context> --name <friendly-name>For example:
argocd cluster add [email protected] --name bankapp-clusterTo integrate your Git repository with ArgoCD:
-
Navigate to Settings > Repositories in the ArgoCD UI.
-
Click on Connect Repo and provide the appropriate repository URL.
-
Select the connection method as HTTPS. If the repository is private:
-
Enter your username and password to authenticate.
-
Otherwise, skip the authentication step for public repositories.
-
-
Choose the default project (or any specific project, if configured) and complete the setup.
Once connected, your repository will be ready for deploying applications via ArgoCD.
Helm is a powerful Kubernetes package manager that simplifies the deployment and management of applications within your Kubernetes clusters. To get started, follow the steps below to install Helm on your local system:
# Download the Helm installation script
curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3
# Change script permissions to make it executable
chmod 700 get_helm.sh
# Run the installation script
./get_helm.shAfter running the script, Helm will be installed, and you can start using it to deploy applications to your Kubernetes cluster.
An Ingress Controller is necessary to manage external HTTP/HTTPS access to your services in Kubernetes. In this step, we will install the NGINX Ingress Controller using Helm.
To install the NGINX Ingress Controller, execute the following commands:
# Add the NGINX Ingress controller Helm repository
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
# Update the Helm repository to ensure you have the latest charts
helm repo update
# Install the ingress-nginx controller in the ingress-nginx namespace
helm install ingress-nginx ingress-nginx/ingress-nginx --namespace ingress-nginx --create-namespaceThis command installs the NGINX Ingress controller into your Kubernetes cluster, creating a new namespace called ingress-nginx. This Ingress controller will handle routing and load balancing for your services.
To enable Horizontal Pod Autoscaling (HPA) in your Kubernetes cluster, the metrics-server is required to collect resource usage data like CPU and memory from the pods. HPA scales your application based on these metrics.
Run the following command to apply the metrics-server:
# Install metrics-server to collect resource usage metrics
kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yamlOnce installed, the metrics-server will start collecting data from your Kubernetes nodes and pods, enabling you to configure HPA based on these metrics.
For securing your application with HTTPS using your custom domain name, you need to generate SSL/TLS certificates. Cert-Manager is a Kubernetes tool that automates the management and issuance of these certificates.
To install Cert-Manager, use the following command:
# Apply Cert-Manager components to your cluster
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.16.2/cert-manager.yamlOnce installed, Cert-Manager will be responsible for automatically issuing and renewing SSL/TLS certificates for your services. You can then configure Cert-Manager to issue a certificate for your application and configure HTTPS with your domain.
-
Application Name: Choose a name for your application.
-
Project Name: Select default.
-
Sync Policy: Choose Automatic.
-
Enable Prune Resources and Self-Heal.
-
Check Auto Create Namespace.
-
Repo URL: Enter the URL of your Git repository.
-
Revision: Select the branch (e.g.,
main). -
Path: Specify the directory containing your Kubernetes manifests (e.g.,
k8s).
-
Cluster: Select your desired cluster.
-
Namespace: Use
bankapp-namespace.
Click Create to finish the setup and deploy your application.
In this step, we will walk through two options to expose your application to the outside world: one using an ALB (Application Load Balancer) with a CNAME record and the other using NodePort if you don't have a domain.
-
Run the following command to get the ALB External-IP of the
ingress-nginx-controller:kubectl get svc -n ingress-nginx
-
Copy the External-IP from the output and create a CNAME record on your domain. For example, use
amitabh.letsdeployit.comas your domain, and replace it in thebankapp-ingress.ymlfile with your domain name, take a reference of below image: -
After updating the
bankapp-ingress.ymlfile, sync the application in ArgoCD. -
Once synchronized, open your browser and access the application via your domain (e.g.,
amitabh.letsdeployit.com). -
If you don't have a domain, you can expose the service using NodePort.
-
Before patching, check the existing services in the
bankapp-namespace:kubectl get svc -n bankapp-namespace
-
Patch the bankapp-service to expose it via NodePort:
kubectl patch svc bankapp-service -n bankapp-namespace -p '{"spec": {"type": "NodePort"}}' -
After patching, check the service again to get the NodePort:
kubectl get svc -n bankapp-namespace
-
Now, access your application in the browser using the URL format:
http://<worker_node_public_ip>:<nodeport>.
Install Jenkins on the master node by following this blog: How to Install Essential DevOps Tools on Ubuntu Linux.
After installation, open port 8080 on the master node and access Jenkins in your browser:
http://<master-node-public-ip>:8080Complete the Jenkins setup by following the on-screen instructions to configure the admin username and password.
To integrate Jenkins with Docker, you need to install Docker and add both the current user and the Jenkins user to the Docker group:
-
Install Docker (if not already installed).
-
Add the current user to the Docker group:
sudo usermod -aG docker $USER && newgrp docker
-
Add the Jenkins user to the Docker group:
sudo usermod -aG docker jenkins
-
Restart Jenkins:
sudo systemctl restart jenkins
Add your DockerHub credentials to Jenkins. You can refer to this blog for detailed steps: Django Notes App using Jenkins CI/CD.
Add GitHub credentials to Jenkins as well to enable seamless integration with your GitHub repository.
To automatically trigger Jenkins builds on changes in your GitHub repository, set up a webhook. Follow the instructions in this blog: Set Up Webhooks for Automatic Deployment.
Create a Jenkins Pipeline job using the reference in this blog: Create a Jenkins Pipeline Job.
While creating the job, ensure that you check the box for This project is parameterized to allow dynamic configuration during the build.
Once everything is set up, trigger the pipeline build by selecting Build with Parameters. Enter the required parameters and start the build process. Monitor the build logs for any errors. If any issues arise, resolve them.
-
Check the Docker Hub for the tagged images after the build.
-
Ensure that the bankapp-deployment is using the correct image tag from Docker Hub. take a reference of below image
-
Start by adding the Prometheus Helm repository:
helm repo add prometheus-community https://prometheus-community.github.io/helm-chartsCreate a dedicated namespace for Prometheus:
kubectl create namespace prometheusInstall the Prometheus and Grafana stack using Helm in the prometheus namespace:
helm install stable prometheus-community/kube-prometheus-stack -n prometheusTo view the services running in the prometheus namespace, use the following command:
kubectl get svc -n prometheusExpose Grafana through NodePort by patching the service:
kubectl patch svc stable-grafana -n prometheus -p '{"spec": {"type": "NodePort"}}'Run the following command again to get the NodePort and open it in your browser using the Master Node's Public IP:
kubectl get svc -n prometheusTo access Grafana, use the admin username and retrieve the password by running the following command:
kubectl get secret --namespace prometheus stable-grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echoNow that Prometheus and Grafana are set up, you can use Grafana to monitor your application metrics. Grafana will pull metrics from Prometheus, and you can create dashboards to visualize various aspects of your application’s performance.
In conclusion, your DevSecOps Mega Project showcases a well-structured and automated pipeline using industry-standard tools. You've effectively integrated AWS, Docker, Kubernetes (EKS), Helm, and ArgoCD for deployment automation. By leveraging Terraform for infrastructure as code and implementing security best practices like IAM roles, SSL certificates, and Horizontal Pod Autoscaling, your setup ensures a secure, scalable, and efficient environment. The project demonstrates strong knowledge in cloud infrastructure, containerization, and CI/CD practices, positioning you well for real-world DevSecOps implementation.
below is for opensource opportunity for contrubutors.
We are thrilled to announce an exciting open-source opportunity for contributors to this project. As a token of appreciation, contributors will receive Free Gift Hampers from TrainWithShubham. The hampers may include a variety of items to make your contribution even more rewarding.
We are looking for contributions in the following areas:
After mapping the domain to the application, there may be issues with user login. If you have experience in troubleshooting or resolving login issues post-domain mapping, we encourage you to contribute a solution. Possible areas for improvement include authentication, DNS configurations, or session management.
Security should be a priority for any application. By incorporating DevSecOps practices into the project, we can significantly improve its security. Contributions may involve:
-
Implementing automated security scans within the CI/CD pipeline.
-
Integrating tools like OWASP ZAP, Trivy, or SonarQube for vulnerability assessments.
-
Securing Kubernetes, Docker, and application infrastructure.
We are always looking for ways to improve the project. If you have ideas for additional features or optimizations, we would love to see them:
-
Enhancing performance and scalability.
-
Improving monitoring and logging capabilities.
-
Streamlining deployment and automation processes.
-
Improving user experience and interface design.
To contribute, simply fork the repository, make your changes, and submit a pull request (PR). If your contribution is accepted, you will receive a Free Gift Hamper from TrainWithShubham as a thank-you for your efforts.
We welcome all developers, whether beginners or experienced, to join the open-source community and contribute to this project. Let’s work together to make this project even better!
We look forward to your innovative contributions, and remember, great work deserves great rewards!