Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BE-INFRA] 무중단 배포 be/release로 브랜치 변경 #424

Merged
merged 26 commits into from
Oct 25, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
bf03d6b
chore: 서브모듈 커밋 시점 변경
dwax1324 Oct 23, 2024
15933e5
docs: 액츄에이터 의존성 추가
dwax1324 Oct 23, 2024
3290af6
infra: 무중단 배포를 위한 cd 스크립트 변경
dwax1324 Oct 23, 2024
792534a
infra: 테스트와 빌드 하나로 통합
dwax1324 Oct 23, 2024
da58be7
[BE-INFRA] 무중단 배포 구현에 따른 action 스크립트 변환 (#399)
dwax1324 Oct 23, 2024
b35acd2
infra: 스크립트 수정
dwax1324 Oct 23, 2024
dd27940
Modify/#398 infra zero down time (#400)
dwax1324 Oct 23, 2024
e7a8c96
infra: 스크립트 수정
dwax1324 Oct 23, 2024
76a6c31
Modify/#398 infra zero down time (#401)
dwax1324 Oct 23, 2024
d1ecaeb
infra: 스크립트 수정
dwax1324 Oct 23, 2024
6b13f9a
Modify/#398 infra zero down time (#403)
dwax1324 Oct 23, 2024
6354c3d
infra: 스크립트 수정
dwax1324 Oct 23, 2024
c84b181
Modify/#398 infra zero down time (#404)
dwax1324 Oct 23, 2024
5fb42d4
infra: 스크립트 수정
dwax1324 Oct 23, 2024
e325332
infra: 스크립트 수정
dwax1324 Oct 24, 2024
35f0747
Modify/#398 infra zero down time (#405)
dwax1324 Oct 24, 2024
4f259f6
infra: 스크립트 수정
dwax1324 Oct 24, 2024
c79aff8
Modify/#398 infra zero down time (#421)
dwax1324 Oct 24, 2024
29c1231
Update Backend-CD-Prod-A.yml
jinchiim Oct 24, 2024
51f6dfd
infra: 서비스 상태 체크 및 종료 로직 추가
dwax1324 Oct 24, 2024
b84c61f
Modify/#398 infra zero down time (#422)
dwax1324 Oct 24, 2024
53c30c5
infra: 에러를 반환하는 코드 수정
dwax1324 Oct 24, 2024
e541551
Modify/#398 infra zero down time (#423)
dwax1324 Oct 24, 2024
f683751
fix: curl 에러 수정
dwax1324 Oct 24, 2024
63061e5
merge
dwax1324 Oct 24, 2024
a226822
modify: 브랜치를 be/release로 변경
dwax1324 Oct 24, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 44 additions & 42 deletions .github/workflows/Backend-CD-Prod-A.yml
Original file line number Diff line number Diff line change
@@ -1,58 +1,26 @@
name: Backend Production Server CD
name: Backend Production Server CD A

on:
push:
branches: [ "be/release" ]
tags:
- 'v*'
workflow_call:

permissions:
contents: read

jobs:
test:
uses: ./.github/workflows/Backend-CI.yml
secrets: inherit


build:
needs: test
runs-on: [cd, app-a]
steps:

- name: Checkout
uses: actions/checkout@v3
with:
token: ${{ secrets.ACTION_TOKEN }}
submodules: true

- name: Set up JDK 17
uses: actions/setup-java@v3
with:
java-version: '17'
distribution: 'temurin'

- name: Build with Gradle
run: ./gradlew bootJar
working-directory: ./backend/pokerogue

- name: Docker build and push
run: |
docker login -u ${{ secrets.DOCKER_USERNAME }} -p ${{ secrets.DOCKER_PASSWORD }}

docker build -t ${{ secrets.DOCKER_SERVER_IMAGE }} -f ./backend/pokerogue/docker/Dockerfile ./backend/pokerogue
docker push ${{ secrets.DOCKER_SERVER_IMAGE }}


deploy:
needs: build
runs-on: [cd, app-a]
runs-on: [ cd, app-a ]
steps:

- name: Change permission
run: |
sudo chown -R ubuntu:ubuntu /home/ubuntu/actions-runner/_work/2024-pokerogue-helper

- name: Port forwarding begin
run: |
sudo sysctl -w net.ipv4.ip_forward=1
sudo iptables -t nat -I PREROUTING 1 -p tcp --dport 80 -j DNAT --to-destination ${{ secrets.BE_PROD_B_PRIVATE_IP }}:80
sudo iptables -t nat -I POSTROUTING 1 -p tcp --dport 80 -j SNAT --to-source ${{ secrets.BE_PROD_A_PRIVATE_IP }}
sudo iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SNAT를 쓴 이유가 궁금해요! DNAT는 서브넷 B WAS로 요청을 전달하는 것 같고 👀

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

이거 지금 봤네요!

응답이 A EC2로 돌아오게 하기 위해서 SNAT을 사용했습니다!
SNAT 을 사용하지 않으면 패킷이 ELB 로부터 온 것으로 착각하고 LoadBalancer로 향하고, A 인스턴스는 패킷 응답을 기다리며 TimeOut이 발생하게 됩니다!

image


- name: Deploy
run: |
cd /home/ubuntu
Expand All @@ -65,9 +33,43 @@ jobs:
echo "No container named 'server' to remove."
fi

sudo docker system prune -f

docker pull ${{ secrets.DOCKER_SERVER_IMAGE }}

docker run -d -p 80:8080 --name server \
-e JAVA_OPTS="-XX:InitialRAMPercentage=70.0 -XX:MaxRAMPercentage=70.0" \
-e TZ=Asia/Seoul \
${{ secrets.DOCKER_SERVER_IMAGE }}

- name: Check service availability
run: |
SUCCESS=false

for i in {1..10}; do
HTTP_STATUS=$(curl -o /dev/null -s -w "%{http_code}" http://localhost:80/actuator/health) || {
echo "Attempt $i: curl command failed. Retrying...";
sleep 3;
continue;
}

if [ $? -eq 0 ] && [ "$HTTP_STATUS" -eq 200 ]; then
echo "Service is up and running"
SUCCESS=true
break
fi
done

- name: Port forwarding end
run: |
sudo iptables -t nat -D PREROUTING 1
sudo iptables -t nat -D POSTROUTING 1
sudo iptables -D INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
sudo iptables -L -v -t nat

- name: Exit if fails
run: |
if [ "$SUCCESS" = false ]; then
echo "Service did not respond successfully after $ATTEMPTS attempts, stopping github actions.."
sudo exit 1
fi
53 changes: 53 additions & 0 deletions .github/workflows/Backend-CD-Prod-All.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
name: Backend Production Server CD All

permissions:
contents: read

on:
merge_group:
branches: [ "be/release" ]
tags:
- 'v*'

jobs:
test:
uses: ./.github/workflows/Backend-CI.yml
secrets: inherit

build:
needs: test
runs-on: ci
steps:
- name: Checkout
uses: actions/checkout@v3
with:
token: ${{ secrets.ACTION_TOKEN }}
submodules: true

- name: Set up JDK 17
uses: actions/setup-java@v3
with:
java-version: '17'
distribution: 'temurin'

- name: Build with Gradle
run: ./gradlew bootJar
working-directory: ./backend/pokerogue

- name: Docker build and push
run: |
docker login -u ${{ secrets.DOCKER_USERNAME }} -p ${{ secrets.DOCKER_PASSWORD }}

docker build -t ${{ secrets.DOCKER_SERVER_IMAGE }} -f ./backend/pokerogue/docker/Dockerfile ./backend/pokerogue
docker push ${{ secrets.DOCKER_SERVER_IMAGE }}

prod_a:
needs: build
name: Backend Production Server CD A
uses: ./.github/workflows/Backend-CD-Prod-A.yml
secrets: inherit
prod_b:
needs: prod_a
name: Backend Production Server CD B
uses: ./.github/workflows/Backend-CD-Prod-B.yml
secrets: inherit
86 changes: 44 additions & 42 deletions .github/workflows/Backend-CD-Prod-B.yml
Original file line number Diff line number Diff line change
@@ -1,58 +1,26 @@
name: Backend Production Server CD
name: Backend Production Server CD B

on:
push:
branches: [ "be/release" ]
tags:
- 'v*'
workflow_call:

permissions:
contents: read

jobs:
test:
uses: ./.github/workflows/Backend-CI.yml
secrets: inherit


build:
needs: test
runs-on: [cd, app-b]
steps:

- name: Checkout
uses: actions/checkout@v3
with:
token: ${{ secrets.ACTION_TOKEN }}
submodules: true

- name: Set up JDK 17
uses: actions/setup-java@v3
with:
java-version: '17'
distribution: 'temurin'

- name: Build with Gradle
run: ./gradlew bootJar
working-directory: ./backend/pokerogue

- name: Docker build and push
run: |
docker login -u ${{ secrets.DOCKER_USERNAME }} -p ${{ secrets.DOCKER_PASSWORD }}

docker build -t ${{ secrets.DOCKER_SERVER_IMAGE }} -f ./backend/pokerogue/docker/Dockerfile ./backend/pokerogue
docker push ${{ secrets.DOCKER_SERVER_IMAGE }}


deploy:
needs: build
runs-on: [cd, app-b]
runs-on: [ cd, app-b ]
steps:

- name: Change permission
run: |
sudo chown -R ubuntu:ubuntu /home/ubuntu/actions-runner/_work/2024-pokerogue-helper

- name: Port forwarding begin
run: |
sudo sysctl -w net.ipv4.ip_forward=1
sudo iptables -t nat -I PREROUTING 1 -p tcp --dport 80 -j DNAT --to-destination ${{ secrets.BE_PROD_A_PRIVATE_IP }}:80
sudo iptables -t nat -I POSTROUTING 1 -p tcp --dport 80 -j SNAT --to-source ${{ secrets.BE_PROD_B_PRIVATE_IP }}
sudo iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT

- name: Deploy
run: |
cd /home/ubuntu
Expand All @@ -65,9 +33,43 @@ jobs:
echo "No container named 'server' to remove."
fi

sudo docker system prune -f

docker pull ${{ secrets.DOCKER_SERVER_IMAGE }}

docker run -d -p 80:8080 --name server \
-e JAVA_OPTS="-XX:InitialRAMPercentage=70.0 -XX:MaxRAMPercentage=70.0" \
-e TZ=Asia/Seoul \
${{ secrets.DOCKER_SERVER_IMAGE }}

- name: Check service availability
run: |
SUCCESS=false

for i in {1..10}; do
HTTP_STATUS=$(curl -o /dev/null -s -w "%{http_code}" http://localhost:80/actuator/health) || {
echo "Attempt $i: curl command failed. Retrying...";
sleep 3;
continue;
}

if [ $? -eq 0 ] && [ "$HTTP_STATUS" -eq 200 ]; then
echo "Service is up and running"
SUCCESS=true
break
fi
done

- name: Port forwarding end
run: |
sudo iptables -t nat -D PREROUTING 1
sudo iptables -t nat -D POSTROUTING 1
sudo iptables -D INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
sudo iptables -L -v -t nat

- name: Exit if fails
run: |
if [ "$SUCCESS" = false ]; then
echo "Service did not respond successfully after 10 attempts, stopping github actions.."
sudo exit 1
fi
3 changes: 3 additions & 0 deletions backend/pokerogue/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,9 @@ dependencies {

// log-back-encoder
implementation 'net.logstash.logback:logstash-logback-encoder:7.4'

// actuator
implementation 'org.springframework.boot:spring-boot-starter-actuator'
}

tasks.named('test') {
Expand Down
2 changes: 1 addition & 1 deletion backend/pokerogue/src/main/resources
Loading