Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

actions: promote release action #142

Merged
merged 4 commits into from
Oct 16, 2024
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions .github/workflows/promote.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
name: Promote Release

on:
# Triggered by the dist server
workflow_dispatch:
ovflowd marked this conversation as resolved.
Show resolved Hide resolved
inputs:
path:
description: 'path to promote'
required: true

jobs:
promote-release:
name: Promote Release
runs-on: ubuntu-latest

steps:
- name: Setup Node
uses: actions/setup-node@v4
with:
node-version: lts/*
cache: 'npm'

- name: Promote Files
env:
CF_ACCESS_KEY_ID: ${{ secrets.CF_ACCESS_KEY_ID }}
CF_SECRET_ACCESS_KEY: ${{ secrets.CF_SECRET_ACCESS_KEY }}
runs: |
node scripts/promote-release.js ${{ inputs.path }}
119 changes: 119 additions & 0 deletions scripts/promote-release.js
flakey5 marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
#!/usr/bin/env node

import {
S3Client,
ListObjectsV2Command,
CopyObjectCommand,
} from '@aws-sdk/client-s3';

if (process.argv.length !== 3) {
console.error(`usage: promote-release <prefix in dist-staging to promote>`);
process.exit(1);
}

if (!process.env.CF_ACCESS_KEY_ID) {
console.error('CF_ACCESS_KEY_ID missing');
process.exit(1);
}

if (!process.env.CF_SECRET_ACCESS_KEY) {
console.error('CF_SECRET_ACCESS_KEY missing');
process.exit(1);
}

const ENDPOINT =
flakey5 marked this conversation as resolved.
Show resolved Hide resolved
'https://07be8d2fbc940503ca1be344714cb0d1.r2.cloudflarestorage.com';
flakey5 marked this conversation as resolved.
Show resolved Hide resolved
const PROD_BUCKET = 'dist-prod';
const STAGING_BUCKET = 'dist-staging';
const RETRY_LIMIT = 3;

const client = new S3Client({
endpoint: ENDPOINT,
region: 'auto',
credentials: {
accessKeyId: process.env.CF_ACCESS_KEY_ID,
secretAccessKey: process.env.CF_SECRET_ACCESS_KEY,
},
});

const path = process.argv[2];
const files = await getFilesToPromote(path);

for (const file of files) {
promoteFile(file);
}

/**
* @param {string} path
* @returns {string[]}
*/
async function getFilesToPromote(path) {
let paths = [];

let truncated = true;
let continuationToken;
while (truncated) {
const data = await retryWrapper(async () => {
return await client.send(
new ListObjectsV2Command({
Bucket: STAGING_BUCKET,
Delimiter: '/',
Prefix: path,
ContinuationToken: continuationToken,
})
);
});

if (data.CommonPrefixes) {
for (const directory of data.CommonPrefixes) {
paths.push(...(await getFilesToPromote(directory.Prefix)));
}
}

if (data.Contents) {
for (const object of data.Contents) {
paths.push(object.Key);
}
}

truncated = data.IsTruncated ?? false;
continuationToken = data.NextContinuationToken;
}

return paths;
}

/**
* @param {string} file
*/
async function promoteFile(file) {
flakey5 marked this conversation as resolved.
Show resolved Hide resolved
console.log(`Promoting ${file}`);

await client.send(
new CopyObjectCommand({
Bucket: PROD_BUCKET,
CopySource: `${STAGING_BUCKET}/${file}`,
Key: file,
})
);
}

/**
* @param {() => Promise<T>} request
* @returns {Promise<T>}
*/
async function retryWrapper(request, retryLimit) {
flakey5 marked this conversation as resolved.
Show resolved Hide resolved
let r2Error;

for (let i = 0; i < RETRY_LIMIT; i++) {
try {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use Promise alike then.catch

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Like

for (let i = 0; i < R2_RETRY_COUNT; i++) {
  result
    .then(/*...*/)
    .catch(/*...*/)
}

?

If so it would send the same request multiple times regardless if it succeeded or not since there's nothing blocking the execution context (like await is doing here)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see you want to await on it. You can still do:

const somethingThatReturnsPromise = invokeSomething.catch();

await somethingThatReturnsPromise;

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Worth noting this is just a copy of this function here

export async function retryWrapper<T>(
request: () => Promise<T>,
retryLimit: number,
sentry?: Toucan
): Promise<T> {
let r2Error: unknown = undefined;
for (let i = 0; i < retryLimit; i++) {
try {
const result = await request();
return result;
} catch (err) {
r2Error = err;
}
}
if (sentry !== undefined) {
sentry.captureException(r2Error);
}
throw r2Error;
}

Why .catch() over the try/catch?

const result = await request();
return result;
} catch (err) {
r2Error = err;
process.emitWarning(`error when contacting r2: ${err}`);
}
}

throw r2Error;
}
Loading