Promotion to R2's dist-prod bucket never happens #3838

targos · 2024-07-17T16:21:32Z

/cc @flakey5

targos · 2024-07-17T16:23:35Z

It's possible that I'm using the wrong search prefix, but I get results with nodejs/release/v20.0.0/ for example.

richardlau · 2024-07-17T16:26:13Z

Suspect it'll be the same as we're seeing with the nightlies: #3826 (comment)

flakey5 · 2024-07-17T16:31:19Z

Suspect it'll be the same as we're seeing with the nightlies

+1

targos · 2024-07-18T05:46:57Z

closing as duplicate

richardlau · 2024-07-18T12:31:19Z

Moving conversation from #3826 for visibility as that PR was merged.

Summary:

build/ansible/www-standalone/tools/promote/upload_to_cloudflare.sh

Line 45 in 533d7e1

    
           aws s3 cp $staging_bucket/$relativedir/$tmpversion/ $dist_bucket/$relativedir/$tmpversion/ --endpoint-url=$cloudflare_endpoint --profile $cloudflare_profile --recursive --no-follow-symlinks

is failing to recursively copy from the staging bucket to prod. It appears that Cloudflare have not fully implemented S3 and aws s3 cp is setting x-amz-tagging-directive which is one of the unimplemented features.

Refs: https://community.cloudflare.com/t/cloudflare-r2-not-supporting-copy-object/672982

flakey5 · 2024-07-18T15:52:35Z

We might be able to make aws s3api copy-object work (docs), but I don't think it supports recursion or pattern matching from my tests

Simplest option might just be having two copy calls, one for the smaller files and one for the larger ones that require multipart. Nowhere near ideal however, gonna try to see if there's anything else we could do

flakey5 · 2024-07-19T15:26:28Z

Could we try aws s3 sync s3://dist-staging/<path> s3://dist-prod/<path>? Or just aws s3 sync s3://dist-staging s3://dist-prod

Also still not sure on why the uploads aren't happening, but can take a look later today

richardlau · 2024-07-19T15:40:12Z

I've changed the recursive aws s3 cp line in upload_to_cloudflare.sh with:

aws s3 sync $staging_bucket/$relativedir/$tmpversion/ $dist_bucket/$relativedir/$tmpversion/ --endpoint-url=$cloudflare_endpoint --profile $cloudflare_profile --no-follow-symlinks

and ran for today's nightly:

root@infra-digitalocean-ubuntu1604-x64-1:/home/staging/tools/promote# bash
root@infra-digitalocean-ubuntu1604-x64-1:/home/staging/tools/promote# site=nodejs
root@infra-digitalocean-ubuntu1604-x64-1:/home/staging/tools/promote# . settings
root@infra-digitalocean-ubuntu1604-x64-1:/home/staging/tools/promote# dstdir=$nightly_dstdir
root@infra-digitalocean-ubuntu1604-x64-1:/home/staging/tools/promote# . upload_to_cloudflare.sh nodejs v23.0.0-nightly20240719a523c345b1
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/SHASUMS256.txt to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/SHASUMS256.txt
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/addons.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/addons.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/addons.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/addons.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/assert.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/assert.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/assert.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/assert.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/assets/hljs.css to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/assets/hljs.css
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/assert.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/assert.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/all.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/all.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/all.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/all.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/assets/style.css to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/assets/style.css
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/assets/api.js to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/assets/api.js
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/async_context.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/async_context.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/addons.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/addons.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/assets/js-flavor-esm.svg to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/assets/js-flavor-esm.svg
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/async_context.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/async_context.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/assets/js-flavor-cjs.svg to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/assets/js-flavor-cjs.svg
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/async_context.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/async_context.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/buffer.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/buffer.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/async_hooks.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/async_hooks.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/buffer.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/buffer.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/child_process.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/child_process.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/child_process.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/child_process.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/async_hooks.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/async_hooks.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/child_process.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/child_process.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/buffer.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/buffer.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/async_hooks.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/async_hooks.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/cli.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/cli.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/cluster.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/cluster.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/cli.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/cli.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/console.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/console.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/cluster.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/cluster.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/cluster.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/cluster.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/console.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/console.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/console.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/console.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/cli.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/cli.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/corepack.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/corepack.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/corepack.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/corepack.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/debugger.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/debugger.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/corepack.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/corepack.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/debugger.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/debugger.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/crypto.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/crypto.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/crypto.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/crypto.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/deprecations.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/deprecations.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/crypto.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/crypto.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/debugger.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/debugger.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/deprecations.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/deprecations.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/dgram.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/dgram.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/deprecations.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/deprecations.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/diagnostics_channel.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/diagnostics_channel.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/dgram.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/dgram.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/dgram.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/dgram.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/diagnostics_channel.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/diagnostics_channel.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/dns.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/dns.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/documentation.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/documentation.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/dns.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/dns.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/domain.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/domain.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/documentation.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/documentation.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/dns.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/dns.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/documentation.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/documentation.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/domain.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/domain.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/diagnostics_channel.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/diagnostics_channel.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/domain.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/domain.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/embedding.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/embedding.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/embedding.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/embedding.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/embedding.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/embedding.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/errors.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/errors.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/errors.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/errors.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/esm.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/esm.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/errors.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/errors.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/esm.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/esm.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/esm.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/esm.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/events.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/events.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/events.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/events.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/events.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/events.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/fs.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/fs.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/fs.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/fs.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/fs.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/fs.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/globals.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/globals.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/globals.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/globals.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/globals.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/globals.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/http.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/http.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/http.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/http.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/http.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/http.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/http2.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/http2.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/https.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/https.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/http2.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/http2.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/https.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/https.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/https.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/https.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/index.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/index.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/http2.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/http2.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/index.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/index.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/inspector.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/inspector.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/inspector.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/inspector.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/intl.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/intl.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/intl.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/intl.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/intl.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/intl.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/module.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/module.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/inspector.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/inspector.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/module.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/module.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/module.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/module.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/modules.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/modules.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/modules.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/modules.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/n-api.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/n-api.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/n-api.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/n-api.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/n-api.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/n-api.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/modules.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/modules.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/net.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/net.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/net.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/net.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/index.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/index.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/net.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/net.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/os.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/os.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/packages.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/packages.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/packages.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/packages.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/os.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/os.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/os.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/os.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/path.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/path.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/packages.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/packages.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/path.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/path.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/path.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/path.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/permissions.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/permissions.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/perf_hooks.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/perf_hooks.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/perf_hooks.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/perf_hooks.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/permissions.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/permissions.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/permissions.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/permissions.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/process.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/process.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/perf_hooks.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/perf_hooks.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/process.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/process.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/punycode.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/punycode.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/process.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/process.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/punycode.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/punycode.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/querystring.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/querystring.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/punycode.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/punycode.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/readline.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/readline.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/querystring.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/querystring.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/readline.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/readline.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/querystring.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/querystring.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/readline.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/readline.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/repl.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/repl.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/repl.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/repl.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/report.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/report.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/report.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/report.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/repl.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/repl.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/single-executable-applications.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/single-executable-applications.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/sqlite.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/sqlite.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/sqlite.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/sqlite.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/single-executable-applications.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/single-executable-applications.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/single-executable-applications.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/single-executable-applications.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/sqlite.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/sqlite.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/report.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/report.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/stream.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/stream.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/stream.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/stream.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/stream.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/stream.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/string_decoder.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/string_decoder.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/string_decoder.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/string_decoder.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/string_decoder.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/string_decoder.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/synopsis.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/synopsis.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/synopsis.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/synopsis.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/synopsis.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/synopsis.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/test.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/test.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/test.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/test.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/timers.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/timers.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/tls.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/tls.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/timers.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/timers.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/tracing.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/tracing.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/tls.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/tls.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/tracing.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/tracing.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/timers.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/timers.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/tls.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/tls.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/tty.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/tty.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/tracing.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/tracing.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/test.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/test.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/tty.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/tty.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/url.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/url.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/url.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/url.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/url.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/url.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/tty.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/tty.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/util.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/util.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/v8.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/v8.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/v8.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/v8.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/util.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/util.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/util.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/util.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/v8.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/v8.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/vm.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/vm.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/wasi.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/wasi.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/vm.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/vm.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/wasi.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/wasi.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/wasi.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/wasi.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/webcrypto.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/webcrypto.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/webcrypto.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/webcrypto.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/vm.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/vm.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/webstreams.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/webstreams.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/webstreams.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/webstreams.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/webstreams.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/webstreams.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/webcrypto.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/webcrypto.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/worker_threads.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/worker_threads.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/worker_threads.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/worker_threads.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/zlib.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/zlib.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/worker_threads.html to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/worker_threads.html
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/zlib.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/zlib.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/zlib.md to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/api/zlib.md
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/apilinks.json to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/docs/apilinks.json
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/win-arm64/node.lib to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/win-arm64/node.lib
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/node-v23.0.0-nightly20240719a523c345b1-headers.tar.xz to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/node-v23.0.0-nightly20240719a523c345b1-headers.tar.xz
copy: s3://dist-staging/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/win-x64/node.lib to s3://dist-prod/nodejs/nightly/v23.0.0-nightly20240719a523c345b1/win-x64/node.lib
copy: s3://dist-staging/nodejs/nightly/index.json to s3://dist-prod/nodejs/nightly/index.json
copy: s3://dist-staging/nodejs/nightly/index.tab to s3://dist-prod/nodejs/nightly/index.tab
root@infra-digitalocean-ubuntu1604-x64-1:/home/staging/tools/promote#

this succeeded without errors (but I guess only copied the files that weren't already there for that nightly). So 🤞 for the next nightly.

richardlau · 2024-07-19T15:59:58Z

Suspect it'll be the same as we're seeing with the nightlies: #3826 (comment)

I think it's less likely this is exactly the same as the nightlies as the nightlies were being partially promoted (prior to #3838 (comment)). But releases don't appear at all (I released Node.js 22.5.1 earlier today).

Unfortunately I've lost my scrollback, but I did answer Y to the Upload files from %s to R2 staging bucket? [y/n] prompt and didn't get any errors. But I've not been able to find the uploaded .asc or .sig file in R2.

richardlau · 2024-07-19T16:19:22Z

oh! I've found where the signing artifacts are being copied to -- under / there's another nodejs dir!

(note this is not the same as the nodejs dir at root level)

So that's the signing artifacts... but still no sign of promoted assets for releases.

flakey5 · 2024-07-19T16:24:06Z

oh! I've found where the signing artifacts are being copied to -- under / there's another nodejs dir!

Ahhhhh okay, is there also the / dir on the staging bucket?

richardlau · 2024-07-19T16:30:33Z

No, only on prod.

richardlau · 2024-07-19T16:33:11Z

I think it's the way we generate r2dir by cutting:
https://github.com/nodejs/node/blob/03f7f8c1b82d0fc0ef2a4e1df8157034661d2c3b/tools/release.sh#L161

e.g.

$ echo /home/dist/nodejs/release/v22.5.1/SHASUMS256.txt | cut -c 11-
/nodejs/release/v22.5.1/SHASUMS256.txt
$

starts with a /. I'll open a PR to cut from position 12.

richardlau · 2024-07-19T16:42:43Z

I'll open a PR to cut from position 12.

nodejs/node#53951

That will hopefully fix the uploads of the signing artifacts -- but we'll still need to work out why the things in staging aren't being promoted to prod during releases.

richardlau · 2024-07-19T17:05:47Z

In terms of release promotion, what is supposed to happen:

tools/release.sh calls dist-promote on the server
dist-promote calls promote_release.sh:

build/ansible/www-standalone/tools/dist/dist-promote

Line 17 in d9c4ea9

/home/staging/tools/promote/promote_release.sh $1 $2
If the releaser proceeds (answers 'Y') then promote_release.sh calls _promote.sh passing in the site and version:

build/ansible/www-standalone/tools/promote/promote_release.sh

Line 38 in d9c4ea9

. ${__dirname}/_promote.sh $site $2

Because version is set (i.e. not empty) then this conditional is skipped (meaning upload_to_cloudflare.sh is not called here):

build/ansible/www-standalone/tools/promote/_promote.sh

Lines 52 to 55 in d9c4ea9

    
           if [ "X${version}" == "X" ] && [ "$resha" == "yes" ]; then 
        
             ${__dirname}/_resha.sh $site $dstdir $subdir 
        
           . ${__dirname}/upload_to_cloudflare.sh $site $subdir 
        
           fi

tools/release.sh then calls dist-sign on the server
dist-sign calls resha_release.sh:

build/ansible/www-standalone/tools/dist/dist-sign

Line 17 in d9c4ea9

/home/staging/tools/promote/resha_release.sh $1 $2
Which calls _resha.sh:

build/ansible/www-standalone/tools/promote/resha_release.sh

Line 26 in d9c4ea9

${__dirname}/_resha.sh $site $dstdir $2

which is supposed to update R2:

build/ansible/www-standalone/tools/promote/_resha.sh

Lines 57 to 59 in d9c4ea9

    
           aws s3 cp ${dstdir}/index.json $staging_bucket/$relativedir/index.json --endpoint-url=$cloudflare_endpoint --profile $cloudflare_profile > /dev/null 
        
           aws s3 cp ${dstdir}/index.tab $staging_bucket/$relativedir/index.tab --endpoint-url=$cloudflare_endpoint --profile $cloudflare_profile > /dev/null 
        
           aws s3 cp ${dstdir}/${version}/SHASUMS256.txt $staging_bucket/$relativedir/${version}/SHASUMS256.txt --endpoint-url=$cloudflare_endpoint --profile $cloudflare_profile > /dev/null

before resha_release.sh is supposed to call upload_to_cloudflare.sh:

build/ansible/www-standalone/tools/promote/resha_release.sh

Line 28 in d9c4ea9

. ${__dirname}/upload_to_cloudflare.sh $site $2
Then tools/release.sh downloads the generated SHASUMS256.txt file, signs it with the releaser's key, and then uploads the signing artifacts to R2 (which I can confirm happens, albeit at an incorrect path).

richardlau · 2024-07-22T11:53:20Z

Could we try aws s3 sync s3://dist-staging/<path> s3://dist-prod/<path>? Or just aws s3 sync s3://dist-staging s3://dist-prod

https://r2.nodejs.org/download/nightly/v23.0.0-nightly2024072179759fa0ac/

root@infra-digitalocean-ubuntu1604-x64-1:/home/staging/tools/promote# bash
root@infra-digitalocean-ubuntu1604-x64-1:/home/staging/tools/promote# site=nodejs
root@infra-digitalocean-ubuntu1604-x64-1:/home/staging/tools/promote# . settings
root@infra-digitalocean-ubuntu1604-x64-1:/home/staging/tools/promote# dstdir=$nightly_dstdir
root@infra-digitalocean-ubuntu1604-x64-1:/home/staging/tools/promote# . upload_to_cloudflare.sh nodejs v23.0.0-nightly2024072179759fa0ac
copy failed: s3://dist-staging/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/node-v23.0.0-nightly2024072179759fa0ac-aix-ppc64.tar.gz to s3://dist-prod/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/node-v23.0.0-nightly2024072179759fa0ac-aix-ppc64.tar.gz An error occurred (NotImplemented) when calling the GetObjectTagging operation: GetObjectTagging not implemented
copy failed: s3://dist-staging/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/node-v23.0.0-nightly2024072179759fa0ac-darwin-x64.tar.gz to s3://dist-prod/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/node-v23.0.0-nightly2024072179759fa0ac-darwin-x64.tar.gz An error occurred (NotImplemented) when calling the GetObjectTagging operation: GetObjectTagging not implemented
copy failed: s3://dist-staging/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/node-v23.0.0-nightly2024072179759fa0ac-darwin-x64.tar.xz to s3://dist-prod/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/node-v23.0.0-nightly2024072179759fa0ac-darwin-x64.tar.xz An error occurred (NotImplemented) when calling the GetObjectTagging operation: GetObjectTagging not implemented
copy failed: s3://dist-staging/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/node-v23.0.0-nightly2024072179759fa0ac-linux-arm64.tar.gz to s3://dist-prod/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/node-v23.0.0-nightly2024072179759fa0ac-linux-arm64.tar.gz An error occurred (NotImplemented) when calling the GetObjectTagging operation: GetObjectTagging not implemented
copy failed: s3://dist-staging/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/node-v23.0.0-nightly2024072179759fa0ac-arm64.msi to s3://dist-prod/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/node-v23.0.0-nightly2024072179759fa0ac-arm64.msi An error occurred (NotImplemented) when calling the GetObjectTagging operation: GetObjectTagging not implemented
copy failed: s3://dist-staging/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/node-v23.0.0-nightly2024072179759fa0ac-headers.tar.gz to s3://dist-prod/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/node-v23.0.0-nightly2024072179759fa0ac-headers.tar.gz An error occurred (NotImplemented) when calling the GetObjectTagging operation: GetObjectTagging not implemented
copy failed: s3://dist-staging/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/node-v23.0.0-nightly2024072179759fa0ac-linux-arm64.tar.xz to s3://dist-prod/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/node-v23.0.0-nightly2024072179759fa0ac-linux-arm64.tar.xz An error occurred (NotImplemented) when calling the GetObjectTagging operation: GetObjectTagging not implemented
copy failed: s3://dist-staging/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/node-v23.0.0-nightly2024072179759fa0ac-linux-armv7l.tar.xz to s3://dist-prod/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/node-v23.0.0-nightly2024072179759fa0ac-linux-armv7l.tar.xz An error occurred (NotImplemented) when calling the GetObjectTagging operation: GetObjectTagging not implemented
copy failed: s3://dist-staging/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/node-v23.0.0-nightly2024072179759fa0ac-linux-armv7l.tar.gz to s3://dist-prod/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/node-v23.0.0-nightly2024072179759fa0ac-linux-armv7l.tar.gz An error occurred (NotImplemented) when calling the GetObjectTagging operation: GetObjectTagging not implemented
copy failed: s3://dist-staging/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/node-v23.0.0-nightly2024072179759fa0ac-linux-s390x.tar.gz to s3://dist-prod/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/node-v23.0.0-nightly2024072179759fa0ac-linux-s390x.tar.gz An error occurred (NotImplemented) when calling the GetObjectTagging operation: GetObjectTagging not implemented
copy failed: s3://dist-staging/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/node-v23.0.0-nightly2024072179759fa0ac-linux-s390x.tar.xz to s3://dist-prod/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/node-v23.0.0-nightly2024072179759fa0ac-linux-s390x.tar.xz An error occurred (NotImplemented) when calling the GetObjectTagging operation: GetObjectTagging not implemented
copy failed: s3://dist-staging/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/node-v23.0.0-nightly2024072179759fa0ac-linux-x64.tar.gz to s3://dist-prod/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/node-v23.0.0-nightly2024072179759fa0ac-linux-x64.tar.gz An error occurred (NotImplemented) when calling the GetObjectTagging operation: GetObjectTagging not implemented
copy failed: s3://dist-staging/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/node-v23.0.0-nightly2024072179759fa0ac-linux-x64.tar.xz to s3://dist-prod/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/node-v23.0.0-nightly2024072179759fa0ac-linux-x64.tar.xz An error occurred (NotImplemented) when calling the GetObjectTagging operation: GetObjectTagging not implemented
copy failed: s3://dist-staging/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/node-v23.0.0-nightly2024072179759fa0ac-win-arm64.7z to s3://dist-prod/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/node-v23.0.0-nightly2024072179759fa0ac-win-arm64.7z An error occurred (NotImplemented) when calling the GetObjectTagging operation: GetObjectTagging not implemented
copy failed: s3://dist-staging/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/node-v23.0.0-nightly2024072179759fa0ac-win-arm64.zip to s3://dist-prod/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/node-v23.0.0-nightly2024072179759fa0ac-win-arm64.zip An error occurred (NotImplemented) when calling the GetObjectTagging operation: GetObjectTagging not implemented
copy failed: s3://dist-staging/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/node-v23.0.0-nightly2024072179759fa0ac-win-x64.zip to s3://dist-prod/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/node-v23.0.0-nightly2024072179759fa0ac-win-x64.zip An error occurred (NotImplemented) when calling the GetObjectTagging operation: GetObjectTagging not implemented
copy failed: s3://dist-staging/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/node-v23.0.0-nightly2024072179759fa0ac-win-x64.7z to s3://dist-prod/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/node-v23.0.0-nightly2024072179759fa0ac-win-x64.7z An error occurred (NotImplemented) when calling the GetObjectTagging operation: GetObjectTagging not implemented
copy failed: s3://dist-staging/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/node-v23.0.0-nightly2024072179759fa0ac-x64.msi to s3://dist-prod/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/node-v23.0.0-nightly2024072179759fa0ac-x64.msi An error occurred (NotImplemented) when calling the GetObjectTagging operation: GetObjectTagging not implemented
copy failed: s3://dist-staging/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/node-v23.0.0-nightly2024072179759fa0ac.pkg to s3://dist-prod/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/node-v23.0.0-nightly2024072179759fa0ac.pkg An error occurred (NotImplemented) when calling the GetObjectTagging operation: GetObjectTagging not implemented
copy failed: s3://dist-staging/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/node-v23.0.0-nightly2024072179759fa0ac.tar.gz to s3://dist-prod/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/node-v23.0.0-nightly2024072179759fa0ac.tar.gz An error occurred (NotImplemented) when calling the GetObjectTagging operation: GetObjectTagging not implemented
copy failed: s3://dist-staging/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/node-v23.0.0-nightly2024072179759fa0ac.tar.xz to s3://dist-prod/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/node-v23.0.0-nightly2024072179759fa0ac.tar.xz An error occurred (NotImplemented) when calling the GetObjectTagging operation: GetObjectTagging not implemented
copy failed: s3://dist-staging/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/win-arm64/node.exe to s3://dist-prod/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/win-arm64/node.exe An error occurred (NotImplemented) when calling the GetObjectTagging operation: GetObjectTagging not implemented
copy failed: s3://dist-staging/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/win-arm64/node_pdb.zip to s3://dist-prod/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/win-arm64/node_pdb.zip An error occurred (NotImplemented) when calling the GetObjectTagging operation: GetObjectTagging not implemented
copy failed: s3://dist-staging/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/win-x64/node.exe to s3://dist-prod/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/win-x64/node.exe An error occurred (NotImplemented) when calling the GetObjectTagging operation: GetObjectTagging not implemented
copy failed: s3://dist-staging/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/win-arm64/node_pdb.7z to s3://dist-prod/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/win-arm64/node_pdb.7z An error occurred (NotImplemented) when calling the GetObjectTagging operation: GetObjectTagging not implemented
copy failed: s3://dist-staging/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/win-x64/node_pdb.7z to s3://dist-prod/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/win-x64/node_pdb.7z An error occurred (NotImplemented) when calling the GetObjectTagging operation: GetObjectTagging not implemented
copy failed: s3://dist-staging/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/win-x64/node_pdb.zip to s3://dist-prod/nodejs/nightly/v23.0.0-nightly2024072179759fa0ac/win-x64/node_pdb.zip An error occurred (NotImplemented) when calling the GetObjectTagging operation: GetObjectTagging not implemented
root@infra-digitalocean-ubuntu1604-x64-1:/home/staging/tools/promote#

richardlau · 2024-07-22T12:07:12Z

So it appears that

aws s3 cp $staging_bucket/$relativedir/$tmpversion/ $dist_bucket/$relativedir/$tmpversion/ --endpoint-url=$cloudflare_endpoint --profile $cloudflare_profile --recursive --no-follow-symlinks --copy-props none

will copy one subset of files across while

aws s3 sync $staging_bucket/$relativedir/$tmpversion/ $dist_bucket/$relativedir/$tmpversion/ --endpoint-url=$cloudflare_endpoint --profile $cloudflare_profile --no-follow-symlinks

will copy the other set of files across.

I guess if we ignored the errors (which is less than ideal) we could call both?
e.g.

aws s3 cp $staging_bucket/$relativedir/$tmpversion/ $dist_bucket/$relativedir/$tmpversion/ --endpoint-url=$cloudflare_endpoint --profile $cloudflare_profile --recursive --no-follow-symlinks --copy-props none || true
aws s3 sync $staging_bucket/$relativedir/$tmpversion/ $dist_bucket/$relativedir/$tmpversion/ --endpoint-url=$cloudflare_endpoint --profile $cloudflare_profile --no-follow-symlinks || true

richardlau · 2024-07-23T12:50:44Z

I guess if we ignored the errors (which is less than ideal) we could call both? e.g.

aws s3 cp $staging_bucket/$relativedir/$tmpversion/ $dist_bucket/$relativedir/$tmpversion/ --endpoint-url=$cloudflare_endpoint --profile $cloudflare_profile --recursive --no-follow-symlinks --copy-props none || true
aws s3 sync $staging_bucket/$relativedir/$tmpversion/ $dist_bucket/$relativedir/$tmpversion/ --endpoint-url=$cloudflare_endpoint --profile $cloudflare_profile --no-follow-symlinks || true

This is deployed and appears to be working for the nightlies.

We'll still need to work out what isn't happening when releases are promoted.

flakey5 · 2024-07-23T14:42:25Z

Apologizes got side tracked - I'll ask with the R2 team again to see if they have any suggestions.

There is Super Slurper, which is a tool that migrates a full bucket over to another and it works with R2. We can call it through Cloudflare's v4 api, but I don't know if any of the endpoints are documented publicly so I don't know how comfortable I'd be with us relying on it.

flakey5 · 2024-07-23T17:16:39Z

Can we try just this command?

aws s3 cp $staging_bucket/$relativedir/$tmpversion/ $dist_bucket/$relativedir/$tmpversion/ --endpoint-url=$cloudflare_endpoint --profile $cloudflare_profile --recursive --no-follow-symlinks --metadata-directive COPY || true

Also been a bit too busy to be able to look into the promotion issue but hoping to be able to tmow

flakey5 · 2024-07-24T14:07:13Z

It looks like promotion worked for v20.16.0? nodejs/release-cloudflare-worker#130

flakey5 · 2024-07-24T14:15:06Z

Aghh it was just the sig files https://r2.nodejs.org/dist/v20.16.0/

richardlau · 2024-07-24T14:20:37Z

Aghh it was just the sig files https://r2.nodejs.org/dist/v20.16.0/

Yes, I kind of expected that. We still don't know what isn't happening when releases are promoted.

Can we try just this command?

aws s3 cp $staging_bucket/$relativedir/$tmpversion/ $dist_bucket/$relativedir/$tmpversion/ --endpoint-url=$cloudflare_endpoint --profile $cloudflare_profile --recursive --no-follow-symlinks --metadata-directive COPY || true

FYI I might not get round to this this week -- and I'm away next week (but if any other build-infra member is up for testing feel free).

flakey5 · 2024-07-24T20:36:33Z

I'm really not sure what the issue is for promoting. Would it be possible to add this to this line for the next release just to see?

ssh ${customsshkey} "$webuser@$webhost" "site=nodejs && . /home/staging/tools/promote/settings && dstdir=$release_dstdir && . /home/staging/tools/promote/upload_to_cloudflare.sh nodejs $version"

flakey5 · 2024-08-25T18:07:31Z

Hmm I'm curious of the worker aws profile exists for the dist and staging users https://github.com/nodejs/node/blob/43f699d4d2799cfc17cbcad5770e1889075d5dbe/tools/release.sh#L22 , I'd imagine it would though

targos · 2024-08-27T08:59:18Z

Verified it exists for dist and staging.

flakey5 · 2024-09-09T02:46:40Z

Grabbed the release.sh and the promotion scripts and did a dry run of a release (aka just adding echo in front of each command that actually does something), and the line that promotes the assets from dist-staging to dist-prod is being called. I don't really know what the issue is here

targos · 2024-09-18T13:40:33Z

I also just found out that not all nightlies are promoted to dist-prod.
For example 20240911 (it exists in dist-staging).

flakey5 · 2024-10-05T02:13:30Z

Got an idea to move the r2 promotion logic to a github action that we trigger manually, similar to what we do in the update release action here for every Node release https://github.com/nodejs/node/blob/32efeea0c046e3b504fee27278a0f7e8da8c86e8/.github/workflows/update-release-links.yml#L18.

Shasums, etc. will still need to be uploaded to the staging bucket by the origin server. But, for actually promoting the release the only command that needs to be ran on the origin server is gh workflow run promote-release.yml --repo nodejs/release-cloudflare-worker --field path=path/to/promote.

Got a pr for the changes needed in the worker repo: nodejs/release-cloudflare-worker#142

Wdyt @targos @richardlau @ovflowd?

See nodejs#3838 Signed-off-by: flakey5 <[email protected]>

richardlau · 2024-10-29T15:41:38Z

I was able to poke this while releasing Node.js 22.11.0.

#3838 (comment):

In terms of release promotion, what is supposed to happen:
...
9. before resha_release.sh is supposed to call upload_to_cloudflare.sh:

build/ansible/www-standalone/tools/promote/resha_release.sh

Line 28 in d9c4ea9

. ${__dirname}/upload_to_cloudflare.sh $site $2

This wasn't happening because that line was commented out all the way back in #3508 (comment). The issue is that the release.sh script is dependent on dist-sign returning the path of the SHASUMS256.txt file and any other output (such as from upload_to_cloudflare.sh) breaks the release script.

My solution was to uncomment the commented out line but redirect all output from upload_to_cloudflare.sh to a log file (so it isn't sent back to the release script but is captured in case we need to check). I'll update #3840, mirroring the files on the server.

AFAICT all the expected assets are present in https://r2.nodejs.org/download/release/v22.11.0/.

ovflowd · 2024-10-29T15:56:49Z

This is awesome!!

targos closed this as not planned Won't fix, can't repro, duplicate, stale Jul 18, 2024

richardlau mentioned this issue Jul 18, 2024

ansible: fix _resha.sh #3826

Merged

richardlau reopened this Jul 18, 2024

richardlau changed the title ~~Promotion to R2's dist-prod bucket never happens for releases~~ Promotion to R2's dist-prod bucket never happens Jul 18, 2024

richardlau mentioned this issue Jul 19, 2024

ansible: fix promotion to R2 dist-prod for releases #3840

Merged

richardlau mentioned this issue Jul 19, 2024

build, tools: drop leading / from r2dir nodejs/node#53951

Merged

flakey5 mentioned this issue Oct 5, 2024

actions: promote release action nodejs/release-cloudflare-worker#142

Merged

flakey5 added a commit to flakey5/nodejs-build that referenced this issue Oct 10, 2024

ansible: promote releases to r2 via actions

aed14c7

See nodejs#3838 Signed-off-by: flakey5 <[email protected]>

flakey5 added a commit to flakey5/nodejs-build that referenced this issue Oct 10, 2024

ansible: promote releases to r2 via actions

069108e

See nodejs#3838 Signed-off-by: flakey5 <[email protected]>

flakey5 mentioned this issue Oct 10, 2024

ansible: promote releases to r2 via actions #3931

Closed

2 tasks

richardlau mentioned this issue Oct 29, 2024

Discuss state of r2 migration on Build WG meeting #3508

Open

richardlau closed this as completed in #3840 Oct 30, 2024

richardlau closed this as completed in 21fc64a Oct 30, 2024

shadowspawn mentioned this issue Nov 2, 2024

Retry when downloading/extraction of Node.js tar fails tj/n#784

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Promotion to R2's dist-prod bucket never happens #3838

Promotion to R2's dist-prod bucket never happens #3838

targos commented Jul 17, 2024

targos commented Jul 17, 2024

richardlau commented Jul 17, 2024

flakey5 commented Jul 17, 2024

targos commented Jul 18, 2024

richardlau commented Jul 18, 2024

flakey5 commented Jul 18, 2024

flakey5 commented Jul 19, 2024

richardlau commented Jul 19, 2024

richardlau commented Jul 19, 2024

richardlau commented Jul 19, 2024

flakey5 commented Jul 19, 2024

richardlau commented Jul 19, 2024

richardlau commented Jul 19, 2024 •

edited

Loading

richardlau commented Jul 19, 2024

richardlau commented Jul 19, 2024 •

edited

Loading

richardlau commented Jul 22, 2024

richardlau commented Jul 22, 2024

richardlau commented Jul 23, 2024

flakey5 commented Jul 23, 2024

flakey5 commented Jul 23, 2024

flakey5 commented Jul 24, 2024

flakey5 commented Jul 24, 2024

richardlau commented Jul 24, 2024

flakey5 commented Jul 24, 2024

flakey5 commented Aug 25, 2024

targos commented Aug 27, 2024

flakey5 commented Sep 9, 2024

targos commented Sep 18, 2024

flakey5 commented Oct 5, 2024 •

edited

Loading

richardlau commented Oct 29, 2024

ovflowd commented Oct 29, 2024

Promotion to R2's dist-prod bucket never happens #3838

Promotion to R2's dist-prod bucket never happens #3838

Comments

targos commented Jul 17, 2024

targos commented Jul 17, 2024

richardlau commented Jul 17, 2024

flakey5 commented Jul 17, 2024

targos commented Jul 18, 2024

richardlau commented Jul 18, 2024

flakey5 commented Jul 18, 2024

flakey5 commented Jul 19, 2024

richardlau commented Jul 19, 2024

richardlau commented Jul 19, 2024

richardlau commented Jul 19, 2024

flakey5 commented Jul 19, 2024

richardlau commented Jul 19, 2024

richardlau commented Jul 19, 2024 • edited Loading

richardlau commented Jul 19, 2024

richardlau commented Jul 19, 2024 • edited Loading

richardlau commented Jul 22, 2024

richardlau commented Jul 22, 2024

richardlau commented Jul 23, 2024

flakey5 commented Jul 23, 2024

flakey5 commented Jul 23, 2024

flakey5 commented Jul 24, 2024

flakey5 commented Jul 24, 2024

richardlau commented Jul 24, 2024

flakey5 commented Jul 24, 2024

flakey5 commented Aug 25, 2024

targos commented Aug 27, 2024

flakey5 commented Sep 9, 2024

targos commented Sep 18, 2024

flakey5 commented Oct 5, 2024 • edited Loading

richardlau commented Oct 29, 2024

ovflowd commented Oct 29, 2024

richardlau commented Jul 19, 2024 •

edited

Loading

richardlau commented Jul 19, 2024 •

edited

Loading

flakey5 commented Oct 5, 2024 •

edited

Loading