Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor to more generic code #12

Merged
merged 4 commits into from
Oct 2, 2023
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ on:
branches: master

env:
NODE_VERSION: 12.x
NODE_VERSION: 18.x
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should keep testing on Node.js 12 until we can upgrade the version of Node.js used to run this tool on our server.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well, that breaks using modern javascript async/await ect :/
What is needed to upgrade node.js on the www server?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Current server is running Ubuntu 16.04 and using nodesource to install Node.js. I'm not sure whether Nodesource provide later versions of Node.js on Ubuntu 16.04. I'd like to update the OS on the server but it's risky given how dependent we are on that droplet at the moment.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so we would either need to adjust this to run on node 12 or upgrade node without using nodesource, since upgrading the OS is not trivial
I prefer the second option, is that possible?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think Node.js 18 is a non-starter on Ubuntu 16.04 due to glibc versions. Node.js 16 is a possibility -- there may even be a Nodesource distribution for it (I'd suggest testing in a separate Ubuntu 16.04 system first).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MoLow Yes, it should be.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on the fact that https://deb.nodesource.com/node_16.x/dists/xenial/Release returns something, I think v16.x can be installed using the NodeSource distribution.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was able to install node16 on ubuntu 16.04 using nodesource. let me check these changes work on node 16

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tests have passed. @richardlau should I go ahead and update www nodejs version?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MoLow Yes (and keep an eye on tomorrow's nightly to check the indexer still runs ok). If you could also update https://github.com/nodejs/build/blob/b8b63ade1e76d8ccf669335ee384ed2ee34a3abe/ansible/www-standalone/tasks/base.yaml#L1-L3 (which says Node.js 8 but we are definitely running with 12 on the server) that would be great.

MoLow marked this conversation as resolved.
Show resolved Hide resolved

jobs:
test:
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
.nyc_output
.tap
node_modules
package-lock.json
123 changes: 123 additions & 0 deletions common.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,123 @@
'use strict'

const path = require('path')
const semver = require('semver')

const ltsNames = {
4: 'argon',
6: 'boron',
8: 'carbon',
10: 'dubnium',
12: 'erbium',
14: 'fermium',
16: 'gallium',
18: 'hydrogen'
}

class Linker {
#links = new Map()
#dirs = []
#baseDir
#docsDir
constructor ({ baseDir, docsDir }) {
this.#baseDir = baseDir
this.#docsDir = docsDir
}

async getLinks (allDirectories, readDir) {
const allDirs = allDirectories
.map((d) => path.basename(d))
.map((d) => {
try {
return semver.parse(d)
/* c8 ignore next 3 */
} catch (e) {
return null
}
})
.filter(Boolean)

this.#makeDocsLinks(allDirs.map((d) => d.raw))

this.#dirs = allDirs.filter((d) => semver.satisfies(d, '~0.10 || ~0.12 || >= 1.0')).map((d) => d.raw)
this.#dirs.sort((d1, d2) => semver.compare(d1, d2))

this.#link('0.10')
this.#link(0.12)

for (let i = 1; ; i++) {
if (!this.#link(i) && i >= 4) {
break
}
}

const max = this.#link(null)
const tbreg = new RegExp(`(\\w+)-${max}.tar.gz`)
const latestDir = path.join(this.#baseDir, 'latest')

let tarball = (await readDir(this.#links.get(latestDir) || latestDir)).filter((f) => tbreg.test(f))

/* c8 ignore next 3 */
if (tarball.length !== 1) {
throw new Error('Could not find latest.tar.gz')
}

tarball = tarball[0]
const name = tarball.match(tbreg)[1]
const dst = path.join(this.#baseDir, `${name}-latest.tar.gz`)
this.#links.set(dst, path.join(this.#baseDir, 'latest', tarball))
return this.#links
}

#makeDocsLinks (versions) {
if (!this.#docsDir) {
return
}

for (const version of versions) {
const src = path.join(this.#baseDir, version, 'docs')
const dst = path.join(this.#docsDir, version)
this.#links.set(dst, src)
}
}

#link (version) {
const line = version && `${version}.x`
const range = version ? `${Number(version) < 1 ? '~' : '^'}${line}` : '*'
const max = semver.maxSatisfying(this.#dirs, range)

if (!max) {
return false
}

const symlink = (name) => {
const dst = path.join(this.#baseDir, name)
const src = path.join(this.#baseDir, max)

this.#links.set(dst, src)

if (!this.#docsDir) {
return
}

const dsrc = path.join(this.#baseDir, max, 'docs')
const ddst = path.join(this.#docsDir, name)
this.#links.set(ddst, dsrc)
}

if (line) {
symlink(`latest-v${line}`)
if (ltsNames[version]) {
symlink(`latest-${ltsNames[version]}`)
}
} else {
symlink('latest')
}

return max
}
}

module.exports = {
Linker
}
158 changes: 18 additions & 140 deletions latest-linker.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,156 +2,34 @@

'use strict'

const fs = require('fs')
const fs = require('fs/promises')
const path = require('path')
const semver = require('semver')
const map = require('map-async')

const ltsNames = {
4: 'argon',
6: 'boron',
8: 'carbon',
10: 'dubnium',
12: 'erbium',
14: 'fermium',
16: 'gallium',
18: 'hydrogen'
}
const { Linker } = require('./common.js')

/* c8 ignore next 3 */
if (process.argv.length < 3) {
throw new Error('Usage: latest-linker.js <downloads directory> [docs directory]')
}

const dir = path.resolve(process.argv[2])
const docsDir = process.argv[3] && path.resolve(process.argv[3])

if (!fs.statSync(dir).isDirectory()) {
throw new Error('Usage: latest-linker.js <downloads directory> [docs directory]')
}
const docsDir = process.argv[3] && path.resolve(process.argv[3]);

if (docsDir && !fs.statSync(docsDir).isDirectory()) {
throw new Error('Usage: latest-linker.js <downloads directory> [docs directory]')
}

map(
fs.readdirSync(dir).map((d) => path.join(dir, d)),
(d, callback) => fs.stat(d, (_, stat) => callback(null, { d, stat })),
afterMap
)

function afterMap (err, allDirs) {
if (err) {
throw err
(async function main () {
/* c8 ignore next 3 */
if (!(await fs.stat(dir)).isDirectory()) {
throw new Error('Usage: latest-linker.js <downloads directory> [docs directory]')
}

allDirs = allDirs.filter((d) => d.stat && d.stat.isDirectory())
.map((d) => path.basename(d.d))
.map((d) => {
try {
return semver.parse(d)
} catch (e) {}
})
.filter(Boolean)

makeDocsLinks(allDirs.map((v) => v.raw))

const dirs = allDirs.filter((d) => semver.satisfies(d, '~0.10 || ~0.12 || >= 1.0'))
.map((d) => d.raw)

dirs.sort((d1, d2) => semver.compare(d1, d2))

link('0.10', dirs)
link(0.12, dirs)

for (let i = 1; ; i++) {
if (!link(i, dirs) && i >= 4) {
break
}
/* c8 ignore next 3 */
if (docsDir && !(await fs.stat(docsDir)).isDirectory()) {
throw new Error('Usage: latest-linker.js <downloads directory> [docs directory]')
}

const max = link(null, dirs)
const tbreg = new RegExp(`(\\w+)-${max}.tar.gz`)

let tarball = fs.readdirSync(path.join(dir, 'latest'))
.filter((f) => tbreg.test(f))

if (tarball.length !== 1) {
throw new Error('Could not find latest.tar.gz')
const allDirs = (await fs.readdir(dir, { withFileTypes: true })).filter((d) => d.isDirectory()).map((d) => d.name)
const linker = new Linker({ baseDir: dir, docsDir })
const links = await linker.getLinks(allDirs, fs.readdir)
for (const [dest, src] of links) {
await fs.unlink(dest).catch(() => {})
await fs.symlink(src, dest)
}

tarball = tarball[0]
const name = tarball.match(tbreg)[1]
const dst = path.join(dir, `${name}-latest.tar.gz`)
try {
fs.unlinkSync(dst)
} catch (e) {}
fs.symlinkSync(path.join(dir, 'latest', tarball), dst)
}

function makeDocsLinks (versions) {
if (!docsDir) {
return
}

versions.forEach((version) => {
const src = path.join(dir, version, 'docs')
const dst = path.join(docsDir, version)

fs.stat(src, (err, stat) => {
if (err || !stat.isDirectory()) {
return
}

fs.unlink(dst, () => {
fs.symlink(src, dst, (err) => {
if (err) {
throw err
}
})
})
})
})
}

function link (version, dirs) {
const line = version && `${version}.x`
const range = version ? `${Number(version) < 1 ? '~' : '^'}${line}` : '*'
const max = semver.maxSatisfying(dirs, range)

if (!max) {
return false
}

function symlink (name) {
const dst = path.join(dir, name)
const src = path.join(dir, max)

try {
fs.unlinkSync(dst)
} catch (e) {}
fs.symlinkSync(src, dst)

if (!docsDir) {
return
}

const dsrc = path.join(dir, max, 'docs')
const ddst = path.join(docsDir, name)

try {
fs.unlinkSync(ddst)
} catch (e) {}
fs.symlinkSync(dsrc, ddst)
}

if (line) {
symlink(`latest-v${line}`)
if (ltsNames[version]) {
symlink(`latest-${ltsNames[version]}`)
}
} else {
symlink('latest')
}

return max
}
})()
7 changes: 3 additions & 4 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
"main": "latest-linker.js",
"scripts": {
"lint": "standard",
"test": "npm run lint && tap test.js"
"test": "npm run lint && tap --allow-incomplete-coverage test.js"
},
"repository": {
"type": "git",
Expand All @@ -14,15 +14,14 @@
"author": "Rod <[email protected]> (http://r.va.gg/)",
"license": "MIT",
"dependencies": {
"map-async": "^0.1.1",
"semver": "^7.3.2"
},
"bin": {
"nodejs-latest-linker": "latest-linker.js"
},
"preferGlobal": true,
"devDependencies": {
"tap": "^14.10.8",
"standard": "^14.3.4"
"standard": "^17.1.0",
"tap": "^18.4.0"
}
}
20 changes: 20 additions & 0 deletions test.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,26 @@ const { execFileSync } = require('child_process')
const fs = require('fs')
const path = require('path')
const tap = require('tap')
const { Linker } = require('./common.js')

tap.test('Linker', async t => {
const linker = new Linker({ baseDir: 'base', docsDir: 'docs' })
const links = await linker.getLinks(
['v19.8.0/', 'v19.8.1/', 'v19.9.0/', 'v20.7.0/', 'v20.8.0/'],
async () => ['docs/', 'win-x64/', 'node-v20.8.0-aix-ppc64.tar.gz', 'node-v20.8.0-arm64.msi', 'node-v20.8.0-headers.tar.gz', 'node-v20.8.0.tar.gz']
)
t.same(links, new Map([
['docs/v19.8.0', 'base/v19.8.0/docs'],
['docs/v19.8.1', 'base/v19.8.1/docs'],
['docs/v19.9.0', 'base/v19.9.0/docs'],
['docs/v20.7.0', 'base/v20.7.0/docs'],
['docs/v20.8.0', 'base/v20.8.0/docs'],
['base/latest', 'base/v20.8.0'],
['docs/latest', 'base/v20.8.0/docs'],
['base/node-latest.tar.gz', 'base/latest/node-v20.8.0.tar.gz']
]))
t.end()
})

tap.test('basic test', t => {
const dir = t.testdir({
Expand Down
Loading