Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docker(install): use undock to extract image #567

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
docker(install): use undock to extract image
Signed-off-by: CrazyMax <[email protected]>
crazy-max committed Jan 19, 2025

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
commit dfe64a3337e6d480114852d9ce9993efbc2678c8
19 changes: 15 additions & 4 deletions __tests__/docker/install.test.itg.ts
Original file line number Diff line number Diff line change
@@ -14,17 +14,25 @@
* limitations under the License.
*/

import {describe, test, expect} from '@jest/globals';
import {beforeAll, describe, test, expect} from '@jest/globals';
import fs from 'fs';
import os from 'os';
import path from 'path';

import {Install, InstallSource, InstallSourceArchive, InstallSourceImage} from '../../src/docker/install';
import {Docker} from '../../src/docker/docker';
import {Undock} from '../../src/undock/undock';
import {Install as UndockInstall} from '../../src/undock/install';
import {Exec} from '../../src/exec';

const tmpDir = () => fs.mkdtempSync(path.join(process.env.TEMP || os.tmpdir(), 'docker-install-itg-'));

beforeAll(async () => {
const undockInstall = new UndockInstall();
const binPath = await undockInstall.download('v0.9.0', true);
await undockInstall.install(binPath);
});

describe('root', () => {
// prettier-ignore
test.each(getSources(true))(
@@ -34,7 +42,8 @@ describe('root', () => {
source: source,
runDir: tmpDir(),
contextName: 'foo',
daemonConfig: `{"debug":true,"features":{"containerd-snapshotter":true}}`
daemonConfig: `{"debug":true,"features":{"containerd-snapshotter":true}}`,
undock: new Undock()
});
await expect(tryInstall(install)).resolves.not.toThrow();
}, 30 * 60 * 1000);
@@ -54,7 +63,8 @@ describe('rootless', () => {
runDir: tmpDir(),
contextName: 'foo',
daemonConfig: `{"debug":true}`,
rootless: true
rootless: true,
undock: new Undock()
});
await expect(
tryInstall(install, async () => {
@@ -79,7 +89,8 @@ describe('tcp', () => {
runDir: tmpDir(),
contextName: 'foo',
daemonConfig: `{"debug":true}`,
localTCPPort: 2378
localTCPPort: 2378,
undock: new Undock()
});
await expect(
tryInstall(install, async () => {
11 changes: 10 additions & 1 deletion __tests__/docker/install.test.ts
Original file line number Diff line number Diff line change
@@ -14,17 +14,25 @@
* limitations under the License.
*/

import {describe, expect, jest, test, beforeEach, afterEach, it} from '@jest/globals';
import {describe, expect, jest, test, beforeEach, afterEach, it, beforeAll} from '@jest/globals';
import fs from 'fs';
import os from 'os';
import path from 'path';
import * as rimraf from 'rimraf';
import osm = require('os');

import {Install, InstallSourceArchive, InstallSourceImage} from '../../src/docker/install';
import {Undock} from '../../src/undock/undock';
import {Install as UndockInstall} from '../../src/undock/install';

const tmpDir = fs.mkdtempSync(path.join(process.env.TEMP || os.tmpdir(), 'docker-install-'));

beforeAll(async () => {
const undockInstall = new UndockInstall();
const binPath = await undockInstall.download('v0.9.0', true);
await undockInstall.install(binPath);
});

afterEach(function () {
rimraf.sync(tmpDir);
});
@@ -63,6 +71,7 @@ describe('download', () => {
const install = new Install({
source: source,
runDir: tmpDir,
undock: new Undock()
});
const toolPath = await install.download();
expect(fs.existsSync(toolPath)).toBe(true);
137 changes: 77 additions & 60 deletions src/docker/install.ts
Original file line number Diff line number Diff line change
@@ -28,11 +28,13 @@ import * as tc from '@actions/tool-cache';

import {Context} from '../context';
import {Docker} from './docker';
import {ImageTools} from '../buildx/imagetools';
import {Undock} from '../undock/undock';
import {Exec} from '../exec';
import {Util} from '../util';
import {limaYamlData, dockerServiceLogsPs1, setupDockerWinPs1} from './assets';

import {GitHubRelease} from '../types/github';
import {HubRepository} from '../hubRepository';
import {Image} from '../types/oci/config';

export interface InstallSourceImage {
@@ -57,6 +59,8 @@ export interface InstallOpts {
daemonConfig?: string;
rootless?: boolean;
localTCPPort?: number;

undock: Undock;
}

interface LimaImage {
@@ -72,6 +76,7 @@ export class Install {
private readonly daemonConfig?: string;
private readonly rootless: boolean;
private readonly localTCPPort?: number;
private readonly undock: Undock;

private _version: string | undefined;
private _toolDir: string | undefined;
@@ -91,76 +96,23 @@ export class Install {
this.daemonConfig = opts.daemonConfig;
this.rootless = opts.rootless || false;
this.localTCPPort = opts.localTCPPort;
this.undock = opts.undock;
}

get toolDir(): string {
return this._toolDir || Context.tmpDir();
}

async downloadStaticArchive(component: 'docker' | 'docker-rootless-extras', src: InstallSourceArchive): Promise<string> {
const release: GitHubRelease = await Install.getRelease(src.version);
this._version = release.tag_name.replace(/^v+|v+$/g, '');
core.debug(`docker.Install.download version: ${this._version}`);

const downloadURL = this.downloadURL(component, this._version, src.channel);
core.info(`Downloading ${downloadURL}`);

const downloadPath = await tc.downloadTool(downloadURL);
core.debug(`docker.Install.download downloadPath: ${downloadPath}`);

let extractFolder;
if (os.platform() == 'win32') {
extractFolder = await tc.extractZip(downloadPath, extractFolder);
} else {
extractFolder = await tc.extractTar(downloadPath, extractFolder);
}
if (Util.isDirectory(path.join(extractFolder, component))) {
extractFolder = path.join(extractFolder, component);
}
core.debug(`docker.Install.download extractFolder: ${extractFolder}`);
return extractFolder;
}

public async download(): Promise<string> {
let extractFolder: string;
let cacheKey: string;
const platform = os.platform();

switch (this.source.type) {
case 'image': {
const tag = this.source.tag;
this._version = tag;
this._version = this.source.tag;
cacheKey = `docker-image`;

core.info(`Downloading docker cli from dockereng/cli-bin:${tag}`);
const cli = await HubRepository.build('dockereng/cli-bin');
extractFolder = await cli.extractImage(tag);

const moby = await HubRepository.build('moby/moby-bin');
if (['win32', 'linux'].includes(platform)) {
core.info(`Downloading dockerd from moby/moby-bin:${tag}`);
await moby.extractImage(tag, extractFolder);
} else if (platform == 'darwin') {
// On macOS, the docker daemon binary will be downloaded inside the lima VM.
// However, we will get the exact git revision from the image config
// to get the matching systemd unit files.
core.info(`Getting git revision from moby/moby-bin:${tag}`);

// There's no macOS image for moby/moby-bin - a linux daemon is run inside lima.
const manifest = await moby.getPlatformManifest(tag, 'linux');

const config = await moby.getJSONBlob<Image>(manifest.config.digest);
core.debug(`Config ${JSON.stringify(config.config)}`);

this.gitCommit = config.config?.Labels?.['org.opencontainers.image.revision'];
if (!this.gitCommit) {
core.warning(`No git revision can be determined from the image. Will use master.`);
this.gitCommit = 'master';
}
core.info(`Git revision is ${this.gitCommit}`);
} else {
core.warning(`dockerd not supported on ${platform}, only the Docker cli will be available`);
}
extractFolder = await this.downloadSourceImage(platform);
break;
}
case 'archive': {
@@ -170,10 +122,10 @@ export class Install {
this._version = version;

core.info(`Downloading Docker ${version} from ${this.source.channel} at download.docker.com`);
extractFolder = await this.downloadStaticArchive('docker', this.source);
extractFolder = await this.downloadSourceArchive('docker', this.source);
if (this.rootless) {
core.info(`Downloading Docker rootless extras ${version} from ${this.source.channel} at download.docker.com`);
const extrasFolder = await this.downloadStaticArchive('docker-rootless-extras', this.source);
const extrasFolder = await this.downloadSourceArchive('docker-rootless-extras', this.source);
fs.readdirSync(extrasFolder).forEach(file => {
const src = path.join(extrasFolder, file);
const dest = path.join(extractFolder, file);
@@ -191,7 +143,9 @@ export class Install {
}
// eslint-disable-next-line @typescript-eslint/no-unused-vars
files.forEach(function (file, index) {
fs.chmodSync(path.join(extractFolder, file), '0755');
if (!Util.isDirectory(path.join(extractFolder, file))) {
fs.chmodSync(path.join(extractFolder, file), '0755');
}
});
});

@@ -203,6 +157,69 @@ export class Install {
return tooldir;
}

private async downloadSourceImage(platform: string): Promise<string> {
const dest = path.join(Context.tmpDir(), 'docker-install-image');
const cliImage = `dockereng/cli-bin:${this._version}`;
const engineImage = `moby/moby-bin:${this._version}`;

core.info(`Downloading Docker CLI from ${cliImage}`);
await this.undock.run({
source: cliImage,
dist: dest
});

if (['win32', 'linux'].includes(platform)) {
core.info(`Downloading Docker engine from ${engineImage}`);
await this.undock.run({
source: engineImage,
dist: dest
});
} else if (platform == 'darwin') {
// On macOS, the docker daemon binary will be downloaded inside the lima VM.
// However, we will get the exact git revision from the image config
// to get the matching systemd unit files. There's no macOS image for
// moby/moby-bin - a linux daemon is run inside lima.
const engineImageConfig = (await new ImageTools().inspectImage(engineImage)) as Record<string, Image>;
core.debug(`docker.Install.downloadSourceImage engineImageConfig: ${JSON.stringify(engineImageConfig)}`);

this.gitCommit = engineImageConfig['linux/arm64'].config?.Labels?.['org.opencontainers.image.revision'];
if (!this.gitCommit) {
core.warning(`No git revision can be determined from the image. Will use default branch as Git revision.`);
this.gitCommit = 'master';
}

core.debug(`docker.Install.downloadSourceImage gitCommit: ${this.gitCommit}`);
} else {
core.warning(`Docker engine not supported on ${platform}, only the Docker cli will be available`);
}

return dest;
}

private async downloadSourceArchive(component: 'docker' | 'docker-rootless-extras', src: InstallSourceArchive): Promise<string> {
const release: GitHubRelease = await Install.getRelease(src.version);
this._version = release.tag_name.replace(/^v+|v+$/g, '');
core.debug(`docker.Install.downloadSourceArchive version: ${this._version}`);

const downloadURL = this.downloadURL(component, this._version, src.channel);
core.info(`Downloading ${downloadURL}`);

const downloadPath = await tc.downloadTool(downloadURL);
core.debug(`docker.Install.downloadSourceArchive downloadPath: ${downloadPath}`);

let extractFolder;
if (os.platform() == 'win32') {
extractFolder = await tc.extractZip(downloadPath, extractFolder);
} else {
extractFolder = await tc.extractTar(downloadPath, extractFolder);
}
if (Util.isDirectory(path.join(extractFolder, component))) {
extractFolder = path.join(extractFolder, component);
}
core.debug(`docker.Install.downloadSourceArchive extractFolder: ${extractFolder}`);
return extractFolder;
}

public async install(): Promise<string> {
if (!this.toolDir) {
throw new Error('toolDir must be set. Run download first.');
174 changes: 0 additions & 174 deletions src/hubRepository.ts

This file was deleted.


Unchanged files with check annotations Beta

# See the License for the specific language governing permissions and
# limitations under the License.
frOM busybox as base

Check warning on line 17 in __tests__/.fixtures/lint.Dockerfile

GitHub Actions / test-itg (ubuntu-latest, buildx/buildx.test.itg.ts)

All commands within the Dockerfile should use the same casing (either upper or lower)

ConsistentInstructionCasing: Command 'frOM' should match the case of the command majority (uppercase) More info: https://docs.docker.com/go/dockerfile/rule/consistent-instruction-casing/
cOpy lint.Dockerfile .

Check warning on line 18 in __tests__/.fixtures/lint.Dockerfile

GitHub Actions / test-itg (ubuntu-latest, buildx/buildx.test.itg.ts)

All commands within the Dockerfile should use the same casing (either upper or lower)

ConsistentInstructionCasing: Command 'cOpy' should match the case of the command majority (uppercase) More info: https://docs.docker.com/go/dockerfile/rule/consistent-instruction-casing/

Check warning on line 18 in __tests__/.fixtures/lint.Dockerfile

GitHub Actions / test-itg (ubuntu-latest, buildx/buildx.test.itg.ts)

All commands within the Dockerfile should use the same casing (either upper or lower)

ConsistentInstructionCasing: Command 'cOpy' should match the case of the command majority (uppercase) More info: https://docs.docker.com/go/dockerfile/rule/consistent-instruction-casing/
# some special chars: distroless/python3-debian12のPythonは3.11
# https://github.com/docker/build-push-action/issues/1204#issuecomment-2274056016
from scratch

Check warning on line 23 in __tests__/.fixtures/lint.Dockerfile

GitHub Actions / test-itg (ubuntu-latest, buildx/buildx.test.itg.ts)

All commands within the Dockerfile should use the same casing (either upper or lower)

ConsistentInstructionCasing: Command 'from' should match the case of the command majority (uppercase) More info: https://docs.docker.com/go/dockerfile/rule/consistent-instruction-casing/
MAINTAINER moby@example.com

Check warning on line 24 in __tests__/.fixtures/lint.Dockerfile

GitHub Actions / test-itg (ubuntu-latest, buildx/buildx.test.itg.ts)

The MAINTAINER instruction is deprecated, use a label instead to define an image author

MaintainerDeprecated: Maintainer instruction is deprecated in favor of using label More info: https://docs.docker.com/go/dockerfile/rule/maintainer-deprecated/

Check warning on line 24 in __tests__/.fixtures/lint.Dockerfile

GitHub Actions / test-itg (ubuntu-latest, buildx/buildx.test.itg.ts)

The MAINTAINER instruction is deprecated, use a label instead to define an image author

MaintainerDeprecated: Maintainer instruction is deprecated in favor of using label More info: https://docs.docker.com/go/dockerfile/rule/maintainer-deprecated/
COPy --from=base \

Check warning on line 25 in __tests__/.fixtures/lint.Dockerfile

GitHub Actions / test-itg (ubuntu-latest, buildx/buildx.test.itg.ts)

All commands within the Dockerfile should use the same casing (either upper or lower)

ConsistentInstructionCasing: Command 'COPy' should match the case of the command majority (uppercase) More info: https://docs.docker.com/go/dockerfile/rule/consistent-instruction-casing/
/lint.Dockerfile \
/
CMD [ "echo", "Hello, Norway!" ]

Check warning on line 29 in __tests__/.fixtures/lint.Dockerfile

GitHub Actions / test-itg (ubuntu-latest, buildx/buildx.test.itg.ts)

Multiple instructions of the same type should not be used in the same stage

MultipleInstructionsDisallowed: Multiple CMD instructions should not be used in the same stage because only the last one will be used More info: https://docs.docker.com/go/dockerfile/rule/multiple-instructions-disallowed/
CMD [ "echo", "Hello, Sweden!" ]
ENTRYPOINT my-program start

Check warning on line 31 in __tests__/.fixtures/lint.Dockerfile

GitHub Actions / test-itg (ubuntu-latest, buildx/buildx.test.itg.ts)

JSON arguments recommended for ENTRYPOINT/CMD to prevent unintended behavior related to OS signals

JSONArgsRecommended: JSON arguments recommended for ENTRYPOINT to prevent unintended behavior related to OS signals More info: https://docs.docker.com/go/dockerfile/rule/json-args-recommended/
# See the License for the specific language governing permissions and
# limitations under the License.
frOM busybox as base

Check warning on line 17 in __tests__/.fixtures/lint-other.Dockerfile

GitHub Actions / test-itg (ubuntu-latest, buildx/buildx.test.itg.ts)

All commands within the Dockerfile should use the same casing (either upper or lower)

ConsistentInstructionCasing: Command 'frOM' should match the case of the command majority (uppercase) More info: https://docs.docker.com/go/dockerfile/rule/consistent-instruction-casing/
cOpy lint-other.Dockerfile .
froM busybox aS notused