Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(image): return error early if total size of layers exceeds limit #8294

Merged
merged 8 commits into from
Jan 30, 2025

Conversation

nikpivkin
Copy link
Contributor

@nikpivkin nikpivkin commented Jan 25, 2025

Description

This PR optimizes the image size check by returning an error if the total size of the layers exceeds the limit, without the need to load the entire image.

Related PRs

Checklist

  • I've read the guidelines for contributing to this repository.
  • I've followed the conventions in the PR title.
  • I've added tests that prove my fix is effective or that my feature works.
  • I've updated the documentation with the relevant information (if needed).
  • I've added usage information (if the PR introduces new options)
  • I've included a "before" and "after" example to the description (if the PR is a user interface change).

@nikpivkin nikpivkin marked this pull request as ready for review January 27, 2025 09:23
imagePath: "../../test/testdata/alpine-311.tar.gz",
imagePath: "../../test/testdata/image2.tar",
artifactOpt: artifact.Option{
ImageOption: types.ImageOptions{MaxImageSize: units.MB * 4.1},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's weird, but my IDE complains 😄
изображение

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GoLand? I think the static analyzer should not show the warning because 4.1*MB is an integer.
https://go.dev/play/p/M1aB0vKfVl1

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, i agree with you.
Just wanted to share 😄

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Strange that the behavior is different from vscode, I think they use the same lsp.

pkg/fanal/artifact/image/image.go Outdated Show resolved Hide resolved
@@ -2255,11 +2255,19 @@ func TestArtifact_Inspect(t *testing.T) {
},
{
name: "sad path, image size is larger than the maximum",
imagePath: "../../test/testdata/alpine-311.tar.gz",
imagePath: "../../test/testdata/image2.tar",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I seem to be missing something
Why are we using an existing image archive (e.g. vuln-image.tar.gz)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I created image2 as it is convenient for testing because it contains two 2mb layers.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But we only test 2 cases:

  • 1st layer size > MaxImageSize.
  • Image size > MaxImageSize.

Why do we need the 2nd layer?

We just don't want to add extra test files (there are too many of them anyway)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The layers are processed in parallel and if a small layer is processed first, we will not check the first case.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if this makes sense for this test, but okay, let's keep this file.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docker: Decimal prefixes (1000)
Docker Hub: Binary prefixes (1024)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nikpivkin It may be worth documenting. Otherwise, users who check the size on Docker Hub may complain about it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW I found an open issue docker/cli#4630

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docker: Decimal prefixes (1000)
Docker Hub: Binary prefixes (1024)

Doesn't the Docker CLI display the actual binary size but uses decimal prefixes like Docker Hub?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added e5574ca

pkg/fanal/artifact/image/image.go Outdated Show resolved Hide resolved
Copy link
Contributor

@DmitriyLewen DmitriyLewen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Comment on lines 525 to 526
An error is returned in the following cases:
- if the compressed image size exceeds the limit,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For me, the most significant difference between markdown and mkdocs is that a newline is required before bullet points.

Suggested change
An error is returned in the following cases:
- if the compressed image size exceeds the limit,
An error is returned in the following cases:
- if the compressed image size exceeds the limit,

It's broken now.

CleanShot 2025-01-29 at 10 04 09

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed 92888b6


An error is returned in the following cases:
- if the compressed image size exceeds the limit,
- if the total size of the layers exceeds the specified limit during their pulling,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- if the total size of the layers exceeds the specified limit during their pulling,
- if the total size of the uncompressed layers exceeds the specified limit during their pulling,

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done 92888b6

Signed-off-by: nikpivkin <[email protected]>
@nikpivkin nikpivkin requested a review from knqyf263 January 29, 2025 15:21
Signed-off-by: knqyf263 <[email protected]>
@knqyf263 knqyf263 enabled auto-merge January 30, 2025 07:41
@knqyf263 knqyf263 added this pull request to the merge queue Jan 30, 2025
Merged via the queue into aquasecurity:main with commit 73bd20d Jan 30, 2025
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants