Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enforce ordered execution of tests #1619

Merged
merged 1 commit into from
Jun 15, 2024
Merged

Conversation

HeavyWombat
Copy link
Contributor

@HeavyWombat HeavyWombat commented Jun 14, 2024

Changes

Mitigation for #1621

Enforce ordered execution since the introduction of the latest test context in cmd/image-processing led to failures when using Ginkgo CLI to drive the tests. Since the PR status checks are driven by go test this regression was not obvious.

$ go run github.com/onsi/ginkgo/v2/ginkgo run --coverprofile=unit.coverprofile --output-dir=build/coverage --randomize-all --trace ./cmd/image-processing

Running Suite: Image Processing Command Suite - /Users/mdiester/go/src/github.com/shipwright-io/build/cmd/image-processing
==========================================================================================================================
Random Seed: 1718369287 - will randomize all specs

Will run 23 of 23 specs
•••••••••••••••
------------------------------
• [FAILED] [0.431 seconds]
Image Processing Resource mutating the image [It] should mutate an image with single annotation
/Users/mdiester/go/src/github.com/shipwright-io/build/cmd/image-processing/main_test.go:256

  Timeline >>
  GET /v2/
  HEAD /v2/temp-image/manifests/nskxt 404 NAME_UNKNOWN Unknown name
  HEAD /v2/temp-image/blobs/sha256:5b943e2b943f6c81dbbd4e2eca5121f4fcc39139e3d1219d6d89bd925b77d9fe 404 BLOB_UNKNOWN Unknown blob
  POST /v2/temp-image/blobs/uploads/
  PATCH /v2/temp-image/blobs/uploads/8973636687925678644
  PUT /v2/temp-image/blobs/uploads/8973636687925678644?digest=sha256%3A5b943e2b943f6c81dbbd4e2eca5121f4fcc39139e3d1219d6d89bd925b77d9fe
  PUT /v2/temp-image/manifests/nskxt
  2024/06/14 14:48:41 Loading the image from the registry "127.0.0.1:50479/temp-image:nskxt"
  GET /v2/
  HEAD /v2/temp-image/manifests/nskxt
  GET /v2/
  GET /v2/temp-image/manifests/nskxt
  2024/06/14 14:48:41 Loaded single image
  2024/06/14 14:48:41 Mutating the image
  GET /v2/
  GET /v2/temp-image/manifests/nskxt
  GET /v2/temp-image/blobs/sha256:5b943e2b943f6c81dbbd4e2eca5121f4fcc39139e3d1219d6d89bd925b77d9fe
  [FAILED] in [It] - /Users/mdiester/go/src/github.com/shipwright-io/build/cmd/image-processing/main_test.go:262 @ 06/14/24 14:48:42.11
  << Timeline

  [FAILED] Unexpected error:
      <*fs.PathError | 0x1400042bbc0>:
      open : no such file or directory
      {
          Op: "open",
          Path: "",
          Err: <syscall.Errno>0x2,
      }
  occurred
  In [It] at: /Users/mdiester/go/src/github.com/shipwright-io/build/cmd/image-processing/main_test.go:262 @ 06/14/24 14:48:42.11

Submitter Checklist

  • Includes tests if functionality changed/was added
  • Includes docs if changes are user-facing
  • Set a kind label on this PR
  • Release notes block has been filled in, or marked NONE

See the contributor guide
for details on coding conventions, github and prow interactions, and the code review process.

Release Notes

NONE

Enforce ordered execution since the introduction of the latest test
context in `cmd/image-processing` led to failures when using Ginkgo CLI
to drive the tests. Since the PR status checks are driven by `go test`
this regression was not obvious.

```
$ go run github.com/onsi/ginkgo/v2/ginkgo run --coverprofile=unit.coverprofile --output-dir=build/coverage --randomize-all --trace ./cmd/image-processing

Running Suite: Image Processing Command Suite - /Users/mdiester/go/src/github.com/shipwright-io/build/cmd/image-processing
==========================================================================================================================
Random Seed: 1718369287 - will randomize all specs

Will run 23 of 23 specs
•••••••••••••••
------------------------------
• [FAILED] [0.431 seconds]
Image Processing Resource mutating the image [It] should mutate an image with single annotation
/Users/mdiester/go/src/github.com/shipwright-io/build/cmd/image-processing/main_test.go:256

  Timeline >>
  GET /v2/
  HEAD /v2/temp-image/manifests/nskxt 404 NAME_UNKNOWN Unknown name
  HEAD /v2/temp-image/blobs/sha256:5b943e2b943f6c81dbbd4e2eca5121f4fcc39139e3d1219d6d89bd925b77d9fe 404 BLOB_UNKNOWN Unknown blob
  POST /v2/temp-image/blobs/uploads/
  PATCH /v2/temp-image/blobs/uploads/8973636687925678644
  PUT /v2/temp-image/blobs/uploads/8973636687925678644?digest=sha256%3A5b943e2b943f6c81dbbd4e2eca5121f4fcc39139e3d1219d6d89bd925b77d9fe
  PUT /v2/temp-image/manifests/nskxt
  2024/06/14 14:48:41 Loading the image from the registry "127.0.0.1:50479/temp-image:nskxt"
  GET /v2/
  HEAD /v2/temp-image/manifests/nskxt
  GET /v2/
  GET /v2/temp-image/manifests/nskxt
  2024/06/14 14:48:41 Loaded single image
  2024/06/14 14:48:41 Mutating the image
  GET /v2/
  GET /v2/temp-image/manifests/nskxt
  GET /v2/temp-image/blobs/sha256:5b943e2b943f6c81dbbd4e2eca5121f4fcc39139e3d1219d6d89bd925b77d9fe
  [FAILED] in [It] - /Users/mdiester/go/src/github.com/shipwright-io/build/cmd/image-processing/main_test.go:262 @ 06/14/24 14:48:42.11
  << Timeline

  [FAILED] Unexpected error:
      <*fs.PathError | 0x1400042bbc0>:
      open : no such file or directory
      {
          Op: "open",
          Path: "",
          Err: <syscall.Errno>0x2,
      }
  occurred
  In [It] at: /Users/mdiester/go/src/github.com/shipwright-io/build/cmd/image-processing/main_test.go:262 @ 06/14/24 14:48:42.11
```
@HeavyWombat HeavyWombat added the kind/failing-test Categorizes issue or PR as related to a consistently or frequently failing test. label Jun 14, 2024
@pull-request-size pull-request-size bot added the size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. label Jun 14, 2024
@openshift-ci openshift-ci bot added the release-note-none Label for when a PR does not need a release note label Jun 14, 2024
Copy link
Member

@SaschaSchwarze0 SaschaSchwarze0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/approve
/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Jun 15, 2024
@SaschaSchwarze0 SaschaSchwarze0 added this to the release-v0.14.0 milestone Jun 15, 2024
Copy link
Contributor

openshift-ci bot commented Jun 15, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: SaschaSchwarze0

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 15, 2024
@openshift-merge-bot openshift-merge-bot bot merged commit ea118e0 into main Jun 15, 2024
15 checks passed
@openshift-merge-bot openshift-merge-bot bot deleted the fix/failing-ginkgo-run branch June 15, 2024 19:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. kind/failing-test Categorizes issue or PR as related to a consistently or frequently failing test. lgtm Indicates that a PR is ready to be merged. release-note-none Label for when a PR does not need a release note size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

2 participants