Images subsequently tagged with a semver aren't picked up by kargo

[darkpeak]

Checklist

• I've searched the issue queue to verify this is not a duplicate bug report.
• I've included steps to reproduce the bug.
• I've pasted the output of kargo version.
• I've pasted logs, if applicable.

Description

Our workflow is this:

• Developers create an image that they tag that looks like link-1234
• This is deployed to our link environment by kargo
• Once they are happy with this image, they tag it with a semantic version - e.g. 1.2.3 - to deploy it through the pipeline to subsequent environments
• Kargo doesn't pick this up, unless we remove the link-1234 tag from the image

Is this a bug, are we misunderstanding how kargo works, or misunderstanding how we should be using it?

As we have to remove the link tag, this leaves us with the original freight with the link tag showing in kargo but not existing in the source image repository any more.

Other customisations to stages we have made that I don't think are relevant are:

• we add an alias to the freight at the start of the link stage to be the tag
• we have a http step at the start of the preprod stage to check if the image tag is a semantic version which will fail and stop if it isn't

Screenshots

Steps to Reproduce

Our warehouse looks like this:

apiVersion: kargo.akuity.io/v1alpha1
kind: Warehouse
metadata:
  name: ecr
  namespace: xxxxx
spec:
  subscriptions:
    - image:
        allowTags: ^(?:\d+\.\d+\.\d+|link-\d+)$
        discoveryLimit: 20
        imageSelectionStrategy: NewestBuild
        repoURL: xxxx

Version

1.2.2

Logs

Paste any relevant application logs here.

[krancour]

This is because of how images are sorted when using NewestBuild:

func sortImagesByDate(images []Image) {
	slices.SortFunc(images, func(lhs, rhs Image) int {
		if comp := rhs.CreatedAt.Compare(*lhs.CreatedAt); comp != 0 {
			return comp
		}
		return strings.Compare(rhs.Tag, lhs.Tag)
	})
}

The two tags point to the same image and therefore have the exact same CreatedAt date, so the tie is broken lexically by name.

If you click the Warehouse and go into "Freight Assembly," you will most likely see the semantically versioned tag you are looking for as no.2 in the list of tags the Warehouse has discovered. You can select that and manually create a piece of Freight referencing it.

With all that being said...

NewestBuild is extremely inefficient. It needs to make at least one additional API call to the registry for every tag is considers. As your repo grows, this is likely to get you rate limited and likely to degrade the performance of all Warehouses with subscriptions to the registry in question.

If you explore different conventions for image tagging, you can not only avoid the drag on performance, but may also solve the issue you're reporting.

It's common for date information to be embedded in tags. If you had one Warehouse subscribed to ^link-\d{8}$ tags, for instance, where \d{8} is a date stamp like 20250131, you can select image based on a lexical ordering to always get the newest. This Warehouse can feed the link stage.

You can use a second Warehouse that selects the image with the semantically greatest tag to feed the other Stages.

This is just one possible approach.

fwiw, there's not much that can be done about making NewestBuild more efficient. We're at the mercy of the registry v2 and OCI specs.

[darkpeak]

Thanks @krancour , that's interesting and helpful.

Sounds like we'll have to look at doing what you suggest, i.e. having a second warehouse looking for semantic tags that feeds into preprod and subsequently prod. I can't think of another approach, as there doesn't seem to be a imageSelectionStrategy that puts semantic versions ahead of our link-1234 tags.