ArtifactHub Github Action

[jon-nfc]

Is your feature request related to a problem? Please describe.

currently in the process of adding my projects to artifacthub and it's a manual process. thanks to github/gitlab the build and release process is automated with CI.

Describe the solution you'd like
A github action that could be used via github CI (workflows in particular) to automate the process of adding/updating artifacthub items you wish to deploy there.

would also be nice to have the action update the available tags/version. (so far, I've noticed containers have a tag limit)

Describe alternatives you've considered
doing it manually although this is cumbersum is not viable when you have lots of projects.

Additional context

• docs: https://docs.github.com/en/actions

[tegioz]

Hi @jon-nfc 👋

Artifact Hub supports multiple kinds of repositories, like Helm charts or Tekton pipelines. These repositories can contain multiple packages, with the exception of the Container image kind, where each repository is expected to contain a single package for a number of reasons.

For all kinds except the Container image, publishers usually add the repository once using the Artifact Hub UI (manually, as you described). Once a repository is added, we scan it periodically for changes, and apply them as needed on our end.

As an example, if you maintain a Helm charts repository, you would just add it once manually to Artifact Hub and you can mostly forget about it. You can release new versions of your charts as often as you need, deprecate existing ones, etc, but there is no action required on your end to keep your chart in Artifact Hub up to date.

This is the reason why, in the majority of cases, a GitHub action wouldn't be helpful or even needed.

The container image kind is a bit different (it's an experimental one actually). For this kind, to keep maintenance simple, we recommend adding a mutable tag (i.e. latest) to have presence on Artifact Hub so that there is no further action required going forward as the image is updated. If you have a large number of images that you'd like to publish and prefer not doing this process manually, you could write a script that using the AH API would add them programatically.

Hope this helps 🙂

[jon-nfc]

Hi @jon-nfc 👋

Artifact Hub supports multiple kinds of repositories, like Helm charts or Tekton pipelines. These repositories can contain multiple packages, with the exception of the Container image kind, where each repository is expected to contain a single package for a number of reasons.

For all kinds except the Container image, publishers usually add the repository once using the Artifact Hub UI (manually, as you described). Once a repository is added, we scan it periodically for changes, and apply them as needed on our end.

As an example, if you maintain a Helm charts repository, you would just add it once manually to Artifact Hub and you can mostly forget about it. You can release new versions of your charts as often as you need, deprecate existing ones, etc, but there is no action required on your end to keep your chart in Artifact Hub up to date.

@tegioz, cheers for taking the time to respond. I had read the docs on what you have mentioned.

This is the reason why, in the majority of cases, a GitHub action wouldn't be helpful or even needed.

I disagree here. An action would still be required. This enables the "artifacthub" item details to be stored as "config as code" and for the initial "Add" of the item to artifacthub. The publishing of builds to external locations is a crucial step within any CI/CD pipeline. This along with reducing human error and now having to manage the permissions to access an external system. Having an action will completely remove the additional burden of the latter.

The container image kind is a bit different (it's an experimental one actually).

The what now? should we be expecting this to be removed?

For this kind, to keep maintenance simple, we recommend adding a mutable tag (i.e. latest) to have presence on Artifact Hub so that there is no further action required going forward as the image is updated.

Deploying "mutable" tags is the worst advice that can be given. This advice leads to those with less knowledge about how the consequences of using this tag can lead to un-predictable consequences; has them scratching their heads on what is or has gone wrong.

I don't see "presence" as the important part of artifacthub. I decided on using artifacthub so that end users could use the "webhook" feature to link into their automation systems so that they could automagically deploy a new version when they have been notified.

re the use of mutable tags for containers. Is this going to be updated in the future to be similar to helm charts wherein artifacthub will track the available tags from the repo/registry?

Without being able to update tags/versions for containers (and part of CI/CD) removes the possibility of someone using the subscriptions/webhooks to be updated when a new version is available.

If you have a large number of images that you'd like to publish and prefer not doing this process manually, you could write a script that using the AH API would add them programmatically.

To bring this into perspective from my point of view. I personally manage 50+ projects across gitlab and github. Most of these projects create a docker image (some more than one), helm chart(s), argoCD templates (some more than one) and kyverno policies (again some more than one). without an action to do and maintain these "artifacthub" items, use of artifact hub for the intent of why I started using it in the first place will be moot.

Officially supported actions are preferred and should be offered as this will increase the uptake of the service, in this case, Artifacthub. Not only this, those that created the software (artifacthub) know if the direction of the "action" is correct. This further cements an additional level of "trust" from consumers.

further to this. Some projects will contain multiple artifacthub items. take a piece of software that you build as an example, that also has a docker image and helm chart. that's now a minimum of two artifact items per project.

[tegioz]

Hi @jon-nfc

We do not have plans to remove support for container images. However, this artifact kind has some limitations and it may not be easy to make them work like the other kinds do, at least in the short term. Please take a look at this comment #1685 (comment) for more context about how it currently works the way it does. Note that we do not recommend "deploying" images using mutable tags. Our advice is to use them for a specific purpose (presence in AH) with the intention of simplifying maintenance given the limitations referenced above.

I don't really think that an action is required, but it could be a nice to have for some particular use cases. There are over 4k repositories registered in Artifact Hub, published by 2.5k organizations and many more users, and this is the first time this has come up in more than 4 years. Well, actually it's the second, both this same week (#3963).

I understand your interest in automating the management of Artifact Hub resources this way, it's interesting and can be definitely useful in some concrete cases (exactly like the Terraform provider request). But given that for most kinds this is usually a one time operation, I find its usefulness limited.

We have limited resources and we need to be very careful about what we focus on, to make sure the time invested on Artifact Hub benefits a large number of publishers and users. We provide a full featured control panel that allows performing all operations that Artifact Hub supports, plus an API that allows doing the same programatically. Providing and supporting more solutions to manage Artifact Hub resources is out of the scope of the project. But we welcome other contributors to create external add-ons using the API, like in the case of the Terraform provider.

I don't know if it's an option for your use case, but I'd suggest you to consider using a single repository for each artifact kind if possible (i.e. No Fuss Computing Helm charts, etc). This is something pretty common in Artifact Hub, specially for publishers with lots of content (i.e. Bitnami).

[tegioz]

Will close this issue for now, please feel free to reopen if needed 🙂

[jon-nfc]

Hi @jon-nfc

We do not have plans to remove support for container images. However, this artifact kind has some limitations and it may not be easy to make them work like the other kinds do, at least in the short term.

good to hear.

Please take a look at this comment #1685 (comment) for more context about how it currently works the way it does.

#1685 (comment) is an interesting read and after reviewing makes more sense to the approach taken with images. It also points out the same issue that I'm going to face nofusscomputing/centurion_erp#19.

Note that we do not recommend "deploying" images using mutable tags. Our advice is to use them for a specific purpose (presence in AH) with the intention of simplifying maintenance given the limitations referenced above.

Presence is not the important fact of artifacthub. For me and I'd guess others whom automate, the artifacthub webhooks is the feature artifacthub offers that others don't.

I don't really think that an action is required, but it could be a nice to have for some particular use cases.

As someone who is on the end user side to artifacthub, an action is paramount. Not only should the action add the project to artifacthub, it should also provide ALL of the artifacthub features. i.e. add/update repo, add/update/remove verified publisher manifest and in the case of the containers provide the latest 10 immutable tags. Without this adding your own project to artifacthub is too time consuming.

Can this issue be re-opened and have it labeled as help wanted so that the community can discover this request and should someone desire creating it, the crux of this issue can be known. Also having it open will allow others whom may be interested to add notes as appropriate. Whilst being able to create this feature is within my skillset, time is of no avail for me to assign to this project. what the future holds is anyone's guess!!

There are over 4k repositories registered in Artifact Hub, published by 2.5k organizations and many more users, and this is the first time this has come up in more than 4 years. Well, actually it's the second, both this same week (#3963).

Me personally, would take interest, like this feature (and #3963) request as a sign that artifacthub may be picking up. I'm of the opinion that the goal of artifact hub is that which we have had lacking prior to its arrival.

I understand your interest in automating the management of Artifact Hub resources this way, it's interesting and can be definitely useful in some concrete cases (exactly like the Terraform provider request). But given that for most kinds this is usually a one time operation, I find its usefulness limited.

see notes above re "it's usefulness". In addition it's no simple feat to get an item on artifacthub. it took me a few hours of time to read the docs, work out how to get it added and another few hours to get it working. whilst most of this is the "learning a new item" maintenance of that learning is required to update an item down he track when you forget what the docs told you. As I'm sure you're aware as a project creator, priority is to your product and without systems to "do for you" (CI/CD), if it's not related to the product development it has no weight placed against working on it, in this case adding your product to artifact hub. simply put it becomes too burdensome to do.

We have limited resources and we need to be very careful about what we focus on, to make sure the time invested on Artifact Hub benefits a large number of publishers and users.

This I can and do understand. As requested above can we re-open this issue as a community developed endeavor?

I don't know if it's an option for your use case, but I'd suggest you to consider using a single repository for each artifact kind if possible (i.e. No Fuss Computing Helm charts, etc). This is something pretty common in Artifact Hub, specially for publishers with lots of content (i.e. Bitnami).

This is the intent. Like yourself time is limited when creating a product. Single repositories, if avail for the item will be used.

Thanks for the detail placed in your responces, its most definitely encouraged discussion

[tegioz]

Thanks for your response @jon-nfc 🙂

Instead of reopening it, it'd probably be best to convert it to a discussion and move it to the ideas category. We mostly use issues to track pending tasks on our side, items that require an action to be taken (it may be a problem to fix or a new feature we'll take care of).

So this would probably fit best in discussions, as an interesting idea. I'd also rather to not label it as help wanted, as it may lead users to think that we'd like that feature to be part of this repository. But we'd prefer it to be an external add-on that is maintained separately.

[jon-nfc]

Thanks for your response @jon-nfc 🙂

Instead of reopening it, it'd probably be best to convert it to a discussion and move it to the ideas category. We mostly use issues to track pending tasks on our side, items that require an action to be taken (it may be a problem to fix or a new feature we'll take care of).

So this would probably fit best in discussions, as an interesting idea. I'd also rather to not label it as help wanted, as it may lead users to think that we'd like that feature to be part of this repository. But we'd prefer it to be an external add-on that is maintained separately.

Agreed