Uploading nightly wheels (for testing and better cross-project integration)

[jarrodmillman]

Would you be interested/willing to upload nightly wheels to:
https://anaconda.org/scientific-python-nightly-wheels/

It would also be great if you would testing against nightly wheels that you depend on (i.e., numpy).

You can read more about why and how we are building / testing against nightly wheels here:
https://scientific-python.org/specs/spec-0004/

We provide a GitHub action to upload nightly builds to the scientific-python nightly channel:
https://github.com/scientific-python/upload-nightly-action/

We @matthewfeickert, @larsoner, @tupui are happy to help.

[ianthomas23]

Yes. It has been on my todo list for a while but I have been waiting for clarity on which packages should and shouldn't be involved. I will take a look and reach out if I need any help.

ContourPy already tests against nightly wheels (#280).

[matthewfeickert]

@ianthomas23 Good news, we have the approval (scientific-python/upload-nightly-action#30 (comment), scientific-python/upload-nightly-action#30 (comment)) to add contourpy to the nightly wheels Anaconda Cloud org https://anaconda.org/scientific-python-nightly-wheels/ !

To get started on this please do the following:

Steps to gain upload access to https://anaconda.org/scientific-python-nightly-wheels/

ANACONDA_ORG_UPLOAD_TOKEN* [x] Create an account on Anaconda Cloud

• Reply here with the link to your account (https://anaconda.org/ianthomas23)
• Make a new Anaconda Cloud API token with permissions described in https://scientific-python.org/specs/spec-0004/#process-for-adding-new-projects

• Save the API token as a repo GitHub Actions secret named ANACONDA_ORG_UPLOAD_TOKEN
• Direct me (@matthewfeickert) to a contourpy nightly wheel for me to upload to create the package in the registry
• Verify that I have added you to the contourpy team "Group" on https://anaconda.org/scientific-python-nightly-wheels
• Verify that I have added the package scientific-python-nightly-wheels/contourpy to the contourpy team "Group"

If you'd also like to use the scientific/upload-nightly-action GitHub Action please check out the action repo's README. If you need to collect wheels from different CI jobs, consider creating a new GitHub Actions workflow with logic similar to matplotlib's or Awkward's.

If you have any questions please just ask here!

[ianthomas23]

@matthewfeickert: I've added the ANACONDA_ORG_UPLOAD_TOKEN as requested. Wheels built today are available at the end of github action run https://github.com/contourpy/contourpy/actions/runs/8039781537, the direct link to the artifact is https://github.com/contourpy/contourpy/actions/runs/8039781537/artifacts/1273387577. I've given you read-only access to this repo as otherwise I don't think you can download the artifacts.

This isn't a busy repo so initially I've opted for a scheduled build of the wheels once a week, but I can also do so manually if there is a lot of activity. I'll add use of the upload-nightly-action when the receiving end is fully set up.

I've separated out the built sdist into its own artifact rather than being in the wheels artifact to avoid ever uploading the sdist to the nightly wheels channel as it is quite big and I presume not required.

The wheel tags are currently 1.2.1.dev1 and do not vary with the git commit hash. Is this going to cause a problem? I've read some discussion about this for awkward but I am not sure what the outcome was.

[matthewfeickert]

I've added the ANACONDA_ORG_UPLOAD_TOKEN as requested. Wheels built today are available at the end of github action run https://github.com/contourpy/contourpy/actions/runs/8039781537, the direct link to the artifact is https://github.com/contourpy/contourpy/actions/runs/8039781537/artifacts/1273387577. I've given you read-only access to this repo as otherwise I don't think you can download the artifacts.

@ianthomas23 Great! I've got the package up now at https://anaconda.org/scientific-python-nightly-wheels/contourpy so you can remove my repo permissions.

This isn't a busy repo so initially I've opted for a scheduled build of the wheels once a week, but I can also do so manually if there is a lot of activity. I'll add use of the upload-nightly-action when the receiving end is fully set up.

Sounds good. If you have any upload infrastructure setup can you manually trigger a run so that we check that everything is in working order? If you have any questions about the upload-nightly-action just let me know.

I've separated out the built sdist into its own artifact rather than being in the wheels artifact to avoid ever uploading the sdist to the nightly wheels channel as it is quite big and I presume not required.

Thanks and correct — just the wheels are of interest here.

The wheel tags are currently 1.2.1.dev1 and do not vary with the git commit hash. Is this going to cause a problem? I've read some discussion about this for awkward but I am not sure what the outcome was.

Thanks for asking, but that's fine! We and Anacond Cloud can handle this. The issue was just in the circumstance where there was a single wheel that was getting reuploaded and overwritten, but that's also been fixed, so all uploads should be fine now.

[ianthomas23]

I've manually triggered a workflow run that builds and uploads the wheels (https://github.com/contourpy/contourpy/actions/runs/8062140403/job/22023379677). It all seems to be working to me. I think that means this is complete now.

I'll keep an eye on it. If anyone needs to contact me about it they can do so here, and I am also lurking on the Scientific Python Discord as well.

[matthewfeickert]

It all seems to be working to me. I think that means this is complete now.

Yup! You're good to go now. 👍