Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: Add optional anaconda_nightly_upload_organization argument #47

Merged
merged 6 commits into from
Jan 24, 2024
Merged

ENH: Add optional anaconda_nightly_upload_organization argument #47

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
e50044e
NF: Add anaconda_nightly_upload_url parameter
skoudoro Nov 29, 2023
66d2324
DOC: add labels documentation
skoudoro Nov 29, 2023
317d7cb
Apply suggestions from code review
skoudoro Nov 29, 2023
9740248
- address stefanv comments.
skoudoro Nov 29, 2023
9b762da
FIX: del list, docker handle only str/bool/number
skoudoro Dec 1, 2023
97a6a9c
Handle comma and new line seperated label lists
matthewfeickert Jan 23, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,26 @@ then generate a token at `https://anaconda.org/scientific-python-nightly-wheels/
with permissions to _Allow write access to the API site_ and _Allow uploads to Standard Python repositories_,
and add the token as a secret to your GitHub repository.

## Using a different channel

This Github Action can upload your nightly builds to a different channel. To do so,
define the `anaconda_nightly_upload_organization` variable. Furthermore,
you can add labels for organizing your artifacts using `anaconda_nightly_upload_labels`
optional parameter. See below:

```yml
jobs:
steps:
...
- name: Upload wheel
uses: scientific-python/upload-nightly-action@5fb764c5bce1ac2297084c0f7161b1919f17c74f # 0.2.0
with:
artifacts_path: dist
anaconda_nightly_upload_organization: my-alternative-organization
anaconda_nightly_upload_token: ${{secrets.UPLOAD_TOKEN}}
anaconda_nightly_upload_labels: dev
```

## Artifact cleanup-policy at the ``scientific-python-nightly-wheels`` channel

To avoid hosting outdated development versions, as well as to clean up space, we do have a
Expand Down
8 changes: 8 additions & 0 deletions action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,14 @@ inputs:
anaconda_nightly_upload_token:
description: 'Token to upload to scientific python org'
required: true
anaconda_nightly_upload_organization:
description: 'Organisation name to upload the wheels to'
required: false
default: scientific-python-nightly-wheels
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
default: scientific-python-nightly-wheels
default: scientific-python-nightly-wheels

I don't know how the action works, but this is confusing since it isn't a URL.
If it can be anything other than a URL, that should either be documented or the field be renamed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So the variable name is wrong! it should be anaconda_nightly_upload_organization

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removing the default will break downstream packages using this action as soon as it is merged like scikit-image

scikit-image is using main. Maybe we can update this in a new PR and let some time to all downstream package to add/update this variable.

anaconda_nightly_upload_labels:
description: 'Labels assigned to the uploaded artifacts'
matthewfeickert marked this conversation as resolved.
Show resolved Hide resolved
required: false
default: [main]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@skoudoro Hey sorry to have one final question here (now that I've arrived), but can you point me to a reference on how you know that anaconda upload --label will accept a list of labels (as opposed to repeated calls of --label like how you would have to do with --channel with micromamba install? micromamba install --channel main --channel conda-forge numpy)?

The anaconda-client docs are not good, so it is possible I'm missing something. Have you tested this with any uploads to your registry yet?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you point me to a reference on how you know that anaconda upload --label will accept a list of labels

As you can see below, I just used the help of the command line (anaconda upload --help.

image

There is also the following documentation : https://docs.anaconda.com/anaconda-repository/user-guide/tutorials/#using-labels-in-the-development-cycle

Have you tested this with any uploads to your registry yet?

Yes, I tried with some artifacts on our repository. I use the following command manually:
anaconda upload --force --user dipy --label dev --label main *1.6.0*.whl

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure how to handle the list with the docker. I am looking forward for your feedback

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

something is still wrong here, see the "Sequence was not expected" error in the CI: https://github.com/scientific-python/upload-nightly-action/actions/runs/7038337722/job/19199259041#step:9:1

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah @bsipocz's point is exactly my concern here. [main] is a sequence but @skoudoro the example you gave of

anaconda upload --force --user dipy --label dev --label main  *1.6.0*.whl

demonstrates the opposite of what you said. This is not a list, this is repeated calls of --label.

@skoudoro can you tell us more about how you're using labels now and why you want to have multiple labels?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, I just updated the code. This is not a list, sorry for the confusion.

how you're using labels now and why you want to have multiple labels?

  • We use only the label dev for now
  • in our case (https://anaconda.org/dipy/dipy), the organization name is not explicit for nightly wheels. I would like to avoid random people downloading nightly wheels and believing that is it a release. So, we tag them as dev instead of main. it force user to be explicit and conscious of what they are trying to get. At the same time, it is not an issue for downstream packages.
  • we do not use multiple labels as the same time. the line above was just an example to show you how it can be done.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
default: [main]
default: main

This should be a string and not a sequence (c.f. #47 (comment))

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, agree.

I just read more about that and make sense


runs:
using: 'docker'
Expand Down
12 changes: 11 additions & 1 deletion cmd.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,17 @@ set -x
# this is to prevent accidental uploads
echo "Getting anaconda token from github secrets..."

ANACONDA_ORG="scientific-python-nightly-wheels"
ANACONDA_ORG="${INPUT_ANACONDA_NIGHTLY_UPLOAD_ORGANIZATION}"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would this break the current behaviour for the core libraries uploading to the default location?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, because I added the default location in this line: https://github.com/scientific-python/upload-nightly-action/pull/47/files#diff-1243c5424efaaa19bd8e813c5e6f6da46316e63761421b3e5f5c8ced9a36e6b6R20

However, I believe that:

  1. this parameter should be required instead of optional
  2. After merging, we should ask the core libraries to add and update this variable
  3. Then, when it is done, create a new PR here to remove this default location and let it empty.

Thank you for your review @bsipocz!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@skoudoro While I'm not against the idea of being explicit about the target channel (I think that's a good idea in general to guard against default changes), I think the original intent for the GitHub Action was to make uploading as easy as possible for the core packages with the idea that they need to configure things as minimally as possible.

I think that this could probably be discussed more broadly outside of this PR though, so maybe you can open up an Issue from your comment to discuss what the target audience and scope of the action configuration should be?

ANACONDA_TOKEN="${INPUT_ANACONDA_NIGHTLY_UPLOAD_TOKEN}"
ANACONDA_LABELS="${INPUT_ANACONDA_NIGHTLY_UPLOAD_LABELS}"

# if the ANACONDA_ORG is empty, exit with status -1
# this is to prevent attempt to upload to the wrong anaconda channel
if [ -z "${ANACONDA_ORG}" ]; then
echo "ANACONDA_ORG is empty , exiting..."
matthewfeickert marked this conversation as resolved.
Show resolved Hide resolved
exit -1
fi


# if the ANACONDA_TOKEN is empty, exit with status -1
# this is to prevent accidental uploads
Expand Down Expand Up @@ -48,5 +57,6 @@ echo "Uploading wheels to anaconda.org..."
anaconda --token "${ANACONDA_TOKEN}" upload \
--force \
--user "${ANACONDA_ORG}" \
--label "${ANACONDA_LABELS}" \
"${INPUT_ARTIFACTS_PATH}"/*.whl
echo "Index: https://pypi.anaconda.org/${ANACONDA_ORG}/simple"
Loading