CI: add automatic publication to pypi.org and test.pypi.org #77
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
hnez
force-pushed
the
auto-pypi
branch
4 times, most recently
from
8edd4f5 to
f277969
Compare
1 task
hnez
force-pushed
the
auto-pypi
branch
2 times, most recently
from
bdeeee1 to
51d9421
Compare
This means we no longer have to update the version number manually when creating a new release because just setting a tag suffices. It does however also add some complexity and magic to the project, like incorrect version numbers when the git history is not complete (hence why we now clone the whole history in the build job) and all files included in the git repository now being shipped in the source distribution (while previously only files mentioned in the MANIFEST.in were included). The beneftis when it comes to automating the release process do however outweight the drawbacks of added compexity. We use `local_scheme = "no-local-version"` to prevent random files in the project directory from causing "+dirty" version numbers. Signed-off-by: Leonard Göhrs <[email protected]>
We want to reduce the potential for human error in our publication process and also streamline the process for everyone with the permission to create tags in the repository. The CI job runs for new commits pushed to the master branch and newly pushed tags, as long as the PUBLISH_PYPI GitHub Action variable is set to "true". This is to prevent CI runs on forked repository from failing because they are not allowed to publish on pypi.org and test.pypi.org. A fork that wants to use the publish logic just has to set the PUBLISH_PYPI variable for their repository. The job does not check out the git repository (hence why it does not use the existing publication logic in the Makefile) and instead downloads the artifacts generated by the build job. All builds are uploaded to test.pypi.org (so they can be tested via pip install) and tagged releases are uploaded to pypi.org as well. Also remove the upload helpers from the Makefile to make it clear that they are replaced by the automated process. Signed-off-by: Leonard Göhrs <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Publishing software manually is a (somewhat) error-prone and (somewhat) tedious process, hence why other projects have already automated this process.
Let's join these other projects and also automate our publication process!
The setup process is surprisingly straight forward and involves setting up trusted publishing on both pypi.org, as well as test.pypi.org. See linux-automation/lxa-iobus#46 for an example on how to do that.
The job also requires a Github Action variable to be set to run (to prevent it from running for forks - for which the publication step would fail due to insufficient permissions). Also see linux-automation/lxa-iobus#46 for an example on how to do that.
TODO before merging:
Note
Due to our date based versioning format some of the versions that
setuptools_scmgenerates will look a bit wonky.The version numbers for un-tagged releases are constructed by taking the previous tagged version, adding one to the
last numeric element of it and appending a
.dev…suffix.For versions like
24.01.1this will look fine (24.01.2.dev123) but for versions with the month as last element it will looka bit strange
24.01->24.2.dev123.These versions are only used on test.pypi.org, so I consider that okay.