Release: new helper script to improve the release process #452
Conversation
|
Hi @portersrc ! That's a great idea to have a script to help on the release. Also it's always nice to see (and learn) new different ways to doing things in bash! |
portersrc
force-pushed
the
release-helper
branch
from
83ceecf
to
81216d1
Compare
|
Hi @wainersm thanks for the helpful feedback. I've tried to address each one. |
|
|
||
| # multi-line grep to find the nydus-snapshotter version in kata's version.yaml | ||
| # then awk/tail/tr and string manipulation to sand it down. | ||
| nydus_version_kata="$(grep -zoP "nydus-snapshotter.*\n.*description.*\n.*url.*\n.*version.*" ${kata_versions_dest} | awk '{print $2}' | tail -n1 | tr -d '\0')" |
There was a problem hiding this comment.
are you deliberately avoiding the use of yq? The use of grep can be fragile in case things re-order or somehow modify... Or just a bit of in-line python, it should be installed everywhere ;-) python -c "import yaml; print(yaml.load(open('$kata_versions_dest'), Loader=yaml.SafeLoader)['externals']['nydus-snapshotter']['version'])"
There was a problem hiding this comment.
In the past we discussed the problem of using yq to write yaml that will result on formatting changes, if I'm not mistaken even with python's lib. However, use yq or python just to read yaml should be acceptable.
There was a problem hiding this comment.
Yes, I left a note in the comments at the top of the script partly for myself about this:
# Note: This script intentionally uses sed. yq (or even python libraries like
# "yaml" or "ruamel.yaml") do not preserve yaml formatting.
I wasted a bit of my life trying to find a good solution for this, and I never found one. It's true that we could read the values out with yq, but in this case we end up writing it back (so, to preserve formatting, we'd still depend on sed, even if we eliminate the grep). In general, I'd prefer this entire script to be python. In fact, I wrote it in python, partly to solve this problem, but then I couldn't find a solution to this issue.
We also considered reading and then writing back to file all of the yaml files with one of these python libraries -- the idea being that once they're formatted by that library, future writes with that library would maintain the same formatting. But this idea has multiple problems, too (introduces a large, ugly PR across at least this repo; when the library version changes, its own formatting can change; and I think I noticed inconsistencies when testing this, too). So, unless I've missed something, grep and sed seems the least bad to me at the moment.
There was a problem hiding this comment.
I's fun watching developers argue about yaml formatting while they use go which enforces gofmt on compile... Anyway it's not on me to judge, I'd prefer using yq at least to get the values but it's a mild suggestion.
As for having this script in python, IMO it'd be more readable and writing could be performed by re string manipulation and writing the full content afterwards. The re.sub should be safer. Anyway I'm a pythonist so I'm probably biased so take this also as a mild suggestion.
As for now the script is fine, just a bit fragile.
hack/release-helper.sh
Outdated
| # | ||
| function update_prereqs() { | ||
| # grab the hash for the latest commit on the pre-install-payload folder | ||
| # in github (which requires figuring out the hash of the merge) |
There was a problem hiding this comment.
Could you please document in code how this works rather than a simple comment that knowing the latest commit helps? IIUC you're looking for the oldest merge commit that contains the latest change to the pre-install-payload, am I right?
There was a problem hiding this comment.
Yes, I added a more verbose comment above the function. I'd be grateful for a cleaner way to do this, too, if you see a way.
Signed-off-by: Chris Porter <[email protected]>
portersrc
force-pushed
the
release-helper
branch
from
81216d1
to
240c985
Compare
|
Thanks @ldoktor for the review! I've addressed the minor things and left a couple comments open for you. |
| # | ||
| function update_prereqs() { | ||
| commit="$(git log -1 --pretty=format:"%H" install/pre-install-payload/)" | ||
| prereqs_latest_hash=$( (git rev-list $commit..HEAD --ancestry-path | cat -n; git rev-list $commit..HEAD --first-parent | cat -n) | sort -k2 -s | uniq -f1 -d | sort -n | tail -1 | cut -f2) |
There was a problem hiding this comment.
Wouldn't:
commit=$(git log -n 1 --pretty=format:"%H" -- install/pre-install-payload/)
prereqs_latest_hash=$(git log --ancestry-path --merges --pretty=format:"%H" $commit^..HEAD | tail -n 1)provide the same result? (not an expert, just remember using it somewhere)
During the CoCo release process, there are a couple of tedious steps in the operator repo that can be automated. The script in this PR attempts to do that.
We just used this to help with the v0.10.0 release. The checklist is here. This script helps with steps 4 and 5.
This script has been used for v0.9.0 and now v0.10.0, so it's been run a couple of rounds and been found useful enough to submit for review.