chore: add towncrier as changelog manager

[fschuch]

This is a POC proposing quality-of-life changes in how we keep changelog files on the project.

The main motivation is the sure merge conflicts we get under on "Unreleased" entry on the changelog files every time we have features developed in parallel, and every time we git-cherrypick to release branches. Both happen quite often and demand rebase and dev time to be spent on them.

The proposed implementation adds towncrier as the changelog manager in the project. With it, unreleased entries are kept on different news fragment files, minimizing conflicts and streamlining the development workflow. Fragments are consolidated into the changelog and deleted when new releases are prepared. There is no major difference when compared to the current way of working, the exact same text we already include under the "Unreleased" header will just go to a different file.

Notice it is a Python package that can be easily installed by pipx and ux, or even included as a dev dependency on poetry. However, all we need is to create the news fragments in the format it expects, our CI takes care of the rest. Devs can install it as they find appropriate. It makes this approach also suitable for non-Python projects.

Jobbergate already contains a GitHub action to backport changes to multiple release branches backport.yaml based on PR labels, but we just can't take advantage of it due to merge conflicts.

It is still up to discussion if we should keep them per subproject or on a unified file with different entries. I went for the former considering we release all components in sync but not all of them include changes every time, so we can better visualize it in a unified file.

Release scripts are also deleted now we delegate the changelog changes, and considering we don't need to perform the checks now all deployments happen from the CI.

Expected outcomes:

• Enable automation when backporting patches to multiple release branches
• Avoid the sure merge conflicts on the changelog files
• Make the dev experience less error-prone and less time-consuming in general (no manual rebases on changelogs)

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 92.93%. Comparing base (ae1dc5a) to head (9005505).
Report is 1 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #690   +/-   ##
=======================================
  Coverage   92.93%   92.93%           
=======================================
  Files          88       88           
  Lines        4842     4842           
=======================================
  Hits         4500     4500           
  Misses        342      342           

Flag	Coverage Δ
agent	94.04% <ø> (ø)
api	95.43% <ø> (ø)
cli	88.62% <ø> (ø)
core	96.31% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[dusktreader]

I'm very interested in this, and I think it's a nice solution for the pain we have in managing subproject CHANGELOGs. I have some questions and thoughts for you:

[dusktreader]

I'm all for this change. I think we should start using it in Jobbergate, see how it goes for a few weeks, and then consider migrating other projects to it as well.

[matheushent]

Overall I like this, I'm 100% in favour of using it.