feat(template): allow to override the template from cli, configuration and plugins

[noirbizarre]

Description

This PR provides full template customization by plugins, configuration and command line.

Plugins can now contribute a template (by providing their own jinja PackageLoader as template_loader attribute) as well as more template variables with their defaults (using the template_extras attribute from `BaseCommitizen). It was my initial use case, I was monkey-patching commitizen like this

Users can now provide their own template (in the format of their choice, like .rst in #384, with extras variables too) by:

• providing a keep_a_changelog.j2 file at the root of their project
• setting the template config to a project root-relative path to their template
• using the --template/-t parameter in both bump and changelog command

They also can provide or override some template variables by:

• setting the extras config which is a dict
• using the --extra/-e parameter (can be used as many times as there are variables) in both bump and changelog

This change is backward compatible, fully tested and documented.

Checklist

• Add test cases to all the changes you introduce
• Run ./scripts/format and ./scripts/test locally to ensure this change passes linter check and test
• Test the changes on the local machine manually
• Update the documentation for the changes
• Make template overridable
• Allow extra template parameters
• Allow incremental with overridden templates
• Provides examples for other syntaxes
  • ReStructuredText
  • AsciiDoc
  • Textile

Additional context

Fixes #132
Fixes #384
Fixes #433
Closes #376 (I didn't keep the tree to releases variable renaming as it is a breaking change (I can however submit this as another PR because I think the releases naming is more explicit)

[codecov]

Codecov Report

Attention: 11 lines in your changes are missing coverage. Please review.

Files	Coverage Δ
commitizen/__version__.py	100.00% <100.00%> (ø)
commitizen/bump.py	100.00% <100.00%> (ø)
commitizen/changelog.py	100.00% <100.00%> (+0.50%)	⬆️
commitizen/changelog_formats/asciidoc.py	100.00% <100.00%> (ø)
commitizen/changelog_formats/base.py	100.00% <100.00%> (ø)
commitizen/changelog_formats/markdown.py	100.00% <100.00%> (ø)
commitizen/changelog_formats/restructuredtext.py	100.00% <100.00%> (ø)
commitizen/changelog_formats/textile.py	100.00% <100.00%> (ø)
commitizen/commands/bump.py	97.72% <100.00%> (+0.08%)	⬆️
commitizen/commands/changelog.py	99.07% <100.00%> (+0.13%)	⬆️
... and 25 more

📢 Thoughts on this report? Let us know!.

[Lee-W]

Hi @noirbizarre for all your amazing contribution! I start to review this one but i've not yet be able to finish it. I'll do so when I have time. Thanks!

[woile]

I like the PR 👍🏻 , though I see an issue, commitizen is using markdown to detect which was the last commit made, and from there create an incremental changelog, without touching the rest of the changelog. How would we guarantee this with the templates feature?

commitizen/commitizen/changelog.py

Lines 150 to 156 in de5c39c

	def parse_version_from_markdown(value: str) -> Optional[str]:
	if not value.startswith("#"):
	return None
	m = re.search(defaults.version_parser, value)
	if not m:
	return None
	return m.groupdict().get("version")

One solution would be to add parameter to the config:

[tool.commitizen]
version = "0.3.0"
latest_changelog_version = "0.1.0"

Now, it still worries me, how would commitizen know where to inject the incremental changelog?
At the moment we use an approximation system tied to markdown, if it is a markdown title, then we try to match to a version format.

If we allow for example templates in sphinx, or any other template system, how would commitizen know where to inject the template?

Even with my provided solution using latest_changelog_version a user could potentially add a version as part of a commit (which we have no control). For example, some adds:

fix: update dep X to 0.1.0

without a way to know if it's a title, commitizen would match that, and it would parse that version, potentially injecting the changelog there.

Does it make sense what I'm saying haha?

Could we have some tests with different templates? For example, using AsciiDoc

= Changelog

== 0.1.0

=== Bug Fixes

-  update dep X to 0.1.0

Textile

h1. Changelog

h2. 0.1.0

h3. Bug Fixes

*  update dep X to 0.1.0

Thanks!

[woile]

We could potentially add a command to cz changelog --export-template that prints the template to stdout 🤔

[noirbizarre]

Oh, I didn't notice the template parsing. That is definitively something I need to handle in this PR.
I'll fix all comments and try adding something to handle changelog reverse parsing.

The problem I see with the latest_changelog_version approach is that it is not backward compatible, and we lose some automation.

What about making parse_version_from_markdown a parse_version_from_changelog method on BaseCommitizen ?

This should be backward compatible, Plugin providing their template could override to provide the parsing. For the template from CLI or conf, I can allow providing a string pointing to a custom method, something like:

[tool.commitizen]
template = "my_template.whatever"
changelog_version_parser = "path.to:my_parser"

I like the cz changelog --export-template idea ! Anyone can bootstrap its customized template from the original one. I'll this too 👍🏼

[Lee-W]

Hi @noirbizarre I notice you have some discussion with @woile . Wondering whether this pr is ready for review now?

[noirbizarre]

I rebased it on latest v3, added the changelog --export-template but it's lacking the incremental part (from the discussion 👆🏼).
I hope to be able to deliver it by the end of the week.
However, it is already possible to review what's in it. I'll add the incremental part as an extra commit to help review it later.

[Lee-W]

Got it. Let me focus on #646 first and will get back to this one afterward 💪

[noirbizarre]

I updated the PR including:

• rebased on style(ruff): enable pyupgrade (UP) rules and fix them all #889 to benefit from Python 3.8+ syntax update (include latest master + ruff pyupgrade enabled)
• applied all suggestion except the -> None because other methods don't have this syntax (I think this should go in another PR where linters are configured accordingly)
• all format/Format have been replaced by changelog_format/ChangelogFormat everywhere
• history squashed to have consistent commits (and so consistent changelog) if merged using Rebase and merge
• defaults.version_parser has been removed to use the new version_scheme.parser

I'll close all Python 3.8+ syntax update suggestion for easier review (most of them being in #889 or automatically fixed with ruff).
#889 can/should be merged before.

[Lee-W]

@noirbizarre I think we're good to merge this one 🎉 but after merging #889, it seems to have some conflicts 😱 Please merge it once you resolve the conflict. Thanks so much for the hard work! this is a really exciting feature!

[noirbizarre]

@Lee-W I think this was a GitHub-only weirdness because I wasn't able to display conflict in the Web UI, and locally I haven't got any (I just did a rebase on the current master without any conflict).

So there it is, PR up to date, waiting for CI to pass, and then it should ready 🚀 🎉

[Lee-W]

hmmm.... not sure what's happening. @noirbizarre do you want to force push to retrigger the github actions again?

[noirbizarre]

Done 👍🏼
🤞🏼

[noirbizarre]

This is driving me crazy !!! 😱
Windows builds are failing on pre-commit (not touched in the PR and not related to the PR).
I think the last pre-commit update is broken on Windows runners (for node-based checks). Excluding this version seems to work.
I think I'll do another separate PR to fix the pre-commit test case by using a check which is not relying on NodeJS (will be faster to execute and safer). Most probably a check based on ruff to have both speed (install and exec) and no dependency management to handle in pre-commit.

EDIT: build succeeded 🎉. The pre-commit fix worked.

EDIT2: I tried a lighter change: just updated prettier to the latest version in the test, and it seems to be enough. I don't know whether Windows worker was broken or if there is really an issue with pre-commit, prettier or the Windows runner. Anyway, it worked

[woile]

Awesome news 🎉 🎉 🚀
Thanks everyone!