release_2.21.0.dev1

pantsbuild.pants release 2.21.0.dev1

release_2.21.0.dev0

New Features

• allow passing extra docker run args to docker_image (#20607)

• Add support for Scalafix (#20394)

• run terraform plan when setting --dry-run flag on experimental-deploy goal (#20488)

User API Changes

• Remove now-expired deprecated python_awslambda alias (#20619)

Bug Fixes

• Resolve adhoc_tool, code_quality_tool execution dependencies relative to target location (#20581)

• Silence warnings from Pex by default, for now (#20590)

• javascript: package yarn v2 (#20540)

• Fix update build files formatter selection (#20580)

• Remove asm -compiling-runtime flag for go 1.22+ compatible versions (#20554)

• Add all new Ruff backends to list of plugins (#20555)

Documentation

• Document the need for migrating tool to user lockfiles for export (#20604)

• Update doc_url calls for new website (#20583)

• docs: export dependency graph as adjacency list (#20566)

• Restructure JVM docs. (#20544)

---

(NB. this release had its tag moved to a different commit and then back on 2024-03-04. The built artifacts are from the first time it was pointing at this commit, not the current tag. ref.)

release_2.20.0rc0

Bug Fixes

• Resolve adhoc_tool, code_quality_tool execution dependencies relative to target location (Cherry-pick of #20581) (#20608)

• Silence warnings from Pex by default, for now (Cherry-pick of #20590) (#20593)

• Fix update build files formatter selection (Cherry-pick of #20580) (#20582)

• Remove asm -compiling-runtime flag for go 1.22+ compatible versions (Cherry-pick of #20554) (#20562)

• Add all new Ruff backends to list of plugins (Cherry-pick of #20555) (#20556)

Documentation

• Document the need for migrating tool to user lockfiles for export (Cherry-pick of #20604) (#20613)

• Update doc_url calls for new website (Cherry-pick of #20583) (#20588)

• docs: export dependency graph as adjacency list (Cherry-pick of #20566) (#20585)

• Restructure JVM docs. (Cherry-pick of #20544) (#20564)

release_2.19.1rc1

Bug Fixes

• Remove asm -compiling-runtime flag for go 1.22+ compatible versions (Cherry-pick of #20554) (#20561)

Documentation

• Document the need for migrating tool to user lockfiles for export (Cherry-pick of #20604) (#20614)

release_2.18.3rc0

Bug Fixes

• python-infer: Avoid false positive strings (Cherry-pick of #20472) (#20599)

• python-infer: respect ignore pragma w/ strings (Cherry-pick of #20477) (#20601)

• Fix direct_url in python PEP660 editable wheels (Cherry-pick of #20486) (#20602)

• Remove asm -compiling-runtime flag for go 1.22+ compatible versions (Cherry-pick of #20554) (#20563)

Documentation

• Document the need for migrating tool to user lockfiles for export (Cherry-pick of #20604) (#20615)

• Fix capitalization of Terraform in left nav (#20489)

release_2.20.0a0

New Features

• add more module mappings for popular packages (#20551)

• Docker: Add full_directory interpolation value for repository configuration. (#20530)

• stats: add output_file option to output the stats to a file (#20512)

• python: respect closed option when exporting dependency graph as JSON (#20523)

• Generate jvm_artifact targets from pom.xml (#20336)

• Add support for the pex --executable argument (#20497)

• python: improve error message when parsing Python interpreter constraints (#20297)

• upgrade known terraform versions (#20469)

• Allow using Ruff to format BUILD files (#20411)

• options: do not ignore .github directory with pants_ignore (#20471)

• Add (optional) support for podman. (#20470)

• python: add _typeshed module to the list of unowned dependencies (#20468)

• Terraform lockfiles (take 2) (#20303)

• Extend dependents goal with output format to support JSON (#20453)

• Extend dependencies goal with output format to support JSON (#20443)

User API Changes

• Update built-in ruff 0.1.6 -> 0.2.1 (#20499)

Plugin API Changes

• Allow AbstractLintRequest subclasses to disable lint rules (#20407)

Bug Fixes

• stop swallowing warnings from Pex by default (#20480)

• javascript: fix running scripts with yarn (#20543)

• docker: fix missing image_id when using containerd-snapshotter (#20533)

• Bump PyO3 to 0.20 to fix backtraces (#20517)

• Fix formatting of the "pex.platforms is deprecated" message (#20514)

• Handle unresolved ambiguous entrypoint dependency for PEX as unowned dependency (#20390)

• plumb through Pex's --check zipapp validation (#20481)

• Allow unmatching "changed" globs (#20505)

• add module mapping overrides for some django-* modules (#20504)

• upgrade Pex to 2.1.163 (#20502)

• upgrade Pex to 2.1.162 (#20496)

• Fix direct_url in python PEP660 editable wheels (#20486)

• python-infer: respect ignore pragma w/ strings (#20477)

• Introduce ruff-check and ruff-format tool ids (#20358)

• python-infer: Avoid false positive strings (#20472)

• Don't eagerly merge configs. (#20459)

• Docker: Update hadolint version to fix segmentation fault issue (#20456)

Documentation

• Fix reference to default_run_goal_use_sandbox option in docs. (#20546)

• fix pants.log location in docs (#20547)

• docs: mention .pants.bootstrap file (#20538)

• docs: provide example of having a --option in [cli.alias] (#20539)

• docs: provide options to refer to a group of targets (#20522)

• docs: mention that when piping xargs may end up invoking Pants goal more than once (#20521)

• docs: provide example how to convert target addresses to source files in rules API (#20524)

• Fix Pex references to point to new home. (#20519)

• Update broken link to default versioning scheme (#20482)

• docs: fix slack channel references (#20475)

• Add a test case for Pants integration testing docs (#20451)

• Add a test case for Pants unit testing docs (#20452)

• Rename tutorials section in the docs (#20449)

release_2.19.1rc0

Bug Fixes

• Fix formatting of the "pex.platforms is deprecated" message (Cherry-pick of #20514) (#20527)

• Bump PyO3 to 0.20 to fix backtraces (Cherry-pick of #20517) (#20532)

• Fix direct_url in python PEP660 editable wheels (Cherry-pick of #20486) (#20494)

• python-infer: respect ignore pragma w/ strings (Cherry-pick of #20477) (#20484)

• python-infer: Avoid false positive strings (Cherry-pick of #20472) (#20483)

• fix: in-repo plugin requirements.txt not loading (Cherry-pick of #20355) (#20381)

Documentation

• Fix capitalization of Terraform in left nav (#20490)

• Actually call bin_name in help string (Cherry-pick of #20434) (#20437)

release_2.19.0

2.19.x Release Series

Pants 2 is a fast, scalable, user-friendly build system for codebases of all sizes. It's currently focused on Python, Go, Java, Scala, Kotlin, Shell, and Docker, with support for other languages and frameworks coming soon.

Individuals and companies can now sponsor Pants financially.

Pants is an open-source project that is not owned or controlled by any one company or organization, and does incur some expenses. These expenses are managed by Pants Build, a non-profit that was established for this purpose. This non-profit's only source of revenue is sponsorship by individuals and companies that use Pants.

We offer formal sponsorship tiers for companies, as well as individual sponsorships via GitHub.

What's New

Highlights

• Test retries for flaky Python tests.
• Using the parametrize to set multiple fields at once when generating new groups of targets.
• buildx support in the Docker backend.
• New backends for running semgrep and openapi-format.

Keep reading to see the details and what's also included.

Overall

The .pids/ internal directory is now under .pants.d/ by default, rather that adjacent to it (this can be controlled via the [GLOBAL].pants_subprocessdir option).

The new [system-binaries] subsystem allows controlling where Pants searches for binaries like bash, zip, tar that is uses as part of normal operation. Set the system_binaries_paths as appropriate, potentially using the <PATH> special value. This can also be set on specific environments using the system_binaries_system_binary_paths field.

Additional features and fixes:

• New versions of the get-pants.sh script install to ~/.local/bin, not ~/bin.
• Environment variables that aren't valid UTF-8 no longer cause Pants to crash (note, PANTS_... environment variables that Pants needs to read but aren't UTF-8 will be ignored).
• Reference documentation for subsystem options includes how to set them in pants.toml, in addition to the environment variable and CLI arguments.
• Environments are now more resilient to being (partially) defined via macros, although this is not recommended.
• Docker environments can now pull public images without credentials, and can now handle docker images without Python.
• The fix goal now partitions files across multiple processes in a way that's more similar to the partitioning of the lint goal, which can work around some spurious linting failures.

BUILD files

The parametrize helper now supports parametrizing multiple fields together, in lock step. For instance:

# Creates two targets:
#
#    example:tests@parametrize=py2
#    example:tests@parametrize=py3

python_test(
    name="tests",
    source="tests.py",
    **parametrize("py2", interpreter_constraints=["==2.7.*"], resolve="lock-a"),
    **parametrize("py3", interpreter_constraints=[">=3.6,<3.7"], resolve="lock-b"),
)

In addition, parametrize can now be used with a single value (field=parametrize("for-consistency")), and now works on target generations in more cases (particularly those with "plugin" fields).

Performance

Some processes are run with more consistent "color" configuration which may result in more cache hits: Pytest, Helm unittest and shunit2 processes now always run with colored output enabled. If [GLOBAL].colors is not enabled, Pants removes the coloring before displaying.

Remote caching/execution

Large files that need to be stored to a remote cache are now streamed to it directly from disk, in a way that should reduce overhead and increase concurrency (#19711).

The deprecation has expired for the [GLOBAL].remote_store_chunk_upload_timeout_seconds and [GLOBAL].remote_cache_read_timeout_millis options. Use [GLOBAL].remote_store_rpc_timeout_millis and [GLOBAL].remote_cache_rpc_timeout_millis instead.

Backends

Docker

The docker backend now has explicit support for using buildx (and thus BuildKit), via the new use_buildx option. This also allows adding few additional fields for docker_image targets, for buildx-specific functionality:

• cache_from and cache_to for using external cache storage backends
• build_platform for configuring multi-platform builds
• output to control how/where buildx outputs an image when running pants package ::

JVM

JVM lock-files are more reliable, handling some cases of Coursier not including dependencies when expected.

Using shading_rules is now supported on deploy_jar targets defined in directories of the build root.

Kotlin

The Kotlin analyzer now runs using the Zulu JDK, to be able to run natively on Apple Silicon (arm64 macOS).

Java

Dependency inference for Java now supports code using recently added syntax like sealed class & permits, switch expressions and yield.

Scala

Scala 3 is now supported when using the BSP in an IDE.

OpenAPI

The new pants.backend.experimental.openapi.lint.openapi_format backend supports running openapi-format to format openapi_... targets.

Protobuf

The pants.backend.codegen.protobuf.python backend now supports:

• protobuf_sources targets that are part of multiple Python resolves, like python_resolve=parametrize("first-resolve", "second-resolve").
• [python-infer].ambiguity_resolution = "by_source_root" to infer dependencies without warnings or errors in more cases.

Version 24.4 of the protoc compiler is included as a known version (but is not the default). It can be selected by setting [protoc].version = "24.4". This version has native support for Apple Silicon (arm64 macOS).

Buf (pants.backend.codegen.protobuf.lint.buf) now correctly ignores buf.work.yaml, buf.lock, buf.gen.lock when running as a linter.

Python

Deprecation: Using the platforms field on pex_binary is now deprecated. It is not a recommended way to build for platforms other than the local one, as it is a lossy abbreviation of the target platform and often leads to binaries that don't work. To replace this, either use complete_platforms to continue cross-compiling or use an environment matching the target platform.

Python tests can now be attempted multiple times, if they fail. This can help with limiting the impact of flaky tests. For instance, to require tests to fail three times before the overall pants test goal fails, set [test].attempts_default = 3. In this case, a very flaky test might fail on the first and second runs but finally pass on the third run, and thus the overall Pants invocation would pass.

Pants now uses version 2.1.148 of the PEX CLI by default, which includes support for Python 3.12. To use Python 3.12, you will need to set [python].pip_version to 23.2 or newer (or latest), and ensure all of your tools use resolves generated with this new version (Pants' built-in ones are not, yet).

Python interpreters provided by ASDF are now searched by default, in addition those provided by pyenv and on the system PATH, by being included in the [python-bootstrap].search_path option.

Many more "groups" of PyPI dependencies have default module mappings that guide how an import statement in code matches particular python_requirements. All packages from the azure-..., django-..., google-cloud-......

release_2.20.0.dev7

New Features

• Add version/local scheme fields to vcs_version (#20446)

• upgrade PEX to 2.1.159 (#20416)

• Add jvm_index Option to Coursier Subsystem (#20271)

• python-protobuf backend now support multiple protoc plugins. (#20387)

User API Changes

• Add remote_provider = "..." option, replacing scheme-look-ups (#20240)

Bug Fixes

• Fix issue with grouping parametrizations on target generators. (#20429)

Documentation

• Mention the Scala repl and its current gotchas in the docs (#20445)

• Fix missing registration for environments rules in pants.core. (#20444)

• Actually call bin_name in help string (#20434)

• Apply latest hand-edits to pantsbuild/pantsbuild.org repo (#20419)

• Fix the weird headings of initial-configuration.mdx (#20422)

• Update docker docs for buildx (#20413)

• What's new in 2.19 (#20310)

• Rename 'environments' docs file/URL to be simpler (#20404)

release_2.19.0rc5

Bug Fixes

• Fix issue with grouping parametrizations on target generators. (Cherry-pick of #20429) (#20433)

Documentation

• Apply latest hand-edits to pantsbuild/pantsbuild.org repo (#20420)

• Fix the weird headings of initial-configuration.mdx (Cherry-pick of #20422) (#20426)

• Update docker docs for buildx (Cherry-pick of #20413) (#20424)

• What's new in 2.19 (Cherry-pick of #20310) (#20409)

• Rename 'environments' docs file/URL to be simpler (cherry-pick of #20404) (#20406)

• Replace this repo's docs content with the update docusaurus content (#20400)