v3.50.1

v3.50.1 (2023-07-20)

Full Changelog

This release contains multiple issues:

Severity	Description	Fixed in
⚠️ Very High	Jobs running on this version of the agent are not cancellable from the UI/API	✅ Fixed in v3.50.2
Medium	When uploading pipelines, if any object in the pipeline YAML contained multiple merge keys, the pipeline would fail to parse. See below for a workaround	✅ Fixed in v3.50.3

Workaround for yaml merge key issue

For example, this pipeline would fail to parse:

default_plugins: &default_plugins
  plugins:
    - docker#4.0.0:
        image: alpine:3.14

default_retry: &default_retry
  retry:
    automatic:
      - exit_status: 42

steps:
  - <<: *default_plugins
    <<: *default_retry
    command: "echo 'hello, world!'"

As a workaround for this, you can use yaml array merge syntax instead:

default_plugins: &default_plugins
  plugins:
    - docker#4.0.0:
        image: alpine:3.14

default_retry: &default_retry
  retry:
    automatic:
      - exit_status: 42

steps:
  - <<: [*default_plugins, *default_retry]
    command: "echo 'hello, world!'"

Fixed

• Empty or zero-length steps is no longer a parser error, and is normalised to [] instead #2225, #2229 (@DrJosh9000)
• Group steps now correctly include the group key #2226 (@DrJosh9000)
• Increases to test coverage for the new parser #2227 (@DrJosh9000)

v3.50.0

v3.50.0 (2023-07-18)

Full Changelog

This release contains multiple issues:

Severity	Description	Fixed in
Medium	When uploading pipelines, some group steps are not correctly parsed, and were ignored.	✅ Fixed in v3.50.1
Low	Uploading pipelines with empty or zero-length steps failed, where they should've been a no-op.	✅ Fixed in v3.50.1
⚠️ Very High	Jobs running on this version of the agent are not cancellable from the UI/API	✅ Fixed in v3.50.2
Medium	When uploading pipelines, if any object in the pipeline YAML contained multiple merge keys, the pipeline would fail to parse. See below for a workaround	✅ Fixed in v3.50.3

Workaround for yaml merge key issue

For example, this pipeline would fail to parse:

default_plugins: &default_plugins
  plugins:
    - docker#4.0.0:
        image: alpine:3.14

default_retry: &default_retry
  retry:
    automatic:
      - exit_status: 42

steps:
  - <<: *default_plugins
    <<: *default_retry
    command: "echo 'hello, world!'"

As a workaround for this, you can use yaml array merge syntax instead:

default_plugins: &default_plugins
  plugins:
    - docker#4.0.0:
        image: alpine:3.14

default_retry: &default_retry
  retry:
    automatic:
      - exit_status: 42

steps:
  - <<: [*default_plugins, *default_retry]
    command: "echo 'hello, world!'"

Added

• We're working on making pipeline signing a feature of the agent! But it's definitely not ready for primetime yet... #2216, #2200, #2191, #2186, #2190, #2181, #2184, #2173, #2180 (@moskyb, @DrJosh9000)
• Add option to configure location of Job Log tmp file #2174 (@yhartanto)
• Add avoid-recursive-trap experiment to avoid a recursive trap #2209 (@triarius)
• Load the AWS Shared Credentials for s3 operations #1730 (@lox)

Fixed

• Add workaround for fatal: bad object errors when fetching from a git mirror #2218 (@DrJosh9000)
• Fix missing fetch when updating git mirrors of submodules (#2203) (@DrJosh9000)
• Use a unique name for each agent started using the systemd template unit file #2205 (@DavidGregory084)
• Polyglot hooks wasn't documented in EXPERIMENTS.md, so we fixed that #2169 (@moskyb)
• De-experimentify wording on the status page #2172 (@DrJosh9000)
• The secrets redactor now properly redacts multi-line secrets and overlapping secrets #2154 (@DrJosh9000)

Changed

• Print agent version and build in debug logs #2211 (@triarius)
• Include the version each experiment was promoted #2199 (@DrJosh9000)

Various code cleanups and meta-fixes

• Fix docker builds for Ubuntu 22.04 #2217 (@moskyb)
• JobRunner cleanup #2207 (@moskyb)
• Simplify command phase #2206 (@triarius)
• Rename Bootstrap struct (and friends) to Executor #2188 (@moskyb)
• Upgrade docker compose plugin to v4.14, use docker compose v2 #2189 (@moskyb)
• Rename package bootstrap -> job #2187 (@moskyb)
• Clarify code around creating a process group #2185 (@triarius)
• Fix docker builds for Ubuntu 22.04 #2217 (@moskyb)

And the usual amount of @dependabot[bot] updates!

v3.49.0

Fixed

• CreateArtifacts & UpdateArtifacts: remove sometimes-too-short timeout after 4 attempts #2159 (@pda)
• Fix submodule mirror repository remote using main repo URL #1998 (@francoiscampbell)
• Update job log file to include line transforms #2157 (@chasestarr)
• Clearer HTTP error logging from API client #2156 (@moskyb)

Changed

• Buildkite-Timeout-Milliseconds API request header #2160 (@pda)
• Extract pipeline parser to package internal/pipeline #2158 (@DrJosh9000)
• Minor dependency updates #2165 #2164 #2162 #2161 #2153 #2152 #2151 (@dependabot[bot])
• Lock library #2145 (@DrJosh9000)

v3.48.0

v3.48.0 (2023-06-06)

Full Changelog

The de-experimentification release!

• The ansi-timestamps experiment is now enabled by default. To remove the
  timestamps from your logs, you can pass the --no-ansi-timestamps flag.
• The flock-file-locks experiment is now enabled by default. Because the old
  and new file lock systems don't interact, we strongly recommend not running
  multiple agents of different versions on the same host.
• The inbuilt-status-page experiment is now enabled by default. For those
  running the agent with --health-check-addr, go to /status to see a
  human-friendly status page.

And whatever happened to git-mirrors? It graduated from experiment-hood in
v3.47.0!

Changed

• De-experimentify ansi-timestamps #2133 (@DrJosh9000)
• Preserve plugin config env var names with consecutive underscores #2116 (@triarius)
• De-experimentify flock-file-locks #2131 (@DrJosh9000)
• Report more AWS metadata #2118 (@david-poirier)
• De-experimentify inbuilt-status-page #2126 (@DrJosh9000)

Fixed

• Fix origin for mirrored submodules #2144 (@DrJosh9000)
• Wipe checkout directory on git checkout and git fetch failure and retry #2137 (@triarius)

v3.47.0

v3.47.0 (2023-05-25)

Full Changelog

Two new and very noteworthy experiments!

1. Have you ever wanted to write hooks in a compiled language? Or in Python or Ruby? Well now you can! With --experiment=polyglot-hooks the agent can run all sorts of hooks and plugins directly. Combined with --experiment=job-api, your hooks-of-a-different-language can alter environment variables through the local Job API!
2. Concurrency groups are great, but have you ever wanted to manage multiple agents running on the same host concurrently accessing a shared resource? Well now you can! With --experiment=agent-api, the agent now has an inbuilt locking service, accessible through new lock subcommands and also via a Unix socket (like the job-api).

Added

• Experiment: Polyglot hooks #2040 (@moskyb)
• Experiment: Local Agent API, with locking service #2042 (@DrJosh9000)
• New flag --upload-skip-symlinks (on artifact upload) allows skipping symlinks when uploading files. --follow-symlinks has been deprecated and renamed to --glob-resolve-follow-symlinks #2072 (@triarius)

Fixed

• The normalised-upload-paths experiment was unintentionally left out of the available experiments list #2076 (@MatthewDolan)

Changed

• The git-mirrors experiment is promoted to full functionality #2032 (@moskyb)
• Errors in the git checkout process are now easier to diagnose #2074 (@moskyb)
• Alpine images updated to Alpine 3.18 #2098 (@moskyb)

v3.46.1

3.46.1 (2023-05-08)

Full Changelog

Fixed

• Avoid long --no-patch arg added to git show in v1.8.4, to e.g. support CentOS 7 #2075 (@pda)

v3.46.0

3.46.0 (2023-05-04)

Full Changelog

Added

• Add *_PRIVATE_KEY to auto-redacted vars #2043 (@moskyb)
• Warn on unknown experiments #2030 (@moskyb)
• More aws tags #1994 (@sj26)
• Add option for outputting structured logs for collection and searching #2009 (@goodspark)
• Include abbrev-commit in buildkite:git:commit meta-data #2054 (@pda)
• Add agent support for getting meta-data by build #2025 (@123sarahj123)

Fixed

• Prevent job cancellation during checkout from retrying #2047 #2068 (@matthewborden + @triarius + @moskyb)
• ArtifactUploader API calls: faster timeout & retry #2028 #2069 (@pda)
• Give a nicer error when empty strings are used as metadata values #2067 (@moskyb)
• Fix BUILDKITE_GIT_CLONE_MIRROR_FLAGS environment variable not working correctly #2056 (@ppatwf)

As always, @dependabot and friends have been deep in the update mines ensuring that all of our dependencies are up to date. Thanks, dependabot!

v3.45.0

3.45.0 (2023-03-16)

Full Changelog

It's a busy one! The major new feature in this release is the job-api experiment, which enables an HTTP API within the agent that allows jobs to inspect and mutate their environment, without using the normal bash-isms that we normally require. This is a big step towards supporting hooks and plugins in other languages, and we're really excited to see what you all do with it!

When this experiment is enabled, the agent will start an HTTP server on a unix domain socket, the address of which will be made available through the BUILDKITE_AGENT_JOB_API_SOCKET environment variable, with a token available through the BUILDKITE_AGENT_JOB_API_TOKEN environment variable. This socket can be used with the buildkite-agent env {get,set,unset} commands on the commandline, or directly through cURL or other HTTP client. Included in this release of the agent is a golang client, which can be imported directly into your Go projects.

Also included is another experimental feature, descending-spawn-priority, which makes agents using the --spawn-with-priority flag spawn agents with a descending priority, rather than the default ascending priority. This is useful when running agents on heterogeneous hardware (ie, having two agents on one machine and four on another), as it means that jobs will be spread more evenly across the agents. For more information, see the original issue, and @DrJosh9000's PR. Huge thanks to @nick-f for bringing this to our attention!

Full changelog follows:

Added

• Add current-job api #1943 #1944 #2013 #2017 (@moskyb + @DrJosh9000)
• Agent docker images now include buildx #2005 (@triarius)
• Add descending-spawn-priority experiment. #2004 (@DrJosh9000)
• We now publish OSS acknowledgements with the agent. You can read them at ACKNOWLEDGEMENTS.md, or by running buildkite-agent acknowledgements #1945 #2000 (@DrJosh9000)
• BUILDKITE_S3_ENDPOINT env var, allowing jobs to upload artifacts to non-S3 endpoints eg minio #1965 (@pda, @nightah)

Fixed

• Avoid holding full job logs, reducing agent memory consumption #2014 (@DrJosh9000)
• ansi-timestamps: Compute prefixes at start of line #2016 (@DrJosh9000)
• Fix DD trace setup warning #2007 (@goodspark)

Changed

• Kubernetes improvements:
  • Set a non-zero exit status when a job is cancelled in Kubernetes #2010 (@triarius)
  • Add tags from env variables provided by the controller in agent-stack-k8s if kuberenetes-exec experiment is enabled #2003 (@triarius)
• Globs parsed by the agent now support negation and bracketing #2001 (@moskyb)
• Allow the use of non-bash shells to execute agent hooks #1995 (@DrJosh9000)
• Don't add custom remotes for submodules when using git-mirrors #1991 (@jonahbull)
• Improve systemd behaviour when updating the agent #1993 (@triarius)
• ... And as always, the usual crop of small fixes, dependency updates, and cleanups (@moskyb, @dependabot, @DrJosh9000, @triarius)

v3.44.0

v3.44.0 (2023-02-27)

Full Changelog

Fixed

• tini is once again available at the old path (/usr/sbin/tini) in the Ubuntu 20.04 image #1934 (@triarius)
• With ansi-timestamps experiment enabled, each line's timestamp is now computed at the end of the line #1940 (@DrJosh9000)
• A panic when the AWS region for an S3 bucket is undiscoverable #1964 (@DrJosh9000)

Added

• An experiment for running jobs under Kubernetes #1884 (@benmoss), #1968 (@triarius)
• Ubuntu 22.04 Docker Image #1966 (@triarius)
• Claims can now be added to OIDC token requests #1951 (@triarius)
• A new flag / environment variable (--git-checkout-flags / BUILDKITE_GIT_CHECKOUT_FLAGS) for passing extra flags to git checkout #1891 (@jmelahman)
• Reference clones can be used for submodules #1959 (@jonahbull)

Changed

• Upstart is no longer supported #1946 (@sj26)
• pipeline upload internally uses a new asynchronous upload flow, reducing the number of connections held open #1927 (@triarius)
• Faster failure when trying to pipeline upload a malformed pipeline #1963 (@triarius)
• Better errors when config loading fails #1937 (@moskyb)
• Pipelines are now parsed with gopkg.in/yaml.v3. This change should be invisible, but involved a non-trivial amount of new code. #1930 (@DrJosh9000)
• Many dependency updates, notably Go v1.20.1 #1955.
• Several minor fixes, improvements and clean-ups (@sj26, @triarius, @jonahbull, @DrJosh9000, @tcptps, @dependabot[bot])

v3.43.1

v3.43.1 (2023-01-20)

Full Changelog

Fixed

• An issue introduced in v3.43.0 where agents running in acquire mode would exit after ~4.5 minutes, failing the job they were running #1923 (@leathekd)