v1.3.0

A feature release on top of 1.2.0, backwards compatible.

Major new features

• New subcommand just serve to start a target-level caching service,
  as described in the corresponding design document.
• just-mr is able to back up and retrieve distribution files
  from a remote execution endpoint. This simplifies usage in an
  environment with restricted internet access.
• just execute now supports blob splitting as new RPC call.
  just install uses this call to reduce traffic if the remote-execution
  endpoint supports blob splitting and the --remember option is given.
  In this way, traffic from the remote-execution endpoint can be reduced
  when subsequently installing artifacts with only small local
  differences.

Other changes

• New script just-deduplicate-repos to avoid blow up of the
  repos.json in the case of chained imports with common dependencies.
• New subcommand add-to-cas to add files and directories to the local
  CAS and optionally also copy them to the remote-execution endpoint.
• The built-in "generic" rule now supports an argument "sh -c",
  allowing to specify the invocation of the shell (defaulting to
  ["sh", "-c"]).
• just describe also shows the values of the implicit dependencies.
• just-mr supports a new form of root, called "foreign file".
• When just-mr executes the action to generate the desired tree of a
  "git tree" repository, it can be specified that certain variables
  of the environment can be inherited.
• The just-mr rc file now supports a field "rc files" to include
  other rc files given by location objects; in particular, it is
  possible to include rc files committed to the workspace.
• Support for fetching archives from FTP and TFTP was added to just-mr
  if it was built with bundled curl. For package builds, libcurl has
  enabled whatever the distro considers suitable.
• The gc subcommand supports an option --no-rotate to carry
  out only local clean up. Part of that local clean up, that is
  also done as part of a full gc, is splitting large files. Note
  that stable versions before 1.3.0 cannot use those split files.
  Hence a downgrade after a gc with 1.3.0 (or higher) requires
  cleaning of cache and CAS.
• The expression language has been extended and, in particular,
  allows indexed access to an array (basically using it as a tuple)
  and a generic form of assertion (to report user errors).
• The analyse subcommand supports a new flag --dump-result to dump
  the analysis result to a file or stdout (if - is given).

Fixes

• The cache key used for an export target is now based on the
  export target itself rather than that of the exported target. The
  latter could lead to spurious cache hits, but only in the case
  where the exported target was an explicit file reference, and a
  regular target with the same name existed as well. Where the new
  cache keys would overlap with the old ones, they would refer to
  the same configured targets. However, we used the fact that we
  changed the target cache key to also clean up the serialization
  format to only contain the JSON object describing repository,
  target, and effective configuration, instead of a singleton list
  containing this object. Therefore, old and new cache keys do not
  overlap at all. In particular, no special care has to be taken
  on upgrading or downgrading. However, old target-level cache
  entries will not be used leading potentially to rebuilding of
  some targets.
• Garbage collection now honors the dependencies of target-level
  caches entries on one another. When upgrading in place, this only
  applies for target-level cache entries written initially after
  the upgrade.
• The taintedness of "configure" targets is now propagated
  correctly in analysis.
• It is no longer incorrectly assumed that every git URL not
  starting with ssh://, http://, nor https:// is a file on the
  local disk. Now, only URLs starting with /, ./, or file://
  are considered file URLs. File URLs, as well as URLs starting
  with git://, http://, or https://, are handled by just-mr
  using libgit2; for every other URL, just-mr shells out to
  git for fetching and the URL is passed to git unchanged.
• Improved portability and update of the bundled dependencies.
• Various minor improvements and typo fixes in the documentation.
• Fixed a race condition in the task queue that could cause (with
  probability roughly 1e-5) a premature termination of the queue
  resulting in spurious analysis failures without explanation (despite
  "failed to analyse target").
• Fixed a race condition in an internal cache of just execute
  used for keeping track of running operations.
• The built-in rule "install" now properly enforces that the
  resulting stage is well-formed, i.e., without tree conflicts.
• Local execution and just execute now correctly create empty
  directories if they are part of the action's input.
• Fixed overwrite of existing symlinks in the output directory
  when using subcommands install and install-cas.
• The format for target-cache shards was changed to a canonical form.
  The new and old formats do not overlap, therefore the correctness
  of the builds is not affected. In particular, no special care has
  to be taken on upgrading or downgrading. However, some target-level
  cache entries will not be used leading potentially to rebuilding of
  some targets.
• The expression "disjoint_map_union" did not verify disjointness
  in all cases; this is fixed now.
• The command line option "--remote-execution-property" can be
  repeated multiple times to list all the properties, but only the
  last one was retained. This is fixed now.