Skip to content

Commit

Permalink
Makes bundle cache key creation more repeatable
Browse files Browse the repository at this point in the history
In CircleCI, different parallel runs could order the files found for the
bundle cache key differently, resulting in cache misses, even though the
files found and the MD5s were the same. Now, the checksums are sorted.

The file that becomes part of the cache key is also added to the run's
artifacts, so future debugging won't have to do so much guesswork.
  • Loading branch information
rotated8 committed Feb 1, 2021
1 parent a4821de commit 57cfb05
Showing 1 changed file with 23 additions and 4 deletions.
27 changes: 23 additions & 4 deletions src/commands/bundle.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,31 @@ parameters:
type: string
default: '1'
steps:
# Generate md5s for any Gemfile*, and *.gemspec files to generate a unique
# cache key representing the contents of all files.
- run:
name: Generate a cache key for the bundle
command: |
echo $(find . -type f \( -name "Gemfile*" -o -name "*.gemspec" \) -exec md5sum {} \;) >> "BUNDLE_CACHE_KEY"
# Let's break this command down step by step.
# `command : >` - YAML magic to concatenate the lines, stripping indentation equal to the next line.
# `find .` - To start, we're going to find things decending from the currect directory.
# `-path "*vendor/bundle" -prune -o` - Prevent find from searching the vendor/bundle directory.
# This helps with idempotentcy, as gems will be installed into vendor/bundle, and they have Gemfiles.
# `-path "*.git" -prune -o` - Prevent find from searching the .git directory.
# `-type f \( -name "Gemfile*" -o -name "*.gemspec" \)` - Match any file beginning with "Gemfile" or
# ending in ".gemspec". No, the '.' is not expanded like with regex.
# `-exec md5sum {} \+` - Execute md5sum on the files found. BUT! The '+' means call md5sum only once,
# with all the filenames one after another.
# `| sort -o "BUNDLE_CACHE_KEY"` send the output of the find command into sort, and write the output to a
# file named 'BUNDLE_CACHE_KEY'. The files will be sorted by their md5 rather than their name.
command: >
find .
-path "*vendor/bundle" -prune -o
-path "*.git" -prune -o
-type f \( -name "Gemfile*" -o -name "*.gemspec" \)
-exec md5sum {} \+
| sort -o "BUNDLE_CACHE_KEY"
- store_artifacts:
path: "BUNDLE_CACHE_KEY"
destination: "BUNDLE_CACHE_KEY"

- restore_cache:
name: Restore bundle from cache
Expand Down

0 comments on commit 57cfb05

Please sign in to comment.