Merge pull request #376 from Shopify/grodowski/flaky-test-fix-round-2 #416

Job	Run time
build-debs (ubuntu-latest)	24s
build-debs (ubuntu-latest)	32s
gh-285 (5.7)	43s
gh-285 (5.7)	42s
go-test (5.7)	2m 12s
go-test (5.7)	2m 15s
build-debs (shopify-ubuntu-arm64)	49s
build-debs (shopify-ubuntu-arm64)	51s
ruby-test (5.7)	2m 39s
ruby-test (5.7)	3m 5s
gh-285 (8.0)	31s
gh-285 (8.0)	36s
go-test (8.0)	2m 2s
go-test (8.0)	2m 8s
ruby-test (8.0)	15m 13s
ruby-test (8.0)	4m 5s
	38m 47s

Provide feedback