Flaky test: request form: unchecking an item when there is another valid item leaves the submit button enabled

[maxkadel]

Expected behavior

Tests consistently pass in CI, unless a branch changes something that relates to the test.

Actual behavior

spec/features/requests/form_spec.rb:865 is flaky - passing and not passing inconsistently, regardless of changes made on a branch.

See, e.g. CircleCI run

"request form a Princeton CAS user When visiting an Alma ID as a CAS User Request button disables and enables unchecking an item when there is another valid item leaves the submit button enabled"

Failure/Error: expect(page).to have_button('Request Selected Items', disabled: false)
  expected to find button "Request Selected Items" that is not disabled but there were no matches
./spec/features/requests/form_spec.rb:880:in `block (5 levels) in <top (required)>'

Impact of this bug

Slows down velocity, causes extra CI runs, which costs time and money.

Implementation notes, if any

It's possible the sheer number of steps in this test is causing issues with the javascript testing library. It's possible adding waits would make the test behave more reliably, but that's also not a great solution.

[maxkadel]

Tried running git bisect on the file; however, we got 8 failing tests but none of them were the one we've seen flaking in CI. Here's what bisect gave as the failing reproduction command

be rspec './spec/features/requests/form_spec.rb[1:2:2:6,1:2:2:33:6,1:3:1:1,1:4:1:3,1:4:1:6,1:5:1,1:5:5,1:5:9]' --seed 52841

[maxkadel]

I was not able to re-create the failure locally, even with the exact order that failed in CI.

[christinach]

we haven't seen this failing test again

[maxkadel]

Still flaky / flaky again - see recent CI run

[rladdusaw]

Possible solution to this issue is to convert the test to a JS test instead of a feature test.