Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: flaky spec #1763

Merged
merged 1 commit into from
Jan 9, 2024
Merged

fix: flaky spec #1763

merged 1 commit into from
Jan 9, 2024

Conversation

roschaefer
Copy link
Collaborator

Motivation

Also this took me way too long to fix. #1729 suffers under some unrelated failing spec. On my machine, even when I git switch main on the 07694a6 I will see sometimes a failing spec, sometimes it passes.

Here is an example:


  1) Configuring Onboarding Channels allows activating and deactivating onboarding channels
     Failure/Error: expect(page).to have_current_path(settings_path(as: admin))
       expected "/settings" to equal "/settings?as=177"

     [Screenshot Image]: tmp/screenshots/failures_r_spec_example_groups_configuring_onboarding_channels_allows_activating_and_deactivating_onboarding_channels_617.png

     # ./spec/system/settings/onboarding_channels_spec.rb:26:in `block (2 levels) in <main>'
     # ./spec/support/better_rails_system_tests.rb:26:in `block (2 levels) in <main>'
     # -e:1:in `<main>'

I don't think it is necessary to check the query params here, maybe it was even an oversight. What I still don't understand: What is some_route_path(as: some_user) even doing? And why did it work in the past?

How to test

  1. git switch main
  2. bin/rspec spec/system/settings/onboarding_channels_spec.rb multiple (!) times
  3. Sometims green, sometimes red

This should not happen on this branch anymore.

@roschaefer roschaefer requested a review from mattwr18 December 13, 2023 16:46
Copy link
Contributor

@mattwr18 mattwr18 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have any problem with this change as long as it works. I have not seen that this has been flaky, it has actually never failed for me either locally or in ci. I believe at some point I was getting failing tests for not having the search params, so I added it to get the test to pass.

The as: user comes from our gem user management gem clearance. See Fast Feature Specs.

There are a few other places in the specs where I have made similar assertions, were none of them flaky? I am wondering if the test in question was actually flaky for the reason you think it was.

If it is flaky for the reason you suggest, I would expect the other specs to be flaky as well. Maybe this PR should update them as well?

@roschaefer roschaefer force-pushed the fix-flaky-onboarding-channels-spec branch from 04ebe77 to 63269d4 Compare January 9, 2024 09:37
 Motivation
 ----------
 Also this took me way too long to fix. #1729 suffers under some unrelated failing spec. On my machine, even when I `git switch main` on the 07694a6 I will see sometimes a failing spec, sometimes it passes.

Here is an example:
```

  1) Configuring Onboarding Channels allows activating and deactivating onboarding channels
     Failure/Error: expect(page).to have_current_path(settings_path(as: admin))
       expected "/settings" to equal "/settings?as=177"

     [Screenshot Image]: tmp/screenshots/failures_r_spec_example_groups_configuring_onboarding_channels_allows_activating_and_deactivating_onboarding_channels_617.png

     # ./spec/system/settings/onboarding_channels_spec.rb:26:in `block (2 levels) in <main>'
     # ./spec/support/better_rails_system_tests.rb:26:in `block (2 levels) in <main>'
     # -e:1:in `<main>'
```

I don't think it is necessary to check the query params here, maybe it was even an oversight. What I still don't understand: What is `some_route_path(as: some_user)` even doing? And why did it work in the past?

 How to test
 -----------
 1. `git switch main`
 2. `bin/rspec spec/system/settings/onboarding_channels_spec.rb` multiple (!) times
 3. Sometims green, sometimes red

 This should not happen on this branch anymore.
@roschaefer roschaefer force-pushed the fix-flaky-onboarding-channels-spec branch from 63269d4 to 9b4d412 Compare January 9, 2024 09:53
@mattwr18 mattwr18 merged commit e74ceb0 into main Jan 9, 2024
1 check passed
@mattwr18 mattwr18 deleted the fix-flaky-onboarding-channels-spec branch January 9, 2024 11:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants