Skip to content
This repository was archived by the owner on Feb 24, 2025. It is now read-only.

Improve sync e2e reliability #3830

Merged
merged 4 commits into from
Jan 20, 2025
Merged

Conversation

loremattei
Copy link
Contributor

Task/Issue URL: https://app.asana.com/0/1204165176092271/1209158713404042/f
Tech Design URL:
CC:

Description:
Sync e2e tests were failing randomly, especially during nightly runs, with an error that the server was not reachable.
After some investigation, I suspect the error is misleading and the actual issue is that the recovery code was invalid when used because the tests got queued or run slower than usual and the GHA timed out, causing the clean up job to run and invalidate the recovery code.

This PR:

  • Changes the schedule of the test, to avoid it overlaps with the other e2e tests and gets queued.
  • Increases the timeout (needed anyway by the switch to Robin which is much slower than Maestro Cloud)
  • Switches to Maestro Action for better reliability of the connection between GHA and Robin (see Switch to using maestro action #3783 for more context about this)

Steps to test this PR:

  1. Run the tests from this branch and validate it's green (you can also check a recent successful run here: https://github.com/duckduckgo/iOS/actions/runs/12828988947 )

Definition of Done (Internal Only):

Copy Testing:

  • Use of correct apostrophes in new copy, ie rather than '

Orientation Testing:

  • Portrait
  • Landscape

Device Testing:

  • iPhone SE (1st Gen)
  • iPhone 8
  • iPhone X
  • iPhone 14 Pro
  • iPad

OS Testing:

  • iOS 15
  • iOS 16
  • iOS 17

Theme Testing:

  • Light theme
  • Dark theme

Internal references:

Software Engineering Expectations
Technical Design Template

@loremattei loremattei marked this pull request as ready for review January 19, 2025 20:57
@loremattei loremattei requested a review from samsymons January 19, 2025 20:58
Copy link
Contributor

@samsymons samsymons left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, the rationale behind the changes makes sense, nice find!

@loremattei loremattei merged commit 3fdb09e into main Jan 20, 2025
19 checks passed
@loremattei loremattei deleted the loremattei/improve-sync-e2e-reliability branch January 20, 2025 08:59
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants