Add GH workflow to run critical autofill tests each night #4186
Conversation
|
This stack of pull requests is managed by Graphite. Learn more about stacking. Join @CDRussell and the rest of your teammates on |
CDRussell
force-pushed
the
feature/craig/automated_autofill_tests
branch
3 times, most recently
from
ef101c1 to
7d96ef6
Compare
There was a problem hiding this comment.
LGTM
Want to throw a proposal we did for Sync. Inside autofill create a folder steps, and move the maestro files that are expected to run as inner flows. Then under autofill just leave the actual tests. (I think you already differentiate them by using a prefix, but that was not clear to me at first look).
Really nice addition to add Pre-requisite. We did the same on sync.
CDRussell
force-pushed
the
feature/craig/automated_autofill_tests
branch
from
7d96ef6 to
92b00db
Compare
CDRussell
force-pushed
the
feature/craig/automated_autofill_tests
branch
from
92b00db to
e3dde7c
Compare
Good idea on |
<!-- Note: This checklist is a reminder of our shared engineering expectations. The items in Bold are required If your PR involves UI changes: 1. Upload screenshots or screencasts that illustrate the changes before / after 2. Add them under the UI changes section (feel free to add more columns if needed) If your PR does not involve UI changes, you can remove the **UI changes** section At a minimum, make sure your changes are tested in API 23 and one of the more recent API levels available. --> Task/Issue URL: https://app.asana.com/0/0/1206622911703638/f ### Description Adds a nightly workflow to execute the autofill e2e maestro tests. Also refactors the autofill test suite into smaller flows as trying to run one large flow exceeds the Maestro timeout threshold (7 mins). ### Steps to test this PR - [ ] Check the workflow file, and make sure that looks ok to run as a nightly job and alert on failure - [ ] They have been running well in my testing (e.g., [test run](https://console.mobile.dev/uploads/4946a190-1df4-4bb2-bfeb-2d636cd88311?teamId=eba4e11d-655d-4e7f-a936-5b56de0edebb&appId=f2da50d2-9691-44f8-9985-74c4555ff437&analysisId=e2e192bc-d29c-43cc-ba02-fe17045fdf40)), but you can optionally also trigger the Maestro tests manually using the following commands: - `./gradlew -Pautofill-disable-auth-requirement assemblePlayRelease` - `version=$(gr -q getVersionName)` - `maestro cloud --include-tags autofillNoAuthTests "app/build/outputs/apk/play/release/duckduckgo-$version-play-release.apk" .maestro`
Task/Issue URL: https://app.asana.com/0/0/1206622911703638/f
Description
Adds a nightly workflow to execute the autofill e2e maestro tests. Also refactors the autofill test suite into smaller flows as trying to run one large flow exceeds the Maestro timeout threshold (7 mins).
Steps to test this PR
./gradlew -Pautofill-disable-auth-requirement assemblePlayReleaseversion=$(gr -q getVersionName)maestro cloud --include-tags autofillNoAuthTests "app/build/outputs/apk/play/release/duckduckgo-$version-play-release.apk" .maestro