-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Test failures on Mac in CI #6508
Comments
Logs of the last successful runs of Firefox on Mac and Edge on Mac:
|
Neither Edge nor Firefox appear in /Applications in CI now. 😭
|
This could be explained by this GitHub blog post:
It is now Q2 FY24, so we may have just been auto-upgraded from macOS 12 to macOS 14 on M1, and they may not pre-install the same browsers on M1 images. |
Indeed, this table now shows:
So if we want x86-64, we need to use For Shaka, we will just have to explicitly install browsers now. |
Installing Firefox & Edge fixes Firefox and shows Edge now failing the same as Chrome, with a series of timeouts. |
Installing Edge & Firefox explicitly solves the failure to start tests. Switching to macOS 13 (x64) seems to resolve the timeouts on Chrome and Edge. It's not a great permanent solution, but I think it's best for now until we understand these timeouts better. |
- Install Edge & Firefox explicitly - Switch to macos-13 for now to resolve Edge & Chrome timeouts Issue shaka-project#6508
- Install Edge & Firefox explicitly - Switch to macos-13 for now to resolve Edge & Chrome timeouts Issue #6508
- Install Edge & Firefox explicitly - Switch to macos-13 for now to resolve Edge & Chrome timeouts Issue #6508
- Install Edge & Firefox explicitly - Switch to macos-13 for now to resolve Edge & Chrome timeouts Issue #6508
- Install Edge & Firefox explicitly - Switch to macos-13 for now to resolve Edge & Chrome timeouts Issue #6508
- Install Edge & Firefox explicitly - Switch to macos-13 for now to resolve Edge & Chrome timeouts Issue #6508
- Install Edge & Firefox explicitly - Switch to macos-13 for now to resolve Edge & Chrome timeouts Issue #6508
Using arm (macos-latest) wherever possible reduces build times by 3 minutes. We have a limited number of macos runners, so if we save time even on Firefox alone (all we can do pending #6508), we can speed up the queue of pending jobs across PRs.
Using arm (macos-latest) wherever possible reduces build times by 3 minutes. We have a limited number of macos runners, so if we save time even on Firefox alone (all we can do pending #6508), we can speed up the queue of pending jobs across PRs.
Using arm (macos-latest) wherever possible reduces build times by 3 minutes. We have a limited number of macos runners, so if we save time even on Firefox alone (all we can do pending #6508), we can speed up the queue of pending jobs across PRs.
Tests are failing or failing to run on Mac, but only in CI. Everything works in the lab.
The last build-and-test job to succeed was this one for the 4.8.0 release PR:
https://github.com/shaka-project/shaka-player/actions/runs/8819839697/job/24212073498
Since then, it always fails, with errors like this for Firefox and Edge:
So webdriver-installer is unable to find installed versions of Firefox and Edge, which likely means they have moved in the CI environment.
Chrome on Mac fails tests instead, with a long cascade of timeouts like:
These are very hard to explain. These tests passed in 4.8.0, but have failed since. Chrome on Mac has not been upgraded, as we can see the same browser & OS version numbers in the logs of the last successful run:
If a solution can't be found quickly, we may need to disable these in CI and rely on lab test runs to gate releases. This may result in Mac-specific regressions that are not found until release time.
The text was updated successfully, but these errors were encountered: