[Reporting] Update Puppeteer to version 8.0.0 and Chromium to r856583 #98688

tsullivan · 2021-04-28T20:38:48Z

Implements #90496

Closes #89555
Closes #89467
Closes #87956

Release note: Updated the headless browser for Kibana Reporting to Chromium r856583 (90.0.4427.0)

Updates Puppeteer to version 8.0.0 and Chromium to r856583
Links to new build of Linux headless_shell in the Kibana team GCS bucket
Links to main download site of Chromium for Mac and Windows
Removes Mac and Windows compatibility from the Chromium build scripts

Version information for r856583:
https://storage.googleapis.com/chromium-find-releases-static/d16.html#d163fd7c6905e456501aba914a24c20b62b4c6dc

Explanation of headless_shell:
https://chromium.googlesource.com/chromium/src/+/d163fd7c6905e456501aba914a24c20b62b4c6dc/headless/#usage-as-a-c_library

How to test this PR
These steps need to happen for every platform: Mac, Windows, Linux x64 and Linux arm64

Make sure the Reporting plugin is fetching the correct version of the browser at start-up time, and that it can successfully unzip it and copy the files to x-pack/plugins/reporting/chromium
Make sure there are no errors when using the Reporting diagnostic tool
All functional and API tests that generate PDF and PNG files should pass
Search for issues in the Puppeteer repo that might apply to our usage

Metrics
Build size changes, before and after:

Package	Before	After
kibana-8.0.0-SNAPSHOT-linux-x86_64.tar.gz	285M	289M
kibana-8.0.0-SNAPSHOT-amd64.deb	289M	293M
kibana-8.0.0-SNAPSHOT-x86_64.rpm	293M	296M
kibana-8.0.0-SNAPSHOT-linux-aarch64.tar.gz	297M	301M
kibana-8.0.0-SNAPSHOT-arm64.deb	301M	306M
kibana-8.0.0-SNAPSHOT-aarch64.rpm	304M	309M
kibana-8.0.0-SNAPSHOT-docker-image.tar.gz	385M	389M
kibana-8.0.0-SNAPSHOT-darwin-x86_64.tar.gz	293M	325M
kibana-8.0.0-SNAPSHOT-windows-x86_64.zip	308M	405M

Checklist

Functional test for "large" dashboard (over 11MP)
Run tests for Mac
Run tests for Windows
Run tests for Linux x64
Run tests for Linux arm64
- Started an EC2 instance using Ubuntu 20.10 ARM, and installed the OS dependencies: sudo apt install -y libnss3 fonts-liberation libfontconfig1

spalger · 2021-04-29T00:03:52Z

Pushing the new build to the GCS bucket caused existing builds to fail so I'm going to revert the update now.

spalger · 2021-04-29T00:04:11Z

We need to plan for how we're going to rotate these correctly

tsullivan · 2021-04-29T00:15:11Z

Pushing the new build to the GCS bucket caused existing builds to fail so I'm going to revert the update now.

This was due to running the test script to build our current/master version of Chromium, while hitting problems trying to build the next version. The test script passed the build but unfortunately auto-uploaded the .zip file to our GCS bucket. It should have been a dry run. Sorry about that!!

tsullivan · 2021-04-29T01:33:54Z

Due to some issues getting the version of chromium to compile for the 7.1.0 version of Puppeteer, I am experimenting to bumping to 8.0.0

watson

Great to see this being upgraded - Thank you! 💪 Just did a drive-by review - I hope you don't mind 😊

package.json

x-pack/plugins/reporting/server/browsers/chromium/driver_factory/index.ts

yarn.lock

elasticmachine · 2021-04-29T19:30:29Z

Pinging @elastic/kibana-reporting-services (Team:Reporting Services)

elasticmachine · 2021-04-29T19:30:29Z

Pinging @elastic/kibana-app-services (Team:AppServices)

dmlemeshko

kibana-qa changes LGTM

tsullivan · 2021-05-05T19:03:29Z

I have found some issues in the Puppeteer repo that could require investigation for this PR:

https://github.com/puppeteer/puppeteer/issues/7043
https://github.com/puppeteer/puppeteer/pull/7063
https://github.com/puppeteer/puppeteer/issues/7038
https://github.com/puppeteer/puppeteer/issues/6926
https://github.com/puppeteer/puppeteer/issues/6875
https://github.com/puppeteer/puppeteer/issues/6641

I will update here after taking more look.

tsullivan · 2021-05-06T02:27:34Z

I ran tests in VMs that have different memory size to track the size of PNG report I was able to create:

RAM	max old space size	max dashboard size in MB
1GB	800	1.75204
2GB	1600	2.175812
4GB	3200	2.877516

jloleysens

I tested this locally on macOS and it worked as expected.

Code changes LGTM, left a few nits.

Now just pending a final check on Windows.

x-pack/build_chromium/README.md

jloleysens · 2021-05-06T12:46:41Z

x-pack/build_chromium/README.md


-In the case of Windows, you can use IE to open `http://localhost:9221` and see if the page loads. In mac/linux you can just curl the JSON endpoints: `curl http://localhost:9221/json/list`.
+- Make sure the Reporting plugin is fetching the correct version of the browser


How best can we determine what the correct version is?

On macOS I could do the following:

./ze-path/Chromium --version Chromium 90.0.4427.0

How do we determine this number is correct or does puppeteer working tell us it is the correct version?

This is a very good question. I think the only way this could work on all platforms is to do something like this:

x-pack/plugins/reporting/chromium/<platform_path>/<binary path> --headless --repl

and then type: navigator.appVersion into the REPL.

Seems like this should be an easier one-liner, or something that is handled by our helper script: src/dev-tools/chromium_version.ts

tsullivan · 2021-05-06T21:20:31Z

@elasticmachine merge upstream

tsullivan · 2021-05-06T23:23:37Z

@elasticmachine merge upstream

Co-authored-by: Jean-Louis Leysens <[email protected]>

kibanamachine · 2021-05-07T02:31:28Z

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

💚 Build #124883 succeeded 5a16f7f
💔 Build #124858 failed 7b529d3
💚 Build #124324 succeeded 20243ba
💚 Build #123969 succeeded d6f02a8
💚 Build #123957 succeeded 31c4cb7

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

…elastic#98688) * Update Puppeteer to 8.0.0 Updates Chromium to r856583 Links to new build of Linux headless_shell in the Kibana team GCS bucket Links to main download site of Chromium for Mac and Windows Removes Mac and Windows compatibility from the Chromium build scripts * add functional tests for large dashboard * ensure png comparison is working * add test for large dashboard pdf * update arm64 binary checksum * update README * more readme update * Update x-pack/build_chromium/README.md Co-authored-by: Jean-Louis Leysens <[email protected]> Co-authored-by: Kibana Machine <[email protected]> Co-authored-by: Jean-Louis Leysens <[email protected]> # Conflicts: # x-pack/test/functional/es_archives/reporting/ecommerce_kibana/data.json

kibanamachine · 2021-05-11T16:57:22Z

Looks like this PR has a backport PR but it still hasn't been merged. Please merge it ASAP to keep the branches relatively in sync.

…856583 (#98688) (#99577) * [Reporting] Update Puppeteer to version 8.0.0 and Chromium to r856583 (#98688) * Update Puppeteer to 8.0.0 Updates Chromium to r856583 Links to new build of Linux headless_shell in the Kibana team GCS bucket Links to main download site of Chromium for Mac and Windows Removes Mac and Windows compatibility from the Chromium build scripts * add functional tests for large dashboard * ensure png comparison is working * add test for large dashboard pdf * update arm64 binary checksum * update README * more readme update * Update x-pack/build_chromium/README.md Co-authored-by: Jean-Louis Leysens <[email protected]> Co-authored-by: Kibana Machine <[email protected]> Co-authored-by: Jean-Louis Leysens <[email protected]> # Conflicts: # x-pack/test/functional/es_archives/reporting/ecommerce_kibana/data.json * fix ts

…r856583 (elastic#98688) (elastic#99577) * [Reporting] Update Puppeteer to version 8.0.0 and Chromium to r856583 (elastic#98688) * Update Puppeteer to 8.0.0 Updates Chromium to r856583 Links to new build of Linux headless_shell in the Kibana team GCS bucket Links to main download site of Chromium for Mac and Windows Removes Mac and Windows compatibility from the Chromium build scripts * add functional tests for large dashboard * ensure png comparison is working * add test for large dashboard pdf * update arm64 binary checksum * update README * more readme update * Update x-pack/build_chromium/README.md Co-authored-by: Jean-Louis Leysens <[email protected]> Co-authored-by: Kibana Machine <[email protected]> Co-authored-by: Jean-Louis Leysens <[email protected]> * fix ts

…r856583 (#98688) (#99577) (#99853) * [Reporting] Update Puppeteer to version 8.0.0 and Chromium to r856583 (#98688) * Update Puppeteer to 8.0.0 Updates Chromium to r856583 Links to new build of Linux headless_shell in the Kibana team GCS bucket Links to main download site of Chromium for Mac and Windows Removes Mac and Windows compatibility from the Chromium build scripts * add functional tests for large dashboard * ensure png comparison is working * add test for large dashboard pdf * update arm64 binary checksum * update README * more readme update * Update x-pack/build_chromium/README.md Co-authored-by: Jean-Louis Leysens <[email protected]> Co-authored-by: Kibana Machine <[email protected]> Co-authored-by: Jean-Louis Leysens <[email protected]> * fix ts

tsullivan · 2021-05-19T19:01:05Z

This PR was backported to 7.13

tsullivan mentioned this pull request Apr 28, 2021

[Reporting] Update Puppeteer to v7.1.0 #98096

Closed

tsullivan force-pushed the reporting/update-chromium-90 branch from 2b4d541 to f36ca90 Compare April 28, 2021 20:39

tsullivan force-pushed the reporting/update-chromium-90 branch 2 times, most recently from 38c7942 to 88b4cd8 Compare April 29, 2021 03:27

watson reviewed Apr 29, 2021

View reviewed changes

package.json Outdated Show resolved Hide resolved

package.json Outdated Show resolved Hide resolved

x-pack/plugins/reporting/server/browsers/chromium/driver_factory/index.ts Outdated Show resolved Hide resolved

yarn.lock Outdated Show resolved Hide resolved

tsullivan changed the title ~~[Reporting] Update Chromium to r848005~~ [Reporting] Update Puppeteer to version 8.0.0 and Chromium to r856583 Apr 29, 2021

tsullivan force-pushed the reporting/update-chromium-90 branch 2 times, most recently from fd48eb5 to d98aed0 Compare April 29, 2021 17:06

tsullivan mentioned this pull request Apr 29, 2021

[Reporting] code cleanup for reporting browser build/install/setup utilities #98799

Merged

tsullivan force-pushed the reporting/update-chromium-90 branch from d98aed0 to 715601c Compare April 29, 2021 18:34

tsullivan marked this pull request as ready for review April 29, 2021 19:28

tsullivan requested review from a team as code owners April 29, 2021 19:28

tsullivan added v7.14.0 v8.0.0 release_note:enhancement Team:AppServices labels Apr 29, 2021

tsullivan marked this pull request as draft April 29, 2021 19:34

tsullivan force-pushed the reporting/update-chromium-90 branch 5 times, most recently from 5f8615c to 1defd02 Compare April 30, 2021 21:22

tsullivan added 2 commits May 4, 2021 18:14

update README

aa26533

more readme update

20243ba

dmlemeshko approved these changes May 5, 2021

View reviewed changes

jloleysens approved these changes May 6, 2021

View reviewed changes

Merge branch 'master' into reporting/update-chromium-90

7b529d3

kibanamachine and others added 2 commits May 6, 2021 19:23

Merge branch 'master' into reporting/update-chromium-90

5a16f7f

Update x-pack/build_chromium/README.md

093f953

Co-authored-by: Jean-Louis Leysens <[email protected]>

tsullivan merged commit f73da42 into elastic:master May 7, 2021

tsullivan deleted the reporting/update-chromium-90 branch May 7, 2021 16:54

tsullivan mentioned this pull request May 7, 2021

[7.x] [Reporting] Update Puppeteer to version 8.0.0 and Chromium to r856583 (#98688) #99577

Merged

kibanamachine added the backport missing Added to PRs automatically when the are determined to be missing a backport. label May 11, 2021

kibanamachine removed the backport missing Added to PRs automatically when the are determined to be missing a backport. label May 11, 2021

tsullivan mentioned this pull request May 11, 2021

[7.13] [Reporting] Update Puppeteer to version 8.0.0 and Chromium to r856583 (#98688) (#99577) #99853

Merged

This was referenced May 14, 2021

[Reporting] Switch Mac Chromium to the internally-hosted custom build #100165

Merged

[Reporting] Upgrade Puppeteer to 8.0.0 #90496

Closed

tsullivan added the v7.13.0 label May 19, 2021

tsullivan mentioned this pull request Sep 30, 2021

[Reporting] Investigate cause of screenshot capture timeouts on Windows #63073

Closed

sophiec20 added the (Deprecated) Feature:Reporting Use Reporting:Screenshot, Reporting:CSV, or Reporting:Framework instead label Aug 21, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Reporting] Update Puppeteer to version 8.0.0 and Chromium to r856583 #98688

[Reporting] Update Puppeteer to version 8.0.0 and Chromium to r856583 #98688

tsullivan commented Apr 28, 2021 •

edited

Loading

spalger commented Apr 29, 2021

spalger commented Apr 29, 2021

tsullivan commented Apr 29, 2021

tsullivan commented Apr 29, 2021

watson left a comment

elasticmachine commented Apr 29, 2021

elasticmachine commented Apr 29, 2021

dmlemeshko left a comment

tsullivan commented May 5, 2021 •

edited

Loading

tsullivan commented May 6, 2021

jloleysens left a comment

jloleysens May 6, 2021

tsullivan May 6, 2021

tsullivan commented May 6, 2021

tsullivan commented May 6, 2021

kibanamachine commented May 7, 2021

kibanamachine commented May 11, 2021

tsullivan commented May 19, 2021


		In the case of Windows, you can use IE to open `http://localhost:9221` and see if the page loads. In mac/linux you can just curl the JSON endpoints: `curl http://localhost:9221/json/list`.
		- Make sure the Reporting plugin is fetching the correct version of the browser

[Reporting] Update Puppeteer to version 8.0.0 and Chromium to r856583 #98688

[Reporting] Update Puppeteer to version 8.0.0 and Chromium to r856583 #98688

Conversation

tsullivan commented Apr 28, 2021 • edited Loading

spalger commented Apr 29, 2021

spalger commented Apr 29, 2021

tsullivan commented Apr 29, 2021

tsullivan commented Apr 29, 2021

watson left a comment

Choose a reason for hiding this comment

elasticmachine commented Apr 29, 2021

elasticmachine commented Apr 29, 2021

dmlemeshko left a comment

Choose a reason for hiding this comment

tsullivan commented May 5, 2021 • edited Loading

tsullivan commented May 6, 2021

jloleysens left a comment

Choose a reason for hiding this comment

jloleysens May 6, 2021

Choose a reason for hiding this comment

tsullivan May 6, 2021

Choose a reason for hiding this comment

tsullivan commented May 6, 2021

tsullivan commented May 6, 2021

kibanamachine commented May 7, 2021

💚 Build Succeeded

Metrics [docs]

History

kibanamachine commented May 11, 2021

tsullivan commented May 19, 2021

tsullivan commented Apr 28, 2021 •

edited

Loading

tsullivan commented May 5, 2021 •

edited

Loading