Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Reporting] Update Puppeteer to version 8.0.0 and Chromium to r856583 #98688

Merged
merged 13 commits into from
May 7, 2021

Conversation

tsullivan
Copy link
Member

@tsullivan tsullivan commented Apr 28, 2021

Implements #90496

Closes #89555
Closes #89467
Closes #87956

Release note: Updated the headless browser for Kibana Reporting to Chromium r856583 (90.0.4427.0)

  • Updates Puppeteer to version 8.0.0 and Chromium to r856583
  • Links to new build of Linux headless_shell in the Kibana team GCS bucket
  • Links to main download site of Chromium for Mac and Windows
  • Removes Mac and Windows compatibility from the Chromium build scripts

Version information for r856583:
https://storage.googleapis.com/chromium-find-releases-static/d16.html#d163fd7c6905e456501aba914a24c20b62b4c6dc

Explanation of headless_shell:
https://chromium.googlesource.com/chromium/src/+/d163fd7c6905e456501aba914a24c20b62b4c6dc/headless/#usage-as-a-c_library


How to test this PR
These steps need to happen for every platform: Mac, Windows, Linux x64 and Linux arm64

  • Make sure the Reporting plugin is fetching the correct version of the browser at start-up time, and that it can successfully unzip it and copy the files to x-pack/plugins/reporting/chromium
  • Make sure there are no errors when using the Reporting diagnostic tool
  • All functional and API tests that generate PDF and PNG files should pass
  • Search for issues in the Puppeteer repo that might apply to our usage

Metrics
Build size changes, before and after:

Package Before After
kibana-8.0.0-SNAPSHOT-linux-x86_64.tar.gz 285M 289M
kibana-8.0.0-SNAPSHOT-amd64.deb 289M 293M
kibana-8.0.0-SNAPSHOT-x86_64.rpm 293M 296M
kibana-8.0.0-SNAPSHOT-linux-aarch64.tar.gz 297M 301M
kibana-8.0.0-SNAPSHOT-arm64.deb 301M 306M
kibana-8.0.0-SNAPSHOT-aarch64.rpm 304M 309M
kibana-8.0.0-SNAPSHOT-docker-image.tar.gz 385M 389M
kibana-8.0.0-SNAPSHOT-darwin-x86_64.tar.gz 293M 325M
kibana-8.0.0-SNAPSHOT-windows-x86_64.zip 308M 405M

Checklist

  • Functional test for "large" dashboard (over 11MP)
  • Run tests for Mac
  • Run tests for Windows
  • Run tests for Linux x64
  • Run tests for Linux arm64
    • Started an EC2 instance using Ubuntu 20.10 ARM, and installed the OS dependencies: sudo apt install -y libnss3 fonts-liberation libfontconfig1

@spalger
Copy link
Contributor

spalger commented Apr 29, 2021

Pushing the new build to the GCS bucket caused existing builds to fail so I'm going to revert the update now.

@spalger
Copy link
Contributor

spalger commented Apr 29, 2021

We need to plan for how we're going to rotate these correctly

@tsullivan
Copy link
Member Author

Pushing the new build to the GCS bucket caused existing builds to fail so I'm going to revert the update now.

This was due to running the test script to build our current/master version of Chromium, while hitting problems trying to build the next version. The test script passed the build but unfortunately auto-uploaded the .zip file to our GCS bucket. It should have been a dry run. Sorry about that!!

@tsullivan
Copy link
Member Author

Due to some issues getting the version of chromium to compile for the 7.1.0 version of Puppeteer, I am experimenting to bumping to 8.0.0

@tsullivan tsullivan force-pushed the reporting/update-chromium-90 branch 2 times, most recently from 38c7942 to 88b4cd8 Compare April 29, 2021 03:27
Copy link
Contributor

@watson watson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great to see this being upgraded - Thank you! 💪 Just did a drive-by review - I hope you don't mind 😊

package.json Outdated Show resolved Hide resolved
package.json Outdated Show resolved Hide resolved
yarn.lock Outdated Show resolved Hide resolved
@tsullivan tsullivan changed the title [Reporting] Update Chromium to r848005 [Reporting] Update Puppeteer to version 8.0.0 and Chromium to r856583 Apr 29, 2021
@tsullivan tsullivan force-pushed the reporting/update-chromium-90 branch 2 times, most recently from fd48eb5 to d98aed0 Compare April 29, 2021 17:06
@tsullivan tsullivan force-pushed the reporting/update-chromium-90 branch from d98aed0 to 715601c Compare April 29, 2021 18:34
@tsullivan tsullivan marked this pull request as ready for review April 29, 2021 19:28
@tsullivan tsullivan requested review from a team as code owners April 29, 2021 19:28
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-reporting-services (Team:Reporting Services)

@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-app-services (Team:AppServices)

@tsullivan tsullivan marked this pull request as draft April 29, 2021 19:34
@tsullivan tsullivan force-pushed the reporting/update-chromium-90 branch 5 times, most recently from 5f8615c to 1defd02 Compare April 30, 2021 21:22
Copy link
Member

@dmlemeshko dmlemeshko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kibana-qa changes LGTM

@tsullivan
Copy link
Member Author

tsullivan commented May 5, 2021

I have found some issues in the Puppeteer repo that could require investigation for this PR:

https://github.com/puppeteer/puppeteer/issues/7043
https://github.com/puppeteer/puppeteer/pull/7063
https://github.com/puppeteer/puppeteer/issues/7038
https://github.com/puppeteer/puppeteer/issues/6926
https://github.com/puppeteer/puppeteer/issues/6875
https://github.com/puppeteer/puppeteer/issues/6641

I will update here after taking more look.

@tsullivan
Copy link
Member Author

I ran tests in VMs that have different memory size to track the size of PNG report I was able to create:

RAM max old space size max dashboard size in MB
1GB 800 1.75204
2GB 1600 2.175812
4GB 3200 2.877516

Copy link
Contributor

@jloleysens jloleysens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested this locally on macOS and it worked as expected.

Code changes LGTM, left a few nits.

Now just pending a final check on Windows.

x-pack/build_chromium/README.md Outdated Show resolved Hide resolved

In the case of Windows, you can use IE to open `http://localhost:9221` and see if the page loads. In mac/linux you can just curl the JSON endpoints: `curl http://localhost:9221/json/list`.
- Make sure the Reporting plugin is fetching the correct version of the browser
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How best can we determine what the correct version is?

On macOS I could do the following:

./ze-path/Chromium --version
Chromium 90.0.4427.0

How do we determine this number is correct or does puppeteer working tell us it is the correct version?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a very good question. I think the only way this could work on all platforms is to do something like this:

x-pack/plugins/reporting/chromium/<platform_path>/<binary path> --headless --repl

and then type: navigator.appVersion into the REPL.

Seems like this should be an easier one-liner, or something that is handled by our helper script: src/dev-tools/chromium_version.ts

@tsullivan
Copy link
Member Author

@elasticmachine merge upstream

@tsullivan
Copy link
Member Author

@elasticmachine merge upstream

@kibanamachine
Copy link
Contributor

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@tsullivan tsullivan merged commit f73da42 into elastic:master May 7, 2021
@tsullivan tsullivan deleted the reporting/update-chromium-90 branch May 7, 2021 16:54
tsullivan added a commit to tsullivan/kibana that referenced this pull request May 7, 2021
…elastic#98688)

* Update Puppeteer to 8.0.0

Updates Chromium to r856583
Links to new build of Linux headless_shell in the Kibana team GCS bucket
Links to main download site of Chromium for Mac and Windows
Removes Mac and Windows compatibility from the Chromium build scripts

* add functional tests for large dashboard

* ensure png comparison is working

* add test for large dashboard pdf

* update arm64 binary checksum

* update README

* more readme update

* Update x-pack/build_chromium/README.md

Co-authored-by: Jean-Louis Leysens <[email protected]>

Co-authored-by: Kibana Machine <[email protected]>
Co-authored-by: Jean-Louis Leysens <[email protected]>
# Conflicts:
#	x-pack/test/functional/es_archives/reporting/ecommerce_kibana/data.json
@kibanamachine
Copy link
Contributor

Looks like this PR has a backport PR but it still hasn't been merged. Please merge it ASAP to keep the branches relatively in sync.

@kibanamachine kibanamachine added the backport missing Added to PRs automatically when the are determined to be missing a backport. label May 11, 2021
@kibanamachine kibanamachine removed the backport missing Added to PRs automatically when the are determined to be missing a backport. label May 11, 2021
tsullivan added a commit that referenced this pull request May 11, 2021
…856583 (#98688) (#99577)

* [Reporting] Update Puppeteer to version 8.0.0 and Chromium to r856583 (#98688)

* Update Puppeteer to 8.0.0

Updates Chromium to r856583
Links to new build of Linux headless_shell in the Kibana team GCS bucket
Links to main download site of Chromium for Mac and Windows
Removes Mac and Windows compatibility from the Chromium build scripts

* add functional tests for large dashboard

* ensure png comparison is working

* add test for large dashboard pdf

* update arm64 binary checksum

* update README

* more readme update

* Update x-pack/build_chromium/README.md

Co-authored-by: Jean-Louis Leysens <[email protected]>

Co-authored-by: Kibana Machine <[email protected]>
Co-authored-by: Jean-Louis Leysens <[email protected]>
# Conflicts:
#	x-pack/test/functional/es_archives/reporting/ecommerce_kibana/data.json

* fix ts
tsullivan added a commit to tsullivan/kibana that referenced this pull request May 11, 2021
…r856583 (elastic#98688) (elastic#99577)

* [Reporting] Update Puppeteer to version 8.0.0 and Chromium to r856583 (elastic#98688)

* Update Puppeteer to 8.0.0

Updates Chromium to r856583
Links to new build of Linux headless_shell in the Kibana team GCS bucket
Links to main download site of Chromium for Mac and Windows
Removes Mac and Windows compatibility from the Chromium build scripts

* add functional tests for large dashboard

* ensure png comparison is working

* add test for large dashboard pdf

* update arm64 binary checksum

* update README

* more readme update

* Update x-pack/build_chromium/README.md

Co-authored-by: Jean-Louis Leysens <[email protected]>

Co-authored-by: Kibana Machine <[email protected]>
Co-authored-by: Jean-Louis Leysens <[email protected]>

* fix ts
tsullivan added a commit that referenced this pull request May 11, 2021
…r856583 (#98688) (#99577) (#99853)

* [Reporting] Update Puppeteer to version 8.0.0 and Chromium to r856583 (#98688)

* Update Puppeteer to 8.0.0

Updates Chromium to r856583
Links to new build of Linux headless_shell in the Kibana team GCS bucket
Links to main download site of Chromium for Mac and Windows
Removes Mac and Windows compatibility from the Chromium build scripts

* add functional tests for large dashboard

* ensure png comparison is working

* add test for large dashboard pdf

* update arm64 binary checksum

* update README

* more readme update

* Update x-pack/build_chromium/README.md

Co-authored-by: Jean-Louis Leysens <[email protected]>

Co-authored-by: Kibana Machine <[email protected]>
Co-authored-by: Jean-Louis Leysens <[email protected]>

* fix ts
@tsullivan
Copy link
Member Author

This PR was backported to 7.13

@sophiec20 sophiec20 added the (Deprecated) Feature:Reporting Use Reporting:Screenshot, Reporting:CSV, or Reporting:Framework instead label Aug 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
(Deprecated) Feature:Reporting Use Reporting:Screenshot, Reporting:CSV, or Reporting:Framework instead release_note:enhancement v7.13.0 v7.14.0 v8.0.0
Projects
None yet
9 participants