Performance improvements

[zinserjan]

There are some things that we can improve to get more screens in time. The goal is to reduce browser idle times as much as possible as the browser is the bottleneck and should theoretically never idle.

There are some optimizations that are safe to do (without breaking browser screenshots):

• parallel image processing while interacting with the browser and requesting the next screenshot(s) (process images parallel while interacting with the browser #69)
• scale images with GraphicsMagick when available (Speed up scaling with gm #70)
• combine image processing calls (e.g. scaling & cropping in a single step)
• crop images only when necessary (currently they will be always cropped, even if this isn't necessary)
• avoid file system calls and handle images in memory (e.g. stop writing images to .tmp directory before stitching)
  • this does not work with GraphicsMagick as it needs to read the images from disk

And some others need further researches or additional/manual tests to avoid errors:

• remove/reduce explicit timeouts in code (browser.pause calls)
• group browser.execute calls together when possible
• Usage of sharp instead of GraphicsMagick
  • sharp is limited to 16383px in width or height so this isn't a real alternative

[lovell]

Hello, I've just started investigating use of this rather handy module and spotted the following comment:

"sharp is limited to 16383px in width or height so this isn't a real alternative"

Did you see the limitInputPixels option? The following should be suitable for resizing large screenshots:

const png = await sharp(input)
  .limitInputPixels(false)
  .resize(width, height)
  .png({ adaptiveFiltering: false })
  .toBuffer();

[zinserjan]

Ah OK, didn't know that. Just saw some issues about this limit and the docs are also mentioning this limit.

But would be really cool if this works.

[lovell]

Ah, that is a most excellent point about the ambiguity of the docs, thank you. I'll attempt to make things clearer these are modifiable defaults.

EDIT: There are currently output limits, soon to be made more relevant to the specific output format - see lovell/sharp#176

[albertogasparin]

FYI sharp v0.18+ now has a per-format limit, so for PNG it should be 32bit