Can't run Puppeteer in AWS Lambda.

[dev1ninja]

I got this error when I run my scripts that use Puppeteer in Node.js.

{
  "errorType": "Error",
  "errorMessage": "Failed to launch the browser process!\n/tmp/chromium: error while loading shared libraries: libnss3.so: cannot open shared object file: No such file or directory\n\n\nTROUBLESHOOTING: https://pptr.dev/troubleshooting\n",
  "trace": [
    "Error: Failed to launch the browser process!",
    "/tmp/chromium: error while loading shared libraries: libnss3.so: cannot open shared object file: No such file or directory",
    "",
    "",
    "TROUBLESHOOTING: https://pptr.dev/troubleshooting",
    "",
    "    at Interface.onClose (/var/task/node_modules/@puppeteer/browsers/lib/cjs/launch.js:259:24)",
    "    at Interface.emit (node:events:525:35)",
    "    at Interface.close (node:internal/readline/interface:533:10)",
    "    at Socket.onend (node:internal/readline/interface:259:10)",
    "    at Socket.emit (node:events:525:35)",
    "    at endReadableNT (node:internal/streams/readable:1359:12)",
    "    at process.processTicksAndRejections (node:internal/process/task_queues:82:21)"
  ]
}

Here is my script.

const Chromium = require('chrome-aws-lambda');
const puppeteer = require('puppeteer-extra');
const StealthPlugin = require('puppeteer-extra-plugin-stealth')
puppeteer.use(StealthPlugin())

const browser = await puppeteer.launch({
    args: Chromium.args,
    defaultViewport: Chromium.defaultViewport,
    executablePath: await Chromium.executablePath,
    headless: Chromium.headless,
    ignoreHTTPSErrors: true,
})
const page = await browser.newPage();
await page.setUserAgent('Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36')

How can I solve this problem?

[pandeysoni]

@dev1ninja I am also facing the same issue.
Any solution you found or still the same issue

[dev1ninja]

@pandeysoni I am still facing the same issue.
It seems to be impossible to pass Cloudflare using chromium.
Just have to use Real Chrome Browser.
For this, the solution is to use Docker to install Google Chrome on AWS Lambda environment.
But I don't know how to do it.
Please let me know once you have solution.
I am also trying to find good solution now.

[sunil-buddala]

I am having the same issue

Failed to launch the browser process!
/tmp/chromium: error while loading shared libraries: libnss3.so: cannot open shared object file: No such file or directory

[kro12]

Looks like there may be an issue in the examples in the readme, I'm guessing it should be

await Chromium.executablePath()

[kcalixto]

Looks like there may be an issue in the examples in the readme, I'm guessing it should be

await Chromium.executablePath()

Heyy, it seems that is not it :(
node is throwing "executablePath is not a function"

What worked for me was downgrading my lambda to Node14.x and use this lambda layer: https://github.com/shelfio/chrome-aws-lambda-layer

hope it helps!

[kro12]

I switched to this and so far it's running (Node 18) on Lambda without any issue.

[Shamanthkolegodu]

chrome-aws-lambda is not working on node 16 and node 18, it is an existing issue.
"@sparticuz/chromium": "^109.0.5",
"puppeteer-core": "^19.4.0"
combination in working for me on node 18

[konarskis]

I finally managed to make it worked and described the solution here, hope it's helpful: https://konarskis.substack.com/p/puppeteer-aws-lambda

The key was to match the versions of puppeteer and chromium, and download the chromium binary at runtime as opposed to trying to put it inside the Lambda itself. This way, we don't need any layers, external dependencies, or other configuration changes.

[Jxmedia]

chrome-aws-lambda is not working on node 16 and node 18, it is an existing issue. "@sparticuz/chromium": "^109.0.5", "puppeteer-core": "^19.4.0" combination in working for me on node 18

This works for me as well on node 18

[alex-andrey]

I finally managed to make it worked and described the solution here, hope it's helpful: https://konarskis.substack.com/p/puppeteer-aws-lambda

The key was to match the versions of puppeteer and chromium, and download the chromium binary at runtime as opposed to trying to put it inside the Lambda itself. This way, we don't need any layers, external dependencies, or other configuration changes.

Thanks for taking the time to write this up @konarskis, was very helpful!

[yogeshwar-chaudhari-20]

I came across many articles on getting Puppeteer to work with Lambda; however, most of them relied on hacks, older versions, or third-party libraries. Wanting to keep my implementation as close to native libraries as possible, I opted to containerize my Lambda function. Almost all of these things are documented in official docs of respective libraries.

Project using Typescript

// tsconfig.json
{
  "compilerOptions": {
    "rootDir": "src",
    "outDir": "dist",
    "strict": true,
    "target": "es6",
    "module": "commonjs",
    "sourceMap": true,
    "esModuleInterop": true,
    "moduleResolution": "node",
    "types": ["jest", "node"]
  },
  "include": ["src/**/*"],
  "exclude": ["node_modules", "**/__test__/*"]
}

Build command

"build": "esbuild index.ts --bundle --minify --sourcemap --platform=node --target=es2020 --outfile=dist/index.js"

Dockerfile

FROM public.ecr.aws/lambda/nodejs:20 as builder
WORKDIR /usr/app
COPY package.json .

# Copy all the code files
COPY common/ ./common/ 
# Copy your code files as per your folder structure.

# Install and build
RUN npm install
RUN npm run build
    
# Build the running container
FROM public.ecr.aws/lambda/nodejs:20
WORKDIR ${LAMBDA_TASK_ROOT}
COPY --from=builder /usr/app/dist/* ./

# Chrome installation 
# Install additional OS dependencies
RUN dnf update -y && \
    dnf install -y \
        ca-certificates \
        liberation-fonts \
        alsa-lib \
        atk \
        cairo \
        cups-libs \
        dbus-glib \
        expat \
        fontconfig \
        libgbm \
        gcc \
        glib2 \
        gtk3 \
        nspr \
        nss \
        pango \
        # SDL \
        libX11 \
        libXcomposite \
        libXcursor \
        libXdamage \
        libXext \
        libXfixes \
        libXi \
        libXrandr \
        libXrender \
        libXScrnSaver \
        libXtst \
        # lsb-release \
        wget \
        xdg-utils
# Chrome installation

RUN npm i puppeteer
ENV YOUR_ENV_VARIABLE=something-value
CMD ["index.handler"]

Build image and run the container:

docker build --platform linux/amd64 -t docker-image:test .  
docker run --platform linux/amd64 -p 9000:8080 docker-image:test
curl "http://localhost:9000/2015-03-31/functions/function/invocations" -d '{"key": ["value"]}'

The docker image is working fine locally. I am still yet to deploy this in Lambda, if someone wants it, I can share that info too.

[kbourro]

I came across many articles on getting Puppeteer to work with Lambda; however, most of them relied on hacks, older versions, or third-party libraries. Wanting to keep my implementation as close to native libraries as possible, I opted to containerize my Lambda function. Almost all of these things are documented in official docs of respective libraries.

Project using Typescript

// tsconfig.json

{

  "compilerOptions": {

    "rootDir": "src",

    "outDir": "dist",

    "strict": true,

    "target": "es6",

    "module": "commonjs",

    "sourceMap": true,

    "esModuleInterop": true,

    "moduleResolution": "node",

    "types": ["jest", "node"]

  },

  "include": ["src/**/*"],

  "exclude": ["node_modules", "**/__test__/*"]

}

Build command

"build": "esbuild index.ts --bundle --minify --sourcemap --platform=node --target=es2020 --outfile=dist/index.js"

Dockerfile

FROM public.ecr.aws/lambda/nodejs:20 as builder

WORKDIR /usr/app

COPY package.json .



# Copy all the code files

COPY common/ ./common/ 

# Copy your code files as per your folder structure.



# Install and build

RUN npm install

RUN npm run build

    

# Build the running container

FROM public.ecr.aws/lambda/nodejs:20

WORKDIR ${LAMBDA_TASK_ROOT}

COPY --from=builder /usr/app/dist/* ./



# Chrome installation 

# Install additional OS dependencies

RUN dnf update -y && \

    dnf install -y \

        ca-certificates \

        liberation-fonts \

        alsa-lib \

        atk \

        cairo \

        cups-libs \

        dbus-glib \

        expat \

        fontconfig \

        libgbm \

        gcc \

        glib2 \

        gtk3 \

        nspr \

        nss \

        pango \

        # SDL \

        libX11 \

        libXcomposite \

        libXcursor \

        libXdamage \

        libXext \

        libXfixes \

        libXi \

        libXrandr \

        libXrender \

        libXScrnSaver \

        libXtst \

        # lsb-release \

        wget \

        xdg-utils

# Chrome installation



RUN npm i puppeteer

ENV YOUR_ENV_VARIABLE=something-value

CMD ["index.handler"]

Build image and run the container:

docker build --platform linux/amd64 -t docker-image:test .  

docker run --platform linux/amd64 -p 9000:8080 docker-image:test

curl "http://localhost:9000/2015-03-31/functions/function/invocations" -d '{"key": ["value"]}'

The docker image is working fine locally. I am still yet to deploy this in Lambda, if someone wants it, I can share that info too.

It worked ? Just tried something similar and chrome fails to start in lambda.

[yogeshwar-woodside]

@kbourro Partly. The containerized approach works in headless mode, but my requests were blocked by the site's CAPTCHA. I concluded that using residential proxies + Lambda or paid scraper services would solve this. In the end, I decided to host it on my local machine for my use case.

It wasn't worth the time for my use case.