Inconsistent Build Outputs and SIGKILL Error with Next

[aaronisrael]

Summary

Issue

When attempting to build our latest Next app on Vercel, we encounter inconsistent build outputs. The build process results in one of the following outcomes:

1. Build Completed

2. Build Failed - "Your deployment's build step did not complete within the maximum of 45 minutes."

3. Build Failed - "Command failed with signal 'SIGKILL'."

Additional Information

We added VERCEL_BUILD_SYSTEM_REPORT=1 to gather additional debug data, but it did not provide any useful information. And adding FORCE_PARALLEL_BUILDS=1 didn't improve our inconstant builds.

Example

No response

Steps to Reproduce

The issue occurs randomly during the build process, making it challenging to pinpoint specific steps to reproduce the problem consistently.

[amyegan]

Thanks for reaching out, Aaron. I have a few questions to help guide investigation into this issue.

• When the build does complete successfully, how long does it take? And does the deployed site function as expected?

  

• What version of Node.js does the project use? It might be set in Project Settings or overridden in in package.json

• Is this a new project, or an existing project that has been upgraded from older package versions?

[aaronisrael]

• Most of the times it takes ~5min but in some cases it can run up to 15min, and our site works like expected.
• We're currently running on node 20.x but changing it to 18.x doesn't improve this issue
• This is an older project that has been converted from /pages to /app router

[aaronisrael]

@amyegan Hi, did you have the time to checkout why these builds are failing?

[DusanStojanovic-gowago]

We are experiencing the same problem with our app. I am not sure if is it after the app router transition or not, but it is exactly as @aaronisrael described.

node: 18.x
next: 14.0.3

The feeling is that every 2nd deployment fails/succeeds.

Only one of the successful deployments from the screenshot is not redeployed.

[amyegan]

Thanks for waiting. It looks like both of you are seeing a pattern of some successful builds, some time-outs, and some builds that fail long before they time out. Please let me know if I missed anything or misunderstood. I have a few more questions. 🙇

• For the builds that fail before they time out, is Command failed with signal 'SIGKILL' the only error message you ever see?
• Do you see this pattern of build errors if you run a local build or deploy from a local build?
• Do either of you have a minimal reproducible example I can use to help with debugging? I haven't been able to replicate this issue in my test project.

If you are able to share a deployment link or the name of the project, I can use that to look for additional log info about the error

[aaronisrael]

For the builds that fail before they time out, is Command failed with signal 'SIGKILL' the only error message you ever see?

Yes for us this is the only error we see even if we add VERCEL_BUILD_SYSTEM_REPORT=1

Do you see this pattern of build errors if you run a local build or deploy from a local build?

Local they always succeed, only if we deploy to Vercel they fail

Do either of you have a minimal reproducible example I can use to help with debugging? I haven't been able to replicate this issue in my test project.

No haven't been able to reproduce this in a different repo. Can I send you the link in private?

[DusanStojanovic-gowago]

For the builds that fail before they time out, is Command failed with signal 'SIGKILL' the only error message you ever see?

The only error before SIGKILL is the sentry error. (we are not uploading source maps for preview branches), so this is kinda expected
Here is the screenshot:

Do you see this pattern of build errors if you run a local build or deploy from a local build?

I tried running the vercel build command multiple times, and it worked every time.

Do either of you have a minimal reproducible example I can use to help with debugging? I haven't been able to replicate this issue in my test project.

I couldn't reproduce it in a different repo, too. I can't share the affected repo, because it is a private repo of a company I am working for, so I am not allowed to do it.

[prashantchothani]

This is a serious issue and no way to identify what is causing the issue. @vercel-infra needs to look into this as locally it builds in less than 100 seconds but on Vercel is shows -SIGKILL after 3-6 seconds.

[aaronisrael]

I believe we've identified the issue: it seems to be related to the runtime edge in conjunction with Dynamic routes. In our rootLayout, the runtime was configured to edge. Upon reverting back to nodejs runtime, all builds stated to be successful again. Interestingly, when we revert the runtime back to edge and make our Dynamic routes as private, the builds also succeed.

[DusanStojanovic-gowago]

Unfortunately, this is almost certainly not the case for my application. We don't use edge, only the app router + pages router without specifying the runtime - so the nodejs is the default.

So, possibly, there are multiple reasons for the same issue.

[DusanStojanovic-gowago]

I don't know if it helps or not, but this is the difference I can see between SIGKILL builds and successful builds:

1. SIGKILL - Build cache 6566MB:

2. Successful build - Build cache < 1MB:

Is it possible to reduce the cache somehow?

[DusanStojanovic-gowago]

Yes, it worked! Now is the time to figure out what are the disadvantages of this 😁

[amyegan]

The disadvantage of using VERCEL_NO_BUILD_CACHE is you'll miss out on faster builds that make use of build cache.

A SIGKILL error usually means the build is exceeding the limits of what is available in the Vercel build container. So another option is to limit the allocated memory in the build command instead. You can do that by overriding the build command with something like NODE_OPTIONS="--max-old-space-size=6144" && yarn run build

[DusanStojanovic-gowago]

Thank you @amyegan for clarification.

Unfortunately, NODE_OPTIONS="--max-old-space-size=8192" didn't help.

I am not sure how to reduce the cache size, so for now, VERCEL_NO_BUILD_CACHE is the only solution that works.

[amyegan]

Did you try with the smaller 6144 limit?

[DusanStojanovic-gowago]

I tried with 6144 and it didn't work.

[stevenqzhang]

I had this same issue for the last few months. Tried everything but what did it for me was setting VERCEL_FORCE_NO_BUILD_CACHE=1

[Lermatroid]

Running into this same issue on this repo. Random builds fail seemingly without reason due to running out of memory.

[DusanStojanovic-gowago]

Unfortunately, this has started happening to me regardless of the VERCEL_FORCE_NO_BUILD_CACHE=1. Do we have any progress on this issue?

[manoranjaninfoviz]

$ set NODE_OPTIONS=--max-old-space-size=4096 && next build 19.50 info - Loaded env from /app/.env.local 29.31 Attention: Next.js now collects completely anonymous telemetry regarding usage. 29.31 This information is used to shape Next.js' roadmap and prioritize features. 29.31 You can learn more, including how to opt-out if you'd not like to participate in this anonymous program, by visiting the following URL: 29.31 https://nextjs.org/telemetry 29.31 33.00 info - Skipping validation of types 33.00 info - Skipping linting 40.47 info - Creating an optimized production build... 222.3 error - No Sentry auth token configured. Source maps will not be uploaded. 222.3 You can find information on how to generate a Sentry auth token here: https://docs.sentry.io/api/auth/ 222.3 After generating a Sentry auth token, set it via the SENTRY_AUTH_TOKEN environment variable during the build. 453.1 (node:27) [DEP_WEBPACK_MODULE_ID] DeprecationWarning: Module.id: Use new ChunkGraph API 453.1 (Use node --trace-deprecation ...` to show where the warning was created)
1135.2 info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

failed to solve: process "/bin/sh -c yarn build" did not complete successfully: exit code: 1`
please help me to solve

[jonahallibone]

I too am having this issue. Removing the build cache does not help.

[plutohan]

We are having this issue too!

[quocvuong2000]

I'm having this issue too!, it's take 15-20 min for the auto-build phrase then have a "SIGKILL" ERROR

[mason-os]

A project I work on (Vercel, NextJS 13.5, Node 18.19) has been seeing flakey SIGKILL errors on our builds since around June this year. It got particularly bad in the past few weeks, so we tried a few of the solutions proposed here:

• Adding the VERCEL_NO_BUILD_CACHE=1 environment var to our project. This slowed down our builds and actually seemed to increase the number of failures.
• With the build cache flag still set to 1, we tried increasing the memory limit to the ~6gb limit suggested in this thread and Vercel's troublehsooting guide. This didn't seem to help, though builds failed slightly faster.
• Toggling build cache (disable, build, re-enable, with memory increased). So far toggling the build cache + increasing the memory seems to have gotten us out of the woods. Today every other build was failing, after re-enabling the build cache + adding the memory bump we have had ~8 successful builds in a row.

The flakiness of the builds was pretty bad, redeploying didn't help and seemed to have the same odds of success as our automated builds.

[github-actions[bot]]

This discussion was automatically locked because the community moved to a new site. Please join us at vercel.community