Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix errors when receiving WebSocket data in Vercel Edge runtime and indefinitely creating new connections #1731

Open
VeskeR opened this issue Apr 15, 2024 · 2 comments
Labels
bug Something isn't working. It's clear that this does need to be fixed.

Comments

@VeskeR
Copy link
Contributor

VeskeR commented Apr 15, 2024

When using an Ably.Realtime instance in the Vercel Edge runtime (to avoid #1732 error replace setImmediate calls in ably-js with setTimeout) - whether within serverless functions or in SSR when rendering the frontend in Edge runtime - the connection to Ably servers sets up correctly. However, we begin to get TypeError: Cannot read properties of null (reading 'length') errors continuously after the connection has been set up. Here's the full error log:

TypeError: Cannot read properties of null (reading 'length')
    at WebSocketTransport.onWsData (webpack-internal:///(middleware)/./node_modules/ably/build/ably.js:6373:131)
    at wsConnection.onmessage (webpack-internal:///(middleware)/./node_modules/ably/build/ably.js:6342:31)
    at [nodejs.internal.kHybridDispatch] (node:internal/event_target:826:20)
    at WebSocket.dispatchEvent (node:internal/event_target:761:26)
    at fireEvent (eval at requireWithFakeGlobalScope (C:\Users\Andrew\dev\ably\ably-js-next-example\node_modules\next\dist\compiled\edge-runtime\index.js:1:657096), <anonymous>:12687:14)
    at websocketMessageReceived (eval at requireWithFakeGlobalScope (C:\Users\Andrew\dev\ably\ably-js-next-example\node_modules\next\dist\compiled\edge-runtime\index.js:1:657096), <anonymous>:12709:7)
    at _ByteParser.run (eval at requireWithFakeGlobalScope (C:\Users\Andrew\dev\ably\ably-js-next-example\node_modules\next\dist\compiled\edge-runtime\index.js:1:657096), <anonymous>:13122:17)
    at _ByteParser._write (eval at requireWithFakeGlobalScope (C:\Users\Andrew\dev\ably\ably-js-next-example\node_modules\next\dist\compiled\edge-runtime\index.js:1:657096), <anonymous>:13001:14)
    at writeOrBuffer (node:internal/streams/writable:564:12)
    at _write (node:internal/streams/writable:493:10)
 ⨯ uncaughtException: TypeError: Cannot read properties of null (reading 'length')
    at WebSocketTransport.onWsData (webpack-internal:///(middleware)/./node_modules/ably/build/ably.js:6373:131)
    at wsConnection.onmessage (webpack-internal:///(middleware)/./node_modules/ably/build/ably.js:6342:31)
    at [nodejs.internal.kHybridDispatch] (node:internal/event_target:826:20)
    at WebSocket.dispatchEvent (node:internal/event_target:761:26)
    at fireEvent (eval at requireWithFakeGlobalScope (C:\Users\Andrew\dev\ably\ably-js-next-example\node_modules\next\dist\compiled\edge-runtime\index.js:1:657096), <anonymous>:12687:14)
    at websocketMessageReceived (eval at requireWithFakeGlobalScope (C:\Users\Andrew\dev\ably\ably-js-next-example\node_modules\next\dist\compiled\edge-runtime\index.js:1:657096), <anonymous>:12709:7)
    at _ByteParser.run (eval at requireWithFakeGlobalScope (C:\Users\Andrew\dev\ably\ably-js-next-example\node_modules\next\dist\compiled\edge-runtime\index.js:1:657096), <anonymous>:13122:17)
    at _ByteParser._write (eval at requireWithFakeGlobalScope (C:\Users\Andrew\dev\ably\ably-js-next-example\node_modules\next\dist\compiled\edge-runtime\index.js:1:657096), <anonymous>:13001:14)
    at writeOrBuffer (node:internal/streams/writable:564:12)
    at _write (node:internal/streams/writable:493:10)
 ⨯ uncaughtException: TypeError: Cannot read properties of null (reading 'length')
    at WebSocketTransport.onWsData (webpack-internal:///(middleware)/./node_modules/ably/build/ably.js:6373:131)
    at wsConnection.onmessage (webpack-internal:///(middleware)/./node_modules/ably/build/ably.js:6342:31)
    at [nodejs.internal.kHybridDispatch] (node:internal/event_target:826:20)
    at WebSocket.dispatchEvent (node:internal/event_target:761:26)
    at fireEvent (eval at requireWithFakeGlobalScope (C:\Users\Andrew\dev\ably\ably-js-next-example\node_modules\next\dist\compiled\edge-runtime\index.js:1:657096), <anonymous>:12687:14)
    at websocketMessageReceived (eval at requireWithFakeGlobalScope (C:\Users\Andrew\dev\ably\ably-js-next-example\node_modules\next\dist\compiled\edge-runtime\index.js:1:657096), <anonymous>:12709:7)
    at _ByteParser.run (eval at requireWithFakeGlobalScope (C:\Users\Andrew\dev\ably\ably-js-next-example\node_modules\next\dist\compiled\edge-runtime\index.js:1:657096), <anonymous>:13122:17)
    at _ByteParser._write (eval at requireWithFakeGlobalScope (C:\Users\Andrew\dev\ably\ably-js-next-example\node_modules\next\dist\compiled\edge-runtime\index.js:1:657096), <anonymous>:13001:14)
    at writeOrBuffer (node:internal/streams/writable:564:12)
    at _write (node:internal/streams/writable:493:10)

It is coming from the WebSocketTransport.onWsData method at this line. Here, data parameter unexpectedly is null, causing an error. Judging by the frequency of errors above, it is highly likely that this error occurs whenever we receive an {"action":0} heartbeat message from the server.

After some time, we encounter the next error:

00:25:01.012 Ably: Transport.onIdleTimerExpire(): No activity seen from realtime in 25075ms; assuming connection has dropped

Transport.onIdleTimerExpire() is triggered because we haven't observed any activity on the opened connection for the maximum idle interval. This is due to the failure of processing the heartbeat.

The above issues are accompanied by the following strange behavior:

  1. Even though we encounter errors for heartbeats, we do not encounter errors when receiving other published messages. For instance, we can publish a message to a channel and receive it correctly in the Ably.Realtime instance running in the Vercel Edge runtime.
  2. We can successfully publish messages via the open WebSocket connection, despite encountering errors when trying to process a heartbeat.
  3. We continue to open new connections to the Ably server after each Transport.onIdleTimerExpire() process, and previous connections are not immediately closed. Consequently, this results in the Ably.Realtime instance continuously opening new connections to Ably servers and eventually exhausting the maximum connection pool per account. This is the most critical problem in this issue, preventing us from allowing customers to use the Ably.Realtime instance in the Vercel Edge runtime at the moment.

┆Issue is synchronized with this Jira Task by Unito

@VeskeR
Copy link
Contributor Author

VeskeR commented Apr 17, 2024

This comment from ably react-hooks repo might be slightly related: ably-labs/react-hooks#8 (comment), user saw his "max connections" linearly increase (just as we did in this issue) when ably-js was run on the server during SSR

@rnbrady
Copy link

rnbrady commented Oct 6, 2024

This may be related to vercel/edge-runtime#983 as that bug is also caused by a heartbeat and also results in a null object being delivered to the message handler.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working. It's clear that this does need to be fixed.
Development

No branches or pull requests

2 participants