metrics: skip empty lines in process cloudflare #3765
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
trivikr
force-pushed
the
skip-processing-empty-lines
branch
from
a98a888
to
c02c46c
Compare
|
cc @targos |
|
Is it normal to have empty lines? |
I don't know. I noticed while processing an example file, and verified the error is not logged after adding the length check to skip processing. #3697 (comment) Once merged and deployed, hopefully most or all of JSON.parse errors go away, making debugging easier. |
trivikr
changed the title
|
This PR is good to merge, and |
|
Deployed, and verified that errors are no longer logged. BeforeEVENT TYPE: OBJECT_FINALIZE
Node version is: v20.14.0
BUCKET cloudflare-logs-nodejs
FILENAME 20240625/20240625T193221Z_20240625T193327Z_121d0e03.log.gz
PROCESSEDFILENAME 20240625/20240625T193221Z_20240625T193327Z
INSIDE CREATE PIPELINE
String length: 23750258
Array Length: 48452
SyntaxError: Unexpected end of JSON input at JSON.parse (<anonymous>) at /usr/src/app/process-cloudflare.js:164:34 at callback (/usr/src/app/node_modules/@google-cloud/storage/build/src/file.js:1624:17) at /usr/src/app/node_modules/@google-cloud/storage/build/src/file.js:1640:87 at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
Upload complete
rss 358.32 MB
heapTotal 57.23 MB
heapUsed 44.96 MB
external 26.86 MB
arrayBuffers 23.75 MBAfterEVENT TYPE: OBJECT_FINALIZE
Node version is: v20.15.0
BUCKET cloudflare-logs-nodejs
FILENAME 20240625/20240625T193633Z_20240625T193744Z_8987830a.log.gz
PROCESSEDFILENAME 20240625/20240625T193633Z_20240625T193744Z
INSIDE CREATE PIPELINE
String length: 9327443
Array Length: 18849
Upload complete
rss 157.49 MB
heapTotal 31.18 MB
heapUsed 25.12 MB
external 12.45 MB
arrayBuffers 9.35 MB |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Refs: #3697 (comment)