Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Send meta_struct properties as byte array #4349

Merged
merged 3 commits into from
May 28, 2024
Merged

Conversation

uurien
Copy link
Collaborator

@uurien uurien commented May 24, 2024

What does this PR do?

Sends meta_struct data as byte array instead of as object.

Motivation

It was expected to be a byte array and I misunderstood the spec it in the first implementation.

Plugin Checklist

  • Unit tests.

Copy link

github-actions bot commented May 24, 2024

Overall package size

Self size: 6.53 MB
Deduped: 60.59 MB
No deduping: 60.87 MB

Dependency sizes

name version self size total size
@datadog/native-iast-taint-tracking 2.1.0 14.91 MB 14.92 MB
@datadog/native-appsec 7.1.1 14.39 MB 14.4 MB
@datadog/pprof 5.3.0 9.85 MB 10.22 MB
protobufjs 7.2.5 2.77 MB 6.56 MB
@datadog/native-iast-rewriter 2.3.1 2.15 MB 2.24 MB
@opentelemetry/core 1.14.0 872.87 kB 1.47 MB
@datadog/native-metrics 2.0.0 898.77 kB 1.3 MB
@opentelemetry/api 1.8.0 1.21 MB 1.21 MB
import-in-the-middle 1.7.4 70.19 kB 739.86 kB
msgpack-lite 0.1.26 201.16 kB 281.59 kB
opentracing 0.14.7 194.81 kB 194.81 kB
semver 7.5.4 93.4 kB 123.8 kB
pprof-format 2.1.0 111.69 kB 111.69 kB
@datadog/sketches-js 2.1.0 109.9 kB 109.9 kB
lodash.sortby 4.7.0 75.76 kB 75.76 kB
lru-cache 7.14.0 74.95 kB 74.95 kB
ignore 5.2.4 51.22 kB 51.22 kB
int64-buffer 0.1.10 49.18 kB 49.18 kB
shell-quote 1.8.1 44.96 kB 44.96 kB
istanbul-lib-coverage 3.2.0 29.34 kB 29.34 kB
tlhunter-sorted-set 0.1.0 24.94 kB 24.94 kB
limiter 1.1.5 23.17 kB 23.17 kB
dc-polyfill 0.1.4 23.1 kB 23.1 kB
retry 0.13.1 18.85 kB 18.85 kB
jest-docblock 29.7.0 8.99 kB 12.76 kB
crypto-randomuuid 1.0.0 11.18 kB 11.18 kB
path-to-regexp 0.1.7 6.78 kB 6.78 kB
koalas 1.0.2 6.47 kB 6.47 kB
module-details-from-path 1.0.3 4.47 kB 4.47 kB

🤖 This report was automatically generated by heaviest-objects-in-the-universe

@pr-commenter
Copy link

pr-commenter bot commented May 24, 2024

Benchmarks

Benchmark execution time: 2024-05-27 13:44:54

Comparing candidate commit 3006ec0 in PR branch ugaitz/fix-meta-struct with baseline commit 95b5a41 in branch master.

Found 1 performance improvements and 0 performance regressions! Performance is the same for 261 metrics, 4 unstable metrics.

scenario:plugin-graphql-with-depth-and-collapse-on-18

  • 🟩 max_rss_usage [-143.362MB; -141.054MB] or [-14.770%; -14.533%]

@uurien uurien marked this pull request as ready for review May 24, 2024 15:38
@uurien uurien requested a review from a team as a code owner May 24, 2024 15:38
simon-id
simon-id previously approved these changes May 27, 2024
szegedi
szegedi previously approved these changes May 27, 2024
Comment on lines 275 to 286
const keys = Array.isArray(value) ? [] : Object.keys(value)
const validKeys = keys.filter(key =>
typeof value[key] === 'string' ||
typeof value[key] === 'number' ||
(value[key] !== null && typeof value[key] === 'object'))

this._encodeMapPrefix(bytes, validKeys.length)

for (const key of validKeys) {
this._encodeString(bytes, key)
this._encodeObjectAsByteArray(bytes, value[key])
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm vacillating on recommending using Object.entries here. On one hand, it creates more temporary small arrays. On the other hand, it eliminates the repetitive evaluation of value[key] and makes the filter predicate context-free, which I think are nice properties for reading comprehension of the code.

At the end of the day, it's a matter of preference, but thought I'll raise it for consideration.

Suggested change
const keys = Array.isArray(value) ? [] : Object.keys(value)
const validKeys = keys.filter(key =>
typeof value[key] === 'string' ||
typeof value[key] === 'number' ||
(value[key] !== null && typeof value[key] === 'object'))
this._encodeMapPrefix(bytes, validKeys.length)
for (const key of validKeys) {
this._encodeString(bytes, key)
this._encodeObjectAsByteArray(bytes, value[key])
}
const entries = Array.isArray(value) ? [] : Object.entries(value)
const validEntries = entries.filter(([, v]) =>
typeof v === 'string' ||
typeof v === 'number' ||
(v !== null && typeof v === 'object'))
this._encodeMapPrefix(bytes, validEntries.length)
for (const [k, v] of validEntries) {
this._encodeString(bytes, k)
this._encodeObjectAsByteArray(bytes, v)
}

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I've done the changes. And changes also the other method that I made in the previous PR to use the same approach. thanks

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I came back to the previous approach. Even if Object.entries is easier to understand, it creates multiple arrays increasing memory and CPU usage.

@uurien uurien dismissed stale reviews from szegedi and simon-id via e56545f May 27, 2024 12:55
CarlesDD
CarlesDD previously approved these changes May 27, 2024
Copy link

codecov bot commented May 27, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 69.19%. Comparing base (3db62e3) to head (e56545f).
Report is 4 commits behind head on master.

Current head e56545f differs from pull request most recent head 3006ec0

Please upload reports for the commit 3006ec0 to get more accurate results.

Additional details and impacted files
@@             Coverage Diff             @@
##           master    #4349       +/-   ##
===========================================
- Coverage   85.94%   69.19%   -16.76%     
===========================================
  Files         117        1      -116     
  Lines        4384      198     -4186     
  Branches       33       33               
===========================================
- Hits         3768      137     -3631     
+ Misses        616       61      -555     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@uurien uurien merged commit 7788968 into master May 28, 2024
115 checks passed
@uurien uurien deleted the ugaitz/fix-meta-struct branch May 28, 2024 07:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants