Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use bytes::Bytes as the HTTP request body in HttpClient. #2485

Merged

Conversation

mstyura
Copy link
Contributor

@mstyura mstyura commented Dec 29, 2024

This will allow pooling of buffers on caller side compared to usage of Vec.

Fixes

Lack of possibility to reuse buffers which are consumed by HttpClient after request has been sent.

Design discussion issue

Should old function be kept with deprecated annotation and hence new function had to have a different name or should I just introduce breaking change by changing signature of a function?

Changes

I propose to change type of body of http request accepted by HttpClient trait from Vec<u8> to bytes::Bytes.
This should unblock pooling of buffers which can be used both for serialization & being http request bodies.
Something like this (pseudo-code to get the idea):

async fn export(&mut self, batch: Vec<SpanData>) -> ExportResult  {
    let mut buffer = pool.get();
    let request = self.build_request(&mut buffer, batch);
    let res = send_request(client, request);
    pool.return(buffer);
    res
}

Merge requirement checklist

  • CONTRIBUTING guidelines followed
  • Unit tests added/updated (if applicable)
  • Appropriate CHANGELOG.md files updated for non-trivial, user-facing changes
  • Changes in public API reviewed (if applicable)

@mstyura mstyura requested a review from a team as a code owner December 29, 2024 09:28
Copy link

codecov bot commented Dec 29, 2024

Codecov Report

Attention: Patch coverage is 0% with 15 lines in your changes missing coverage. Please review.

Project coverage is 79.6%. Comparing base (17cce83) to head (d898ccb).
Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
opentelemetry-http/src/lib.rs 0.0% 6 Missing ⚠️
opentelemetry-otlp/src/exporter/http/logs.rs 0.0% 2 Missing ⚠️
opentelemetry-otlp/src/exporter/http/metrics.rs 0.0% 2 Missing ⚠️
opentelemetry-otlp/src/exporter/http/trace.rs 0.0% 2 Missing ⚠️
opentelemetry-zipkin/src/exporter/uploader.rs 0.0% 2 Missing ⚠️
...try-sdk/src/trace/sampler/jaeger_remote/sampler.rs 0.0% 1 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##            main   #2485     +/-   ##
=======================================
- Coverage   79.6%   79.6%   -0.1%     
=======================================
  Files        118     118             
  Lines      22464   22467      +3     
=======================================
  Hits       17897   17897             
- Misses      4567    4570      +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@mstyura
Copy link
Contributor Author

mstyura commented Jan 2, 2025

Hello @cijothomas
Could you please take a look at this PR once you have time?
Thanks a lot in advance!

This will allow pooling of buffers on caller side compared to usage of Vec<u8>.
@mstyura mstyura force-pushed the http-client-request-body-bytes branch from 2e19563 to e0156e6 Compare January 2, 2025 11:22
Copy link
Member

@lalitb lalitb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I believe it is safe to have dependency on Bytes crate (as of now, it was only used for header injection), as it well maintained, and widely used (eg, by tokio, reqwest, and hyper).

@mstyura
Copy link
Contributor Author

mstyura commented Jan 9, 2025

And actually Bytes was already used in as type of response body :)

@mstyura
Copy link
Contributor Author

mstyura commented Jan 29, 2025

Is there anything I can do to facilitate merging this?

@cijothomas cijothomas merged commit 8520610 into open-telemetry:main Jan 29, 2025
20 of 21 checks passed
@cijothomas
Copy link
Member

Is there anything I can do to facilitate merging this?

Sorry for the delay! Missed this un-intentionally!

Would you be sending a PR to do buffer pooling for otlp?

@mstyura
Copy link
Contributor Author

mstyura commented Jan 29, 2025

I don't have prepared PR to implement pooling for oltp, I've planned to implement it for Datadog (in contrib repo), because I have app which uses Datadog exporter and I see in profiler it will benefit if pooling. Regarding oltp I could try implementing it, but not very soon.

@cijothomas
Copy link
Member

I think Datadog now supports OTLP natively, so you should be able to leverage OTLP Exporter. We were discussing the possibility of sunsetting the DD exporter in favor of OTLP (can't find that discussion anymore :( )

@scottgerring @julianocosta89 shared this doc : https://www.datadoghq.com/blog/monitor-rust-otel/

@julianocosta89
Copy link
Member

I think Datadog now supports OTLP natively, so you should be able to leverage OTLP Exporter. We were discussing the possibility of sunsetting the DD exporter in favor of OTLP (can't find that discussion anymore :( )

@scottgerring @julianocosta89 shared this doc : https://www.datadoghq.com/blog/monitor-rust-otel/

Hello hello, just for reference, this thread had the initial discussion: https://cloud-native.slack.com/archives/C03GDP0H023/p1737396437600899

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants