Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GrpcTransportFactory fails to create when max retries is set to 0 #1514

Open
barell opened this issue Feb 19, 2025 · 5 comments
Open

GrpcTransportFactory fails to create when max retries is set to 0 #1514

barell opened this issue Feb 19, 2025 · 5 comments
Labels
bug Something isn't working

Comments

@barell
Copy link
Contributor

barell commented Feb 19, 2025

Describe your environment
PHP 8.3
open-telemetry/transport-grpc v1.1.2

Steps to reproduce
Set maxRetries to 0 when creating grpc transport.

What is the expected behavior?
Grpc transport is working and it does not attempt to retry on fail (the same configuration work on http transport).

What is the actual behavior?
Got error

ArithmeticError {#173
  #message: "Bit shift by negative number"
  #code: 0
  #file: "/app/vendor/open-telemetry/transport-grpc/GrpcTransportFactory.php"
  #line: 124
  trace: {
    /app/vendor/open-telemetry/transport-grpc/GrpcTransportFactory.php:124 { …}

I am not familiar with Otel ecosystem and do not know much grpc but I guess we should allow maxRetries = 0?

@barell barell added the bug Something isn't working label Feb 19, 2025
@brettmc
Copy link
Collaborator

brettmc commented Feb 19, 2025

I am not familiar with Otel ecosystem and do not know much grpc but I guess we should allow maxRetries = 0?

I think you're right, and it should mean only one attempt is made. The test coverage could be better for GrpcTransportFactory :(

@barell
Copy link
Contributor Author

barell commented Feb 19, 2025

I might have spot another related issue - after setting max retries to 1 another error popped out:

channel stack builder failed: INVALID_ARGUMENT: errors validating service config: [field:methodConfig[0].retryPolicy.maxAttempts error:must be at least 2]

I guess it comes from grpc directly https://github.com/grpc/grpc-proto/blob/master/grpc/service_config/service_config.proto

@brettmc
Copy link
Collaborator

brettmc commented Feb 24, 2025

I guess it comes from grpc directly

Right, then it seems like we could at least add some validation (and a test!) that maxRetries >= 2. Do you feel like contributing a PR for this?

@barell
Copy link
Contributor Author

barell commented Feb 24, 2025

I think adding a check and exception is a little improvement to developer experience as this will pop up earlier with clearer message. But I really would like to understand where does this grpc restriction come from.

I'm not familiar with grpc concepts and maybe I miss some context. I also tried to look around but couldn't find any explanation.

In the mean time I will try to put together a little PR.

Edit:
I found this document https://github.com/grpc/proposal/blob/master/A6-client-retries.md#limits-on-retries-and-hedges and as I understand, maxAttempts does include the original request. So this would explain why grpc requires at least one attempt but maybe the second attempt is required when retry policy is enabled.

So in other words, when we create grpc transport and set maxRetries to 3, we should set maxAttempts inside grpc retry policy to 4. Because it will make first attempt, then it will retry it 3 times. Does it make sense @brettmc ?

I think we could also support maxRetries with 0 and it would mean that we do not use any retry policy in grpc. Then we only make one attempt. But I want to test this as this is just my assumption.

@brettmc
Copy link
Collaborator

brettmc commented Feb 25, 2025

That makes sense, @barell - I think that the PR doesn't break anything new, and it could be an enhancement where maxRetries=0 drops the retry policy.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants