Additional GRPC headers #4357

YurySuslovScalyr · 2023-03-31T16:30:34Z

YurySuslovScalyr
Mar 31, 2023

I'm implementing a new storage backend for Jaeger, and our backend requires some custom authentication/routing GRPC header. Is there a way to add custom GRPC headers to calls that Jaeger query and collector make?

Answered by yurishkuro

Mar 31, 2023

We don't support custom headers, only Authorization: bearer xxx. Can you not use this mechanism?

View full answer

yurishkuro · 2023-03-31T23:06:21Z

yurishkuro
Mar 31, 2023
Maintainer

We don't support custom headers, only Authorization: bearer xxx. Can you not use this mechanism?

1 reply

YurySuslovScalyr Apr 4, 2023
Author

Yes, I think I can. But how can I configure the token to be used, for example by jaeger all-in-one, to access the storage? I don't see any relevant configuration options.

YurySuslovScalyr · 2023-04-04T15:29:18Z

YurySuslovScalyr
Apr 4, 2023
Author

One more question, I am trying to connect Jaeger to the storage backend for testing purposes. But I get "Connection refused" errors from the jaeger-all-in-one container trying to connect to the storage. On the other hand, Postman GRPC request connects to storage just fine.

Here's a config I am using:

docker run \
  -e SPAN_STORAGE_TYPE=grpc-plugin \
  -e GRPC_STORAGE_SERVER="localhost:4774" \
  jaegertracing/all-in-one:1.43

Log output from Jaeger

2023/04/04 15:12:45 maxprocs: Leaving GOMAXPROCS=5: CPU quota undefined
{"level":"info","ts":1680621165.788154,"caller":"flags/service.go:119","msg":"Mounting metrics handler on admin server","route":"/metrics"}
{"level":"info","ts":1680621165.7882202,"caller":"flags/service.go:125","msg":"Mounting expvar handler on admin server","route":"/debug/vars"}
{"level":"info","ts":1680621165.78844,"caller":"flags/admin.go:129","msg":"Mounting health check on admin server","route":"/"}
{"level":"info","ts":1680621165.7884731,"caller":"flags/admin.go:143","msg":"Starting admin HTTP server","http-addr":":14269"}
{"level":"info","ts":1680621165.7884877,"caller":"flags/admin.go:121","msg":"Admin server started","http.host-port":"[::]:14269","health-status":"unavailable"}
{"level":"info","ts":1680621165.7912998,"caller":"channelz/funcs.go:340","msg":"[core][Channel #1] Channel created","system":"grpc","grpc_log":true}
{"level":"info","ts":1680621165.7913654,"caller":"channelz/funcs.go:340","msg":"[core][Channel #1] original dial target is: \"localhost:4774\"","system":"grpc","grpc_log":true}
{"level":"info","ts":1680621165.7913787,"caller":"channelz/funcs.go:340","msg":"[core][Channel #1] parsed dial target is: {Scheme:localhost Authority: URL:{Scheme:localhost Opaque:4774 User: Host: Path: RawPath: OmitHost:false ForceQuery:false RawQuery: Fragment: RawFragment:}}","system":"grpc","grpc_log":true}
{"level":"info","ts":1680621165.7913833,"caller":"channelz/funcs.go:340","msg":"[core][Channel #1] fallback to scheme \"passthrough\"","system":"grpc","grpc_log":true}
{"level":"info","ts":1680621165.791389,"caller":"channelz/funcs.go:340","msg":"[core][Channel #1] parsed dial target is: {Scheme:passthrough Authority: URL:{Scheme:passthrough Opaque: User: Host: Path:/localhost:4774 RawPath: OmitHost:false ForceQuery:false RawQuery: Fragment: RawFragment:}}","system":"grpc","grpc_log":true}
{"level":"info","ts":1680621165.7913933,"caller":"channelz/funcs.go:340","msg":"[core][Channel #1] Channel authority set to \"localhost:4774\"","system":"grpc","grpc_log":true}
{"level":"info","ts":1680621165.7914593,"caller":"channelz/funcs.go:340","msg":"[core][Channel #1] Resolver state updated: {\n  \"Addresses\": [\n    {\n      \"Addr\": \"localhost:4774\",\n      \"ServerName\": \"\",\n      \"Attributes\": null,\n      \"BalancerAttributes\": null,\n      \"Type\": 0,\n      \"Metadata\": null\n    }\n  ],\n  \"ServiceConfig\": null,\n  \"Attributes\": null\n} (resolver returned new addresses)","system":"grpc","grpc_log":true}
{"level":"info","ts":1680621165.7914803,"caller":"channelz/funcs.go:340","msg":"[core][Channel #1] Channel switches to new LB policy \"pick_first\"","system":"grpc","grpc_log":true}
{"level":"info","ts":1680621165.79149,"caller":"channelz/funcs.go:340","msg":"[core][Channel #1 SubChannel #2] Subchannel created","system":"grpc","grpc_log":true}
{"level":"info","ts":1680621165.7915692,"caller":"channelz/funcs.go:340","msg":"[core][Channel #1] Channel Connectivity change to CONNECTING","system":"grpc","grpc_log":true}
{"level":"info","ts":1680621165.791585,"caller":"channelz/funcs.go:340","msg":"[core][Channel #1 SubChannel #2] Subchannel Connectivity change to CONNECTING","system":"grpc","grpc_log":true}
{"level":"info","ts":1680621165.7915897,"caller":"channelz/funcs.go:340","msg":"[core][Channel #1 SubChannel #2] Subchannel picks a new address \"localhost:4774\" to connect","system":"grpc","grpc_log":true}
{"level":"info","ts":1680621165.7916536,"caller":"grpclog/component.go:71","msg":"[core]pickfirstBalancer: UpdateSubConnState: 0x4000128840, {CONNECTING }","system":"grpc","grpc_log":true}
{"level":"info","ts":1680621165.7919557,"caller":"grpclog/component.go:71","msg":"[core]Creating new client transport to \"{\\n  \\\"Addr\\\": \\\"localhost:4774\\\",\\n  \\\"ServerName\\\": \\\"localhost:4774\\\",\\n  \\\"Attributes\\\": null,\\n  \\\"BalancerAttributes\\\": null,\\n  \\\"Type\\\": 0,\\n  \\\"Metadata\\\": null\\n}\": connection error: desc = \"transport: Error while dialing: dial tcp 127.0.0.1:4774: connect: connection refused\"","system":"grpc","grpc_log":true}
{"level":"warn","ts":1680621165.791982,"caller":"channelz/funcs.go:342","msg":"[core][Channel #1 SubChannel #2] grpc: addrConn.createTransport failed to connect to {\n  \"Addr\": \"localhost:4774\",\n  \"ServerName\": \"localhost:4774\",\n  \"Attributes\": null,\n  \"BalancerAttributes\": null,\n  \"Type\": 0,\n  \"Metadata\": null\n}. Err: connection error: desc = \"transport: Error while dialing: dial tcp 127.0.0.1:4774: connect: connection refused\"","system":"grpc","grpc_log":true}
{"level":"info","ts":1680621165.7919986,"caller":"channelz/funcs.go:340","msg":"[core][Channel #1 SubChannel #2] Subchannel Connectivity change to TRANSIENT_FAILURE","system":"grpc","grpc_log":true}
{"level":"info","ts":1680621165.7920103,"caller":"grpclog/component.go:71","msg":"[core]pickfirstBalancer: UpdateSubConnState: 0x4000128840, {TRANSIENT_FAILURE connection error: desc = \"transport: Error while dialing: dial tcp 127.0.0.1:4774: connect: connection refused\"}","system":"grpc","grpc_log":true}
{"level":"info","ts":1680621165.7920146,"caller":"channelz/funcs.go:340","msg":"[core][Channel #1] Channel Connectivity change to TRANSIENT_FAILURE","system":"grpc","grpc_log":true}
{"level":"info","ts":1680621166.7930028,"caller":"channelz/funcs.go:340","msg":"[core][Channel #1 SubChannel #2] Subchannel Connectivity change to IDLE","system":"grpc","grpc_log":true}
{"level":"info","ts":1680621166.7934513,"caller":"grpclog/component.go:71","msg":"[core]pickfirstBalancer: UpdateSubConnState: 0x4000128840, {IDLE connection error: desc = \"transport: Error while dialing: dial tcp 127.0.0.1:4774: connect: connection refused\"}","system":"grpc","grpc_log":true}
{"level":"info","ts":1680621166.7934887,"caller":"channelz/funcs.go:340","msg":"[core][Channel #1] Channel Connectivity change to IDLE","system":"grpc","grpc_log":true}
{"level":"info","ts":1680621166.7936356,"caller":"channelz/funcs.go:340","msg":"[core][Channel #1 SubChannel #2] Subchannel Connectivity change to CONNECTING","system":"grpc","grpc_log":true}
{"level":"info","ts":1680621166.7936642,"caller":"channelz/funcs.go:340","msg":"[core][Channel #1 SubChannel #2] Subchannel picks a new address \"localhost:4774\" to connect","system":"grpc","grpc_log":true}
{"level":"info","ts":1680621166.7946947,"caller":"grpclog/component.go:71","msg":"[core]pickfirstBalancer: UpdateSubConnState: 0x4000128840, {CONNECTING }","system":"grpc","grpc_log":true}
{"level":"info","ts":1680621166.794715,"caller":"channelz/funcs.go:340","msg":"[core][Channel #1] Channel Connectivity change to CONNECTING","system":"grpc","grpc_log":true}
{"level":"info","ts":1680621166.7954063,"caller":"grpclog/component.go:71","msg":"[core]Creating new client transport to \"{\\n  \\\"Addr\\\": \\\"localhost:4774\\\",\\n  \\\"ServerName\\\": \\\"localhost:4774\\\",\\n  \\\"Attributes\\\": null,\\n  \\\"BalancerAttributes\\\": null,\\n  \\\"Type\\\": 0,\\n  \\\"Metadata\\\": null\\n}\": connection error: desc = \"transport: Error while dialing: dial tcp 127.0.0.1:4774: connect: connection refused\"","system":"grpc","grpc_log":true}
{"level":"warn","ts":1680621166.7954555,"caller":"channelz/funcs.go:342","msg":"[core][Channel #1 SubChannel #2] grpc: addrConn.createTransport failed to connect to {\n  \"Addr\": \"localhost:4774\",\n  \"ServerName\": \"localhost:4774\",\n  \"Attributes\": null,\n  \"BalancerAttributes\": null,\n  \"Type\": 0,\n  \"Metadata\": null\n}. Err: connection error: desc = \"transport: Error while dialing: dial tcp 127.0.0.1:4774: connect: connection refused\"","system":"grpc","grpc_log":true}
{"level":"info","ts":1680621166.7954907,"caller":"channelz/funcs.go:340","msg":"[core][Channel #1 SubChannel #2] Subchannel Connectivity change to TRANSIENT_FAILURE","system":"grpc","grpc_log":true}
{"level":"info","ts":1680621166.795683,"caller":"grpclog/component.go:71","msg":"[core]pickfirstBalancer: UpdateSubConnState: 0x4000128840, {TRANSIENT_FAILURE connection error: desc = \"transport: Error while dialing: dial tcp 127.0.0.1:4774: connect: connection refused\"}","system":"grpc","grpc_log":true}
{"level":"info","ts":1680621166.795705,"caller":"channelz/funcs.go:340","msg":"[core][Channel #1] Channel Connectivity change to TRANSIENT_FAILURE","system":"grpc","grpc_log":true}
{"level":"info","ts":1680621168.5843632,"caller":"channelz/funcs.go:340","msg":"[core][Channel #1 SubChannel #2] Subchannel Connectivity change to IDLE","system":"grpc","grpc_log":true}
{"level":"info","ts":1680621168.5847013,"caller":"grpclog/component.go:71","msg":"[core]pickfirstBalancer: UpdateSubConnState: 0x4000128840, {IDLE connection error: desc = \"transport: Error while dialing: dial tcp 127.0.0.1:4774: connect: connection refused\"}","system":"grpc","grpc_log":true}
{"level":"info","ts":1680621168.584748,"caller":"channelz/funcs.go:340","msg":"[core][Channel #1] Channel Connectivity change to IDLE","system":"grpc","grpc_log":true}
{"level":"info","ts":1680621168.584867,"caller":"channelz/funcs.go:340","msg":"[core][Channel #1 SubChannel #2] Subchannel Connectivity change to CONNECTING","system":"grpc","grpc_log":true}
{"level":"info","ts":1680621168.584943,"caller":"channelz/funcs.go:340","msg":"[core][Channel #1 SubChannel #2] Subchannel picks a new address \"localhost:4774\" to connect","system":"grpc","grpc_log":true}
{"level":"info","ts":1680621168.5869415,"caller":"grpclog/component.go:71","msg":"[core]pickfirstBalancer: UpdateSubConnState: 0x4000128840, {CONNECTING }","system":"grpc","grpc_log":true}
{"level":"info","ts":1680621168.586981,"caller":"channelz/funcs.go:340","msg":"[core][Channel #1] Channel Connectivity change to CONNECTING","system":"grpc","grpc_log":true}
{"level":"info","ts":1680621168.587721,"caller":"grpclog/component.go:71","msg":"[core]Creating new client transport to \"{\\n  \\\"Addr\\\": \\\"localhost:4774\\\",\\n  \\\"ServerName\\\": \\\"localhost:4774\\\",\\n  \\\"Attributes\\\": null,\\n  \\\"BalancerAttributes\\\": null,\\n  \\\"Type\\\": 0,\\n  \\\"Metadata\\\": null\\n}\": connection error: desc = \"transport: Error while dialing: dial tcp 127.0.0.1:4774: connect: connection refused\"","system":"grpc","grpc_log":true}
{"level":"warn","ts":1680621168.5879736,"caller":"channelz/funcs.go:342","msg":"[core][Channel #1 SubChannel #2] grpc: addrConn.createTransport failed to connect to {\n  \"Addr\": \"localhost:4774\",\n  \"ServerName\": \"localhost:4774\",\n  \"Attributes\": null,\n  \"BalancerAttributes\": null,\n  \"Type\": 0,\n  \"Metadata\": null\n}. Err: connection error: desc = \"transport: Error while dialing: dial tcp 127.0.0.1:4774: connect: connection refused\"","system":"grpc","grpc_log":true}
{"level":"info","ts":1680621168.588017,"caller":"channelz/funcs.go:340","msg":"[core][Channel #1 SubChannel #2] Subchannel Connectivity change to TRANSIENT_FAILURE","system":"grpc","grpc_log":true}
{"level":"info","ts":1680621168.588053,"caller":"grpclog/component.go:71","msg":"[core]pickfirstBalancer: UpdateSubConnState: 0x4000128840, {TRANSIENT_FAILURE connection error: desc = \"transport: Error while dialing: dial tcp 127.0.0.1:4774: connect: connection refused\"}","system":"grpc","grpc_log":true}
{"level":"info","ts":1680621168.5880938,"caller":"channelz/funcs.go:340","msg":"[core][Channel #1] Channel Connectivity change to TRANSIENT_FAILURE","system":"grpc","grpc_log":true}
{"level":"info","ts":1680621170.7927165,"caller":"channelz/funcs.go:340","msg":"[core][Channel #1] Channel Connectivity change to SHUTDOWN","system":"grpc","grpc_log":true}
{"level":"info","ts":1680621170.7929137,"caller":"channelz/funcs.go:340","msg":"[core][Channel #1 SubChannel #2] Subchannel Connectivity change to SHUTDOWN","system":"grpc","grpc_log":true}
{"level":"info","ts":1680621170.7930245,"caller":"channelz/funcs.go:340","msg":"[core][Channel #1 SubChannel #2] Subchannel deleted","system":"grpc","grpc_log":true}
{"level":"info","ts":1680621170.7930362,"caller":"channelz/funcs.go:340","msg":"[core][Channel #1] Channel deleted","system":"grpc","grpc_log":true}
{"level":"fatal","ts":1680621170.7930896,"caller":"./main.go:109","msg":"Failed to init storage factory","error":"grpc-plugin builder failed to create a store: error connecting to remote storage: context deadline exceeded","stacktrace":"main.main.func1\n\t./main.go:109\ngithub.com/spf13/cobra.(*Command).execute\n\tgithub.com/spf13/[email protected]/command.go:916\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\tgithub.com/spf13/[email protected]/command.go:1044\ngithub.com/spf13/cobra.(*Command).Execute\n\tgithub.com/spf13/[email protected]/command.go:968\nmain.main\n\t./main.go:240\nruntime.main\n\truntime/proc.go:250"}

1 reply

YurySuslovScalyr Apr 4, 2023
Author

Figured this out, need to use a different host for connecting to the host system.

Itchimonji · 2024-06-03T22:52:00Z

Itchimonji
Jun 3, 2024

Hey,
how do you support grafana-mimir as SPM storage, when you do not support headers? Is there a workaround for this?
When you want to query grafana-mimir for SPM Metrics, you need to add a X-Scope-OrgID header with a tenant. Without this, I can't use the SPM feature of Jaeger :( .

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Jaeger - Distributed Tracing Platform

Additional GRPC headers #4357

{{title}}

Replies: 3 comments 2 replies

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

Select a reply

Jaeger - Distributed Tracing Platform

Additional GRPC headers #4357

YurySuslovScalyr Mar 31, 2023

Replies: 3 comments · 2 replies

yurishkuro Mar 31, 2023 Maintainer

YurySuslovScalyr Apr 4, 2023 Author

YurySuslovScalyr Apr 4, 2023 Author

YurySuslovScalyr Apr 4, 2023 Author

Itchimonji Jun 3, 2024

YurySuslovScalyr
Mar 31, 2023

Replies: 3 comments 2 replies

yurishkuro
Mar 31, 2023
Maintainer

YurySuslovScalyr Apr 4, 2023
Author

YurySuslovScalyr
Apr 4, 2023
Author

YurySuslovScalyr Apr 4, 2023
Author

Itchimonji
Jun 3, 2024