[receiver/awsxrayreceiver] Add option to pull x-ray traces from AWS

[Simon-Boyer]

Component(s)

receiver/awsxray

Is your feature request related to a problem? Please describe.

I am trying to pull traces directly from AWS, so that I do not have to setup layers or other configs for all my lambda functions. From my understanding, the x-ray receiver can only receive and process x-ray traces, but it does not actively pull them from AWS.

Describe the solution you'd like

An option that can enable a "pulling" mode: instead of passing the x-ray traces to the receiver, the receiver pulls data directly from AWS every X interval. Could look something like this:

receivers:
  awsxray:
    pull:
      enabled: true
      interval: 30s
    endpoint: 0.0.0.0:2000
    transport: udp
    ...

Describe alternatives you've considered

Writing a separate binary that pulls data from X-Ray and forwards it to the current receiver. Would work, but I think it makes more sense to have it in the receiver.

Additional context

Basically trying to reproduce this datadog integration: https://docs.datadoghq.com/integrations/amazon_xray/

[github-actions]

Pinging code owners:

• receiver/awsxray: @wangzlei @srprash

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[crobert-1]

so that I do not have to setup layers or other configs for all my lambda functions

I'm interested in hearing about the complexity of this, as I'm unfamiliar. What all is involved in setting this up? Are there performance implications here?

Does AWS x-ray expose a way to pull traces from it?

The code owners would have much more context and information here, so they may have better questions and thoughts here.

[Simon-Boyer]

What all is involved in setting this up?

I havent looked in details, but i'm guessing simply querying the API endpoint

Are there performance implications here?

Probably, that's why you would change the pull rate according to your needs

Does AWS x-ray expose a way to pull traces from it?

Looks like yes: https://docs.aws.amazon.com/xray/latest/devguide/xray-api-gettingdata.html#xray-api-traces

[crobert-1]

What all is involved in setting this up?

I havent looked in details, but i'm guessing simply querying the API endpoint

I'm wondering how complicated it is to have AWS x-ray export traces to the collector. It sounds like, from what you're saying, it's some amount of work to accomplish this, so I'm wondering what all is involved in making this happen with the collector's existing functionality? Sorry for the confusion there, I didn't word my question very clearly.

[Simon-Boyer]

I'm not sure if I fully understand the question, but I don't think it is really difficult to have AWS exports to the collector.

If we were to use the collector existing functionality, I think we could write something like a small script/program on a cron job that scrapes the last traces on the AWS X-Ray API endpoint, then export them (without changing the format) to an OTEL collector with the xray receiver configured.

But I believe having that feature available, builtin, into the receiver would make sense instead of having it completely separate. As an example, the cloudwatch receiver (for logs) does offers polling data instead of simply receiving it: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/awscloudwatchreceiver

[crobert-1]

That's fair, a cron job is definitely not ideal. I'll defer to code owners for better insights into possibly adding this 👍

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• receiver/awsxray: @wangzlei @srprash

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• receiver/awsxray: @wangzlei @srprash

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[github-actions]

This issue has been closed as inactive because it has been stale for 120 days with no activity.

[Mjb141]

@Simon-Boyer Did you end up writing something to scrape X-Ray and send the traces to the receiver? I've just found this after using the Cloudwatch receiver to scrape logs only to find the X-Ray receiver can't scrape...

[Simon-Boyer]

Unfortunatly no. I was documenting OTEL support for a client use cases and I explained the scraper idea but because of this and some other unsupported stuff they decided to not go forward with OTEL.

[jholm117]

@Mjb141 I recently put together a script for this with a helm chart for my own purposes. The script will scrape X-ray for traces and forward them to an X-ray receiver. It doesn't have much in the way of documentation at the moment but maybe you'll find it useful and I'm happy to answer any questions or take feedback.

https://github.com/jholm117/awsxray-exporter

[Mjb141]

All good, thanks anyway - got what we needed from https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/awsxrayreceiver/receiver_test.go#L101-L104 and just wrote a quick and dirty script to do it.