jaeger traces/spans is not able to export to kafka as json format

[edificeinsightworksapp]

jaeger spans is not able to export to kafka as json format, We have elasticsearch span storare type here traces from jaeger collector is being sent in json format with es cli flags defined fields. We need to send it to kafka and from kafka to Elastisearch. Below is my jaeger collector config file. I am getting data in otlp format only and not json format for traces.

Request to provide a solution how can we use es.fields CLI flags and send to Kafka or send traces in json format to kafka queue?

receivers:
  # ...
  otlp:
    protocols:
      grpc:

processors:
  resource:
    attributes:
    - key: process.executable.path
      action: delete
  filter:
    error_mode: ignore
    traces:
      span:
        - 'attributes["process.serviceName"] == "TRADE_SINGLETON"'
        - 'attributes["process.service.name"] == "TRADE_SINGLETON"'
        - 'attributes["serviceName"] == "TRADE_SINGLETON"'
        - 'attributes["service.name"] == "TRADE_SINGLETON"'
        - 'resource.attributes["serviceName"] == "TRADE_SINGLETON"'
        - 'resource.attributes["service.name"] == "TRADE_SINGLETON"'
        - 'resource.attributes["process.service.name"] == "TRADE_SINGLETON"'
        - 'resource.attributes["process.serviceName"] == "TRADE_SINGLETON"'
        - 'serviceName == "TRADE_SINGLETON"'
        - 'process.serviceName == "TRADE_SINGLETON"'
    spans:
      exclude:
        match_type: regexp
        span_names: CreateWork
  memory_limiter:
    check_interval: 1s
    limit_mib: 2000
  batch:

exporters:
  debug:
    verbosity: detailed
  #jaeger_elasticsearch:
  #  endpoint: "http://asert:9200"
   # tls:
    #  insecure: true
  kafka:
    brokers:
      - localhost:9092
    protocol_version: 2.0.0
    encoding: jaeger_json
  logging:
    verbosity: detailed

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [resource,filter]
      #exporters: [debug, jaeger_elasticsearch]
      exporters: [debug, kafka, logging]
  telemetry:
    logs:
      level: "debug"
      encoding: "json"
      development: true

Thanks

[yurishkuro]

Below is my jaeger collector config file

this is OTEL collector, not Jaeger.

[yurishkuro]

As I already said, the config file you have is not Jaeger-compatible in any way, its an OTEL Collector config. Jaeger doesn't complain about it because unlike with CLI flags there is no validation in the library we use for the syntax of the config. So all the entries in your config are completely ignored.

[edificeinsightworksapp]

Thanks a lot for your inputs. Will update to jaeger config and check

[edificeinsightworksapp]

Hello, I am unable to find jaeger config file template to enable custom processing and output formats definitions. Pls do let me know how can we use Open telemetry type of config in jaeger collector?

[yurishkuro]

The primary configuration mechanism for Jaeger is CLI flags and env vars. https://www.jaegertracing.io/docs/1.58/cli/

You can translate those into corresponding YAML config by creating sections at . separators in the property names.

[yurishkuro]

@dosu-bot