Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Performance test: trace-ingestion load generator #983

Closed
wants to merge 8 commits into from
Closed

Performance test: trace-ingestion load generator #983

wants to merge 8 commits into from

Conversation

chenqi0805
Copy link
Collaborator

@chenqi0805 chenqi0805 commented Feb 7, 2022 •
edited
Loading

Description

This PR

  • adds TraceTemplates for generating otel-proto compatble http/json request. The sample data schema is as follows
    • Spans in each traceId is of size batchSize * # of endpoints
    • Each trace group is splitted and assigned evenly across endpoints
  • adds TraceChain to send otel-proto compatble http/json requests according to the above.
  • creates slowburn and fixed-clients simulation in analogy with log ingestion

Note: manually tested with the following pipelines.yaml:

entry-pipeline:
  source:
    otel_trace_source:
      ssl: false
      unframed_requests: true
  sink:
    - pipeline:
        name: "raw-pipeline"
raw-pipeline:
  source:
    pipeline:
      name: "entry-pipeline"
  prepper:
    - otel_trace_raw_prepper:
        root_span_flush_delay: 1
        trace_flush_interval: 5
    - otel_trace_group_prepper:
        hosts: [ "https://node-0.example.com:9200" ]
        username: "admin"
        password: "admin"
  sink:
    - opensearch:
        hosts: [ "https://node-0.example.com:9200" ]
        username: "admin"
        password: "admin"
        trace_analytics_raw: true

Issues Resolved

Resolves #940

Check List

  • New functionality includes testing.
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed with a real name per the DCO

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@chenqi0805 chenqi0805 requested a review from a team as a code owner February 7, 2022 15:47
cmanning09
cmanning09 reviewed Feb 7, 2022
View reviewed changes
...src/gatling/java/org/opensearch/dataprepper/test/performance/FixedClientTraceSimulation.java
.during(duration)
.on(TraceChain.sendUnframedExportTraceServiceRequest(endpoints, batchSize));

{
Copy link
Contributor

@cmanning09 cmanning09 Feb 7, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noob question, why are we using a block?

...st/src/gatling/java/org/opensearch/dataprepper/test/performance/SlowBurnTraceSimulation.java
import java.util.Arrays;
import java.util.List;

public class SlowBurnTraceSimulation extends Simulation {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The naming choice is a little confusing. FixedClientTraceSimulation and SlowBurnTraceSimulation use Slow Burn scenario. Yet the difference is fixed and ramping clients. I might recommend FixedClientTraceSimulation and RampUpClientTraceSimulation

...ance-test/src/gatling/java/org/opensearch/dataprepper/test/performance/tools/TraceChain.java

import java.util.List;

public class TraceChain {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's call this: TraceChainBuilder.

If we are introducing another Chain, let's rename the existing one to LogChainBuilder

...ance-test/src/gatling/java/org/opensearch/dataprepper/test/performance/tools/TraceChain.java

public class TraceChain {
private static final JsonFormat.Printer PRINTER = JsonFormat.printer().omittingInsignificantWhitespace();

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's create a private constructor here:

private TraceChainBuilder() {}

...-test/src/gatling/java/org/opensearch/dataprepper/test/performance/tools/TraceTemplates.java
@@ -0,0 +1,135 @@
/*
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you run the formatter on this code? I am noticing some spacing patterns that are making it difficult to read.

...-test/src/gatling/java/org/opensearch/dataprepper/test/performance/tools/TraceTemplates.java
for (int i=0; i<peerListSize*batchSize; i+=batchSize) {
results.add(getExportTraceServiceRequest(resourceSpansList.subList(i, i+batchSize)));
}
Collections.shuffle(results);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Won't this impact reproducibility with the workload?

...-test/src/gatling/java/org/opensearch/dataprepper/test/performance/tools/TraceTemplates.java
}).collect(Collectors.toList());
}

public static List<ExportTraceServiceRequest> getExportTraceServiceRequests(final int peerListSize, final int batchSize) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do any of the static functions below need to be pubic?

@chenqi0805 chenqi0805 closed this Sep 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@cmanning09 cmanning09 cmanning09 left review comments

@sbayer55 sbayer55 Awaiting requested review from sbayer55

@graytaylor0 graytaylor0 Awaiting requested review from graytaylor0 graytaylor0 is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Load generator for trace ingestion
2 participants
@chenqi0805 @cmanning09