Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Attach exceptions to spans in error handlers for better Observability #2782

Open
michalcukierman opened this issue Oct 3, 2024 · 0 comments

Comments

@michalcukierman
Copy link
Contributor

michalcukierman commented Oct 3, 2024

It appears that attaching exceptions to spans in error handlers could be implemented relatively easily, improving the observability of failures.

The approach taken in the Quarkus handler (AttachExceptionHandler) uses the following line to record exceptions:

LocalRootSpan.current().recordException(throwable);

We could adopt a similar approach in PulsarIncomingChannel to enhance traceability. For example, modifying the method here: smallrye-reactive-messaging-pulsar/PulsarIncomingChannel.java#L199:

public synchronized void reportFailure(Throwable failure, boolean fatal) {
    // Don't keep all the failures, only keep them for reporting.
    if (failures.size() == 10) {
        failures.remove(0);
    }
    failures.add(failure);
    
    // Attach the exception to the current span for observability
    LocalRootSpan.current().recordException(failure);

    if (fatal) {
        close();
    }
}

Adding this simple modification would provide a more comprehensive view of exceptions for observability tools like OpenTelemetry, Jaeger, or similar, enabling better tracking and debugging of issues across the system.

I can create the PR, if you think that the investigation is correct.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant