Integration with Pydantic AI

[nguyenth-sieuviet]

Describe the feature or potential improvement

I am using Pydantic AI for orchestrating LLM workflows. I love it.
However, it supports primarily Pydantic Logfire, which is an monitoring platform also developed by the Pydantic team.
Unfortunately, it lacks the intuitive trace analysis and dashboard that LangFuse is good good at.

Since Pydantic is so popular for Structured Output, and Pydantic AI is also gaining momentum, I wonder if LangFuse has plan to develop integration components for it?

Additional information

Pydantic AI's section on Debugging and Monitoring:
https://ai.pydantic.dev/logfire/#using-logfire

[marcklingen]

Thanks for sharing! On the documentation page of PydanticAI only their commercial product is highlighted. Are you aware of any open instrumentation hooks here? Otherwise this might be a limitation of PydanticAI.

[mstrathman]

I believe you can pass in the langfuse client when creating the model, then pass that model into the agent. This should allow for at least basic compatibility.

For example:

from langfuse.openai import AsyncOpenAI

# Initialize the OpenAI client for asynchronous operations
client = AsyncOpenAI()

# Initialize the OpenAI model with the specified version
model = OpenAIModel("gpt-4o", openai_client=client)

# create agent with model
agent = Agent(
    model=model
)

[marcklingen]

Update: Pydantic AI supports instrumentation via OpenTelemetry via the logfire sdk according to this page

Since Logfire is build on OpenTelemetry, you can use the Logfire Python SDK to send data to any OpenTelemetry collector.

I was not aware that the logfire instrumentation for Pydantic AI allows for this as the documentation is focused on the Logfire service.

Adding an OpenTelemetry collector to Langfuse is planned, this should solve for automated tracing of Pydantic AI in Langfuse. We track this here: https://github.com/orgs/langfuse/discussions/2509