Blog post for OpenTelemetry Generative AI updates

.cspell/en-words.txt

@@ -144,3 +144,6 @@ wordpress
 WSGI
 zend
 zipkin
+Liudmila
+Molkova
+GENAI

content/en/blog/2024/otel-generative-ai/index.md

@@ -0,0 +1,163 @@
+---
+title: OpenTelemetry for Generative AI
+linkTitle: OpenTelemetry for Generative AI
+date: 2024-11-09
+author: >-
+  [Drew Robbins](https://github.com/drewby) (Microsoft),  [Liudmila
+  Molkova](https://github.com/lmolkova) (Microsoft)
+issue: [#5581](https://github.com/open-telemetry/opentelemetry.io/issues/5581)
+sig: SIG GenAI Observability
+---
+
+As organizations increasingly adopt Large Language Models (LLMs) and other
+generative AI technologies, ensuring reliable performance, efficiency, and
+safety is essential to meet user expectations, optimize resource costs, and
+safeguard against unintended outputs. Effective observability for AI operations,
+behaviors, and outcomes can help meet these goals. OpenTelemetry is being
+enhanced to support these needs specifically for generative AI.
+
+Two primary assets are in development to make this possible: **Semantic
+Conventions** and an **Instrumentation Library**. The first instrumentation library targets OpenAI in Python.
+
+[**Semantic Conventions**](https://opentelemetry.io/docs/concepts/semantic-conventions/)
+establish standardized guidelines for how telemetry data is structured and
+collected across platforms, defining inputs, outputs, and operational details.
+For generative AI, these conventions streamline monitoring, troubleshooting, and
+optimizing AI models by standardizing attributes such as model parameters,
+response metadata, and token usage. This consistency supports better
+observability across tools, environments, and APIs, helping organizations track
+performance, cost, and safety with ease.
+
+The
+[**Instrumentation Library**](https://opentelemetry.io/docs/specs/otel/overview/#instrumentation-libraries)
+is being developed within the
+[OpenTelemetry Python Contrib](https://github.com/open-telemetry/opentelemetry-python-contrib) under [instrumentation-genai](https://github.com/open-telemetry/opentelemetry-python-contrib/tree/main/instrumentation-genai)
+project to automate telemetry collection for generative AI applications. The
+first release is a Python library for instrumenting OpenAI client calls, given
+Python's widespread use in AI development and the popularity of OpenAI. Designed
+to integrate seamlessly with OpenAI's API, this library captures spans and
+events, gathering essential data like model inputs, response metadata, and token
+usage in a structured format.
+
+## Key Signals for Generative AI
+
+The
+[Semantic Conventions for Generative AI](https://github.com/open-telemetry/semantic-conventions/tree/v1.28.0/docs/gen-ai)
+focus on capturing insights into AI model behavior through three primary
+signals: [Traces](https://opentelemetry.io/docs/concepts/signals/traces/),
+[Metrics](https://opentelemetry.io/docs/concepts/signals/metrics/), and
+[Events](https://opentelemetry.io/docs/specs/otel/logs/event-api/).
+
+Together, these signals provide a comprehensive monitoring framework, enabling
+better cost management, performance tuning, and request tracing.
+
+### Traces: Tracing Model Interactions
+
+Traces track each model interaction’s lifecycle, covering input parameters (for
+example, temperature, top_p) and response details like token count or errors.
+They provide visibility into each request, aiding in identifying bottlenecks and
+analyzing the impact of settings on model output.
+
+### Metrics: Monitoring Usage and Performance
+
+Metrics aggregate high-level indicators like request volume, latency, and token
+counts, essential for managing costs and performance. This data is particularly
+critical for API-dependent AI applications with rate limits and cost
+considerations.
+
+### Events: Capturing Detailed Interactions
+
+Events log detailed moments during model execution, such as user prompts and
+model responses, providing a granular view of model interactions. These insights
+are invaluable for debugging and optimizing AI applications where unexpected
+behaviors may arise.
+
+{{% alert title="Note" color="info" %}} Note that we decided to use the
+newer Events API (https://opentelemetry.io/docs/specs/otel/logs/event-api/)
+specification in the Semantic Conventions for Generative AI. The events API
+allows for us to define specific
+[semantic conventions](https://opentelemetry.io/docs/specs/semconv/general/events/)
+for the user prompts and model responses that we capture. {{% /alert %}}
+
+### Extending Observability with Vendor-Specific Attributes
+
+The Semantic Conventions also define vendor-specific attributes for platforms
+like OpenAI and Azure Inference API, ensuring telemetry captures both general
+and provider-specific details. This added flexibility supports multi-platform
+monitoring and in-depth insights.
+
+## Building the Python Instrumentation Library for OpenAI
+
+This Python-based library for OpenTelemetry captures key telemetry signals for
+OpenAI models, providing developers with an out-of-the-box observability
+solution tailored to AI workloads. The library,
+[hosted within the OpenTelemetry Python Contrib repository](https://github.com/open-telemetry/opentelemetry-python-contrib/tree/opentelemetry-instrumentation-openai-v2%3D%3D2.0b0/instrumentation-genai/opentelemetry-instrumentation-openai-v2),
+automatically collects telemetry from OpenAI model interactions, including
+request and response metadata and token usage.
+
+As generative AI applications grow, additional instrumentation libraries for
+other languages will follow, extending OpenTelemetry support across more tools
+and environments. The current library’s focus on OpenAI highlights its
+popularity and demand within AI development, making it a valuable initial
+implementation.
+
+### Example Usage
+
+Here’s an example of using the OpenTelemetry Python library to monitor a
+generative AI application with the OpenAI client. Make sure you first install
+the library:
+
+```bash
+pip install opentelemetry-instrumentation-openai-v2
+```
+
+Then include the following code in your Python application:
+
+```python
+from openai import OpenAI
+from opentelemetry.instrumentation.openai_v2 import OpenAIInstrumentor
+
+OpenAIInstrumentor().instrument()
+
+client = OpenAI()
+response = client.chat.completions.create(
+    model="gpt-4-mini",
+    messages=[{"role": "user", "content": "Write a short poem on OpenTelemetry."}],
+)
+
+# The library captures telemetry, including request and response metadata, token usage, and more.
+```
+
+With this simple instrumentation, one can begin capture traces from their
+generative AI application. Here is an example from the
+[Aspire Dashboard](https://learn.microsoft.com/dotnet/aspire/fundamentals/dashboard/standalone?tabs=bash)
+for local debugging.
+
+![Chat trace in Aspire Dashboard](aspire-dashboard-trace.png)
+
+Here is a similar trace captured in
+[Jaeger](https://www.jaegertracing.io/docs/next-release-v2/getting-started/#running):
+
+![Chat trace in Jaeger](jaeger-trace.png)
+
+It's also easy to capture the content history of the chat for debugging and
+improving your application. Simply set the environment variable
+`OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT` as follows:
+
+```bash
+export OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=True
+```
+
+This will turn on content capture which collects OpenTelemetry events containing
+the payload:
+
+![Content Capture Aspire Dashboard](aspire-dashboard-content-capture.png)
+
+## Join Us in Shaping the Future of Generative AI Observability
+
+Community collaboration is key to OpenTelemetry’s success. We invite developers,
+AI practitioners, and organizations to contribute, share feedback, or
+participate in discussions. Explore the OpenTelemetry Python Contrib project,
+contribute code, or help shape observability for AI as it continues to evolve.
+More information can be found at the
+[Generative AI Observability project page](https://github.com/open-telemetry/community/blob/main/projects/gen-ai.md).