diff --git a/bootcamp/tutorials/integration/evaluation_with_phoenix.ipynb b/bootcamp/tutorials/integration/evaluation_with_phoenix.ipynb index ef249ff0b..7716af7d1 100644 --- a/bootcamp/tutorials/integration/evaluation_with_phoenix.ipynb +++ b/bootcamp/tutorials/integration/evaluation_with_phoenix.ipynb @@ -444,16 +444,65 @@ " - **QA Explanation**: Details why an answer is correct or incorrect." ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Phoenix Tracing Overview\n", + "\n", + "Phoenix provides **OTEL-compatible tracing** for LLM applications, with integrations for frameworks like **Langchain**, **LlamaIndex**, and SDKs such as **OpenAI** and **Mistral**. Tracing captures the entire request flow, offering insights into:\n", + "\n", + "- **Application Latency**: Identify and optimize slow LLM invocations and component performance.\n", + "- **Token Usage**: Break down token consumption for cost optimization.\n", + "- **Runtime Exceptions**: Capture critical issues like rate-limiting.\n", + "- **Retrieved Documents**: Analyze document retrieval, score, and order.\n", + "\n", + "By utilizing Phoenix’s tracing, you can **identify bottlenecks**, **optimize resources**, and **ensure system reliability** across various frameworks and languages." + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "WARNI [phoenix.session.session] Existing running Phoenix instance detected! Shutting it down and starting a new instance...\n", + "WARNI [opentelemetry.instrumentation.instrumentor] Attempting to instrument while already instrumented\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "🌍 To view the Phoenix app in your browser, visit http://localhost:6006/\n", + "πŸ“– For more information on how to use Phoenix, check out https://docs.arize.com/phoenix\n" + ] + } + ], + "source": [ + "import phoenix as px\n", + "from phoenix.trace.openai import OpenAIInstrumentor\n", + "\n", + "# To view traces in Phoenix, you will first have to start a Phoenix server. You can do this by running the following:\n", + "session = px.launch_app()\n", + "\n", + "# Initialize OpenAI auto-instrumentation\n", + "OpenAIInstrumentor().instrument()" + ] + }, { "cell_type": "code", - "execution_count": 7, + "execution_count": 11, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ - "run_evals |β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 6/6 (100.0%) | ⏳ 00:04<00:00 | 1.38it/s\n" + "run_evals |β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 6/6 (100.0%) | ⏳ 00:04<00:00 | 1.46it/s\n" ] } ],