Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(llm-example): updated test files #4032

Merged
merged 1 commit into from
Sep 30, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion examples/quick-start-llm-python/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ opentelemetry-bootstrap -a install

# add openai api key
echo "OPENAI_API_KEY={your-open-ai-api-key}" >> .env
# add google gemini api key
# add google gemini api key (optional)
echo "GOOGLE_API_KEY={your-google-gemini-api-key}" >> .env

# add tracetest agent keys
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,16 @@

from langchain_google_genai import ChatGoogleGenerativeAI

import os

class GoogleGeminiProvider:
def provider_name(self):
return "Google (Gemini)"

def enabled(self):
gemini_api_key = os.getenv("GOOGLE_API_KEY", "")
return gemini_api_key != ""

def summarize(self, text):
chat = ChatGoogleGenerativeAI(model="gemini-pro")

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,16 @@

from langchain_openai import ChatOpenAI

import streamlit as st
import os

class OpenAIChatGPTProvider:
def provider_name(self):
return "OpenAI (ChatGPT)"

def enabled(self):
openai_api_key = os.getenv("OPENAI_API_KEY", "")
return openai_api_key != ""

def summarize(self, text):
# Get OpenAI API key and URL to be summarized
openai_api_key = os.getenv("OPENAI_API_KEY", "")
Expand Down
8 changes: 7 additions & 1 deletion examples/quick-start-llm-python/app/llm/providers.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,13 @@
]

def get_providers():
return list(map(lambda p: p.provider_name(), _providers))
providers = []

for provider in _providers:
if provider.enabled():
providers.append(provider.provider_name())

return providers

def get_provider(provider_name):
for provider in _providers:
Expand Down
4 changes: 1 addition & 3 deletions examples/quick-start-llm-python/docker-compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -53,9 +53,7 @@ services:
ports:
- 4317:4317
volumes:
- type: bind
source: ./observability/otelcollector.config.yaml
target: /otel-local-config.yaml
- ./observability/otelcollector.config.yaml:/otel-local-config.yaml

jaeger:
healthcheck:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,30 +1,10 @@
// @ts-check
const { test, expect } = require('@playwright/test');

const geminiTraceBasedTest = require('./definitions/gemini');
const chatgptTraceBasedTest = require('./definitions/chatgpt');

const { runTracebasedTest } = require('./tracetest');

test('generated summarized test for Gemini', async ({ request }) => {
const result = await request.post(`http://localhost:8800/summarizeText`, {
data: {
provider: "Google (Gemini)",
text: "Born in London, Turing was raised in southern England. He graduated from King's College, Cambridge, and in 1938, earned a doctorate degree from Princeton University. During World War II, Turing worked for the Government Code and Cypher School at Bletchley Park, Britain's codebreaking centre that produced Ultra intelligence. He led Hut 8, the section responsible for German naval cryptanalysis. Turing devised techniques for speeding the breaking of German ciphers, including improvements to the pre-war Polish bomba method, an electromechanical machine that could find settings for the Enigma machine. He played a crucial role in cracking intercepted messages that enabled the Allies to defeat the Axis powers in many crucial engagements, including the Battle of the Atlantic.\n\nAfter the war, Turing worked at the National Physical Laboratory, where he designed the Automatic Computing Engine, one of the first designs for a stored-program computer. In 1948, Turing joined Max Newman's Computing Machine Laboratory at the Victoria University of Manchester, where he helped develop the Manchester computers[12] and became interested in mathematical biology. Turing wrote on the chemical basis of morphogenesis and predicted oscillating chemical reactions such as the Belousov–Zhabotinsky reaction, first observed in the 1960s. Despite these accomplishments, he was never fully recognised during his lifetime because much of his work was covered by the Official Secrets Act."
}
});

const jsonResult = await result.json();
expect(jsonResult).not.toBe(null);
expect(jsonResult.summary).not.toBe(null);

const traceID = jsonResult.trace_id;
expect(traceID).not.toBe(null);

// run trace-based test
await runTracebasedTest(geminiTraceBasedTest, traceID);
});

test('generated summarized test for OpenAI', async ({ request }) => {
const result = await request.post(`http://localhost:8800/summarizeText`, {
data: {
Expand Down
25 changes: 25 additions & 0 deletions examples/quick-start-llm-python/tests/e2e/gemini.api.spec.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
// @ts-check
const { test, expect } = require('@playwright/test');

const geminiTraceBasedTest = require('./definitions/gemini');

const { runTracebasedTest } = require('./tracetest');

test('generated summarized test for Gemini', async ({ request }) => {
const result = await request.post(`http://localhost:8800/summarizeText`, {
data: {
provider: "Google (Gemini)",
text: "Born in London, Turing was raised in southern England. He graduated from King's College, Cambridge, and in 1938, earned a doctorate degree from Princeton University. During World War II, Turing worked for the Government Code and Cypher School at Bletchley Park, Britain's codebreaking centre that produced Ultra intelligence. He led Hut 8, the section responsible for German naval cryptanalysis. Turing devised techniques for speeding the breaking of German ciphers, including improvements to the pre-war Polish bomba method, an electromechanical machine that could find settings for the Enigma machine. He played a crucial role in cracking intercepted messages that enabled the Allies to defeat the Axis powers in many crucial engagements, including the Battle of the Atlantic.\n\nAfter the war, Turing worked at the National Physical Laboratory, where he designed the Automatic Computing Engine, one of the first designs for a stored-program computer. In 1948, Turing joined Max Newman's Computing Machine Laboratory at the Victoria University of Manchester, where he helped develop the Manchester computers[12] and became interested in mathematical biology. Turing wrote on the chemical basis of morphogenesis and predicted oscillating chemical reactions such as the Belousov–Zhabotinsky reaction, first observed in the 1960s. Despite these accomplishments, he was never fully recognised during his lifetime because much of his work was covered by the Official Secrets Act."
}
});

const jsonResult = await result.json();
expect(jsonResult).not.toBe(null);
expect(jsonResult.summary).not.toBe(null);

const traceID = jsonResult.trace_id;
expect(traceID).not.toBe(null);

// run trace-based test
await runTracebasedTest(geminiTraceBasedTest, traceID);
});
67 changes: 0 additions & 67 deletions examples/quick-start-llm-python/tests/e2e/ui.spec.js

This file was deleted.

3 changes: 1 addition & 2 deletions examples/quick-start-llm-python/tests/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,7 @@
"main": "index.js",
"scripts": {
"record-test": "npx playwright codegen localhost:8501",
"api-tests": "npx playwright test **/*api.spec.js",
"ui-tests": "npx playwright test **/*ui.spec.js"
"api-tests": "npx playwright test **/*api.spec.js"
},
"keywords": [],
"author": "",
Expand Down
Loading