Skip to content

Commit

Permalink
allow script deperndencies in test step summary prompt
Browse files Browse the repository at this point in the history
  • Loading branch information
aluedeke committed Feb 26, 2025
1 parent dc67c26 commit 9acbfe0
Show file tree
Hide file tree
Showing 2 changed files with 7 additions and 2 deletions.
1 change: 1 addition & 0 deletions src/prompts/checkly.eval.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -473,6 +473,7 @@ test('visit page and take screenshot', async ({ page }) => {
name,
scriptPath,
script,
[],
errors,
);
const { text: summary } = await generateText({
Expand Down
8 changes: 6 additions & 2 deletions src/prompts/checkly.ts
Original file line number Diff line number Diff line change
Expand Up @@ -213,7 +213,8 @@ export function clusterCheckResults(
export function summarizeTestStepsPrompt(
testName: string,
scriptName: string,
script: string,
scriptPath: string,
dependencies: { script: string; scriptPath: string }[],
errors: string[],
): [string, PromptConfig] {
return [
Expand All @@ -223,9 +224,12 @@ export function summarizeTestStepsPrompt(
Do not refer to technical details of the test, use the domain language from the application under test.
Test name: ${testName}
Script name: ${scriptName}
Script: ${script}
Script: ${scriptPath}
Error stack: ${errors}
Dependent scripts of the main script:
${dependencies.map((d) => `- ${d.scriptPath}\n ${d.script}`).join("\n")}
Summarize the steps executed by the test using high level domain language. Focus on the user flow omit technical details. Use max 5 words per step.
Identify the step which failed by match the script code with the stack of the error. Include details about the test failure.
Expand Down

0 comments on commit 9acbfe0

Please sign in to comment.