Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: update prompts and invalidate cache #493

Merged
merged 25 commits into from
Jan 28, 2025
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
78dbaff
feat: update prompts and invalidate cache
maxdeichmann Jan 20, 2025
3a35096
feat: update prompts and invalidate cache
maxdeichmann Jan 20, 2025
cd3fb2b
feat: update prompts and invalidate cache
maxdeichmann Jan 20, 2025
e03ee35
feat: update prompts and invalidate cache
maxdeichmann Jan 20, 2025
3faa622
feat: update prompts and invalidate cache
maxdeichmann Jan 20, 2025
5622d7c
feat: update prompts and invalidate cache
maxdeichmann Jan 20, 2025
509d547
feat: update prompts and invalidate cache
maxdeichmann Jan 20, 2025
512a35e
feat: update prompts and invalidate cache
maxdeichmann Jan 20, 2025
272766b
feat: update prompts and invalidate cache
maxdeichmann Jan 20, 2025
bc7162f
feat: update prompts and invalidate cache
maxdeichmann Jan 20, 2025
4cf0625
feat: update prompts and invalidate cache
maxdeichmann Jan 20, 2025
611d816
Update integration-test/langfuse-integration-node.spec.ts
hassiebp Jan 21, 2025
b983d5f
Update langfuse-core/src/openapi/server.ts
hassiebp Jan 21, 2025
d5fd31f
Merge branch 'main' into max/lfe-3587-update-prompt-versions
maxdeichmann Jan 24, 2025
3c820da
Merge branch 'main' into max/lfe-3587-update-prompt-versions
maxdeichmann Jan 24, 2025
a1d9a9a
fix
maxdeichmann Jan 27, 2025
e4c6d90
push
maxdeichmann Jan 27, 2025
bfe3dd0
Merge branch 'main' into max/lfe-3587-update-prompt-versions
maxdeichmann Jan 27, 2025
f2a171a
fix
maxdeichmann Jan 27, 2025
0cf97b9
Merge branch 'max/lfe-3587-update-prompt-versions' of github.com:lang…
maxdeichmann Jan 27, 2025
70e52db
fix
maxdeichmann Jan 27, 2025
d9acb0f
fix
maxdeichmann Jan 27, 2025
0168fec
push
maxdeichmann Jan 27, 2025
27bf6ab
fix
maxdeichmann Jan 27, 2025
6b8d300
push
maxdeichmann Jan 27, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions integration-test/langfuse-integration-node.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -432,6 +432,26 @@ describe("Langfuse Node.js", () => {
version: expect.any(Number),
});
});

it.only("update a prompt", async () => {
hassiebp marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove it.only to ensure all tests run.

const promptName = "test-prompt" + Math.random().toString(36);
await langfuse.createPrompt({
name: promptName,
prompt: "This is a prompt with a {{variable}}",
isActive: true,
labels: ["john"],
});

const updatedPrompt = await langfuse.updatePrompt({
name: promptName,
version: 1,
newLabels: ["john", "doe"],
});
maxdeichmann marked this conversation as resolved.
Show resolved Hide resolved

const prompt = await langfuse.getPrompt(promptName);
expect(prompt.labels).toEqual(expect.arrayContaining(["john", "doe"]));
expect(updatedPrompt.labels).toEqual(expect.arrayContaining(["john", "doe"]));
});
});

it("link prompt to generation", async () => {
Expand Down
79 changes: 76 additions & 3 deletions langfuse-core/openapi-spec/openapi-server.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -723,6 +723,9 @@ paths:
Notes:


- Introduction to data model:
https://langfuse.com/docs/tracing-data-model

- Batch sizes are limited to 3.5 MB in total. You need to adjust the
number of events per batch accordingly.

Expand Down Expand Up @@ -1423,6 +1426,76 @@ paths:
schema: {}
security:
- BasicAuth: []
/api/public/v2/prompts/{promptName}/version/{version}:
patch:
description: Update labels for a specific prompt version
operationId: promptVersion_update
tags:
- PromptVersion
parameters:
- name: promptName
in: path
description: The name of the prompt
required: true
schema:
type: string
- name: version
in: path
description: Version of the prompt to update
required: true
schema:
type: integer
responses:
'200':
description: ''
content:
application/json:
schema:
$ref: '#/components/schemas/Prompt'
'400':
description: ''
content:
application/json:
schema: {}
'401':
description: ''
content:
application/json:
schema: {}
'403':
description: ''
content:
application/json:
schema: {}
'404':
description: ''
content:
application/json:
schema: {}
'405':
description: ''
content:
application/json:
schema: {}
security:
- BasicAuth: []
requestBody:
required: true
content:
application/json:
schema:
type: object
properties:
newLabels:
type: array
items:
type: string
description: >-
New labels for the prompt version. Labels are unique across
versions. The "latest" label is reserved and managed by
Langfuse.
required:
- newLabels
/api/public/v2/prompts/{promptName}:
get:
description: Get a prompt
Expand Down Expand Up @@ -3183,11 +3256,12 @@ components:
to exact match, use `(?i)^modelname$`
startDate:
type: string
format: date
format: date-time
nullable: true
description: Apply only to generations which are newer than this ISO date.
unit:
$ref: '#/components/schemas/ModelUsageUnit'
nullable: true
description: Unit used by this model.
inputPrice:
type: number
Expand Down Expand Up @@ -3223,7 +3297,6 @@ components:
- id
- modelName
- matchPattern
- unit
- isLangfuseManaged
ModelUsageUnit:
title: ModelUsageUnit
Expand Down Expand Up @@ -4276,6 +4349,7 @@ components:
description: Apply only to generations which are newer than this ISO date.
unit:
$ref: '#/components/schemas/ModelUsageUnit'
nullable: true
description: Unit used by this model.
inputPrice:
type: number
Expand Down Expand Up @@ -4308,7 +4382,6 @@ components:
required:
- modelName
- matchPattern
- unit
Observations:
title: Observations
type: object
Expand Down
16 changes: 16 additions & 0 deletions langfuse-core/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@ import {
type UpdateLangfuseGenerationBody,
type UpdateLangfuseSpanBody,
type GetMediaResponse,
UpdatePromptBody,
} from "./types";
import { LangfuseMedia, type LangfuseMediaResolveMediaReferencesParams } from "./media/LangfuseMedia";
import {
Expand Down Expand Up @@ -527,6 +528,15 @@ abstract class LangfuseCoreStateless {
);
}

async updatePromptStateless(
body: UpdatePromptBody & { name: string; version: number }
): Promise<LangfusePromptClient> {
return this.fetchAndLogErrors(
`${this.baseUrl}/api/public/v2/prompts/${encodeURIComponent(body.name)}/versions/${encodeURIComponent(body.version)}`,
this._getFetchOptions({ method: "PATCH", body: JSON.stringify(body) })
);
}

async getPromptStateless(
name: string,
version?: number,
Expand Down Expand Up @@ -1382,6 +1392,12 @@ export abstract class LangfuseCore extends LangfuseCoreStateless {
return new TextPromptClient(promptResponse);
}

async updatePrompt(body: { name: string; version: number; newLabels: string[] }): Promise<LangfusePromptClient> {
const newPrompt = await this.updatePromptStateless(body);
this._promptCache.invalidate(body.name);
return newPrompt;
}

async getPrompt(
name: string,
version?: number,
Expand Down
97 changes: 93 additions & 4 deletions langfuse-core/src/openapi/server.ts
Original file line number Diff line number Diff line change
Expand Up @@ -190,6 +190,7 @@ export interface paths {
*
* Notes:
*
* - Introduction to data model: https://langfuse.com/docs/tracing-data-model
* - Batch sizes are limited to 3.5 MB in total. You need to adjust the number of events per batch accordingly.
* - The API does not return a 4xx status code for input errors. Instead, it responds with a 207 status code, which includes a list of the encountered errors. */
post: operations["ingestion_batch"];
Expand Down Expand Up @@ -338,6 +339,23 @@ export interface paths {
patch?: never;
trace?: never;
};
"/api/public/v2/prompts/{promptName}/version/{version}": {
parameters: {
query?: never;
header?: never;
path?: never;
cookie?: never;
};
get?: never;
put?: never;
post?: never;
delete?: never;
options?: never;
head?: never;
/** @description Update labels for a specific prompt version */
patch: operations["promptVersion_update"];
trace?: never;
};
"/api/public/v2/prompts/{promptName}": {
parameters: {
query?: never;
Expand Down Expand Up @@ -949,12 +967,12 @@ export interface components {
/** @description Regex pattern which matches this model definition to generation.model. Useful in case of fine-tuned models. If you want to exact match, use `(?i)^modelname$` */
matchPattern: string;
/**
* Format: date
* Format: date-time
* @description Apply only to generations which are newer than this ISO date.
*/
startDate?: string | null;
/** @description Unit used by this model. */
unit: components["schemas"]["ModelUsageUnit"];
unit?: components["schemas"]["ModelUsageUnit"];
/**
* Format: double
* @description Price (USD) per input unit
Expand Down Expand Up @@ -988,7 +1006,7 @@ export interface components {
*/
ObservationLevel: "DEBUG" | "DEFAULT" | "WARNING" | "ERROR";
/** MapValue */
MapValue: (string | null) | (number | null) | (boolean | null) | (string[] | null) | undefined;
MapValue: (string | null) | (number | null) | (boolean | null) | (string[] | null);
hassiebp marked this conversation as resolved.
Show resolved Hide resolved
/**
* CommentObjectType
* @enum {string}
Expand Down Expand Up @@ -1477,7 +1495,7 @@ export interface components {
*/
startDate?: string | null;
/** @description Unit used by this model. */
unit: components["schemas"]["ModelUsageUnit"];
unit?: components["schemas"]["ModelUsageUnit"];
/**
* Format: double
* @description Price (USD) per input unit
Expand Down Expand Up @@ -3317,6 +3335,77 @@ export interface operations {
};
};
};
promptVersion_update: {
parameters: {
query?: never;
header?: never;
path: {
/** @description The name of the prompt */
promptName: string;
/** @description Version of the prompt to update */
version: number;
};
cookie?: never;
};
requestBody: {
content: {
"application/json": {
/** @description New labels for the prompt version. Labels are unique across versions. The "latest" label is reserved and managed by Langfuse. */
newLabels: string[];
};
};
};
responses: {
200: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": components["schemas"]["Prompt"];
};
};
400: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": unknown;
};
};
401: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": unknown;
};
};
403: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": unknown;
};
};
404: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": unknown;
};
};
405: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": unknown;
};
};
};
};
prompts_get: {
parameters: {
query?: {
Expand Down
8 changes: 8 additions & 0 deletions langfuse-core/src/prompts/promptCache.ts
Original file line number Diff line number Diff line change
Expand Up @@ -50,4 +50,12 @@ export class LangfusePromptCache {
public isRefreshing(key: string): boolean {
return this._refreshingKeys.has(key);
}

public invalidate(promptName: string): void {
for (const key of this._cache.keys()) {
if (key.startsWith(promptName)) {
this._cache.delete(key);
}
}
}
maxdeichmann marked this conversation as resolved.
Show resolved Hide resolved
maxdeichmann marked this conversation as resolved.
Show resolved Hide resolved
}
3 changes: 3 additions & 0 deletions langfuse-core/src/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -161,6 +161,9 @@ export type GetLangfuseDatasetRunsResponse = FixTypes<
export type CreateLangfusePromptBody = FixTypes<
paths["/api/public/v2/prompts"]["post"]["requestBody"]["content"]["application/json"]
>;
export type UpdatePromptBody = FixTypes<
paths["/api/public/v2/prompts/{promptName}/version/{version}"]["patch"]["requestBody"]["content"]["application/json"]
>;
export type CreateLangfusePromptResponse =
paths["/api/public/v2/prompts"]["post"]["responses"]["200"]["content"]["application/json"];

Expand Down
Loading