Support LLM token count and metadata for Vercel AI SDK

[zirkelc]

Feature request

LangSmith does not show the token count for traces created from wrapAISDKModel for the Vercel AI SDK. The counts are sent to LangSmith:

usage:
  promptTokens: 872
  completionTokens: 28

However, the counts are not rendered in the UI. Also, any metadata added via experimental_telemetry.metadata is not sent to LangSmith.

Motivation

I'm exploring the usage of Vercel AI SDK while keeping the good DX from LangSmith.

[agola11]

Thanks for flagging @zirkelc. We are hoping to fix at least the token counting issue very soon

[Boohi]

Any updates on this? This is a quite critical feature to have on customer-facing applications

[jacoblee93]

Hey @Boohi, we've just shipped 0.2.1 with a new OTEL-based approach:

#1104

Docs will follow soon, but here's an example in Node.js:

/**
 * OpenTelemetry trace exporter for Vercel AI SDK.
 *
 * @example
 * ```ts
 * import { AISDKExporter } from "langsmith/vercel";
 * import { Client } from "langsmith";
 *
 * import { generateText } from "ai";
 * import { openai } from "@ai-sdk/openai";
 *
 * import { NodeSDK } from "@opentelemetry/sdk-node";
 * import { getNodeAutoInstrumentations } from "@opentelemetry/auto-instrumentations-node";
 *
 * const client = new Client();
 *
 * const sdk = new NodeSDK({
 *   traceExporter: new AISDKExporter({ client }),
 *   instrumentations: [getNodeAutoInstrumentations()],
 * });
 *
 * sdk.start();
 *
 * const res = await generateText({
 *   model: openai("gpt-4o-mini"),
 *   messages: [
 *     {
 *       role: "user",
 *       content: "What color is the sky?",
 *     },
 *   ],
 *   experimental_telemetry: AISDKExporter.getSettings({
 *     runName: "langsmith_traced_call",
 *     metadata: { userId: "123", language: "english" },
 *   }),
 * });
 *
 * await sdk.shutdown();
 * ```
 */

If you are using any LangChain packages, we recommend bumping all to latest along with this.