Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new LLM provider Novita AI #7231

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions docs/core_docs/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -268,6 +268,8 @@ docs/integrations/toolkits/openapi.md
docs/integrations/toolkits/openapi.mdx
docs/integrations/text_embedding/togetherai.md
docs/integrations/text_embedding/togetherai.mdx
docs/integrations/text_embedding/pinecone.md
docs/integrations/text_embedding/pinecone.mdx
docs/integrations/text_embedding/openai.md
docs/integrations/text_embedding/openai.mdx
docs/integrations/text_embedding/ollama.md
Expand Down Expand Up @@ -328,12 +330,16 @@ docs/integrations/llms/azure.md
docs/integrations/llms/azure.mdx
docs/integrations/llms/arcjet.md
docs/integrations/llms/arcjet.mdx
docs/integrations/chat/xai.md
docs/integrations/chat/xai.mdx
docs/integrations/chat/togetherai.md
docs/integrations/chat/togetherai.mdx
docs/integrations/chat/openai.md
docs/integrations/chat/openai.mdx
docs/integrations/chat/ollama.md
docs/integrations/chat/ollama.mdx
docs/integrations/chat/novita.md
docs/integrations/chat/novita.mdx
docs/integrations/chat/mistral.md
docs/integrations/chat/mistral.mdx
docs/integrations/chat/ibm.md
Expand Down
207 changes: 207 additions & 0 deletions docs/core_docs/docs/integrations/chat/novita.ipynb
Original file line number Diff line number Diff line change
@@ -0,0 +1,207 @@
{
"cells": [
{
"cell_type": "raw",
"metadata": {},
"source": [
"---\n",
"sidebar_label: Novita\n",
"---"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# ChatNovita\n",
"\n",
"[Novita AI](https://novita.ai/model-api/product/llm-api?utm_source=github_langchain&utm_medium=github_readme&utm_campaign=link): The most cost-effective, stable, and scalable inference platform — delivering production-ready performance at a lower cost.\n",
"\n",
"This guide will help you getting started with `ChatNovitaAI` [chat models](/docs/concepts/chat_models). For detailed documentation of all `ChatNovitaAI` features and configurations head to the [API reference](https://novita.ai/docs/model-api/reference/llm/llm.html?utm_source=github_langchain&utm_medium=github_readme&utm_campaign=link)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Overview\n",
"\n",
"### Model features\n",
"| [Tool calling](../../how_to/tool_calling.ipynb) | [Structured output](../../how_to/structured_output.ipynb) | JSON mode | [Image input](../../how_to/multimodal_inputs.ipynb) | Audio input | Video input | [Token-level streaming](../../how_to/chat_streaming.ipynb) | Native async | [Token usage](../../how_to/chat_token_usage_tracking.ipynb) | [Logprobs](../../how_to/logprobs.ipynb) |\n",
"| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n",
"| ❌ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | ❌ |"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Setup\n",
"\n",
"To access Novita AI models you'll need to create a Novita account and get an API key.\n",
"\n",
"### Credentials\n",
"\n",
"Head to [this page](https://novita.ai/settings#key-management?utm_source=github_langchain&utm_medium=github_readme&utm_campaign=link) to sign up to Novita AI and generate an API key. Once you've done this set the NOVITA_API_KEY environment variable:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"vscode": {
"languageId": "shellscript"
}
},
"outputs": [],
"source": [
"export NOVITA_API_KEY=\"your-api-key\""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Installation\n",
"\n",
"The LangChain Novita integration lives in the `@langchain-community` package:\n",
"\n",
"```{=mdx}\n",
"import IntegrationInstallTooltip from \"@mdx_components/integration_install_tooltip.mdx\";\n",
"import Npm2Yarn from \"@theme/Npm2Yarn\";\n",
"\n",
"<IntegrationInstallTooltip></IntegrationInstallTooltip>\n",
"\n",
"<Npm2Yarn>\n",
" @langchain/community @langchain/core\n",
"</Npm2Yarn>\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Instantiation\n",
"\n",
"Now we can instantiate our model object and generate chat completions:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"vscode": {
"languageId": "javascript"
}
},
"outputs": [],
"source": [
"import { ChatNovitaAI } from \"../novita.js\";\n",
"\n",
"const llm = new ChatNovitaAI({\n",
" model: \"gryphe/mythomax-l2-13b\",\n",
" temperature: 0,\n",
" // other params...\n",
"})"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Invocation"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"vscode": {
"languageId": "javascript"
}
},
"outputs": [],
"source": [
"const aiMsg = await llm.invoke([\n",
" [\n",
" \"system\",\n",
" \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n",
" ],\n",
" [\"human\", \"I love programming.\"],\n",
"])\n",
"aiMsg"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"vscode": {
"languageId": "javascript"
}
},
"outputs": [],
"source": [
"console.log(aiMsg.content)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Chaining\n",
"\n",
"We can [chain](../../how_to/sequence.ipynb) our model with a prompt template like so:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"vscode": {
"languageId": "javascript"
}
},
"outputs": [],
"source": [
"import { ChatPromptTemplate } from \"@langchain/core/prompts\"\n",
"\n",
"const prompt = ChatPromptTemplate.fromMessages(\n",
" [\n",
" [\n",
" \"system\",\n",
" \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n",
" ],\n",
" [\"human\", \"{input}\"],\n",
" ]\n",
")\n",
"\n",
"const chain = prompt.pipe(llm);\n",
"await chain.invoke(\n",
" {\n",
" input_language: \"English\",\n",
" output_language: \"German\",\n",
" input: \"I love programming.\",\n",
" }\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## API reference\n",
"\n",
"For detailed documentation of Novita AI LLM APIs, head to [Novita AI API reference](https://novita.ai/docs/model-api/reference/llm/llm.html?utm_source=github_langchain&utm_medium=github_readme&utm_campaign=link)\n"
]
}
],
"metadata": {
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
51 changes: 51 additions & 0 deletions docs/core_docs/docs/integrations/chat/novita.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
---
sidebar_label: Novita
---

import CodeBlock from "@theme/CodeBlock";

# ChatNovita

[Novita AI](https://novita.ai/model-api/product/llm-api?utm_source=github_langchain&utm_medium=github_readme&utm_campaign=link) is the most cost-effective, stable, and scalable inference platform — delivering production-ready performance at a lower cost.

This guide will help you get started with `ChatNovitaAI` [chat models](/docs/concepts/chat_models). For detailed documentation of all `ChatNovitaAI` features and configurations head to the [API reference](https://novita.ai/docs/model-api/reference/llm/llm.html?utm_source=github_langchain&utm_medium=github_readme&utm_campaign=link).

## Setup

To access Novita AI models you'll need to create a Novita account and get an API key.

### Credentials

Head to [this page](https://novita.ai/settings#key-management?utm_source=github_langchain&utm_medium=github_readme&utm_campaign=link) to sign up to Novita AI and generate an API key. Once you've done this set the NOVITA_API_KEY environment variable:

```bash
export NOVITA_API_KEY="your-api-key"
```

### Installation

The LangChain Novita integration lives in the `@langchain-community` package:

import IntegrationInstallTooltip from "@mdx_components/integration_install_tooltip.mdx";

<IntegrationInstallTooltip></IntegrationInstallTooltip>

```bash npm2yarn
npm install @langchain/community @langchain/core
```

## Basic usage

import Novita from "@examples/models/chat/integration_novita.ts";

<CodeBlock language="typescript">{Novita}</CodeBlock>

## Chain model calls

import NovitaChain from "@examples/models/chat/integration_novita_chain.ts";

<CodeBlock language="typescript">{NovitaChain}</CodeBlock>

## API reference

For detailed documentation of Novita AI LLM APIs, head to [Novita AI API reference](https://novita.ai/docs/model-api/reference/llm/llm.html?utm_source=github_langchain&utm_medium=github_readme&utm_campaign=link)
25 changes: 25 additions & 0 deletions examples/src/models/chat/integration_novita.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
import { ChatNovita } from "@langchain/community/chat_models/novita";
import { HumanMessage } from "@langchain/core/messages";

// Use gryphe/mythomax-l2-13b
const chat = new ChatNovita({
apiKey: "YOUR-API-KEY", // In Node.js defaults to process.env.NOVITA_API_KEY
model: "gryphe/mythomax-l2-13b", // Check available models at https://novita.ai/llm-api
temperature: 0.3,
});

const messages = [new HumanMessage("Hello")];

const res = await chat.invoke(messages);
/*
AIMessage {
content: "Hello! How can I help you today? Is there something you would like to talk about or ask about? I'm here to assist you with any questions you may have.",
}
*/

const res2 = await chat.invoke(messages);
/*
AIMessage {
text: "Hello! How can I help you today? Is there something you would like to talk about or ask about? I'm here to assist you with any questions you may have.",
}
*/
29 changes: 29 additions & 0 deletions examples/src/models/chat/integration_novita_chain.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
import { ChatNovita } from "@langchain/community/chat_models/novita";
import { LLMChain } from "langchain/chains";
import { PromptTemplate } from "@langchain/core/prompts";


const chat = new ChatNovita({
apiKey: "YOUR-API-KEY", // In Node.js defaults to process.env.NOVITA_API_KEY
model: "gryphe/mythomax-l2-13b", // Check available models at https://novita.ai/llm-api
temperature: 0.3,
});

const prompt = PromptTemplate.fromTemplate(
"What is a good name for a company that makes {product}?"
);
const chain = new LLMChain({ llm: chat, prompt });

const response = await chain.invoke({ product: "colorful socks" });

console.log({ response });

/*
{
text: `I'm not sure what you mean by "colorful socks" but here are some ideas:\n` +
'\n' +
'- Sock-it to me!\n' +
'- Socks Away\n' +
'- Fancy Footwear'
}
*/
1 change: 1 addition & 0 deletions libs/langchain-community/langchain.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,7 @@ export const config = {
"llms/writer": "llms/writer",
"llms/yandex": "llms/yandex",
"llms/layerup_security": "llms/layerup_security",
"llms/novita": "llms/novita",
// vectorstores
"vectorstores/analyticdb": "vectorstores/analyticdb",
"vectorstores/astradb": "vectorstores/astradb",
Expand Down
2 changes: 1 addition & 1 deletion libs/langchain-community/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@
"author": "LangChain",
"license": "MIT",
"dependencies": {
"@langchain/openai": ">=0.2.0 <0.4.0",
"@langchain/openai": "workspace:*",
"binary-extensions": "^2.2.0",
"expr-eval": "^2.0.2",
"flat": "^5.0.2",
Expand Down
Loading