Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

all[minor]: Add support for openai tools in all tool calling models #5737

Closed
wants to merge 20 commits into from

Conversation

bracesproul
Copy link
Collaborator

@bracesproul bracesproul commented Jun 12, 2024

anthropic[patch]: Add support for openai tool formats
community[patch]: Add support for openai tool formats with bedrock
google-common[patch]: Add support for openai tool formats
google-genai[patch]: Add support for openai tool formats
google-groq[patch]: Add support for openai tool formats
google-mistral[patch]: Add support for openai tool formats

TODO: update createToolCallingAgent to accept openai tools (and any other agent construction func we use)

Copy link

vercel bot commented Jun 12, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
langchainjs-api-refs ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jun 21, 2024 10:10pm
langchainjs-docs ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jun 21, 2024 10:10pm

@bracesproul bracesproul marked this pull request as ready for review June 12, 2024 21:48
@dosubot dosubot bot added size:XL This PR changes 500-999 lines, ignoring generated files. auto:improvement Medium size change to existing code to handle new use-cases labels Jun 12, 2024
@@ -66,6 +66,7 @@ export const config = {
"utils/testing": "utils/testing/index",
"utils/tiktoken": "utils/tiktoken",
"utils/types": "utils/types/index",
"utils/is_openai_tool": "utils/is_openai_tool",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we put this in an existing entrypoint? types maybe?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Want to avoid fragmentation

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved to @langchain/core/language_models/base so it's exported from the same place as ToolDefinition

}

return (tools as StructuredToolInterface[]).map((tool) => ({
name: tool.name,
description: tool.description,
input_schema: zodToJsonSchema(tool.schema),
input_schema: zodToJsonSchema(tool.schema) as AnthropicTool.InputSchema,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why necessary now?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're technically using a different type since we import from the SDK now. Anthropic enforces type: string in their json schemas which are always there, but they aren't typed as explicitly including it.

} else if (data.type === "content_block_delta") {
} else if (
data.type === "content_block_delta" &&
data.delta.type === "text_delta"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why necessary now? We don't support streaming tool calls yet.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SDK types bump req it to access text below

@@ -976,7 +993,7 @@ export class ChatAnthropicMessages<
name: functionName,
description:
jsonSchema.description ?? "A function available to call.",
input_schema: jsonSchema,
input_schema: jsonSchema as AnthropicTool.InputSchema,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Prefer as few changes as possible

@michaelozery
Copy link
Contributor

Hi @jacoblee93 ,
Thank you for your work on this project! I was wondering if there's any prediction on when this PR might be merged.

Thanks!

@bracesproul bracesproul changed the title core[minor]: Add base implementation for WSO all[minor]: Add support for openai tools in all tool calling models Jun 20, 2024
@bracesproul
Copy link
Collaborator Author

hey @michaelozery this PR will be merged & released today

@bracesproul
Copy link
Collaborator Author

Closing in favor of:
#5849
#5852
#5853
#5854
#5855
#5856

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto:improvement Medium size change to existing code to handle new use-cases size:XL This PR changes 500-999 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants