Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support 128k max tokens for openrouter and vertex thinking #1338

Merged
merged 2 commits into from
Mar 3, 2025

Conversation

mrubens
Copy link
Collaborator

@mrubens mrubens commented Mar 3, 2025

#1327 for OpenRouter and Vertex too @cte @monotykamary

This sure seems to imply that the max is 128k on Vertex (no mention of a beta).

Screenshot 2025-03-03 at 9 49 32 AM


Important

Increase maxTokens to 128,000 for anthropic/claude-3.7-sonnet:thinking in OpenRouter API and update tests accordingly.

  • Behavior:
    • Increase maxTokens to 128,000 for anthropic/claude-3.7-sonnet:thinking in openrouter.ts and openrouter.test.ts.
  • Tests:
    • Update OpenRouterHandler test in openrouter.test.ts to reflect new maxTokens limit.
  • Implementation:
    • Modify getOpenRouterModels() in openrouter.ts to set maxTokens to 128,000 for anthropic/claude-3.7-sonnet:thinking.

This description was created by Ellipsis for 63cecf8. It will automatically update as commits are pushed.

@mrubens mrubens requested a review from cte as a code owner March 3, 2025 14:39
Copy link

changeset-bot bot commented Mar 3, 2025

⚠️ No Changeset found

Latest commit: 2b7df5e

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@dosubot dosubot bot added size:XS This PR changes 0-9 lines, ignoring generated files. enhancement New feature or request labels Mar 3, 2025
@mrubens mrubens changed the title Support 128k max tokens for openrouter thinking Support 128k max tokens for openrouter and vertex thinking Mar 3, 2025
@mrubens mrubens merged commit dba32ef into main Mar 3, 2025
9 checks passed
@mrubens mrubens deleted the openrouter_think_big branch March 3, 2025 14:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request size:XS This PR changes 0-9 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant