Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set Vertex maxTokens to 64000 #1348

Merged
merged 1 commit into from
Mar 3, 2025
Merged

Set Vertex maxTokens to 64000 #1348

merged 1 commit into from
Mar 3, 2025

Conversation

mrubens
Copy link
Collaborator

@mrubens mrubens commented Mar 3, 2025

Context

Follow-up to #1338 - it turns out that vertex can't handle more than 64k.


Important

Set maxTokens to 64,000 for claude-3-7-sonnet@20250219:thinking in vertexModels to comply with Vertex AI's limit.

  • Behavior:
    • Update maxTokens for claude-3-7-sonnet@20250219:thinking in vertexModels from 128,000 to 64,000 to comply with Vertex AI's 64k limit.

This description was created by Ellipsis for af06af1. It will automatically update as commits are pushed.

@mrubens mrubens requested a review from cte as a code owner March 3, 2025 21:03
Copy link

changeset-bot bot commented Mar 3, 2025

⚠️ No Changeset found

Latest commit: af06af1

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@dosubot dosubot bot added the size:XS This PR changes 0-9 lines, ignoring generated files. label Mar 3, 2025
@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Mar 3, 2025
@mrubens mrubens merged commit 3168b1e into main Mar 3, 2025
15 of 16 checks passed
@mrubens mrubens deleted the mrubens-patch-4 branch March 3, 2025 21:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
lgtm This PR has been approved by a maintainer size:XS This PR changes 0-9 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants