Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add reasoning effort control for Claude 3.7 #2540

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

DeJeune
Copy link
Contributor

@DeJeune DeJeune commented Feb 28, 2025

  • Add reasoning effort settings with low/medium/high options
  • Implement reasoning effort for Claude 3.7 Sonnet models with OpenAI and Anthropic Provider
  • Update localization tips for reasoning effort
  • Enhance provider handling of reasoning effort parameters

Resolve #2304

…c Provider and OpenAI Provider

- Add reasoning effort settings with low/medium/high options
- Implement reasoning effort for Claude 3.7 Sonnet models
- Update localization tips for reasoning effort
- Enhance provider handling of reasoning effort parameters
@DeJeune
Copy link
Contributor Author

DeJeune commented Feb 28, 2025

大概说明一下,对于推理相关参数的处理复用了新版OpenRouterAPI: https://openrouter.ai/docs/use-cases/reasoning-tokens

{
"model": "your-model",
"messages": [],
"reasoning": {
// One of the following (not both):
"effort": "high", // Can be "high", "medium", or "low" (OpenAI-style)
"max_tokens": 2000, // Specific token limit (Anthropic-style)

// Optional: Default is false. All models support this.
"exclude": false // Set to true to exclude reasoning tokens from response
  }
}
  1. reasoning_effort是走 openai,如果anthropic这种不支持effort的模型设置了effort参数会按百分比转换成max_token:budget_tokens = max(min(global_max_tokens * {effort_ratio}, 32000), 1024) effort比例等于0.2, 0.5, 0.8
  2. reasoning里面的max_tokens是指推理的最大token,也对应了 anthropic的budget_token,如果openai这种不支持的模型会按照max_tokens/global_max_tokens换算成effort

所以我们只需要2选1,最好的方法就是复用本身有的UI即可,这样一个配置就能支持

image

在这里 如果是o3系列和o1模型或者是OpenRouter的claude-sonnet-3.7,就会直接使用reasoning_effort
如果是其他中转商或者Anthropic原生接口的Claude-sonnet-3.7 就会换算成buget_token
我直接在代码里把所有的o3系列和o1模型,以及claude-sonnet-3.7全部标记成了Reasoning模型,这样才走得到我上述的逻辑

如果要Claude开启推理,只需要打开思维链长度即可

关闭思维链时:

image

打开思维链时

image

如果要使用满血思考,可以把max_tokens拉到官方推荐的128k token

然后这位老哥 @preszzz 提到的直接在渲染进程发起请求这问题,我感觉没必要单独处理一个Claude-3.7。如果确实需要处理,需要把所有api请求搬到主进程,然后为APIKey加密存储,防止恶意软件。后面可以考虑整体进行改动

总之这样在前端进行了最小的改动,把budget_token的使用交给程序,这样也避免用户设置超过max_token的budget_token

@DeJeune
Copy link
Contributor Author

DeJeune commented Feb 28, 2025

@ousugo 麻烦看一下,感谢🙏

@DeJeune
Copy link
Contributor Author

DeJeune commented Feb 28, 2025

然后下面是一些测试结果

openrouter

dmxapi

原生anthropic接口测试,该代码里面aihubmix的type,没有anthropic API😭

image

@DeJeune DeJeune marked this pull request as ready for review February 28, 2025 06:25
@ousugo
Copy link
Collaborator

ousugo commented Feb 28, 2025

我觉得可以 👍 ,让猫哥再看一下就可以了

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature]: Claude3.7 sonnet 推理功能适配
2 participants