Align Handling of reasoning_content and content for DeepSeek R1 Models with Official API

[autherlj]

Feature request / 功能建议

Current Implementation:
In the existing code, tokens and content (including reasoning steps) are combined into a single content field, with tags used to distinguish reasoning content. For example:
delta = chunk["choices"][0]["delta"] if "content" not in delta: continue else: response_content += html.escape(delta["content"]) yield response_content
This approach causes reasoning content (e.g., ...) and regular content to be merged, which may lead to rendering issues in frontend interfaces (e.g., escaped HTML tags not being parsed correctly).
Proposed Solution:
Align the handling of reasoning_content and content with the DeepSeek R1 official API, where these fields are processed separately. Specifically:

Add logic to extract reasoning_content and content from the API response independently.
Stop combining them into a single field with tags.
Example implementation based on the official API:

reasoning_content = ""  
content = ""  

for chunk in response:  
    delta = chunk["choices"][0]["delta"]  
    if "reasoning_content" in delta:  
        reasoning_content += delta["reasoning_content"]  
    elif "content" in delta:  
        content += delta["content"]  
    ## Yield or process the separated content as needed  

Motivation / 动机

Consistency with Official API: Ensures compatibility with DeepSeek R1’s specifications, simplifying integration for third-party tools and frontend interfaces.
Improved Frontend Handling: Separating reasoning_content (e.g., internal reasoning steps) from content (final output) allows frontends to render these appropriately (e.g., hiding/showing reasoning steps or formatting them differently).
Avoid Workarounds: Eliminates the need for manual HTML escaping and parsing of tags, reducing code complexity and potential errors.
The DeepSeek-R1 official API explicitly distinguishes reasoning_content and content in its response schema. Adopting this structure will future-proof the implementation and align with best practices for model output handling.

Your contribution / 您的贡献

https://api-docs.deepseek.com/zh-cn/guides/reasoning_model

[qinxuye]

If reasoning_content is added, some users may break due to this change.

Just wonder if the reasoning_content is de facto now.

[qinxuye]

After discussion online, we reach an agreement that we can enable reasoning_content via

xinference launch xxx --reasoning_content True

and it's disabled by default to not make too much impact.

[qinxuye]

If anyone is interested in implementing this feature, please let us know.

[oubeichen]

vllm has supported both the --enable-reasoning and --reasoning-parser options since version 0.7.1. If xinference supports them as well, does it also need to distinguish the reasoning-parser?

[qinxuye]

vllm has supported both the --enable-reasoning and --reasoning-parser options since version 0.7.1. If xinference supports them as well, does it also need to distinguish the reasoning-parser?

I don't think we need to rely on vllm, the parser itself is quite simple.

I think we need to add reasoning_start_tag and reasong_end_tag etc to llm_family.json, and just parse it according to the reasoning_content option if opened. This would be general for all engines.

[zoupengfeiagent]

请问reasoning_content字段什么时候可以发布啊。@qinxuye

[qinxuye]

请问reasoning_content字段什么时候可以发布啊。@qinxuye

1.3.0 发布吧

社区有感兴趣实现的及时说。

[zoupengfeiagent]

vllm部署方式已支持返回reasoning_content。 @qinxuye 是不是考虑升级下vllm版本.
vllm serve deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B
--enable-reasoning --reasoning-parser deepseek_r1
https://docs.vllm.ai/en/latest/features/reasoning_outputs.html

[George-TQL]

请问reasoning_content字段什么时候可以发布啊。@qinxuye

1.3.0 发布吧

社区有感兴趣实现的及时说。

请问下大概什么时候会发布呢。有什么临时解决办法吗。

[Zhangtiande]

同问什么时候实现，不然客户端要做这个标签的提取，到时候xinference一更新又要把提取去掉，还挺麻烦的

[qinxuye]

同问什么时候实现，不然客户端要做这个标签的提取，到时候xinference一更新又要把提取去掉，还挺麻烦的

下周发版，这个特性默认不会开，需要 launch 的时候指定。