Stable Function Calling in open-r1

[ATaylorAerospace]

Hello Everyone,

My goal is to help create stable Function Calling in open-R1 so model pipelines can properly and reliably integrate external tools.

This is the current status of Function Calling in deepseek-chat that can use as a foundation to start: https://api-docs.deepseek.com/guides/function_calling.

Look forward to collaborating on this with the community so feel free to drop me a message

[aymeric-roucher]

I can support adding Code agentic behaviour! Rationale for using code rather than the JSON blobs traditionally used in function calling are here: https://huggingface.co/docs/smolagents/en/conceptual_guides/intro_agents#code-agents

[ATaylorAerospace]

I can support adding Code agentic behaviour! Rationale for using code rather than the JSON blobs traditionally used in function calling are here: https://huggingface.co/docs/smolagents/en/conceptual_guides/intro_agents#code-agents

Thanks for posting. Yes I think using code based interface would for sure be a better more stable approach for REST payloads to separate endpoints for a resource or action but question would be whether R1 supports queries and or mutations to a single endpoint for GraphQL?

Also one of the downstream use cases that will come out of Deep Seek R1 and open-r1 is the ability to run smaller models cheaply on firmware or embedded devices. The overhead of running code for external calls is ok for cloud based or desktop reasoning agents but would be too much for firmware applications since when using code for external calls you have additional memory usage to run a python compiler for example and definitely have more security risks.

Maybe we can approach this in two ways for open-r1,

1. For the current larger foundation R1 model the community is working on now we can come up with a couple of code based interfaces for calls.
2. For future edge models for r1 we can create a more stable JSON based interface for REST and GraphQL calls.

[WhiteGiver-Plus]

Have you considered integrating function calling capabilities within long CoT? An r1 model that can obtain tool results in real-time during the internal thinking process would be exciting, although it might be challenging.

[ATaylorAerospace]

Integrated function calling within long chain of thought might work for less rigid REST payloads but would not work consistently for more rigid GraphQL payloads and or schemas.

When R1 or 03-mini etc use chain of thought the llm produces reasoning steps and these reasoning steps can be mixed up with the final output and corrupt a strict GraphQL payload. Also long chain of thought is open ended and could also introduce extra characters or tokens that would break the required payload structure.

You could perhaps with hidden chain of thought separate the R1 or 03-mini reasoning steps from the output generation but would then need post processing and validation to check the payload against the GraphQL schema which introduces more overhead which could be an issue if you are trying to run a scaled down model on firmware for example.

[tyfeng1997]

I built a web app to test the tools plan and tool use capabilities of different LLMs. I found that DeepSeek v3 is also prone to errors. For example, it is very easy to fall into a function call loop, which Claude 3.5 Sonnet will not make. As DeepSeek said, function calls need to be improved.