compensate for LLMs misunderstanding a prompt that should result in a tool call

[codefromthecrypt]

It is unlikely that a prompt that should result in a tool call will misfire using gpt-4o, at least according to this blog, using gorilla to test calls. the success rate is very high.

However, even if we don't see in practice gpt-4o mistaking a tool call for a question (and returning text instead), it is certainly possible. It is much more likely in local inference, where sometimes 1/10 will misfire or will misfire based on not understanding tool or prompts exactly how gpt does.

I suggest we make an approach where, when we know there's a tool call expected, we retry when the response from the LLM is text instead. Right now, we have retries, but only on HTTP failure. This is application layer and would help make local LLMS more usable.