Wrong and random answers for question: How many 'R's are in the word strawberry?

[dibu28]

Wrong and random answers for question: "How many 'R's are in the word strawberry?"

Using model llama3.1 8B in ollama on Windows 11.

Model always gives different answers like 2 or 4. only sometimes it gives correct 3.

[tcsenpai]

I think is either a LLM limitation or a prompting issues. I'll further test this with the same model/provider as llama3.1 8B should indeed be able to do it

[tcsenpai]

This should now be way better. Unfortunately, LLMs are unpredictable in their results but the current iteration should be way better. Can you confirm this @dibu28 ?

[dibu28]

Pulled changes. Tried 5 times but the result allways the same 2 letters:
And also error with title appears sometimes.

[tcsenpai]

I'll keep this open and try to fix it from a prompt point of view. If you manage to, feel free to push a PR.