DeepSeek-R1 GGUF can't be loaded

[kbradsha]

Bug Report

GPT4All downloads the DeepSeek-R1 library, but receive an error attempting to load the model for chat.

Steps to Reproduce

1. Navigate to Models
2. Select DeepSeek-R1 bartowski
3. Upon successful download the model can be loaded by ollama
4. GPT4All reports "Could not load model due to invalid model file for DeepSeek-R1-Distill-Qwen-14B-Q4_0.gguf"

Expected Behavior

GGuf file should load as with all of the others. Ollama loads the same file fine.

Your Environment

• GPT4All version: v3.6.1
• Operating System: Ubuntu 24.04
• Chat model used (if applicable): DeepSeek-R1

I notice in the Chat Template settings an error is present:
Syntax error: 1:267: error: This feature has not been supported yet

[dantrez]

I am having the same issue. Version 3.6.1, Windows 10 Pro.

[QohoZ]

I am having the same issue. Version 3.7.0, Windows 11 Pro. I tried many models of several people to run DeepSeek-R1, but without success so far.

[kbradsha]

I suspect the chat template needs modification to accommodate the new output format.

[adgu]

Same here. W11H, v3.7.0

[BrushAway]

Good morning all,

Just to confirm, I tried some of the lower parameter versions of bartowski's Deepseek models (7B and 14B) but was receiving an error when trying to load the model into gpt4all. However, I tried loading in the larger model DeepSeek-R1-Distill-Llama-70B-Q5_K_S which did load in correctly and is usable. That isn't much solace to many of you who don't have the RAM but perhaps this could rule out gpt4all, possibly.

Hope this helps.

[breathless19]

Same here on Bazzite Linux

[kbradsha]

I wonder if there is a compatible chat template generated for the 70B model which could be used for 14B or 32B even.
32B seems to be the limit taking just 21 of the 24 gigabyte available to Nvidia 90 series cards.

[kbradsha]

I wonder if the Llama 8B distill works as well and it's only the QWEN models experiencing the issue.

[2602lim]

I have the same problem. The software is unable to directly open the downloaded gguf file.
. Version 3.7.0，

[fieldequation]

Same here, v3.7.0, W11H. Can't load 14B and 7B model:

Could not load model due to invalid model file for DeepSeek-R1-Distill-Qwen-14B-Q8_0.gguf

[fieldequation]

I wonder if the Llama 8B distill works as well and it's only the QWEN models experiencing the issue.

I think so, both Llama models (8B and 70B) work fine on my device. Qwen based DeepSeek models may not compatible with current version of GPT4All

[ilgrank]

Also possible solution here:
https://huggingface.co/IntelligentEstate/Die_Walkure-R1-Distill-Llama-8B-iQ4_K_M-GGUF

[eliuha]

the solution did not work for me
version 3.7.0

[realasking]

Same problem here. v3.7.0, Win11. Can't load 7B, 8B, 14B DeepSeek-R1 Llama and Qwen models.

[kaisernova]

Same problem, Win 11

[tomkpt]

PLEASE MAKE DEEPSEEK compatible to GPT4All, As I had tried all GGFUF of Deepseek, but none are working.

LMStudio is working super fast and completely fine with Deepseek, so make it compatible too.

[TooShyTo]

Same issue here Windows 11 Pro 23H2

[ilgrank]

the solution did not work for me version 3.7.0

Edit 1: Also the 70B parameter model works, but I got 0.019T/s, which is unusable for me (too big to fit in VRAM)
Edit 2: Just the LLAMA models do work, QWEN do not.

[brynrmrz]

This is not an issue, GPT4ALL docs clearly say custom models are not supported OOTB and it is up to user to additional configuration to work. This issue should close.

[nobody5050]

A pr was merged into the repo a few hours ago that fixes this, but if you want to get it working right now use this chat template which is based on that pr:

{%- if not add_generation_prompt is defined %}
    {%- set add_generation_prompt = false %}
{%- endif %}
{%- if messages[0]['role'] == 'system' %}
    {{- messages[0]['content'] }}
{%- endif %}
{%- for message in messages %}
    {%- if message['role'] == 'user' %}
        {{- '<｜User｜>' + message['content'] }}
    {%- endif %}
    {%- if message['role'] == 'assistant' %}
        {%- set content = message['content'].split('</think>', 1) | last %}
        {{- '<｜Assistant｜>' + content + '<｜end▁of▁sentence｜>' }}
    {%- endif %}
{%- endfor -%}
{%- if add_generation_prompt %}
    {{- '<｜Assistant｜>' }}
{%- endif %}

[tomkpt]

{%- if not add_generation_prompt is defined %}
{%- set add_generation_prompt = false %}
{%- endif %}
{%- if messages[0]['role'] == 'system' %}
{{- messages[0]['content'] }}
{%- endif %}
{%- for message in messages %}
{%- if message['role'] == 'user' %}
{{- '<｜User｜>' + message['content'] }}
{%- endif %}
{%- if message['role'] == 'assistant' %}
{%- set content = message['content'].split('', 1) | last %}
{{- '<｜Assistant｜>' + content + '<｜end▁of▁sentence｜>' }}
{%- endif %}
{%- endfor -%}
{%- if add_generation_prompt %}
{{- '<｜Assistant｜>' }}
{%- endif %}

Thanks you so much for this, it works now. But can it be possible or any plan of UI changes for Thinking Part, as it looks very good in DeepSeek online or LM Studio, the THINKING-REASONING PART should look beautifully in small box with small font with brain ICON blinking.