Reducing RAM usage?

[PointlessCircle]

Hey, I've been trying ComfyUI and Stable Diffusion for a while now and it's been really interesting, however I'm constantly running into Out Of Memory errors when using bigger workflows. Is there any way I could reduce ComfyUI's RAM/VRAM usage? I read a bit into it and found out about quantizing models, however I can't figure out how to get that to work with my checkpoint of choice (LEOSAM's HelloWorld XL 7.0).
I have a RTX 3050 mobile with 4GB of VRAM, an AMD Ryzen 7 6000, and 16GB of RAM. Thanks in advance!

[Rogala]

You can increase the swap file in the system to 40-50 GB if you have space and enable unloading from VRAM to RAM through the driver settings, but the program should be selected for python from Comfy X:...\ComfyUI\python_embedded\python.exe.

System Memory Fallback for Stable Diffusion

[PointlessCircle]

Thank you so much for your quick response! Since I was using an outdated Nvidia driver, it automatically enabled sysmem fallback, and I was using all 16GB of RAM. As for the swap file, I added 45GB to it, but my computer only ended up allocating 13GB from it before hitting OOM again, so I'm pretty sure that ComfyUI just won't use all of it even if provided, for whatever reason, so that, unfortunately, did not work either.

[Rogala]

It's very strange I have a configuration no better than yours. Only the RAM is twice as much at 32GB. I tried LEOSAM's HelloWorld XL 7.0 model, the picture is created without errors.

`** ComfyUI startup time: 2025-01-29 19:58:24.705
** Platform: Windows
** Python version: 3.11.9 (tags/v3.11.9:de54cf5, Apr 2 2024, 10:12:12) [MSC v.1938 64 bit (AMD64)]

Checkpoint files will always be loaded safely.
Total VRAM 4096 MB, total RAM 32599 MB
pytorch version: 2.5.1+cu124
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce GTX 1650 SUPER : native
Using pytorch attention
ComfyUI version: 0.3.12
[Prompt Server] web root: c:\ai\ComfyUI\ComfyUI\web
[Crystools INFO] Crystools version: 1.21.0
[Crystools INFO] CPU: 12th Gen Intel(R) Core(TM) i3-12100F - Arch: AMD64 - OS: Windows 11
[Crystools INFO] Pynvml (Nvidia) initialized.
[Crystools INFO] GPU/s:
[Crystools INFO] 0) NVIDIA GeForce GTX 1650 SUPER
[Crystools INFO] NVIDIA Driver: 566.36`

But I've never had any problems with SDXL models, I also run FLUX, GGUF Q5 and FP8 models with peace of mind. It's time consuming but it works without “Out Of Memory errors”.

[PointlessCircle]

It's probably the IPAdapter nodes then, I was trying to use this workflow, which loads a ton of IPAdapter models. I can run the checkpoint without those nodes fine though.

[LukeG89]

Today I did some tests, just for fun, and separate diffusion model, text encoders and vae (by using the ModelSave, CLIPSave and VAESave nodes) from one checkpoint (in my case, I used epiCRealism XL VXII) and then converted the diffusion model with GGUF Convertor (Alpha) from gguf custom node.

I've run a simple default workflow with the same parameters. There have been a reduction of ~3GB of RAM with GGUF, not a big deal, but it's something. Also I can't see any quality loss.

Checkpoint:

RAM: 14.9 GB
VRAM: 5.7 GB
TOTAL MEMORY: 20.6 GB

Diffusion model GGUF:

RAM: 11.9 GB
VRAM: 5.7 GB
TOTAL MEMORY: 17.6 GB

Test with fp8

I also tried with fp8 mode in Load Diffusion Model (with the non-GGUF model) and by using the command arguments --fp8_e4m3fn-unet and --fp8_e4m3fn-text-enc with the GGUF model.
There have been a little memory reduction, but in these cases the quality loss is a little visible.

Diffusion Model in fp8 mode:

RAM: 12.9 GB
VRAM: 4.6 GB
TOTAL MEMORY: 17.5 GB

Diffusion GGUF with --fp8_e4m3fn-unet and --fp8_e4m3fn-text-enc:

RAM: 10.9 GB
VRAM: 5.8 GB
TOTAL MEMORY: 16.7 GB

@PointlessCircle As you can see from my tests, your specs are pretty limited for SDXL models, it's a miracle if you can generate just with a simple workflow, loading other models like ControlNet or IPAdapter is not possible.
You should consider upgrading your RAM or using SD1.5 models that are lighter.