Major Updates 2024.09.21 Announcement

[ffxvs]

Project Rework and Forge with Flux support

Installing

Setting up instance will take a bit longer than usual because I move most WebUI dependecies to the container which make the container size is increased. For RunPod, I recommend to choose instance with extreme internet speed. For Paperspace, it should be fine once they start caching the container in their server. And since I move the dependencies to the container, launching WebUI will be much much faster now.

RunPod

For RunPod users, you can use the current templates as usual. I recommend creating a new installation rather than using an existing one.

• SD WebUI Auto1111
• SD WebUI Forge

Paperspace

For Paperspace users, because Paperspace caches the old container in their server, you have to create a new installation with specific container version. Which means that when there is a new version of the container, you have to create new installation again. There will be a date on the container tag indicating the creation date. I will try to do updates only if necessary.

Current container version are :

ffxvs/sd-webui-containers:forge-2024.09.21

ffxvs/sd-webui-containers:auto1111-2024.09.21

Command is still the same :

bash /paperspace-start.sh

Updating

To help you understand how the update process works, this project consists of :

1. Container, which contains the main dependencies to run Stable Diffusion, such as python, pytorch, xformers, cuda, jupyter lab, etc. In RunPod, the container will be updated automatically. While in Paperspace, as I explained earlier, Paperspace caches old version of the container so you have to update it manually by creating a new installation.
2. Notebooks, to install/update WebUI and download the necessary resources. Notebooks will be updated automatically every time you start the machine. When there is a new version, your old notebook will be move to old-notebooks directory.
3. WebUI, it will be updated every time you run cell 1.1. Initialization in notebook setup.
4. Assets, consists of modules for running notebooks and resource lists when you want to download models and others. Assets are also updated automatically when you start the machine.

CivitAI API Key

Previously, we had to input the api key to the civitai_token variable every time the notebook was updated. Now, the CivitAI API Key will be stored in shared storage so you don't have to input civitai_token repeatedly.

Temporary Storage Settings (Paperspace Only)

Store your models, LoRA, controlNet and text encoder in temporary storage. Don't be afraid of getting charged. Temporary storage now has a nice UI. Settings will persist even if you restart the machine.

Extensions

Now the extension list has an UI for installing and updating extensions. But please remember that Forge just went through a major changes, so many extensions won't work.

Resource Lists

Just like extensions, right now resource lists also have an UI for users to select which models and version they want to download. Flux resource lists also has been added. You can also request me to add your favorite models to the list.

ControlNet and Install from URLs ?

I haven't changed them yet. They still use the old method. But I will update them gradually.

Closing

I just wanted to say thank you for using my work. I'm sorry if this took too long.
I've been through trials and errors over the past month. But it's been so much fun.
Your suggestions and criticisms are very much appreciated.

[ffxvs]

I found this helpful table from this comment.
Too many models make users confused. But recommended models do not always mean good. Everyone can experience it differently.
I will add Q3_K_S, Q4_K_S, Q5_K_S to the list. Need to hear your opinion on this GGUF models. I may remove some models later.
After all, we are still in the early stages of the Flux era.

Allowed quantization types:
   2  or  Q4_0   :  3.50G, +0.2499 ppl @ 7B - small, very high quality loss - legacy, prefer using Q3_K_M
   3  or  Q4_1   :  3.90G, +0.1846 ppl @ 7B - small, substantial quality loss - legacy, prefer using Q3_K_L
   8  or  Q5_0   :  4.30G, +0.0796 ppl @ 7B - medium, balanced quality - legacy, prefer using Q4_K_M
   9  or  Q5_1   :  4.70G, +0.0415 ppl @ 7B - medium, low quality loss - legacy, prefer using Q5_K_M
  10  or  Q2_K   :  2.67G, +0.8698 ppl @ 7B - smallest, extreme quality loss - not recommended
  12  or  Q3_K   : alias for Q3_K_M
  11  or  Q3_K_S :  2.75G, +0.5505 ppl @ 7B - very small, very high quality loss
  12  or  Q3_K_M :  3.06G, +0.2437 ppl @ 7B - very small, very high quality loss
  13  or  Q3_K_L :  3.35G, +0.1803 ppl @ 7B - small, substantial quality loss
  15  or  Q4_K   : alias for Q4_K_M
  14  or  Q4_K_S :  3.56G, +0.1149 ppl @ 7B - small, significant quality loss
  15  or  Q4_K_M :  3.80G, +0.0535 ppl @ 7B - medium, balanced quality - *recommended*
  17  or  Q5_K   : alias for Q5_K_M
  16  or  Q5_K_S :  4.33G, +0.0353 ppl @ 7B - large, low quality loss - *recommended*
  17  or  Q5_K_M :  4.45G, +0.0142 ppl @ 7B - large, very low quality loss - *recommended*
  18  or  Q6_K   :  5.15G, +0.0044 ppl @ 7B - very large, extremely low quality loss
   7  or  Q8_0   :  6.70G, +0.0004 ppl @ 7B - very large, extremely low quality loss - not recommended
   1  or  F16    : 13.00G              @ 7B - extremely large, virtually no quality loss - not recommended
   0  or  F32    : 26.00G              @ 7B - absolutely huge, lossless - not recommended

[jakeluo734]

thanks for your great work, this is amazing.
However there is one issue I am experiencing, when using API to switch checkpoint, nothing really happens. But it works if I switch checkpoint in the webui

[ffxvs]

Sorry, I enabled the API but didn't change anything regarding the API.
Did that work before the last update?
How do you call API to switch checkpoint?

[ffxvs]

Ok, I know the culprit.
Since we group models in subdirectories (sd, sdxl, flux). We must include the subdir before the checkpoint name.
For example :

{"sd_model_checkpoint": "flux/flux1-dev-Q5_K_S.gguf"}

Othewise, error will be thrown in the console log saying "model not found"

[jakeluo734]

i doubt it though. I put all my models within /models instead of the subdirectories.
Seems like this is a known issue with forge: lllyasviel/stable-diffusion-webui-forge#1421
Wondering what version of forge is being used here.

[ffxvs]

It always use the latest commit.
This is what I just tried if you don't believe me :

curl -X 'POST' \
  'https://xxxxxxxxxx-3001.proxy.runpod.net/sdapi/v1/options' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{"sd_model_checkpoint": "sdxl/epicrealismXL_v8Kiss.safetensors"}'

When it success :

Model selected: {'checkpoint_info': {'filename': '/notebooks/stable-diffusion-webui-forge/models/Stable-diffusion/sdxl/epicrealismXL_v8Kiss.safetensors', 'hash': 'bcae2f3a'}, 'additional_modules': [], 'unet_storage_dtype': None}
Using online LoRAs in FP16: False

When it error because I didn't include sub directory :

  File "/notebooks/stable-diffusion-webui-forge/modules/api/api.py", line 689, in set_config
    raise RuntimeError(f"model {checkpoint_name!r} not found")
RuntimeError: model 'epicrealismXL_v8Kiss.safetensors' not found

[jakeluo734]

I see, so you need to include the subdirectories. Interestingly this was required before the update, thanks for addressing this!