Problems getting Flux working in SD.Next

[panton41]

I followed the directions in a related post - installed a Flux-based model via the built-in download utility - but can't get anything to work. Not even the official Flux model downloaded via Hugging Face. I also referenced the main Flux page linked from the main Git page, but it didn't actually have any sort of directions to get it working. (Lots of theory, but no practice.)

I understand that most Flux models are only part of the overall system, but for the life of me, I can't even begin to figure out how to get the other parts installed and working. I understand there's a separate VAE to install, and I've done that. And that there are several text parsers, and I've selected one to use. However, on the console, I still get the error stating that the model is incomplete and nothing renders. I feel like I'm missing an important step, and I'm not sure what it is. I feel like it might be related to a Clip parser, but I'm not certain and can't find anything related to getting that to work.

(Honestly, Hugging Face is a dumpster fire of a website, and the people who designed it should be mortified by how unintuitive it is to use. So my problem might simply be the fact the website I need to use was made by people who failed Web Design CISC-C101 and I simply can't navigate it.)

Part of the problem, I assume, is that I'm coming from a very slick Stable Diffusion program on macOS called Draw Things that, while very slow on my Apple Silicon hardware, "just works" with Flux models downloaded from Civital. I have to imagine it's doing a lot automatically that I have to do manually in this program.

For what it's worth, I'm using SD.Next on Linux with an AMD GPU using a AppImage-based installer called Stability Matrix that installs dependencies into an isolated environment mostly independent of the main OS. Getting any Stable Diffusion program working natively on Linux with is an exercise in madness, and I've spent my time in the nuthouse and don't plan on going back.

[vladmandic]

did you check flux docs? https://vladmandic.github.io/sdnext-docs/FLUX/

in short, don't try to "assemble the model" - you don't need to and thats not how sdnext works.
if you click on flux in networks -> models -> reference, that is all that is needed.
replacing vae, unet, transformers, encoders, etc. - all those things are optional and only used if you want to use a specific fine-tuned component.

you don't need to do anything on huggingface other than "accept terms and conditions" (because black-forrest-labs requires it, not because of sdnext does)

now, if you do just this and nothing else, what happens?

[panton41]

So that works for the official model. I do a use a Flux model that ISN'T the bowdlerized official one?

[vladmandic]

for finetunes, it depends:

• if its a full model, you load it by selecting it in the model list
• if its just a unet, you load model (official or 3rd party, but must start with full model) and THEN load finetuned unet using unet loader
• if its just encoder, same as above
• if its just a vae, same as above
  basically, you always start with full model - official or tuned and then replace components as needed. you don't try to manually assemble a model from all the components.
  this should be in the doc i've sent.