Unusable When You Have MANY Loras Locally

[Ameno-Sagiri]

Expected Behavior

The UI Is usable

Actual Behavior

The UI runs at extremely low framerates and frequently locks up, sometimes to the point that the entire tab dies (Chrome, Degoogled Chromium, Vivaldi, Firefox, makes no difference). Sometimes I will try to type in a prompt and have to wait a full 10+ seconds before what I typed actually appears, etc, or before it lets me click on anything, or before my click from 20 seconds ago finally has an effect. It's like working with a very slow slideshow. I have a great deal of patience, and I can firmly say this is in an unusable state for how trial-and-error AI is.

Steps to Reproduce

Have 2183 loras installed locally (Or seemingly over 2000?)

Debug Logs

https://paste.denizenscript.com/View/126818

Other

I've used SwarmUI for almost 2 months now, and had nearly no issues with it (other than some metadata related issues with loras). I'm actually not specifically sure when this problem started, but it happened somewhere between having 1900-2200 loras - nothing I do in the UI will alleviate the issues (other than if I search for something in the lora tab that has very few results, though the lag still somewhat persists in some of the menus, I'm assuming it's because it's not trying to load up the entire compendium of loras all at once when I filter them?). I tried all the lora views (cards/thumbnails/etc) and that seemed to have no impact. I did notice it slowly got laggier, and after going on a hunt for more loras over the last week, it is now unusable.

Unrelated, but I started using SwarmUI after InvokeAI because Invoke became unusable for the very same reason... just that took far less loras before I had problems with it. I get this is a very... unique and possibly niche issue, but it's still a problem.

[brendanhoar]

I'm in the same pickle, @mcmonkey4eva says I should have fewer models, lol. Don't have a fix, just some suggestions.

One thing to try might be to use the Reset All Metadata button. Sometimes there's some problems with the cached metadata in the directory/central ldb files.

Also, if you have remote/slow storage, you can try enabling the DoBackendDataCache option in the Server/Server Configuration page - it may or may not help.

Lastly, I've mostly given up on the search capability of the gui picker, instead I use the <lora: and <embed: invocations almost exclusively. The text search on those works pretty well.

[Ameno-Sagiri]

I'm in the same pickle, @mcmonkey4eva says I should have fewer models, lol. Don't have a fix, just some suggestions.

One thing to try might be to use the Reset All Metadata button. Sometimes there's some problems with the cached metadata in the directory/central ldb files.

Also, if you have remote/slow storage, you can try enabling the DoBackendDataCache option in the Server/Server Configuration page - it may or may not help.

Lastly, I've mostly given up on the search capability of the gui picker, instead I use the <lora: and <embed: invocations almost exclusively. The text search on those works pretty well.

Thanks for the reply.

In regards to the storage speed, I do have every model on a Samsung Pro 980 NVME drive, so speed technically shouldn't be the issue when it comes to filesystem. An alternative I found that may work for you is I have deployed SwarmUI with Stability Matrix which has its own built in Lora browser that, while not quite as detailed as the one in SwarmUI, still works, and it's just an extra step of sorting and clicking. I cannot memorize the lora tags for everything, and like visual aid, which is why I insist on using a lora browser, mainly for the sake of better organization without any guesswork.

I suspected an official answer (not yours) I might have gotten would be "have less models", but I would rather not consider that a valid solution from someone in development, and find that to be a bit of a cop-out answer to ignore a problem rather than fix it, just because it doesn't directly affect most people and isn't on a priority list. I feel that if it can be done, it should be made to work the way it's intended. I get that priorities lie elsewhere and there are maybe more important things to do, but I don't agree with writing this off for that reason, as it can be addressed at some point, even if not now.

I just wanted to point out that Stability Matrix's lora browser is functional still and seems to handle any amount of local models I have, but I do realize that's more of a standalone desktop application and is not running through a webUI, so there must be a multitude of limitations I am just not aware of by running things through a browser rather than a standalone application like SM, though I've been around programmers a lot of my life, enough to know some of the problems they may face, it's not alien to me.

I have attempted your two solutions. I am not quite sure at the moment if it was a permanent fix, but toggling DoBackendDataCache has possibly made a difference, it's not lagging all that badly even after loading up a checkpoint and some loras and generating a few things. Will update if it goes back to heavy lag again.

[mcmonkey4eva]

DoBackendDataCache is a wrapper around the fact that no matter how bad you might think Swarm's handling is, Comfy's is 100x slower. I've made multiple PRs to ComfyUI recently to try to reduce the pain of comfy's model listing - one's already been pulled, the other's still pending.

That shouldn't alter performance of the browser page at all, other than very specifically (A) the time it takes to hit the refresh button, and (B) the time it takes to load the Comfy Workflow tab.

The browser framerate lag you're describing is ... odd. That shouldn't happen. If you eg type <lora: that will then try to search the lora list for your prompt, so maybe that goes slow, or if specifically the lora list section lags that'd make sense, but an overall interface lag when you're just interacting with the general UI doesn't quite make sense.

What's your browser, and what extensions? Have you tried a different browser?

I note I had an issue reported before, where users of the Chrome 1Password extension had pages going horrifically slow, which I investigated and discovered that ... yeah the 1password extension does exponential scale processing on every webpage any time anything on the page updates ever, so yeah any page with a sufficiently high volume of content is gonna murder your browser.