Integration of `from_pretrained` and `from_single_file` #10208

suzukimain · 2024-12-13T09:55:32Z

What does this PR do?

This PR integrates the from_single_file function into from_pretrained

pip install -q git+https://github.com/suzukimain/diffusers.git@loading_method_integration

from diffusers import StableDiffusionPipeline

# Load a single file checkpoint
pipe = StableDiffusionPipeline.from_pretrained("https://huggingface.co/stable-diffusion-v1-5/stable-diffusion-v1-5/blob/main/v1-5-pruned-emaonly.safetensors")

# Load a multi-folder Diffusers format checkpoint
pipe = StableDiffusionPipeline.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5")

Additionally, since AutoPipeline doesn't have a method to load single file checkpoints, I've adapted a part of the model_search functionality to enable this capability.

from diffusers import AutoPipelineForText2Image

# Load a single file checkpoint
pipe = AutoPipelineForText2Image.from_pretrained("https://huggingface.co/stable-diffusion-v1-5/stable-diffusion-v1-5/blob/main/v1-5-pruned-emaonly.safetensors")

# Load a multi-folder Diffusers format checkpoint
pipe = AutoPipelineForText2Image.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5")

Edit:
Reflected changes

Before submitting

This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
Did you read the contributor guideline?
Did you read our philosophy doc (important for complex PRs)?
Was this discussed/approved via a GitHub issue or the forum? Please add a link to it if that's the case.
Did you make sure to update the documentation with your changes? Here are the
documentation guidelines, and
here are tips on formatting docstrings.
Did you write any new necessary tests?

Who can review?

@yiyixuxu ,@asomoza

suzukimain · 2024-12-14T03:56:03Z

This PR allows these issues to be corrected.
#6986 #7496 #7979

…loading_method_integration

suzukimain · 2024-12-17T03:05:48Z

Updated dictionary used when loading the single file checkpoint in AutoPipeline

suzukimain · 2024-12-19T09:29:03Z

Can anyone please review this PR?

…loading_method_integration

…e map for loading single file checkpoints.

suzukimain · 2024-12-23T00:52:12Z

Hi, Could you please review this PR if possible?
I hope to not have caused any inconvenience. Thank you for your cooperation.
cc: @hlky @DN6

bghira · 2025-01-03T16:10:00Z

cc @yiyixuxu @a-r-r-o-w

…loading_method_integration

DN6 · 2025-01-07T14:43:39Z

Hi @suzukimain thank you for taking the time to put this together.

I think if we were to unify single file and from pretrained loading it would be by placing all the loading logic under from_pretrained rather than create a new loading method.

We can look into how to unify the two loading methods, but I don't think we can merge this PR in its current state.

cc'ing: @yiyixuxu for her thoughts as well.

…tegration

…Integrate-load-method-into-from_pretrained

suzukimain · 2025-01-10T08:20:38Z

src/diffusers/loaders/single_file.py

-            local_files_only=local_files_only,
-            revision=revision,
-        )
+        pipeline_class = _get_pipeline_class(cls, class_name=cls.__name__, config=None)


If you don't pass the class_name when loading a single file checkpoint in DiffusionPipeline, it will result in a TypeError.

suzukimain · 2025-01-10T08:31:59Z

src/diffusers/pipelines/pipeline_utils.py

+        # Load the pipeline from a single file
+        elif load_method_name == "from_single_file":
+            # The arguments for the __init__ method of `DiffusionPipeline` are keyword arguments, so they cannot be loaded from `from_single_file`
+            if cls.__name__ == "DiffusionPipeline":
+                # import it here to avoid circular import
+                from .stable_diffusion.convert_from_ckpt import download_from_original_stable_diffusion_ckpt
+
+                checkpoint = load_single_file_checkpoint(
+                    pretrained_model_name_or_path,
+                    force_download=force_download,
+                    proxies=proxies,
+                    token=token,
+                    cache_dir=cache_dir,
+                    local_files_only=local_files_only,
+                    revision=revision,
+                )
+
+                return download_from_original_stable_diffusion_ckpt(checkpoint, **kwargs)
+            else:
+                return cls.from_single_file(pretrained_model_name_or_path, **kwargs)


I'm separating the process because executing DiffusionPipeline.from_single_file results in the following error:

ValueError: Pipeline <class 'diffusers.pipelines.pipeline_utils.DiffusionPipeline'> expected {'kwargs', 'args'}, but only set() were passed.

suzukimain · 2025-01-10T08:38:42Z

Hello, I have made corrections based on your advice.
cc @DN6 @yiyixuxu

…loading_method_integration

suzukimain and others added 16 commits December 13, 2024 10:15

update

d8dc461

update

b239d4f

update

abf20e1

update

67b1f40

update

4132129

Added pipeline mappings for loading a single file checkpoint

ef77efb

update

349726f

fix

5e9d485

update

2475b86

Merge branch 'huggingface:main' into load_Method_Integration

e791d62

fix

4788002

fix

fdf5020

fix

b9449df

Fixed duplicate checkpoint loading

8c25f96

Merge branch 'main' into loading_method_integration

cb525df

Merge branch 'main' into loading_method_integration

f545353

suzukimain and others added 5 commits December 15, 2024 11:51

Merge branch 'main' into loading_method_integration

ef96367

Code formatting

891b11c

Merge branch 'main' of https://github.com/huggingface/diffusers into …

e072f90

…loading_method_integration

fix

7b22b18

Add keys and sort alphabetically

eb09cf0

suzukimain added 2 commits December 18, 2024 20:49

Merge branch 'main' into loading_method_integration

b9b62ea

Merge branch 'main' into loading_method_integration

ca764d4

suzukimain added 2 commits December 23, 2024 00:03

Merge branch 'main' of https://github.com/huggingface/diffusers into …

0874ab6

…loading_method_integration

Added flux-depth, flux-fill, and mochi-1-preview to the pipelin…

6488693

…e map for loading single file checkpoints.

suzukimain added 2 commits January 4, 2025 01:59

Merge branch 'main' of https://github.com/huggingface/diffusers into …

970a7ff

…loading_method_integration

update

75e510d

suzukimain and others added 21 commits January 8, 2025 11:27

Merge branch 'main' into loading_method_integration

118ce7f

update

0e41022

update

384bd7f

update

56403a6

update

e9a48fb

update

0ec85e0

update

41f960f

update

8c10847

fix

6a79268

make quality

8e845c2

update

6b252aa

update

d10e3f1

update

ceae059

update

18fce79

update

a5604da

update

69ba240

update

d82b9a9

make quality

b0f18ae

Merge https://github.com/huggingface/diffusers into loading_method_in…

8e58877

…tegration

Merge branch 'main' of https://github.com/huggingface/diffusers into …

32ce3a6

…Integrate-load-method-into-from_pretrained

Integrate from_pretrained with from_single_file functionality

bf0f979

suzukimain commented Jan 10, 2025

View reviewed changes

Merge branch 'main' of https://github.com/huggingface/diffusers into …

0a7aa0e

…loading_method_integration

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Integration of `from_pretrained` and `from_single_file` #10208

Integration of `from_pretrained` and `from_single_file` #10208

suzukimain commented Dec 13, 2024 •

edited

Loading

suzukimain commented Dec 14, 2024

suzukimain commented Dec 17, 2024

suzukimain commented Dec 19, 2024

suzukimain commented Dec 23, 2024 •

edited

Loading

bghira commented Jan 3, 2025

DN6 commented Jan 7, 2025

suzukimain Jan 10, 2025

suzukimain Jan 10, 2025

suzukimain commented Jan 10, 2025

Integration of from_pretrained and from_single_file #10208

Are you sure you want to change the base?

Integration of from_pretrained and from_single_file #10208

Conversation

suzukimain commented Dec 13, 2024 • edited Loading

What does this PR do?

Before submitting

Who can review?

suzukimain commented Dec 14, 2024

suzukimain commented Dec 17, 2024

suzukimain commented Dec 19, 2024

suzukimain commented Dec 23, 2024 • edited Loading

bghira commented Jan 3, 2025

DN6 commented Jan 7, 2025

suzukimain Jan 10, 2025

Choose a reason for hiding this comment

suzukimain Jan 10, 2025

Choose a reason for hiding this comment

suzukimain commented Jan 10, 2025

Integration of `from_pretrained` and `from_single_file` #10208

Integration of `from_pretrained` and `from_single_file` #10208

suzukimain commented Dec 13, 2024 •

edited

Loading

suzukimain commented Dec 23, 2024 •

edited

Loading