Can I train lora for img2img style synthesis?

[tekmen0]

1 image is not enough to capture my desired style. I want to leverage latent knowledge of instant style and fine-tune it with 300 input-output style image pairs. One way can be adding lora to existing layers of instantstyle.

Is it right way to go, or should I do something different?

Thanks for great repo :)

[haofanwang]

Here are two potential ways, I believe both can work if you already have style image pairs.

(1) Finetune InstantStyle with only style block updated. You can load the pre-trained weight.

(2) Train a LoRA, but only add LoRA at style block. At this time, you are actually training a B-LoRA.