[refactor embeddings] gligen + ip-adapter #6244
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
This reverts commit 85f27fa.
|
|
||
|
|
||
| class PositionNet(nn.Module): | ||
| class GLIGENTextBoundingboxProjection(nn.Module): |
There was a problem hiding this comment.
@patrickvonplaten
Any rules around when we call it "Projection" vs "Embedding"?
There was a problem hiding this comment.
I follow this rule of thumb.
When you're representing some feature (time, bbox coordinates, tokens, etc.) in the latent space for the first time, it's better to call those representations embeddings. The subsequent operations performed on those representations are projections (more aligned with the literature in linear algebra).
There was a problem hiding this comment.
I like GLIGENTextBoundingboxProjection - it's nicely specific
|
The remote_code tests fail due to the class name change. What should we do about this? |
|
|
@yiyixuxu I think it's okay to introduce as a breaking change. I don't seem to see it imported directly in a lot of public repos And importing the GLIGEN pipeline will be unaffected. |
* refactor ip-adapter-imageproj, gligen --------- Co-authored-by: yiyixuxu <yixu310@gmail,com>
* refactor ip-adapter-imageproj, gligen --------- Co-authored-by: yiyixuxu <yixu310@gmail,com>
* refactor ip-adapter-imageproj, gligen --------- Co-authored-by: yiyixuxu <yixu310@gmail,com>
continue refactoring embeddings: GLIGEN + IP-Adapter