New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

image patch embedding #66

Open

frederhoon opened this issue Jan 6, 2025 · 1 comment

frederhoon commented Jan 6, 2025

大佬您好，目前我用alphaclip输出的image_embedding是一维的，请问我要怎么获得image打成patch过程中二维的patch embedding？是否能有中间这一部分的输出呢？

Owner

SunzeY commented Jan 7, 2025

可以将被拉长的feature reshape还原成二维，参考这里

nh = attentions.shape[1]
attentions = attentions[0, :, 0, 1:].reshape(nh, -1)
p_s = int(math.sqrt(attentions.shape[1]))
attentions = attentions.reshape(nh, p_s, p_s)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment