You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have problem understanding how I can use this library to implement cross attention
for instance if tensor x=torch.rand(100,14,64) is key, tensor y=torch.rand(100,11,64) is value and tensorz=torch.rand(100,14,1) is query, how can I use TransformerDecoderBuilder to compute the cross attention for this example?
I have difficulty to comprehend how I can use LinearTransformerCausalDecoder for computing cross attention. I will appreciate if anyone can clarify it for this example key, query and value ? Thanks.
The text was updated successfully, but these errors were encountered:
Hello,
I have problem understanding how I can use this library to implement cross attention
for instance if tensor
x=torch.rand(100,14,64)
is key, tensory=torch.rand(100,11,64)
is value and tensorz=torch.rand(100,14,1)
is query, how can I useTransformerDecoderBuilder
to compute the cross attention for this example?Here is how I built encoder and decoder class:
I have difficulty to comprehend how I can use
LinearTransformerCausalDecoder
for computing cross attention. I will appreciate if anyone can clarify it for this example key, query and value ? Thanks.The text was updated successfully, but these errors were encountered: