Recurrent Attention Model using Transformer?

[bemoregt]

Hi, @clvcooke @kevinzakka @malashinroman

Visual Recurrent Attention Model using Transformer is not yet?

That is possible?

I wonder ...

Thanks.

Best,

@bemoregt.

[clvcooke]

Hey @bemoregt, maybe not a great place to ask (and I don't think any of the people you pinged are collaborators), but it certainly is possible. The recent VIT paper shows how to use transformers for image classification, so you could probably put them in place of the RNN cell used in RAM.

https://openreview.net/forum?id=YicbFdNTTy