Implementing tensorflow version of CausalDotProduct

[JamesDeAntonis]

May we create a tensorflow version of CausalDotProduct? This is necessary for the transformers implementation of Performers, which implements for both pytorch and tf.

If you don't want this done, we would instead copy-and-paste the code into transformers, then make the necessary changes to tensorflow. While that is fine, I think it's cleaner to do it all in this repo.

[angeloskath]

Hi James,

Sorry for the late reply. I understand why you would want to keep it in this repo, it is all updated together and tested together.

However, how would you go about integrating it? Where would the code reside? In the main fast_transformers package?

Let me know your thoughts, I am totally open to suggestions.

Cheers,
Angelos