Exphormer Implementation

[phoeenniixx]

Implementing Exphormer based on this paper

See also #9751

[phoeenniixx]

Hi @rusty1s, @xnuohz, @wsad1, @EdisonLeeeee, I think this PR is now ready for review
Sorry it took a while to understand the coding standards here😅
Now only the linting and documentation build is failing, I tried solving the linting issue but when I added annotations, it now shows new errors, please can you give me a little debrief on what are the standards...

Thanks in advance!

[phoeenniixx]

Hi @rusty1s, @xnuohz, @wsad1, @EdisonLeeeee, I think the PR is ready to review, all conflicts are resolved!!

[phoeenniixx]

Hi @rusty1s, @xnuohz, @wsad1, @EdisonLeeeee, please provide an ETA for review...

[xnuohz]

Hi, @phoeenniixx I just submitted a PR for SGFormer. You can check it and determine how your work could be better integrated into torch_geometric. Like but not limited

• Your test cases are a little complicated
• Add an example for training a model on a special dataset and log the metrics
• ...
  Thanks:)

[phoeenniixx]

Thanks for the reply @xnuohz, been a little busy lately, I will make the changes in few days :)

[puririshi98]

hi @phoeenniixx, i can get this merged after review. to start off can you share a log of the example run?

[puririshi98]

1)please put the model under torch_geometric.nn.models and the test under test/nn/model

2. then please make an example of running this as an additional model option for examples/ogbn-train.py. do this with "--gnn-choice" arg parse option, with choices ["sage, gat, exphormer_graph_transformer"].
   then run all 3 and share the logs. make the default the highest scoring one

you also need a unit test for the submodules under test/nn/attention
(i will have other PRs for GraphTrasnformers follow this design pattern so its unified)