Skip to content

Commit

Permalink
Grouped Query Attention + Refactor Attn (#492)
Browse files Browse the repository at this point in the history 
Adds support for GQA, and refactors MHA and MQA as special cases of GQA.

---------

Co-authored-by: root <Sasha Doubov>
Co-authored-by: Vitaliy Chiley <[email protected]>
Co-authored-by: Daniel King <[email protected]>
  • Loading branch information
@sashaDoubov @vchiley @dakinggg
3 people authored Aug 11, 2023
1 parent 7ac554d commit d2fbc3b
Show file tree
Hide file tree
Showing 5 changed files with 293 additions and 155 deletions.
Loading

0 comments on commit d2fbc3b

Please sign in to comment.