Tensor-parallel like FeedForward to lower memory requirements#10623

Open

a-r-r-o-w wants to merge 2 commits intomainfrom memory-optimizations/tensor-parallel-like-ff

+196

Commits on Jan 21, 2025

memory-optimized ff
a-r-r-o-w
committed

Commits on Jan 25, 2025

fix geglu and swiglu gate computation
a-r-r-o-w
committed