Code

Add FlashMLA#74

Code

Open

EricLBuehler wants to merge 27 commits intomainEricLBuehler/candle:mainfrom flash_mlaEricLBuehler/candle:flash_mlaCopy head branch name to clipboard

+2,561

Commits

Commits on Feb 24, 2025

• Add FlashMLA

  EricLBuehler

  committed
  aad8187

  Copy full SHA for aad8187

• Add flash attn rust ffi

  EricLBuehler

  committed
  d16c27f

  Copy full SHA for d16c27f

Commits on Feb 25, 2025

• Automatic computation of mla metadata

  EricLBuehler

  committed
  ca880a0

  Copy full SHA for ca880a0

• Add some shape checks

  EricLBuehler

  committed
  7f2a72d

  Copy full SHA for 7f2a72d

• Fix .a name

  EricLBuehler

  committed
  4b007d0

  Copy full SHA for 4b007d0

• Fix linking name mha_fwd_kvcache_mla

  EricLBuehler

  committed
  e3f5f25

  Copy full SHA for e3f5f25

• extern "C"

  EricLBuehler

  committed
  aed2a32

  Copy full SHA for aed2a32

• Merge branch 'main' into flash_mla

  EricLBuehler

  committed
  03399ec

  Copy full SHA for 03399ec

• Handle CUDA_NVCC_FLAGS

  EricLBuehler

  committed
  a104e68

  Copy full SHA for a104e68

• Include flash_fwd_mla_kernel.h again

  EricLBuehler

  committed
  6070fd2

  Copy full SHA for 6070fd2

• Include flash_fwd_mla_kernel.h only once

  EricLBuehler

  committed
  778b932

  Copy full SHA for 778b932

• Tweak

  EricLBuehler

  committed
  a682c41

  Copy full SHA for a682c41

• Add flash_fwd_mla_kernel.h

  EricLBuehler

  committed
  8b2151b

  Copy full SHA for 8b2151b

• Use cute::bfloat16_t

  EricLBuehler

  committed
  dd81c1a

  Copy full SHA for dd81c1a

• Move CUDA_NVCC_FLAGS to last

  EricLBuehler

  committed
  45c9163

  Copy full SHA for 45c9163

• Fix reshape

  EricLBuehler

  committed
  117286a

  Copy full SHA for 117286a

Commits on Feb 26, 2025

• Only k_c_k_pe cache, no k/v cache

  EricLBuehler

  committed
  782f066

  Copy full SHA for 782f066

• Fix passing head_size_v

  EricLBuehler

  committed
  3b3cd2f

  Copy full SHA for 3b3cd2f

• Remove check for "v"

  EricLBuehler

  committed
  64a8cde

  Copy full SHA for 64a8cde

• Fix out shape

  EricLBuehler

  committed
  d8101c2

  Copy full SHA for d8101c2

• out-accum should be f32

  EricLBuehler

  committed
  294dcfd

  Copy full SHA for 294dcfd

Commits on Mar 3, 2025

• Remove references to flashattnv3

  EricLBuehler

  committed
  79f4c56

  Copy full SHA for 79f4c56

Commits on Mar 4, 2025

• Add test

  EricLBuehler

  committed
  66e7c17

  Copy full SHA for 66e7c17

• Some fixes

  EricLBuehler

  committed
  72cb652

  Copy full SHA for 72cb652

• Use repeat interleave

  EricLBuehler

  committed
  bb391ee

  Copy full SHA for bb391ee

• Maybe some progress...

  EricLBuehler

  committed
  3a683a5

  Copy full SHA for 3a683a5

• Tests pass!

  EricLBuehler

  committed
  3e5c023

  Copy full SHA for 3e5c023