Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add flash attention #19

Draft
wants to merge 13 commits into
base: main
Choose a base branch
from
Draft

Add flash attention #19

wants to merge 13 commits into from

Commits on Apr 13, 2023

  1. Add back experimental features

    @jlamypoirier
    jlamypoirier committed Apr 13, 2023
    Configuration menu
    Copy the full SHA
    0b4585c View commit details
    Browse the repository at this point in the history

  2. Fixes

    @jlamypoirier
    jlamypoirier committed Apr 13, 2023
    Configuration menu
    Copy the full SHA
    4187786 View commit details
    Browse the repository at this point in the history

Commits on Apr 14, 2023

  1. Error message

    @jlamypoirier
    jlamypoirier committed Apr 14, 2023
    Configuration menu
    Copy the full SHA
    10f4a98 View commit details
    Browse the repository at this point in the history

Commits on Apr 20, 2023

  1. Attn implementations

    @jlamypoirier
    jlamypoirier committed Apr 20, 2023
    Configuration menu
    Copy the full SHA
    d08ce17 View commit details
    Browse the repository at this point in the history

Commits on Apr 21, 2023

  1. Optional jit

    @jlamypoirier
    jlamypoirier committed Apr 21, 2023
    Configuration menu
    Copy the full SHA
    23eacc7 View commit details
    Browse the repository at this point in the history

  2. No flash for graphs

    @jlamypoirier
    jlamypoirier committed Apr 21, 2023
    Configuration menu
    Copy the full SHA
    5dd048f View commit details
    Browse the repository at this point in the history

Commits on Apr 22, 2023

  1. Unpad for flash attn, remove flash/torch for decode, misc improvements

    @jlamypoirier
    jlamypoirier committed Apr 22, 2023
    Configuration menu
    Copy the full SHA
    a6efba9 View commit details
    Browse the repository at this point in the history

Commits on Apr 24, 2023

  1. cleanup

    @jlamypoirier
    jlamypoirier committed Apr 24, 2023
    Configuration menu
    Copy the full SHA
    a2efad2 View commit details
    Browse the repository at this point in the history

  2. Flash attention

    @jlamypoirier
    jlamypoirier committed Apr 24, 2023
    Configuration menu
    Copy the full SHA
    424f4e7 View commit details
    Browse the repository at this point in the history

  3. style

    @jlamypoirier
    jlamypoirier committed Apr 24, 2023
    Configuration menu
    Copy the full SHA
    71b14a1 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' into add_flash_attention

    @jlamypoirier
    jlamypoirier committed Apr 24, 2023
    Configuration menu
    Copy the full SHA
    c8743f0 View commit details
    Browse the repository at this point in the history

  5. Reduce diff

    @jlamypoirier
    jlamypoirier committed Apr 24, 2023
    Configuration menu
    Copy the full SHA
    237fea3 View commit details
    Browse the repository at this point in the history

  6. cleanup

    @jlamypoirier
    jlamypoirier committed Apr 24, 2023
    Configuration menu
    Copy the full SHA
    67693fd View commit details
    Browse the repository at this point in the history