Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bfloat zero #1

Open
wants to merge 16 commits into
base: master
Choose a base branch
from
Open

Bfloat zero #1

wants to merge 16 commits into from

Commits on Sep 7, 2022

  1. Allreduce bfloat in fp32 in zero

    Samuel Weinbach committed Sep 7, 2022
    Configuration menu
    Copy the full SHA
    b0eb1c5 View commit details
    Browse the repository at this point in the history

  2. Tied grad reduce for bfloat

    Samuel Weinbach committed Sep 7, 2022
    Configuration menu
    Copy the full SHA
    ecb85a5 View commit details
    Browse the repository at this point in the history

Commits on Sep 20, 2022

  1. Correct op_builder import

    @rrutmann
    rrutmann committed Sep 20, 2022
    Configuration menu
    Copy the full SHA
    55bf9f8 View commit details
    Browse the repository at this point in the history

  2. Correct import from git_version_info

    @rrutmann
    rrutmann committed Sep 20, 2022
    Configuration menu
    Copy the full SHA
    f58fc1b View commit details
    Browse the repository at this point in the history

Commits on Oct 24, 2022

  1. Test communication type

    @mali-git
    mali-git committed Oct 24, 2022
    Configuration menu
    Copy the full SHA
    e794448 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'bfloat_zero' of https://github.com/OpenGPTX/DeepSpeed i… 

    …nto bfloat_zero
    @rrutmann
    rrutmann committed Oct 24, 2022
    Configuration menu
    Copy the full SHA
    2dfdce5 View commit details
    Browse the repository at this point in the history

Commits on Oct 25, 2022

  1. Test parameters' type & communication type

    @mali-git
    mali-git committed Oct 25, 2022
    Configuration menu
    Copy the full SHA
    1466e72 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'bfloat_zero' of https://github.com/OpenGPTX/DeepSpeed i… 

    …nto bfloat_zero
    @rrutmann
    rrutmann committed Oct 25, 2022
    Configuration menu
    Copy the full SHA
    390562e View commit details
    Browse the repository at this point in the history

  3. Test Zero stage 0 and 1

    @mali-git
    mali-git committed Oct 25, 2022
    Configuration menu
    Copy the full SHA
    5ee97ef View commit details
    Browse the repository at this point in the history

Commits on Oct 26, 2022

  1. Test _exec_reduce_tied_grads

    @rrutmann
    rrutmann committed Oct 26, 2022
    Configuration menu
    Copy the full SHA
    86d5b42 View commit details
    Browse the repository at this point in the history

  2. Add TiedLinearStackPipe

    @rrutmann
    rrutmann committed Oct 26, 2022
    Configuration menu
    Copy the full SHA
    c6ba9ae View commit details
    Browse the repository at this point in the history

  3. Resolved merge conflicts

    @rrutmann
    rrutmann committed Oct 26, 2022
    Configuration menu
    Copy the full SHA
    2a39442 View commit details
    Browse the repository at this point in the history

Commits on Nov 8, 2022

  1. Initial draft of test__exec_backward_pass

    @rrutmann
    rrutmann committed Nov 8, 2022
    Configuration menu
    Copy the full SHA
    12c4315 View commit details
    Browse the repository at this point in the history

  2. Adapt assertion

    @mali-git
    mali-git committed Nov 8, 2022
    Configuration menu
    Copy the full SHA
    d5d8ea1 View commit details
    Browse the repository at this point in the history

Commits on Nov 29, 2022

  1. Add comments

    @mali-git
    mali-git committed Nov 29, 2022
    Configuration menu
    Copy the full SHA
    2d343ff View commit details
    Browse the repository at this point in the history

  2. Add comment

    @mali-git
    mali-git committed Nov 29, 2022
    Configuration menu
    Copy the full SHA
    256ca0b View commit details
    Browse the repository at this point in the history