Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Draft : API Server #18

Open
wants to merge 27 commits into
base: v1.1
Choose a base branch
from
Open

Draft : API Server #18

wants to merge 27 commits into from

Commits on Oct 24, 2023

  1. hydra-moe vllm init

    @fearnworks
    fearnworks committed Oct 24, 2023
    Configuration menu
    Copy the full SHA
    a9a31ca View commit details
    Browse the repository at this point in the history

Commits on Oct 25, 2023

  1. Pivot to standalone api server

    @fearnworks
    fearnworks committed Oct 25, 2023
    Configuration menu
    Copy the full SHA
    94e1329 View commit details
    Browse the repository at this point in the history

  2. Succesful api model inference

    @fearnworks
    fearnworks committed Oct 25, 2023
    Configuration menu
    Copy the full SHA
    e23751b View commit details
    Browse the repository at this point in the history

  3. Streaming fix

    @fearnworks
    fearnworks committed Oct 25, 2023
    Configuration menu
    Copy the full SHA
    a859eb2 View commit details
    Browse the repository at this point in the history

  4. Add api notebook

    @fearnworks
    fearnworks committed Oct 25, 2023
    Configuration menu
    Copy the full SHA
    fe7986e View commit details
    Browse the repository at this point in the history

  5. Newline fixes

    @fearnworks
    fearnworks committed Oct 25, 2023
    Configuration menu
    Copy the full SHA
    81c19bf View commit details
    Browse the repository at this point in the history

Commits on Oct 26, 2023

  1. Decouple model loading from chat

    @fearnworks
    fearnworks committed Oct 26, 2023
    Configuration menu
    Copy the full SHA
    560d184 View commit details
    Browse the repository at this point in the history

  2. Add installable hydramoe base

    @fearnworks
    fearnworks committed Oct 26, 2023
    Configuration menu
    Copy the full SHA
    ce32263 View commit details
    Browse the repository at this point in the history

  3. Fix memory hog on model load

    @fearnworks
    fearnworks committed Oct 26, 2023
    Configuration menu
    Copy the full SHA
    b1c9d69 View commit details
    Browse the repository at this point in the history

  4. Update to run api from compose

    @fearnworks
    fearnworks committed Oct 26, 2023
    Configuration menu
    Copy the full SHA
    61cbf76 View commit details
    Browse the repository at this point in the history

Commits on Oct 27, 2023

  1. ModelWorker & RabbitMQ Init, First Handshake

    @fearnworks
    fearnworks committed Oct 27, 2023
    Configuration menu
    Copy the full SHA
    9c0cf52 View commit details
    Browse the repository at this point in the history

  2. Streaming Inference over RabbitMQ

    @fearnworks
    fearnworks committed Oct 27, 2023
    Configuration menu
    Copy the full SHA
    670acd8 View commit details
    Browse the repository at this point in the history

Commits on Oct 30, 2023

  1. Integrate fixes for eos stream

    @fearnworks
    fearnworks committed Oct 30, 2023
    Configuration menu
    Copy the full SHA
    9cc1c04 View commit details
    Browse the repository at this point in the history

  2. Skeleton for openai routing and schemas

    @fearnworks
    fearnworks committed Oct 30, 2023
    Configuration menu
    Copy the full SHA
    b0fb4db View commit details
    Browse the repository at this point in the history

Commits on Oct 31, 2023

  1. Manual merge from pharouks fork, openai api init

    @fearnworks
    fearnworks committed Oct 31, 2023
    Configuration menu
    Copy the full SHA
    4993241 View commit details
    Browse the repository at this point in the history

  2. Add validation for max_tokens api param

    @fearnworks
    fearnworks committed Oct 31, 2023
    Configuration menu
    Copy the full SHA
    7f07de7 View commit details
    Browse the repository at this point in the history

  3. Add read of model_info from inference yaml file

    @fearnworks
    fearnworks committed Oct 31, 2023
    Configuration menu
    Copy the full SHA
    94a8beb View commit details
    Browse the repository at this point in the history

  4. Model Info Endpoint provided by ModelWorker

    @fearnworks
    fearnworks committed Oct 31, 2023
    Configuration menu
    Copy the full SHA
    d644712 View commit details
    Browse the repository at this point in the history

  5. Add cli option for choosing a new config file

    @fearnworks
    fearnworks committed Oct 31, 2023
    Configuration menu
    Copy the full SHA
    bfa6a83 View commit details
    Browse the repository at this point in the history

  6. Connect max_tokens api input to model gen

    @fearnworks
    fearnworks committed Oct 31, 2023
    Configuration menu
    Copy the full SHA
    45f417e View commit details
    Browse the repository at this point in the history

  7. A params for temp, top_p, top_k, and rep penalty

    @fearnworks
    fearnworks committed Oct 31, 2023
    Configuration menu
    Copy the full SHA
    77119eb View commit details
    Browse the repository at this point in the history

  8. Update chat_service.py

    @pharaouk
    pharaouk authored Oct 31, 2023
    Configuration menu
    Copy the full SHA
    93224a0 View commit details
    Browse the repository at this point in the history

  9. Update chat.py

    @pharaouk
    pharaouk authored Oct 31, 2023
    Configuration menu
    Copy the full SHA
    ae39630 View commit details
    Browse the repository at this point in the history

  10. Updates to infer script and new tokenizer logic

    @fearnworks
    fearnworks committed Oct 31, 2023
    Configuration menu
    Copy the full SHA
    115fe56 View commit details
    Browse the repository at this point in the history

  11. Add check for max_token limit being breached

    @fearnworks
    fearnworks committed Oct 31, 2023
    Configuration menu
    Copy the full SHA
    7192317 View commit details
    Browse the repository at this point in the history

Commits on Nov 2, 2023

  1. a

    pharaouk committed Nov 2, 2023
    Configuration menu
    Copy the full SHA
    e0dd9dc View commit details
    Browse the repository at this point in the history

  2. Revert "a" 

    This reverts commit e0dd9dc.
    pharaouk committed Nov 2, 2023
    Configuration menu
    Copy the full SHA
    5846241 View commit details
    Browse the repository at this point in the history