Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix same family model issue and add test #9

Open
wants to merge 63 commits into
base: rebase_mm
Choose a base branch
from
Open

Fix same family model issue and add test #9

wants to merge 63 commits into from

Conversation

xuechendi
Copy link

No description provided.

@github-actions GitHub Actions
Copy link

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can do one of these:

  • Add ready label to the PR
  • Enable auto-merge.

🚀

youkaichao and others added 27 commits November 27, 2024 19:54
@youkaichao
[misc] upgrade filelock version (vllm-project#10731)
cb4e1c3 
Signed-off-by: youkaichao <[email protected]>
@zixuanzhang226
[Model] support bitsandbytes quantization with minicpm3 model (vllm-p…
70dc14f 
…roject#10682)

Signed-off-by: Ubuntu <[email protected]>
@spacewander
[Doc] Update model in arch_overview.rst to match comment (vllm-projec…
278be67 
…t#10701)

Signed-off-by: spacewander <[email protected]>
@rickyyx
[Bug][CLI] Allow users to disable prefix caching explicitly (vllm-pro…
d9b4b3f 
…ject#10724)

Signed-off-by: rickyx <[email protected]>
@WoosukKwon
[V1] Do not allocate beyond the max_model_len (vllm-project#10730)
a79b122 
Signed-off-by: Woosuk Kwon <[email protected]>
@WoosukKwon
[Kernel] Update vllm-flash-attn version (vllm-project#10736)
9a8bff0 
Signed-off-by: Woosuk Kwon <[email protected]>
@richardsliu
[TPU] Update requirements-tpu (vllm-project#10726)
3ed5e73 
Signed-off-by: Richard Liu <[email protected]>
@sixsixcoder @Isotr0py @DarkLight1337
[Model] Added GLM-4 series hf format model support vllm==0.6.4 (vllm-…
5fc5ce0 
…project#10561)

Signed-off-by: Isotr0py <[email protected]>
Co-authored-by: Isotr0py <[email protected]>
Co-authored-by: Cyrus Leung <[email protected]>
@WoosukKwon
[Kernel] Update vllm-flash-attn version to reduce CPU overheads (vllm…
8c1e77f 
…-project#10742)

Signed-off-by: Woosuk Kwon <[email protected]>
@WoosukKwon
[V1] Optimize the CPU overheads in FlashAttention custom op (vllm-pro…
98f47f2 
…ject#10733)

Signed-off-by: Woosuk Kwon <[email protected]>
@Isotr0py
[Model] Add Internlm2 LoRA support (vllm-project#5064)
c83919c 
Signed-off-by: Isotr0py <[email protected]>
@DarkLight1337
[Model] Clean up MiniCPMV (vllm-project#10751)
fa6ecb9 
Signed-off-by: DarkLight1337 <[email protected]>
@noooop
[Misc] typo find in sampling_metadata.py (vllm-project#10740)
c82b432
@jeejeelee
[Bugfix] Fix Idefics3 bug (vllm-project#10778)
3132aac 
Signed-off-by: Jee Jee Li <[email protected]>
@wangxiyuan
[platform] Add verify_quantization in platform. (vllm-project#10757)
661175b 
Signed-off-by: wangxiyuan <[email protected]>
@NickLucche @DarkLight1337
[Bugfix] Fix OpenVino/Neuron driver_worker init (vllm-project#10779)
40bc242 
Signed-off-by: NickLucche <[email protected]>
Signed-off-by: Cyrus Leung <[email protected]>
Co-authored-by: Cyrus Leung <[email protected]>
@Isotr0py
[Model] Refactor Molmo weights loading to use AutoWeightsLoader (vllm…
16ee07f 
…-project#10771)

Signed-off-by: Isotr0py <[email protected]>
@patrickvonplaten @youkaichao
[Interleaved ATTN] Support for Mistral-8B (vllm-project#10591)
e7cfc4e 
Signed-off-by: youkaichao <[email protected]>
Co-authored-by: youkaichao <[email protected]>
@wangxiyuan
[doc] format fix (vllm-project#10789)
7e4bbda 
Signed-off-by: wangxiyuan <[email protected]>
@DarkLight1337
[Model] Replace embedding models with pooling adapter (vllm-project#1…
1337071 
…0769)

Signed-off-by: DarkLight1337 <[email protected]>
@DarkLight1337
[Misc] Improve type annotations for support_torch_compile (vllm-pro…
f877a7d 
…ject#10763)

Signed-off-by: DarkLight1337 <[email protected]>
@DarkLight1337
[Misc] Rename embedding classes to pooling (vllm-project#10801)
d2f058e 
Signed-off-by: DarkLight1337 <[email protected]>
@youkaichao
[doc] add warning about comparing hf and vllm outputs (vllm-project#1…
169a0ff 
…0805)

Signed-off-by: youkaichao <[email protected]>
@ywang96 @heheda12345 @Isotr0py
[Misc] Adding MMMU-Pro vision dataset to serving benchmark (vllm-pr…
c11f172 
…oject#10804)

Signed-off-by: Roger Wang <[email protected]>
Co-authored-by: Chen Zhang <[email protected]>
Co-authored-by: Isotr0py <[email protected]>
@KuntaiDu
[Core] Implement disagg prefill by StatelessProcessGroup (vllm-projec…
0590ec3 
…t#10502)

This PR provides initial support for single-node disaggregated prefill in 1P1D scenario.
Signed-off-by: KuntaiDu <[email protected]>
Co-authored-by: ApostaC <[email protected]>
Co-authored-by: YaoJiayi <[email protected]>
@Isotr0py
[Model] Add BNB support to Llava and Pixtral-HF (vllm-project#10795)
b18c9bb 
Signed-off-by: Isotr0py <[email protected]>
@cduk
[core] Avoid metrics log noise when idle - include speculative decodi… (
b795477 
vllm-project#10809)
@xuechendi xuechendi force-pushed the multi_models_rebase branch from 168af3c to 3133992 Compare December 2, 2024 22:44
@xuechendi xuechendi force-pushed the multi_models_rebase branch from 3133992 to 0915219 Compare December 2, 2024 23:31
mgoin and others added 21 commits December 3, 2024 02:26
@mgoin
[Bugfix] Prevent benchmark_throughput.py from using duplicated random…
4433195 
… prompts (vllm-project#10753)
@zixuanzhang226
[Model] support bitsandbytes quantization with minicpm model (vllm-pr…
d746268 
…oject#10842)

Signed-off-by: Ubuntu <[email protected]>
@jeejeelee
[Bugfix] Fix QKVParallelLinearWithShardedLora bias bug (vllm-project#…
a4cf256 
…10844)

Signed-off-by: Jee Jee Li <[email protected]>
@youkaichao
[core][distributed] add pynccl broadcast (vllm-project#10843)
21fe7b4 
Signed-off-by: youkaichao <[email protected]>
@youkaichao
[torch.compile] remove compilation_context and simplify code (vllm-pr…
dc5ce86 
…oject#10838)

Signed-off-by: youkaichao <[email protected]>
@russellb @DarkLight1337
[Doc] Add github links for source code references (vllm-project#10672)
ef51831 
Signed-off-by: Russell Bryant <[email protected]>
Signed-off-by: DarkLight1337 <[email protected]>
Co-authored-by: DarkLight1337 <[email protected]>
@DarkLight1337
[Misc] Remove deprecated names (vllm-project#10817)
3257d44 
Signed-off-by: DarkLight1337 <[email protected]>
@aarnphm @mgoin
[Core][Performance] Add XGrammar support for guided decoding and set …
9323a31 
…it as default (vllm-project#10785)

Signed-off-by: Aaron Pham <[email protected]>
Signed-off-by: mgoin <[email protected]>
Co-authored-by: mgoin <[email protected]>
@zhengy001
[Speculative Decoding] Move indices to device before filtering output (…
f6084f6 
…vllm-project#10850)

Co-authored-by: Yang Zheng(SW)(Alex) <[email protected]>
@alexm-redhat @mgoin @ywang96
[V1] VLM - Run the mm_mapper preprocessor in the frontend process (vl…
3bc94ca 
…lm-project#10640)

Signed-off-by: Roger Wang <[email protected]>
Co-authored-by: Michael Goin <[email protected]>
Co-authored-by: Roger Wang <[email protected]>
@yma11
[MISC][XPU] quick fix for XPU CI (vllm-project#10859)
2f2cdc7 
Signed-off-by: yan ma <[email protected]>
@mgoin
[Bugfix] Only require XGrammar on x86 (vllm-project#10865)
7090c27 
Signed-off-by: mgoin <[email protected]>
@tomeras91
[Frontend] correctly record prefill and decode time metrics (vllm-pro…
7c32b68 
…ject#10853)

Signed-off-by: Tomer Asida <[email protected]>
@gshtras
[Build][Bugfix] Using the correct type hint (vllm-project#10866)
a061fe6 
Signed-off-by: Gregory Shtrasberg <[email protected]>
@xuechendi @aarnphm
[Benchmark] Benchmark structured output with datasets (vllm-project#1…
381ac93 
…0557)

Signed-off-by: Aaron Pham <[email protected]>
Signed-off-by: Chendi Xue <[email protected]>
Co-authored-by: Aaron Pham <[email protected]>
@tlrmchlsmth
[CI/Build] Replace mean with torch.all in test_pynccl.py (vllm-projec…
d2bd88b 
…t#10876)

Signed-off-by: Tyler Michael Smith <[email protected]>
@wangxiyuan
Drop ROCm load format check (vllm-project#10767)
b5b647b 
Signed-off-by: wangxiyuan <[email protected]>
@khluu
[ci/build] Change queue name for Release jobs (vllm-project#10875)
fa2dea6
@khluu
[ci/build] Job to build and push release image (vllm-project#10877)
c9ca4fc
@o2363286
[bugfix] fixed parameter “n” when set parameter “bestof” > 1 (vllm-pr…
8db957e 
…oject#10854)

Signed-off-by: jianzheng <[email protected]>
@khluu
[ci/build] Update vLLM postmerge ECR repo (vllm-project#10887)
c92acb9
@xuechendi xuechendi force-pushed the multi_models_rebase branch from 0915219 to 655b610 Compare December 4, 2024 17:16
jikunshang and others added 4 commits December 4, 2024 20:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.