Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Added logprobs to vllm and nemo #264

Open
wants to merge 66 commits into
base: main
Choose a base branch
from
Open

Added logprobs to vllm and nemo #264

wants to merge 66 commits into from
+257 −34

Conversation

i-vainn
Copy link
Collaborator

@i-vainn i-vainn commented Nov 25, 2024

No description provided.

shtoshni and others added 19 commits November 5, 2024 10:08
@shtoshni
Allow for not passing prompt config/template
691a0c6
@shtoshni
Allow for not passing prompt config/template
c1d49d5
@shtoshni
Merge branch 'main' into gen_rm
0f24697
@shtoshni
Prompt config for math generative RM
805ed64
@shtoshni
Merge branch 'main' into gen_rm
2df0e4b
@i-vainn
added include_generation parameter to generation config
73a77a1
@i-vainn
modified conf template
52e4290
@i-vainn
make vllm return logprobs
9d7e9ae
@i-vainn
Merge branch 'main' into imoshkov/continue_generation
d03a33f
@i-vainn
new prompt template
34185ad
@i-vainn
merged with main
8e06822
@i-vainn
added logprobs param
ea6691e
@i-vainn
added logprobs param
f55bfc2
@i-vainn
Merge branch 'main' into imoshkov/continue_generation
fca3203
@i-vainn
Merge branch 'imoshkov/continue_generation' into imoshkov/debiasing_a…
0244e1b 
…nswer
@i-vainn
added logprobs inference param
c25368f
@i-vainn
attempted to add nemo logprobs
12add35
@i-vainn
removed exp files from commit
1bbaf6d
@i-vainn
added template
d779ef1
@Kipok
Copy link
Collaborator

Kipok commented Dec 4, 2024

@i-vainn please mark as non-draft when this is ready for review

Kipok
Kipok approved these changes Feb 11, 2025
View reviewed changes
Copy link
Collaborator

@Kipok Kipok left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As long as this is well tested I'm good with merging. Left a few comments to clean up

nemo_skills/inference/server/model.py Show resolved Hide resolved
nemo_skills/inference/server/model.py Show resolved Hide resolved
nemo_skills/inference/server/serve_trt.py Outdated Show resolved Hide resolved
nemo_skills/inference/server/serve_trt.py Outdated Show resolved Hide resolved
nemo_skills/inference/server/serve_trt.py Outdated Show resolved Hide resolved
nemo_skills/inference/server/serve_trt.py Outdated Show resolved Hide resolved
nemo_skills/inference/server/serve_trt.py Outdated Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@shtoshni shtoshni shtoshni left review comments

@Kipok Kipok Kipok approved these changes

Assignees
No one assigned
Labels
run GPU tests
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@i-vainn @Kipok @shtoshni