You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
vLLM supports structured output capability with guided decoding, but the vLLM backend does not have this capability.
Describe the solution you'd like
Add Guided Decoding for vllm backend, so we can using it throught generate endpoint.
Describe alternatives you've considered
We could write our own backend modification, but it would be nice if it worked out of the box.
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
vLLM supports structured output capability with guided decoding, but the vLLM backend does not have this capability.
Describe the solution you'd like
Add Guided Decoding for vllm backend, so we can using it throught generate endpoint.
Describe alternatives you've considered
We could write our own backend modification, but it would be nice if it worked out of the box.
The text was updated successfully, but these errors were encountered: