[Feature] batch concurrent requests while streaming responses

[moxiegushi]

Checklist

• 1. If the issue you raised is not a feature but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
• 2. Please use English, otherwise it will be closed.

Motivation

Does sglang support handling batch concurrent requests while streaming responses for each one ?

Related resources

No response

[zhaochenyang20]

You could directly try this and try to test it on yourselves. Hope you can share your results, thanks!

I also feel interested.😂