Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add context window size information to client capabilities #3641

Open
ekzhu opened this issue Oct 3, 2024 · 0 comments
Open

Add context window size information to client capabilities #3641

ekzhu opened this issue Oct 3, 2024 · 0 comments

Comments

@ekzhu
Copy link
Collaborator

ekzhu commented Oct 3, 2024

What feature would you like to be added?

Right now, model capabilities include:

        ModelCapabilities(
            function_calling=True, json_output=True, vision=True
        )

We should add other information that might be needed upstream such as the context window size, the max number of output tokens, perhaps the cost (input and output), as well as the max number of images, the max size of images, etc.

Why is this needed?

This metadata allows application to make decision about context input size, and other parts of the framework to manage context.

@ekzhu ekzhu removed the needs-triage label Oct 3, 2024
@rysweet rysweet added this to the future milestone Oct 22, 2024
@jackgerrits jackgerrits modified the milestones: future, 0.4 Oct 22, 2024
@fniedtner fniedtner modified the milestones: 0.4, future Oct 22, 2024
@fniedtner fniedtner removed the feature label Oct 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants