Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[backend] Expose suggested models #419

Merged
merged 11 commits into from
Nov 25, 2024
Merged

[backend] Expose suggested models #419

merged 11 commits into from
Nov 25, 2024

Conversation

dpoulopoulos
Copy link
Contributor

@dpoulopoulos dpoulopoulos commented Nov 21, 2024 •
edited
Loading

What's changing

This PR introduces a new /models endpoint that returns a list of suggested models for a given task. Currently, the only supported task is "summarization."

The suggested models are stored in a YAML file, along with details such as number of parameters, memory footprint, and default values. Not all information is available for every model. For example, OpenAI models do not provide memory footprint data, while other models may not include summarization-specific default parameters.

Additionally, this PR adds a new SDK function for calling the /models endpoint via Python, along with the corresponding documentation.

How to test it

  1. Run a local Lumigator deployment: make local-up
  2. Run the following command: 
    curl -s http://localhost:8000/api/v1/models/summarization | jq

Additional notes for reviewers

If you request the suggested models for a task that Lumigator doesn't support (e.g., curl -s http://localhost:8000/api/v1/models/translation | jq), you should get an HTTP 400 error:

{
  "detail": "Unsupported task. Choose from: ['summarization']"
}

I already...

  • added some tests for any new functionality
  • updated the documentation
  • [NA] checked if a (backend) DB migration step was required and included it if required

Closes #381

@dpoulopoulos dpoulopoulos linked an issue Nov 21, 2024 that may be closed by this pull request
[ENH] Expose a list of supported models #381
Closed
@dpoulopoulos dpoulopoulos self-assigned this Nov 21, 2024
@github-actions github-actions bot added documentation Improvements or additions to documentation sdk backend api Changes which impact API/presentation layer schemas Changes to schemas (which may be public facing) labels Nov 21, 2024
@dpoulopoulos dpoulopoulos added enhancement New feature or request and removed schemas Changes to schemas (which may be public facing) labels Nov 21, 2024
@dpoulopoulos dpoulopoulos force-pushed the 381-expose-suggested-models branch from c8d49c9 to dca4861 Compare November 21, 2024 11:44
@github-actions github-actions bot added the schemas Changes to schemas (which may be public facing) label Nov 21, 2024
@dpoulopoulos dpoulopoulos force-pushed the 381-expose-suggested-models branch 9 times, most recently from 0d76054 to 562fe31 Compare November 21, 2024 14:13
@dpoulopoulos dpoulopoulos marked this pull request as ready for review November 21, 2024 14:17
@dpoulopoulos dpoulopoulos force-pushed the 381-expose-suggested-models branch 2 times, most recently from 1784aa6 to dad88a2 Compare November 21, 2024 14:47
veekaybee
veekaybee reviewed Nov 21, 2024
View reviewed changes
docs/source/get-started/suggested-models.md Outdated Show resolved Hide resolved
lumigator/python/mzai/backend/backend/api/routes/models.py Outdated Show resolved Hide resolved
lumigator/python/mzai/backend/backend/models.yaml Outdated Show resolved Hide resolved
lumigator/python/mzai/backend/backend/tests/api/routes/test_models.py Outdated Show resolved Hide resolved
lumigator/python/mzai/backend/backend/tests/api/routes/test_models.py Outdated Show resolved Hide resolved
lumigator/python/mzai/backend/backend/tests/api/routes/test_models.py Outdated Show resolved Hide resolved
lumigator/python/mzai/schemas/lumigator_schemas/models.py Outdated Show resolved Hide resolved
lumigator/python/mzai/sdk/tests/data/models.json Show resolved Hide resolved
@dpoulopoulos dpoulopoulos force-pushed the 381-expose-suggested-models branch from dad88a2 to 01d488a Compare November 22, 2024 10:44
@dpoulopoulos dpoulopoulos changed the title 381 expose suggested models [backend] Expose suggested models Nov 22, 2024
@dpoulopoulos dpoulopoulos force-pushed the 381-expose-suggested-models branch 2 times, most recently from a28f6e3 to e27b724 Compare November 22, 2024 13:54
veekaybee
veekaybee approved these changes Nov 22, 2024
View reviewed changes
Copy link
Contributor

@veekaybee veekaybee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good after the final round of addressed changes, thanks for adding this!

@dpoulopoulos dpoulopoulos force-pushed the 381-expose-suggested-models branch 3 times, most recently from a1426d8 to 12da903 Compare November 22, 2024 15:56
@dpoulopoulos
backend: Introduce models endpoint
a5db6ac 
Introduce a new `/models` endpoint that returns a list of suggested
models for a given task.

Currently, the only supported task is that of summarization.

Refs #381

Signed-off-by: Dimitris Poulopoulos <[email protected]>
@dpoulopoulos
backend: Move suggested models in a YAML file
3daa40e 
Move the suggested models list in a YAML file. Read the YAML file during
runtime to respond appropriately.

Refs #381

Signed-off-by: Dimitris Poulopoulos <[email protected]>
@dpoulopoulos
models: Replace GPT-4-Turbo with GPT-4o
1649033 
GPT-4-Turbo is a legacy OpenAI model.

Replace it with GPT-4o, which is the latest iteration of OpenAI's
flagship model.

Signed-off-by: Dimitris Poulopoulos <[email protected]>
@dpoulopoulos
backend: Extend the models YAML file
ead2fe2 
Extend the models YAML file to include model-specific information.

Add model-specific details, such as the number of parameters, memory
footprint, and tensor type (e.g., FP32, BF16, etc.), along with a set
of default values used by Lumigator when calling the models.

Not all parameters are applicable to each model. For example, Hugging
Face models have default values for parameters like `max_length`, while
API models include parameters like `temperature`. In general, parameters
are model-specific, so there is no universal set of parameter names.

The defaults are inferred either by checking the documentation for each
model or by examining the configuration file on the Hugging Face Hub.

Refs #381

Signed-off-by: Dimitris Poulopoulos <[email protected]>
@dpoulopoulos
backend: Get supported tasks from models YAML file
364669d 
Use the models YAML file to retrieve the list of supported tasks,
instead of hardcoding it in the code.

Refs #381

Signed-off-by: Dimitris Poulopoulos <[email protected]>
@dpoulopoulos
schemas: Validate Models Response object
0a9b13f 
Validate the response of the ``/models`` endpoint using Pydantic.

For each model, there are a few required fields:

* name
* uri
* description

There are also some optional fields, since not all models have available
information (e.g., we don't have the model size for OpenAI models) or
default parameters.

Finally, the default parameters dictionary is a generic `dict`, as there
is no guaranteed uniformity.

Refs #381

Signed-off-by: Dimitris Poulopoulos <[email protected]>
@dpoulopoulos
backend: Test models endpoint
087b568 
Signed-off-by: Dimitris Poulopoulos <[email protected]>
@dpoulopoulos
sdk: Retrieve suggested models via the SDK
9b460a0 
Retrieve the suggested models per task via the Lumigator SDK client.

Refs #381

Signed-off-by: Dimitris Poulopoulos <[email protected]>
@dpoulopoulos
sdk: Test the models route
10487de 
Signed-off-by: Dimitris Poulopoulos <[email protected]>
@dpoulopoulos
docs: Add suggested models documentation
3220ce6 
Add a page to the documentation listing the suggested models
and their details, along with instructions on how to retrieve
them using the API.

Closes #381

Signed-off-by: Dimitris Poulopoulos <[email protected]>
@dpoulopoulos
docs: Fix SDK and Schema reference guide
8861585 
Signed-off-by: Dimitris Poulopoulos <[email protected]>
@dpoulopoulos dpoulopoulos force-pushed the 381-expose-suggested-models branch from 12da903 to 8861585 Compare November 25, 2024 10:29
@dpoulopoulos dpoulopoulos merged commit 19b9ade into main Nov 25, 2024
7 checks passed
@dpoulopoulos dpoulopoulos deleted the 381-expose-suggested-models branch November 25, 2024 10:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ividal ividal ividal left review comments

@veekaybee veekaybee veekaybee approved these changes

@aittalam aittalam Awaiting requested review from aittalam

Assignees

@dpoulopoulos dpoulopoulos

Labels
api Changes which impact API/presentation layer backend documentation Improvements or additions to documentation enhancement New feature or request schemas Changes to schemas (which may be public facing) sdk
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[ENH] Expose a list of supported models
4 participants
@dpoulopoulos @ividal @veekaybee @aittalam