You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Images must be in binary or base64-encoded formats to be understood by popular MM embedding models. As such, we should add helper UI around producing such formats, based on raw images users can drag into the UI. And/or, provide a URL to some image.
The rough flow can be:
Users opens a portal for selecting an image from local filesystem, dragging directly, or providing a url
User selects what format they want to convert it to (can be auto-populated based on the associated model, ideally)
UI handles conversion and injecting into the sample document (if ingest) or query (if search)
Furthermore, URL support is targeted to be an optional input by 2.19 - see opensearch-project/ml-commons#3054. As such, we can add URL with no further conversion as an option once that is available.
The text was updated successfully, but these errors were encountered:
Im trying to think the big picture what are some use cases a user would do once they have this requirement. For example searching an image by text "a orange cat". But what else are we aiming for?
This issue is around allowing users on the UI to search with an image, such as to find related images, as one example. This may / may not also include text.
This issue is just around the UI supporting searching with an image, and converting it to the supported formats. The open ML commons issue is just expanding on the formats/types that can be used as input for an image (a url).
Images must be in binary or base64-encoded formats to be understood by popular MM embedding models. As such, we should add helper UI around producing such formats, based on raw images users can drag into the UI. And/or, provide a URL to some image.
The rough flow can be:
Furthermore, URL support is targeted to be an optional input by 2.19 - see opensearch-project/ml-commons#3054. As such, we can add URL with no further conversion as an option once that is available.
The text was updated successfully, but these errors were encountered: