diff --git a/docs/ai/extai.rst b/docs/ai/extai.rst index 69c0ab34453..aa2d0fba2b5 100644 --- a/docs/ai/extai.rst +++ b/docs/ai/extai.rst @@ -1,11 +1,10 @@ .. _ref_ai_extai_reference: -============ -AI Extension -============ +======= +ext::ai +======= -This reference documents the |Gel| AI extension's components, configuration -options, and APIs. +This reference documents the |Gel| ``ext::ai`` extension components, configuration options, and database APIs. Enabling the Extension @@ -43,7 +42,8 @@ Reset configuration: .. code-block:: edgeql - configure current branch reset ext::ai::Config::indexer_naptime; + configure current branch + reset ext::ai::Config::indexer_naptime; .. _ref_ai_extai_reference_ui: @@ -93,6 +93,7 @@ Enables management of API configurations for AI API providers. :width: 100% + .. _ref_ai_extai_reference_index: Index @@ -134,48 +135,6 @@ Parameters: * ``truncate_to_max``: bool (Default: False) -Built-in resources -================== - -.. _ref_ai_extai_reference_embedding_models: - -Embedding models ----------------- - -OpenAI (`documentation `__) - -* ``text-embedding-3-small`` -* ``text-embedding-3-large`` -* ``text-embedding-ada-002`` - -Mistral (`documentation `__) - -* ``mistral-embed`` - - -.. _ref_ai_extai_reference_text_generation_models: - -Text generation models ----------------------- - -OpenAI (`documentation `__) - -* ``gpt-3.5-turbo`` -* ``gpt-4-turbo-preview`` - -Mistral (`documentation `__) - -* ``mistral-small-latest`` -* ``mistral-medium-latest`` -* ``mistral-large-latest`` - -Anthropic (`documentation `__) - -* ``claude-3-haiku-20240307`` -* ``claude-3-sonnet-20240229`` -* ``claude-3-opus-20240229`` - - Functions ========= @@ -272,8 +231,8 @@ Functions } -Types -===== +Scalar and Object Types +======================= Provider Configuration Types ---------------------------- @@ -350,6 +309,45 @@ Example provider configuration: Model Types ----------- +.. _ref_ai_extai_reference_embedding_models: + +Embedding models +^^^^^^^^^^^^^^^^ + +OpenAI (`documentation `__) + +* ``text-embedding-3-small`` +* ``text-embedding-3-large`` +* ``text-embedding-ada-002`` + +Mistral (`documentation `__) + +* ``mistral-embed`` + + +.. _ref_ai_extai_reference_text_generation_models: + +Text generation models +^^^^^^^^^^^^^^^^^^^^^^ + +OpenAI (`documentation `__) + +* ``gpt-3.5-turbo`` +* ``gpt-4-turbo-preview`` + +Mistral (`documentation `__) + +* ``mistral-small-latest`` +* ``mistral-medium-latest`` +* ``mistral-large-latest`` + +Anthropic (`documentation `__) + +* ``claude-3-haiku-20240307`` +* ``claude-3-sonnet-20240229`` +* ``claude-3-opus-20240229`` + + .. list-table:: :class: funcoptable @@ -385,6 +383,7 @@ Model Types * ``embedding_model_supports_shortening`` - Input shortening support flag + --------- .. eql:type:: ext::ai::TextGenerationModel diff --git a/docs/ai/extvectorstore.rst b/docs/ai/extvectorstore.rst new file mode 100644 index 00000000000..973be9d7aae --- /dev/null +++ b/docs/ai/extvectorstore.rst @@ -0,0 +1,81 @@ +.. _ref_extvectorstore_reference: + +================ +ext::vectorstore +================ + + +The ``ext::vectorstore`` extension package provides simplified vectorstore +workflows for |Gel|, built on top of the pgvector integration. It includes +predefined vector dimensions and a base schema for vector storage records. + + +Enabling the extension +====================== + +The extension package can be installed using the :gelcmd:`extension` CLI +command: + +.. code-block:: bash + + $ gel extension install vectorstore + + +It can be enabled using the :ref:`extension ` +mechanism: + +.. code-block:: sdl + + using extension vectorstore; + + +The Vectorstore extension is designed to be used in combination with the +:ref:`Vectostore Python binding ` or other +integrations, rather than on its own. + + +Types +===== + +Vector Types +------------ + +The extension provides two pre-defined vector types with different dimensions: + +- ``ext::vectorstore::vector_1024``: 1024-dimensional vector +- ``ext::vectorstore::vector_1536``: 1536-dimensional vector + +All vector types extend ``ext::pgvector::vector`` with their respective dimensions. + + +Record Types +------------ + +.. eql:type:: ext::vectorstore::BaseRecord + + Abstract type that defines the basic structure for vector storage records. + + Properties: + + * ``collection: str`` (required): Identifies the collection the record belongs to + * ``text: str``: Associated text content + * ``embedding: ext::pgvector::vector``: The vector embedding + * ``external_id: str``: External identifier with unique constraint + * ``metadata: json``: Additional metadata in JSON format + + +.. eql:type:: ext::vectorstore::DefaultRecord + + Extends :eql:type:`ext::vectorstore::BaseRecord` with specific + configurations. + + Properties: + + * Inherits all properties from :eql:type:`ext::vectorstore::BaseRecord` + * Specializes ``embedding`` to use ``vector_1536`` type + * Includes an HNSW cosine similarity index on the embedding with: + + * ``m = 16`` + * ``ef_construction = 128`` + + diff --git a/docs/ai/index.rst b/docs/ai/index.rst index e8bc9f65903..e1e4547325d 100644 --- a/docs/ai/index.rst +++ b/docs/ai/index.rst @@ -1,8 +1,10 @@ .. _ref_ai_overview: -====== -Gel AI -====== +== +AI +== + +:edb-alt-title: Using Gel AI .. toctree:: :hidden: @@ -12,11 +14,11 @@ Gel AI http python javascript - -:edb-alt-title: Using Gel AI + extvectorstore + vectorstore_python |Gel| AI is a set of tools designed to enable you to ship AI-enabled apps with -practically no effort. This is what comes in the box: +practically no effort. 1. ``ext::ai``: this Gel extension automatically generates embeddings for your data. Works with OpenAI, Mistral AI, Anthropic, and any other provider with a @@ -30,7 +32,5 @@ practically no effort. This is what comes in the box: 3. Python library: ``gel.ai``. Access all Gel AI features straight from your Python application. -4. JavaScript library: ``@gel/ai``. - - - +4. JavaScript library: ``@gel/ai``. Access all Gel AI features right from your + JavaScript backend application. diff --git a/docs/ai/javascript.rst b/docs/ai/javascript.rst index 6b54192104a..8a81431a6e1 100644 --- a/docs/ai/javascript.rst +++ b/docs/ai/javascript.rst @@ -1,110 +1,72 @@ -.. _ref_ai_javascript: +.. _ref_ai_javascript_reference: ============== JavaScript API ============== -``@gel/ai`` offers a convenient wrapper around ``ext::ai``. Install it with -npm or via your package manager of choice: +``@gel/ai`` is a wrapper around the :ref:`AI extension +` in |Gel|. .. tabs:: .. code-tab:: bash + :caption: npm $ npm install @gel/ai .. code-tab:: bash + :caption: yarn $ yarn add @gel/ai .. code-tab:: bash + :caption: pnpm $ pnpm add @gel/ai .. code-tab:: bash + :caption: bun $ bun add @gel/ai -Usage -===== +Overview +======== -Start by importing ``createClient`` from ``gel`` and ``createAI`` from -``@gel/ai``: +The AI package is built on top of the regular |Gel| client objects. + +**Example**: .. code-block:: typescript import { createClient } from "gel"; - import { createAI } from "@gel/ai"; - -Create a |Gel| client. Create an instance of the AI client by passing in the -Gel client and any options for the AI provider (like the text generation -model): + import { createRAGClient } from "@gel/ai"; -.. code-block:: typescript const client = createClient(); - const gpt4Ai = createAI(client, { + const gpt4Ai = createRAGClient(client, { model: "gpt-4-turbo-preview", }); -You may use any of the supported :ref:`text generation models -`. Add your query as context: - -.. code-block:: typescript - const astronomyAi = gpt4Ai.withContext({ query: "Astronomy" }); -This "query" property doesn't have to be a proper query at all. It can be any -expression that produces a set of objects, like ``Astronomy`` in the example -above which will return all objects of that type. On the other hand, if you -want to narrow the field more, you can give it a query like ``select Astronomy -filter .topic = "Mars"``. - -The default text generation prompt will ask your selected provider to limit -answer to information provided in the context and will pass the queried -objects' AI index as context along with that prompt. - -Call your AI client's ``queryRag`` method, passing in a text query. - -.. code-block:: typescript - console.log( await astronomyAi.queryRag("What color is the sky on Mars?") ); -You can chain additional calls of ``withContext`` or ``withConfig`` to create -additional AI clients, identical except for the newly specified values. - -.. code-block:: typescript - - const fastAstronomyAi = astronomyAi.withConfig({ - model: "gpt-3.5-turbo", - }); - console.log( - await fastAstronomyAi.queryRag("What color is the sky on Mars?") - ); - - const fastChemistryAi = fastAstronomyAi.withContext({ - query: "Chemistry" - }); - console.log( - await fastChemistryAi.queryRag("What is the atomic number of gold?") - ); +Factory functions +================= -API Reference -============= - -.. js:function:: createAI( \ +.. js:function:: createRAGClient( \ client: Client, \ - options: Partial = {} \ - ): GelAI + options: Partial = {} \ + ): RAGClient - Creates an instance of ``GelAI`` with the specified client and options. + Creates an instance of ``RAGClient`` with the specified client and options. :param client: A |Gel| client instance. @@ -120,18 +82,20 @@ API Reference interactions. The default is the built-in system prompt. -GelAI ------ +Core classes +============ + + +.. js:class:: RAGClient -Instances of ``GelAI`` offer methods for client configuration and utilizing -RAG. + Instances of ``RAGClient`` offer methods for client configuration and utilizing RAG. -Public methods -^^^^^^^^^^^^^^ + :ivar client: + An instance of |Gel| client. -.. js:method:: withConfig(options: Partial): GelAI +.. js:method:: withConfig(options: Partial): RAGClient - Returns a new ``GelAI`` instance with updated configuration options. + Returns a new ``RAGClient`` instance with updated configuration options. :param string options.model: Required. Specifies the AI model to use. This could be a version of GPT @@ -143,9 +107,9 @@ Public methods structure that includes roles and content for more complex interactions. The default is the built-in system prompt. -.. js:method:: withContext(context: Partial): GelAI +.. js:method:: withContext(context: Partial): RAGClient - Returns a new ``GelAI`` instance with an updated query context. + Returns a new ``RAGClient`` instance with an updated query context. :param string context.query: Required. Specifies an expression to determine the relevant objects and diff --git a/docs/ai/python.rst b/docs/ai/python.rst index e752d9a43ca..a4bf3b58bfe 100644 --- a/docs/ai/python.rst +++ b/docs/ai/python.rst @@ -4,7 +4,8 @@ Python API ========== -The ``gel.ai`` package is an optional binding of the AI extension in |Gel|. +The ``gel.ai`` package is an optional binding of the :ref:`AI extension +` in |Gel|. .. code-block:: bash diff --git a/docs/ai/vectorstore_python.rst b/docs/ai/vectorstore_python.rst new file mode 100644 index 00000000000..4ae8240807a --- /dev/null +++ b/docs/ai/vectorstore_python.rst @@ -0,0 +1,313 @@ +.. _ref_ai_vectorstore_python: + +====================== +Vectorstore Python API +====================== + + +Core Classes +============ + +.. py:class:: GelVectorstore + + A framework-agnostic interface for interacting with |Gel's| ext::vectorstore. + + This class provides methods for storing, retrieving, and searching + vector embeddings. It follows vector database conventions and supports + different embedding models. + + Args: + + * ``embedding_model`` (:py:class:`BaseEmbeddingModel`): The embedding model used to generate vectors. + * ``collection_name`` (str): The name of the collection. + * ``record_type`` (str): The schema type (table name) for storing records. + * ``client_config`` (dict | None): The config for the |Gel| client. + + +.. py:method:: add_items(self, items: list[InsertItem]) + + Add multiple items to the vector store in a single transaction. Embeddings + will be generated and stored for all items. + + Args: + + * ``items`` (list[:py:class:`InsertItem`]): List of items to add. Each contains: + + * ``text`` (str): The text content to be embedded + * ``metadata`` (dict[str, Any]): Additional data to store + + Returns: + + * List of database record IDs for the inserted items. + + +.. py:method:: add_vectors(self, records: list[InsertRecord]) + + Add pre-computed vector embeddings to the store. Use this method when you have + already generated embeddings and want to store them directly without re-computing them. + + Args: + + * ``records`` (list[:py:class:`InsertRecord`]): List of records. Each contains: + + * ``embedding`` (list[float]): Pre-computed embeddings + * ``text`` (Optional[str]): Original text content + * ``metadata`` (dict[str, Any]): Additional data to store + + Returns: + + * List of database record IDs for the inserted items. + +.. py:method:: delete(self, ids: list[uuid.UUID]) + + Delete records from the vector store by their IDs. + + Args: + + * ``ids`` (list[uuid.UUID]): List of record IDs to delete. + + Returns: + + * List of deleted record IDs. + +.. py:method:: get_by_ids(self, ids: list[uuid.UUID]) -> list[Record] + + Retrieve specific records by their IDs. + + Args: + + * ``ids`` (list[uuid.UUID]): List of record IDs to retrieve. + + Returns: + + * List of retrieved records. Each result contains: + + * ``id`` (uuid.UUID): The record's unique identifier + * ``text`` (Optional[str]): The original text content + * ``embedding`` (Optional[list[float]]): The stored vector embedding + * ``metadata`` (Optional[dict[str, Any]]): Any associated metadata + +.. py:method:: search_by_item(self, item: Any, filters: Optional[CompositeFilter] = None, limit: Optional[int] = 4) -> list[SearchResult] + + Search for similar items in the vector store. + + This method: + + 1. Generates an embedding for the input item + 2. Finds records with similar embeddings + 3. Optionally filters results based on metadata + 4. Returns the most similar items up to the specified limit + + Args: + + * ``item`` (Any): The query item to find similar matches for. Must be compatible with the embedding model's target_type. + * ``filters`` (Optional[:py:class:`CompositeFilter`]): Metadata-based filters to use. + * ``limit`` (Optional[int]): Max number of results to return. Defaults to 4. + + Returns: + + * List of similar items, ordered by similarity. Each result contains: + + * ``id`` (uuid.UUID): The record's unique identifier + * ``text`` (Optional[str]): The original text content + * ``embedding`` (list[float]): The stored vector embedding + * ``metadata`` (Optional[dict[str, Any]]): Any associated metadata + * ``cosine_similarity`` (float): Similarity score (higher is more similar) + +.. py:method:: search_by_vector(self, vector: list[float], filter_expression: str = "", limit: Optional[int] = 4) -> list[SearchResult] + + Search using a pre-computed vector embedding. Useful when you have already computed + the embedding or want to search with a modified/combined embedding vector. + + Args: + + * ``vector`` (list[float]): The query embedding to search with. Must match the dimensionality of stored embeddings. + * ``filter_expression`` (str): Filter expression for metadata filtering. + * ``limit`` (Optional[int]): Max number of results to return. Defaults to 4. + + Returns: + + * List of similar items, ordered by similarity. Each result contains: + + * ``id`` (uuid.UUID): The record's unique identifier + * ``text`` (Optional[str]): The original text content + * ``embedding`` (list[float]): The stored vector embedding + * ``metadata`` (Optional[dict[str, Any]]): Any associated metadata + * ``cosine_similarity`` (float): Similarity score (higher is more similar) + +.. py:method:: update_record(self, record: Record) -> Optional[uuid.UUID] + + Update an existing record in the vector store. Only specified fields will be updated. + If text is provided but not embedding, a new embedding will be automatically generated. + + Args: + + * ``record`` (:py:class:`Record`): + + * ``id`` (uuid.UUID): The ID of the record to update + * ``text`` (Optional[str]): New text content. If provided without embedding, a new embedding will be generated. + * ``embedding`` (Optional[list[float]]): New vector embedding. + * ``metadata`` (Optional[dict[str, Any]]): New metadata to store with the record. Completely replaces existing metadata. + + Returns: + + * The updated record's ID if found and updated, None if no record was found with the given ID. + + Raises: + + * ValueError: If no fields are specified for update. + + +.. py:class:: BaseEmbeddingModel + + Abstract base class for embedding models. + Any embedding model used with :py:class:`GelVectorstore` must implement this + interface. The model is expected to convert input data (text, images, etc.) + into a numerical vector representation. + + .. py:method:: __call__(self, item) -> list[float] + + Convert an input item into a list of floating-point values (vector + embedding). Must be implemented in subclasses. + + Args: + + * ``item``: Input item to be converted to an embedding + + Returns: + + * list[float]: Vector embedding of the input item + + .. py:method:: dimensions(self) -> int + + Return the number of dimensions in the embedding vector. + Must be implemented in subclasses. + + Returns: + + * int: Number of dimensions in the embedding vector + + .. py:method:: target_type(self) -> TypeVar + + Return the expected data type of the input (e.g., str for text, image + for vision models). Must be implemented in subclasses. + + Returns: + + * TypeVar: Expected input data type + + +Data Classes +============ + +.. py:class:: InsertItem + + An item whose embedding will be created and stored alongside the item in the vector store. + + Args: + + * ``text`` (str): The text content to be embedded + * ``metadata`` (dict[str, Any]): Additional data to store. Defaults to empty dict. + +.. py:class:: InsertRecord + + A record to be added to the vector store with embedding pre-computed. + + Args: + + * ``embedding`` (list[float]): Pre-computed embeddings + * ``text`` (str | None): Original text content. Defaults to None. + * ``metadata`` (dict[str, Any]): Additional data to store. Defaults to empty dict. + +.. py:class:: Record + + A record retrieved from the vector store, or an update record. + Custom ``__init__`` so we can detect which fields the user passed + (even if they pass None or {}). + + Args: + + * ``id`` (uuid.UUID): The record's unique identifier + * ``text`` (str | None): The text content. Defaults to None. + * ``embedding`` (list[float] | None): The vector embedding. Defaults to None. + * ``metadata`` (dict[str, Any]): Additional data stored with the record. Defaults to empty dict. + +.. py:class:: SearchResult + + A search result from the vector store. + + Inherits from :py:class:`Record` + + Args: + + * ``cosine_similarity`` (float): Similarity score for the search result. Defaults to 0.0. + + +Metadata Filtering +================== + +.. py:class:: FilterOperator + + Enumeration of supported filter operators for metadata filtering. + + Values: + + * ``EQ``: Equal to (=) + * ``NE``: Not equal to (!=) + * ``GT``: Greater than (>) + * ``LT``: Less than (<) + * ``GTE``: Greater than or equal to (>=) + * ``LTE``: Less than or equal to (<=) + * ``IN``: Value in array + * ``NOT_IN``: Value not in array + * ``LIKE``: Pattern matching + * ``ILIKE``: Case-insensitive pattern matching + * ``ANY``: Any array element matches + * ``ALL``: All array elements match + * ``CONTAINS``: String contains value + * ``EXISTS``: Field exists + +.. py:class:: FilterCondition + + Enumeration of conditions for combining multiple filters. + + Values: + + * ``AND``: All conditions must be true + * ``OR``: Any condition must be true + +.. py:class:: MetadataFilter + + Represents a single metadata filter condition. + + Args: + + * ``key`` (str): The metadata field key to filter on + * ``value`` (int | float | str): The value to compare against + * ``operator`` (:py:class:`FilterOperator`): The comparison operator. Defaults to FilterOperator.EQ. + +.. py:class:: CompositeFilter + + Allows grouping multiple MetadataFilter instances using AND/OR conditions. + + Args: + + * ``filters`` (list[:py:class:`CompositeFilter` | :py:class:`MetadataFilter`]): List of filters to combine + * ``condition`` (:py:class:`FilterCondition`): How to combine the filters. Defaults to FilterCondition.AND. + +.. py:function:: get_filter_clause(filters: CompositeFilter) -> str + + Get the filter clause for a given CompositeFilter. + + Args: + + * ``filters`` (:py:class:`CompositeFilter`): The composite filter to convert to a clause + + Returns: + + * str: The filter clause string for use in queries + + Raises: + + * ValueError: If an unknown operator or condition is encountered + diff --git a/docs/intro/guides/ai_edgeql.rst b/docs/intro/guides/ai_edgeql.rst index 940420206bc..afe465a7e92 100644 --- a/docs/intro/guides/ai_edgeql.rst +++ b/docs/intro/guides/ai_edgeql.rst @@ -277,10 +277,12 @@ You are now sufficiently equipped to use |Gel| AI in your applications. If you'd like to build something on your own, make sure to check out the :ref:`Reference manual ` in order to learn the details about using different APIs and models, configuring prompts or using the UI. -Make sure to also check out the |Gel| AI bindings in Python and JavaScript if -those languages are relevant to you. +Make sure to also check out the |Gel| AI bindings in :ref:`Python +` and :ref:`JavaScript ` +if those languages are relevant to you. And if you would like more guidance for how |Gel| AI can be fit into an -application, take a look at the FastAPI Gel AI Tutorial, where we're building a -search bot using features you learned about above. +application, take a look at the :ref:`FastAPI Gel AI Tutorial +`, where we're building a search bot using +features you learned about above. diff --git a/docs/intro/guides/ai_python.rst b/docs/intro/guides/ai_python.rst index db4835b2cbd..57bd33986ad 100644 --- a/docs/intro/guides/ai_python.rst +++ b/docs/intro/guides/ai_python.rst @@ -359,11 +359,14 @@ Keep going! You are now sufficiently equipped to use |Gel| AI in your applications. If you'd like to build something on your own, make sure to check out the -:ref:`Reference manual ` in order to learn the details -about using different APIs and models, configuring prompts or using the UI. +:ref:`Reference manual ` for the AI extension in order +to learn the details about using different APIs and models, configuring prompts +or using the UI. Make sure to take a look at the :ref:`Python binding reference +`, too. And if you would like more guidance for how |Gel| AI can be fit into an -application, take a look at the FastAPI Gel AI Tutorial, where we're building a -search bot using features you learned about above. +application, take a look at the :ref:`FastAPI Gel AI Tutorial +`, where we're building a search bot using +features you learned about above. diff --git a/docs/intro/quickstart/ai/fastapi.rst b/docs/intro/quickstart/ai/fastapi.rst index f7be6a8c677..ada703299cc 100644 --- a/docs/intro/quickstart/ai/fastapi.rst +++ b/docs/intro/quickstart/ai/fastapi.rst @@ -6,12 +6,16 @@ Using the built-in RAG .. edb:split-section:: - In this section we'll use |Gel|'s built-in vector search and - retrieval-augmented generation capabilities to decorate our flashcard app - with a couple AI features. We're going to create a ``/fetch_similar`` - endpoint that's going to look up flashcards similar to a text search query, - as well as a ``/fetch_rag`` endpoint that's going to enable us to talk to - an LLM about the content of our flashcard deck. + In this section we'll learn about |Gel's| built-in vector search and + retrieval-augmented generation capabilities. We'll be continuing from where + we left off in the :ref:`main quickstart `. Feel free to browse the + complete flascards app code in this `repo + `_. + + In this tutorial we'll focus on creating a ``/fetch_similar`` endpoint for + looking up flashcards similar to a text search query, as well as a + ``/fetch_rag`` endpoint that's going to enable us to talk to an LLM about + the content of our flashcard deck. We're going to start with the same schema we left off with in the primary quickstart. @@ -33,17 +37,17 @@ Using the built-in RAG type Deck extending Timestamped { required name: str; description: str; - cards := ( - select .`, or build an LLM-powered search bot from the + ground up with the :ref:`FastAPI Gel AI tutorial + `. diff --git a/docs/intro/tutorials/ai_fastapi_searchbot.rst b/docs/intro/tutorials/ai_fastapi_searchbot.rst index 5c96250f170..3cead353a74 100644 --- a/docs/intro/tutorials/ai_fastapi_searchbot.rst +++ b/docs/intro/tutorials/ai_fastapi_searchbot.rst @@ -20,12 +20,6 @@ human-friendly answer. After that, we'll use Gel to implement chat history so that the bot remembers previous interactions with the user. We'll finish it off with semantic search-based cross-chat memory. -The end result is going to look something like this: - -.. image:: - /docs/tutorials/placeholder.png - :alt: Placeholder - :width: 100% 1. Initialize the project =========================