diff --git a/README.md b/README.md index 48e6dbd7..a6fb62dd 100644 --- a/README.md +++ b/README.md @@ -1,88 +1,143 @@ -# SuperRag - -Super-performant RAG pipeline for AI Agents/Assistants. - -## API - -### POST /api/v1/ingest - -Input example: -```json -{ - "files": [ - { - "type": "PDF", - "url": "https://path-to-my-file.pdf" - } - ], - "vector_database": { - "type": "qdrant", - "config": { - "api_key": "my_api_key", - "host": "my_qdrant_host" - } - }, - "index_name": "my_index", - "encoder": { - "type": "openai", - "name": "text-embedding-3-small", - "dimensions": 1536 # encoder depends on the provider and model - }, - "webhook_url": "https://my-webhook-url" -} -``` - -### POST /api/v1/query - -Input example: -```json -{ - "input": "A query", - "vector_database": { - "type": "qdrant", - "config": { - "api_key": "my_api_key", - "host": "my_qdrant_host" - } - }, - "index_name": "my_index", - "encoder": { - "type": "openai", - "name": "text-embedding-3-small", - }, - "interpreter_mode": False, # Set to True if you wish to run computation Q&A with a code interpreter - "session_id": "my_session_id" # keeps micro-vm sessions and enables caching -} -``` - -### DELETE /api/v1/delete - -Input example: -```json -{ - "file_url": "A file url to delete", - "vector_database": { - "type": "qdrant", - "config": { - "api_key": "my_api_key", - "host": "my_qdrant_host" - } - }, - "index_name": "my_index", -} -``` - -## Supported file types - -- PDF -- TXT -- MARKDOWN -- PPTX -- DOCX +
+ +

Super-Rag 🥷

+

+ Super performant RAG pipeline for AI apps. +

+
+
+
+

+ Features • + Installation • + How to use + Docs +

+
-## Supported vector databases -- qdrant -- pinecone -- weaviate -- astra +## Key features + - Supports multiple document formats and vector databases. + - Provides a production ready REST API. + - Includes options for encoding data using different encoding models both propriatory and open source. + - Built in code interpreter mode for computational question & answer scenarios. + - Allows session management through unique IDs for caching purposes. + +## Installation + +1. Clone the repository + ```bash + git clone https://github.com/superagent-ai/super-rag + cd super-rag + ``` + +2. Setup virtual environment + ```bash + # Using virtualenv + virtualenv env + source env/bin/activate + + # Or using venv + python3 -m venv env + source env/bin/activate + ``` + +3. Install requried packages + ```bash + poetry install + ``` + +4. Rename `.env.example` to `.env` and set your environment variables + +5. Run server + ```bash + uvicorn main:app --reload + ``` + +## How to use +Super-Rag comes with a built in REST API powered by FastApi. + +### Ingest documents + ```json + // POST: /api/v1/ingest + + // Payload + { + "files": [{ + "url": "https://arxiv.org/pdf/2210.03629.pdf" + }], + "vector_database": { + "type": "qdrant", + "config": { + "api_key": "YOUR API KEY", + "host": "THE QDRANT HOST" + } + }, + "encoder": { + "type": "openai", + "name": "text-embedding-3-small", + "dimensions": 1536 // encoder depends on the provider and model + }, + "index_name": "YOUR INDEX", + "webhook_url": "https://webhook.site/0e217d1c-49f1-424a-9992-497db09f7793" + } + ``` + +### Query documents + ```json + // POST: /api/v1/query + + // Payload + { + "input": "What is ReAct", + "vector_database": { + "type": "qdrant", + "config": { + "api_key": "YOUR API KEY", + "host": "THE QDRANT HOST" + } + }, + "index_name": "YOUR INDEX", + "interpreter_mode": true, + "encoder": { + "type": "cohere", + "name": "embed-multilingual-light-v3.0", + "dimensions": 384 + }, + "exclude_fields": ["metadata"], + "session_id": "test" + } + ``` + +### Delete document + ```json + // POST: /api/v1/delete + + // Payload + { + "file_url": "A file url to delete", + "vector_database": { + "type": "qdrant", + "config": { + "api_key": "YOUR API KEY", + "host": "THE QDRANT HOST" + } + }, + "index_name": "my_index", + } + + ``` + +## Supportd encoders +- OpenAi +- Cohere +- HuggingFace +- FastEmbed + + +## Supported vector databases +- Weaviate +- Qdrant +- Weaviate +- Astra +- Supabase (coming soon)