Welcome to Atlas

This platform acts as a starter template for building a knowledgebase and chatbot system. It's not perfect but it's a good starting point for building a more complex system.

Process

Authenticate using next-auth:

Upload a file to the uploadthing API and process:

• Upload a file to the uploadthing API Process selected files in the knowledgebase UI to:
• Parse and chunk them into smaller pieces, using the Unstructured.io API, or Unstructured local container
• Embed the chunks using OpenAi text-embedding-3-large
• Store the embedded chunks in Pinecone Serverless, or Qdrant local container

Query your knowledgebase

• Proceed to the chat route and query your documents
• Each user message will go through the process of:
  • Embedding the message using OpenAi text-embedding-3-large
  • Perform a query to Pinecone Serverless, or Qdrant local container and retrieve the top K results
  • Rerank the results using Cohere's reranking model and return the top N result subset
  • Reranked chunks are used to enrich the user message and sent to OpenAi for a response

Chat Options

• In the chat window, you can see the chat options on the bottom right of the screen

• Start a new conversation (You can view your conversations by clicking the chevron on the middle right of the screen)

• Save your conversation

• Switch to the Analysis Assistant, which uses the code Interpreter (This currently disables access to your knowledgebase)

• Speech is not currently implemented

Settings / Personalization

• Select Settings
• You can now see the setings window, which allows you to customize the application
• The Document Processing and Vectorization tabs in the Forge allow detailed customization of how the files get processed and stored in your knowledgebase.
• The Knowledgebase settings allow customization of the response retrieval
• Finally, the Profile settings allow personalization of assistant responses, if enabled
• You can switch your theme by clicking the highlighted button

Telemetry/Metrics

• If you've configured grafana-alloy-example, you can navigate to the Dashboard/Analytics tab. How and what you want to view in the Analytics Dashboard is highly configurable and covered in grafana-alloy-example

Open source alternatives

You can use Qdrant and Unstructured local containers to replace Pinecone and Unstructured.io respectively, by deploying these containers and selecting the appropriate settings in the Forge.

services:
  qdrant:
    image: qdrant/qdrant:latest
    container_name: qdrant
    ports:
      - '6333:6333'
    volumes:
      - qdrant_data:/qdrant/storage
    environment:
      QDRANT__SERVICE__GRPC_PORT: 6334
      QDRANT__SERVICE__HTTP_PORT: 6333

  unstructured-api:
    image: downloads.unstructured.io/unstructured-io/unstructured-api:latest
    container_name: unstructured-api
    ports:
      - '8000:8000'
    depends_on:
      - qdrant

volumes:
  qdrant_data:
    driver: local

UI Inspired by

Shadcn Dashboard Starter

That's it! Make sure to set all required environment variables before starting your application.