Skip to content

Latest commit

 

History

History
115 lines (76 loc) · 4.31 KB

README.md

File metadata and controls

115 lines (76 loc) · 4.31 KB

RAG-based Local Chat Box

Author: Md Mutasim Billah Abu Noman Akanda, Machine Learning Engineer
Updated on: 1 October, 2024

Project Description

RAG-based Local Chat Box is a Streamlit-based web application designed to integrate a Local Language Model (LLM) with Retrieval-Augmented Generation (RAG) capabilities. This project allows users to index documents, create embeddings, and interact with their data through an intuitive chat interface powered by state-of-the-art language models. Ideal for researchers and developers, rag-based-local-chat-box facilitates efficient data retrieval and conversational interactions within a local environment.

Features

  • Model Selection: Choose from a variety of local LLM models (Mistral, llama) to suit your needs.
  • Document Indexing: Easily load and index documents from a specified directory.
  • Embedding Creation: Utilize the nomic-embed-text model to create meaningful embeddings for your documents.
  • Interactive Chat Interface: Engage with your indexed data through a seamless chat interface.
  • Session Management: Maintains chat history and model selections across user sessions.
  • Real-time Responses: Receive streaming responses from the LLM for a dynamic interaction experience.

Project Structure

rag-based-local-chat-box/
├── ui.py                              # Main Streamlit application script
├── document_loader.py                 # Handles loading and indexing of documents
├── models.py                          # Retrieves and manages the list of available LLM models
├── llm.py                             # Manages interactions with the Language Model, including streaming responses
├── requirements.txt                   # Python package dependencies
├── README.md                          # Project documentation
└── Research/                          # Default directory for storing and indexing documents

Prerequisites

Before setting up the project, ensure you have the following prerequisites installed:

  • Python: Version 3.8 or higher
  • Streamlit: For running the web application
  • Ollama: A local LLM framework
  • Git: For cloning the repository
  • Pip: Python package installer

Setup Instructions

1. Clone the Repository

Begin by cloning the rag-based-local-chat-box repository to your local machine:

git clone https://github.com/noman024/rag-based-local-chat-box.git
cd rag-based-local-chat-box

2. Set Up a Virtual Environment

It's recommended to use a virtual environment to manage dependencies:

sudo chmod 777 setup_env.sh
./setup_env.sh

3. Prepare Your Documents

Place the documents you wish to index in the Research/ directory or specify a different folder path via the application's sidebar.

Usage

Running the Application

Start the Streamlit application by navigating to the project directory and executing:

streamlit run ui.py

This command will launch the RAG-based Local Chat Box web interface in your default web browser.

Using RAG-based Local Chat Box

1. Select a Model:

  • Navigate to the sidebar.
  • Use the dropdown menu to select your desired local LLM model from the available options.

2. Specify Folder Path:

  • Enter the path to the folder containing the documents you wish to index.
  • he default path is set to Research, but you can modify this as needed.

3. Index Documents:

  • Click the Index Documents button.
  • The application will validate the directory and begin creating embeddings.
  • Once indexing is complete, a confirmation message will appear.

4. Interact via Chat:

  • Use the chat input at the bottom of the interface to ask questions.
  • The assistant will provide answers based on the indexed documents.
  • Chat history is maintained throughout your session for continuity.

Dependencies

RAG-based Local Chat Box relies on several Python packages and external tools:

  • Streamlit: For building the web interface.
  • LangChain Community: Provides utilities for working with language models.
  • Ollama: Facilitates interactions with local LLMs.
  • nomic-embed-text: Embedding model for creating document embeddings.

For the full list, see the requirements.txt file.

Contact

If you have any questions or issues, please open an issue on the GitHub repository.