Loading in Vector Database where I am continuously updating using Langchain Indexing API

[akashsonowal]

hi @hwchase17 and team,

I have a streaming data source and am using the Langchain Indexing API to keep the most update content and delete old.

I have a separate instance which is doing this indexing and the vector database is stored in AWS EFS.

When I want to use this database for RAG (in different instance than the instance that is building the database), The RAG instance uses the vector database through the EFS. I am currently loading this vector database as daemon process every 5 mins. But this slows down my RAG server. Is there any better way to do this or any other suggestion.

Looking forward to your reply.

Thanks,

Akash