Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement references + PDF viewer over source documents for RAG agent [New Feature] #81

Open
danXyu opened this issue Oct 9, 2024 · 0 comments

Comments

@danXyu
Copy link
Contributor

danXyu commented Oct 9, 2024

  • The RAG agent should be able to both synthesize an answer and provide its references (the metadata / context from the vectors used to generate the answer). We should also have a PDF viewer that jumps to the page where those vectors came from and highlights the passage / chunk that was used in that vector.

  • This is a complex feature.

  • The implementation steps would look as follows:

  1. Use layoutparser instead of textsplitter to generate chunks for the embeddings. Instead of doing a simple x-character chunk, we would use this DL library to extract the text elements and also pull out a bounding box for each element that we are chunking alongside the page number. Store this in the embedding metadata.
  2. Use a react PDF viewer library to display the PDF whenever the RAG agent is invoked, and one of its references is clicked (which should be in the uploads directory on the server) that will show up in a right side panel next to the chat UI.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant