Changed Directory structure, small fix, prompts #3850
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
There was a problem hiding this comment.
Directory structure looks a lot better! Asked ChatGPT about a few I took issue with, but mostly it's time to focus on filename/function/variable renames now. Check out the comments on the main PR #3749 before you start that, since Yuhong had some things to say on that front
| @@ -3,10 +3,10 @@ | |||
|
|
|||
| from langgraph.types import Send | |||
|
|
|||
| from onyx.agents.agent_search.deep_search_a.initial__individual_sub_answer__subgraph.states import ( | |||
| from onyx.agents.agent_search.deep_search_a.initial.individual_sub_answer_generation.states import ( | |||
There was a problem hiding this comment.
LLM recommends individual_sub_answer
There was a problem hiding this comment.
Here was the prompt:
For a langgraph graph created with the purpose of answering a subquestion, choose the best folder name or create your own that you believe is better:
- individual_sub_answer
- individual_sub_answer_generation
- answer_subquestion
There was a problem hiding this comment.
^^ the above was using Cursor with knowledge of our current codebase, plain ChatGPT picked answer_subquestion. Seems like either is fine
| @@ -102,7 +102,9 @@ def answer_generation( | |||
| ) | |||
|
|
|||
| answer_citation_ids = get_answer_citation_ids(answer_str) | |||
| cited_docs = [context_docs[id] for id in answer_citation_ids] | |||
| cited_docs = [ | |||
| context_docs[id] for id in answer_citation_ids if id < len(context_docs) | |||
There was a problem hiding this comment.
out of bounds citation should never happen unless the LLM is really weak, is this a difference in the docs we give the LLM vs the context docs here? Probably we prefer failing loudly
| generate_initial_answer, | ||
| ) | ||
| from onyx.agents.agent_search.deep_search_a.initial__retrieval_sub_answers__subgraph.nodes.initial_answer_quality_check import ( | ||
| from onyx.agents.agent_search.deep_search_a.initial.initial_answer_generation.nodes.initial_answer_quality_check import ( |
There was a problem hiding this comment.
Cursor thinks we should go with "individual_sub_answer" which I think is pretty weird, ChatGPT likes your choice but also thinks "generate_first_answer" is good. I'm inclined to stick with your choice since it's backed up by ChatGPT and doesn't seem clearly wrong like individual_sub_answer
prompt: For a langgraph graph created with the purpose of creating the first answer to a user question, choose the best folder name or create your own that you believe is better (keep in mind that this will be a subfolder of a new directory "initial" that contains the subgraphs used for gathering information and generating the first answer to the user question):
- retrieval_sub_answers
- initial_answer_generation
- answer_question
| AnswerQuestionState, | ||
| ) | ||
| from onyx.agents.agent_search.deep_search_a.refinement__consolidate_sub_answers__subgraph.edges import ( | ||
| from onyx.agents.agent_search.deep_search_a.refininement.sub_answer_consolidation.edges import ( |
There was a problem hiding this comment.
refininement -> refinement
|
|
||
| Make sure that you keep all relevant information, specifically as it concerns to the ultimate goal. | ||
| (But keep other details as well.) | ||
| It is a matter of life and death that you do NOT use your internal knowledge, just the provided` |
There was a problem hiding this comment.
provided has a ` at the end
| So a good idea would be to \n use the sub-questions to resolve ambiguities and/or to separate the | ||
| question for different entities that may be involved in the original question. | ||
| For an initial user question, please generate at 5-10 individual sub-questions whose answers would help | ||
| \n to answer the initial question. The individual questions should be answerable by a good RAG system. |
| question for different entities that may be involved in the original question. | ||
| For an initial user question, please generate at 5-10 individual sub-questions whose answers would help | ||
| \n to answer the initial question. The individual questions should be answerable by a good RAG system. | ||
| So a good idea would be to \n use the sub-questions to resolve ambiguities and/or to separate the |
| @@ -673,9 +712,9 @@ | |||
| {history} | |||
|
|
|||
| Here is the initial question: | |||
| ------- | |||
| \n -------\n | |||
There was a problem hiding this comment.
When you add \n in a triple quoted string, you're adding extra newlines in addition to the ones already present, i.e. it looks like
Here is the initial question:
/-------/
question
/--------/
If that's what you're going for though, that's fine
| - new bullet points | ||
| - substantially more document citations ([[D1]](), [[D2]](), [[D3]](), etc.) | ||
|
|
||
| Put yourself in the shoes of the user and think about whether the refined answer is really substantially | ||
| better than the initial answer. | ||
| better and delivers really new insights than the initial answer. |
There was a problem hiding this comment.
better than and delivers really new insights from the initial answer
| You need to summarize the key parts of the history of a conversation between a user and an agent. The | ||
| summary has two purposes: | ||
| 1) providing the suitable context for a new question, and | ||
| 2) To capture the key information that was discussed and that the user may have a follow-up question about. |
There was a problem hiding this comment.
For consistency, change 2) to "capturing" or 1) to "To provide"
joachim-danswer
force-pushed
the
agent-search-feature-jr6
branch
from
f87c173 to
f90bac5
Compare
- adjusted prompt - adjusted citation removal - length cutoff by words, not characters
joachim-danswer
force-pushed
the
agent-search-feature-jr6
branch
from
2805c0a to
194ec2e
Compare
Description
Changed Directory structure, small fix, prompts
How Has This Been Tested?
Locally
Backporting (check the box to trigger backport action)
Note: You have to check that the action passes, otherwise resolve the conflicts manually and tag the patches.