Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Custom rag output #63

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

Custom rag output #63

wants to merge 6 commits into from

Conversation

hummerichsander
Copy link
Contributor

@hummerichsander hummerichsander commented Dec 3, 2024

PR: Analysis Questions and Grammar handling

This PR adds analysis questions to the RAG job; analysis questions allow to specify additional questions the LLM should answer following a grammar the user can select. Furthermore, we introduce the Grammar model, that allows to specify additional grammars besides the usual yes/no grammar.

Changes

  1. Added model AnalysisQuestion in the rag app. This model allows adding analysis questions to the RAG job.
  2. Renamed model QuestionFilterQuestion in the rag app. This model holds the usual rag filter questions.
  3. Adapt RagTaskProcessor class to handle both filter- and analysis-questions.
  4. Added model Grammar in the chats app. This model captures the grammar string that allows to query a specific output. Besides the grammar string, one can provide a llm_instruction that is inserted into the message sent to the LLM. Example:
YesNoGrammar = Grammar(
    name="YES_NO",
    human_readable_name="Yes or No",
    grammar= dedent("""\
      root ::= Answer
      Answer ::= "Yes" | "No"
    """),
    llm_instruction="Answer each question in English faithfully with 'Yes' or 'No'."
)
  1. The AsyncChatClient now requires a grammar argument in its ask_report_question and ask_question methods, which allows asking a yes-or-no question or a free-text question with the same method but with different grammar arguments.
  2. Add CHAT_DEFAULT_GRAMMARS variable in the base settings. This list of default grammars is added to the database on compose-up via the create_default_grammars management command.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant