Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: remove initial tool call messages #73

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

gregnr
Copy link
Collaborator

@gregnr gregnr commented Aug 15, 2024

Background

When this project was first created as a POC, chat message history did not persist, but your DB tables did. So we added a hidden initial tool call message at the beginning of every conversation that told the LLM which tables were available. This allowed the LLM to know your schema right away without asking.

Since making the POC, we added the ability to persist chat messages which makes this feature unnecessary (DB will always have no tables at the start of the conversation).

Change

Remove these initial hidden tool call messages. They add a decent amount of tokens every request, so if they don't have a good purpose anymore lets remove them.

@gregnr
Copy link
Collaborator Author

gregnr commented Aug 15, 2024

Holding off on merging this as the initial chat behaviour seems to regressed (can't just say "books" and a books table appears immediately). Going to test a few prompt updates.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants