Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] search only operation support #445

Merged
merged 1 commit into from
Jun 26, 2023
Merged

Conversation

wybryan
Copy link
Contributor

@wybryan wybryan commented Jun 26, 2023

What is it: adds a switch to configure GPTCache in search only mode, i.e. if cache misses, return None directly without calling LLM API.

Motivation: In certain application, we want to have full control on what to cache and when to update the cache, therefore whenever a cache miss happens, we don't want to call LLM API directly because the query used in cache search may be totally different to the query used in actual LLM call. if cache misses, we just want the code return with None so we know cache missd.

@sre-ci-robot sre-ci-robot requested a review from cxie June 26, 2023 13:07
@sre-ci-robot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: wybryan
To complete the pull request process, please assign simfg after the PR has been reviewed.
You can assign the PR to them by writing /assign @simfg in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@sre-ci-robot
Copy link
Collaborator

Welcome @wybryan! It looks like this is your first PR to zilliztech/GPTCache 🎉

@wybryan
Copy link
Contributor Author

wybryan commented Jun 26, 2023

/assign @SimFG

@codecov
Copy link

codecov bot commented Jun 26, 2023

Codecov Report

Merging #445 (1fd33fb) into dev (10dcfae) will decrease coverage by 0.03%.
The diff coverage is 66.66%.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##              dev     #445      +/-   ##
==========================================
- Coverage   93.89%   93.87%   -0.03%     
==========================================
  Files          90       90              
  Lines        3327     3330       +3     
==========================================
+ Hits         3124     3126       +2     
- Misses        203      204       +1     
Impacted Files Coverage Δ
gptcache/adapter/adapter.py 86.66% <66.66%> (-0.32%) ⬇️

@SimFG SimFG merged commit 1106f08 into zilliztech:dev Jun 26, 2023
5 of 7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants