Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Enhance ai-cache Plugin with Vector Similarity-Based LLM Cache Recall and Multi-DB Support #1248

Merged
merged 92 commits into from
Nov 21, 2024

Conversation

EnableAsync
Copy link
Contributor

@EnableAsync EnableAsync commented Aug 25, 2024

Ⅰ. Describe what this PR did

This PR extends the functionality of the ai-cache plugin, enabling more efficient AI application development by introducing vector similarity-based caching and recall mechanisms.

Ⅱ. Does this pull request fix one issue?

Please refer to issue #1040 and #1041.

Ⅲ. Why don't you add test cases (unit test/integration test)?

Test cases will be added later.

Ⅳ. Describe how to verify it

After filling in the apikey and ChromaCollectionID in docker-compose-test/envoy.yaml, execute the following code:

cd docker-compose-test/
docker compose up

Then test it by accessing the LLM via cURL:

curl http://172.17.0.1:10000/v1/chat/completions -X POST -d '{"model":"gpt-4o-mini","messages":[{"content":"今天中午吃什么","role":"user"}]}' -H "Content-Type: application/json"

Ⅴ. Special notes for reviews

johnlanni and others added 9 commits August 1, 2024 15:09
update

update: 注意在使用http协议的时候不要用tls

update: add lobechat

add: makefile for ai-proxy

fix bugs

fix bugs

fix: redis connection

fix: dashvector and dashscope cluster

fix: change vdb collection

feat: add chroma logic

docs: 增加 api 说明

update: no callback version

fix: change to callback

fix: finish chrome

remove: key

update: gitignore
@CLAassistant
Copy link

CLAassistant commented Aug 25, 2024

CLA assistant check
All committers have signed the CLA.

@EnableAsync EnableAsync changed the title feat: Add Chroma vector database support to ai-cache WASM Plugin feat: Add ai-cache WASM Plugin Aug 25, 2024
@EnableAsync EnableAsync changed the title feat: Add ai-cache WASM Plugin feat: Enhance ai-cache Plugin with Vector Similarity-Based LLM Cache Recall and Multi-DB Support Aug 25, 2024
@codecov-commenter
Copy link

codecov-commenter commented Sep 6, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 43.52%. Comparing base (ef31e09) to head (134aecc).
Report is 204 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1248      +/-   ##
==========================================
+ Coverage   35.91%   43.52%   +7.61%     
==========================================
  Files          69       76       +7     
  Lines       11576    12320     +744     
==========================================
+ Hits         4157     5362    +1205     
+ Misses       7104     6622     -482     
- Partials      315      336      +21     

see 69 files with indirect coverage changes

---- 🚨 Try these New Features:

Copy link
Collaborator

@CH3CHO CH3CHO left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM


## Weaviate
Weaviate 所对应的 `vector.type` 为 `weaviate`。它并无特有的配置字段。
需要提前创建 Collection。需要注意的是 Weaviate 会设置首字母自动大写,在填写配置 `collectionID` 的时候需要将首字母设置为大写。
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个 Weaviate 里好像是叫 class,不叫 collection。

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

其他几个 provider 的对应描述也调整一下吧。需要告知用户实际创建的是什么,需要把什么填入 collectionId 配置里。

Copy link
Collaborator

@CH3CHO CH3CHO left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Collaborator

@johnlanni johnlanni left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@johnlanni johnlanni merged commit c2d405b into alibaba:main Nov 21, 2024
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants