Support Reranker: setence-sf and infinity, and some bugs fixed #306
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
wzh1994
reviewed
Oct 17, 2024
|
|
||
|
|
||
| @Reranker.register_reranker(batch=True) | ||
| def ModuleReranker( |
There was a problem hiding this comment.
这里为什么从注册列表里面去掉了，而是变成了if self._name == "ModuleReranker":。我个人感觉可能是想在Rerank进行update的时候，把TrainableModule也Update了。
我在想Reranker可以支持对类进行注册。在__new__的时候，如果发现name: str = "ModuleReranker"在注册表中且是个class，则直接返回注册的类的实例化对象，看看能不能满足需求
@classmethod
def register_reranker(
cls: "Reranker", func: Optional[Callable] = None, batch: bool = False
):
def decorator(f):
if isinstance(f, type):
if batch: raise NotImplementedError('...')
cls.registered_reranker[f.__name__] = f
return f
else:
def wrapper(nodes, **kwargs):
if batch:
return f(nodes, **kwargs)
else:
results = [f(node, **kwargs) for node in nodes]
return [result for result in results if result]
cls.registered_reranker[f.__name__] = wrapper
return wrapper
return decorator(func) if func else decorator
class Reranker(ModuleBase, _PostProcess):
pass
@Reranker.register_reranker(batch=True)
class ModuleReranker(Reranker):
def __init__(self, ...):
pass
def forward():
pass
| if type == 'embed' or ModelManager.get_model_type(model_name) == 'embed': | ||
| if not type: | ||
| type = ModelManager.get_model_type(model_name) | ||
| if type in ('embed', 'reranker'): | ||
| if lazyllm.config['default_embedding_engine'] == 'transformers' or not check_requirements('infinity_emb'): |
There was a problem hiding this comment.
check_requirements可以加个@functools.lru_cache
| if self._model_type == "reranker": | ||
| target_name = 'rerank' | ||
| else: | ||
| target_name = 'embeddings' | ||
| if lazyllm.config['mode'] == lazyllm.Mode.Display: |
There was a problem hiding this comment.
可以找个时间，把Display模式完整的删掉，因为作为应用开发框架，Display模式没啥意义
There was a problem hiding this comment.
Added TODO list.
lazyllm/tools/rag/document.py
Outdated
| @@ -80,4 +80,7 @@ def forward(self, *args, **kw) -> List[DocNode]: | |||
| return self._impl.retrieve(*args, **kw) | |||
|
|
|||
| def __repr__(self): | |||
| return lazyllm.make_repr("Module", "Document", manager=bool(self._manager)) | |||
There was a problem hiding this comment.
return lazyllm.make_repr("Module", "Document", manager=hasattr(self, '_manager'))
wzh1994
reviewed
Oct 18, 2024
lazyllm/tools/rag/rerank.py
Outdated
|
|
||
|
|
||
| class Reranker(ModuleBase, _PostProcess): | ||
| registered_reranker = dict() | ||
|
|
||
| def __new__(cls, name: str = "ModuleReranker", *args, **kwargs): | ||
| if name == "ModuleReranker": | ||
| return super(Reranker, cls).__new__(cls.registered_reranker["ModuleReranker"]) |
There was a problem hiding this comment.
这里要判断name在注册表且为class，然后断言这个class要是Reranker的subclass
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Support Reranker:
Bugs fixed:
Documentno attr:_managerModuleBaseobject,isinstance(xx, ModuleBase)= False, this will cause cannot add into submodules. example: