-
Notifications
You must be signed in to change notification settings - Fork 26
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
实际测试效果较差 #33
Comments
您好,我们这边测试了Qwen1.5-7B-chat,在32k文本上依旧能够正常生成~ 请问具体您遇到的问题是什么样呢 |
那 尝试过 64k文本 or 100K文本吗? 我的测试结果显示 当超出窗口后,效果变差 |
您好,我采样了部分128K的passkey retrieval的数据,使用Qwen1.7-7B-chat同样能够准确地从文本中抽取passkey |
哦哦哦,感谢您的回复。请问你抽取的passkey 所在位置是否比较固定,比如文章开头 or 末尾处。您方便提供一个样例给我,辅助我测试吗? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
您好,您的工作确实很有创意。我剥离了原有的评估代码,使用Qwen1.5-14B-chat 进行测试。 当提供的文本长度达到30000字符长度时,InfLLM已经开始无法准确的回答问题。当文本继续增大时,输出错误加剧。能帮忙看看调整哪些参数可以优化结果
yaml 文件如下:
model:
type: inf-llm
path: /data/public/LLM/basemodels/qwen_1_5/Qwen1.5-14B-Chat/
block_size: 128
n_init: 128
n_local: 4096
topk: 16
repr_topk: 4
max_cached_block: 32
exc_block_size: 512
fattn: True
base: 1000000
distance_scale: 1.0
max_len: 2147483647
chunk_size: 2048
conv_type: qwen
server 脚本
lnFLLM_server.txt
The text was updated successfully, but these errors were encountered: