number of hits for a given query is not as specified in the retrieval command

[NourOM02]

I run the bm25 for the cqadupstack/english dataset using the following command :

command = python -m pyserini.search.lucene --threads 16 --batch-size 128 --index beir-v1.0.0-cqadupstack-english.flat --topics beir-v1.0.0-cqadupstack-english-test --output run.beir.bm25-flat.cqadupstack-english.txt --output-format trec --hits 1000 --bm25 --remove-query

It looks like for every query the results as expected, but the query with the id : 84177 only returns 1 hit (as you can see in the image)

[lintool]

Hi @NourOM02 - thanks for noting this. This is likely because there aren't any more hits to retrieve, e.g., there are no more matching docs in the corpus.

[NourOM02]

Indeed @lintool , I conducted more tests and found that some queries may even return 0 hits. To explain more, if $Q=q_1, q_2, \text{...}, q_n$ is the query with $n$ keywords, and $C=D_1, D_2, \text{...}, D_k$ is the corpus containing $k$ documents, and $h$ is the number of hits specified by the user's command, then the number of returned hits is :
$n_{hits} = min(|${ $D_i$ s.t $\exists q \in Q$, $\exists D \in C$ s.t $q \in D|$}, $h)$'

This is probably the case for BM25 only since it works on exact term matching, I believe the same problem is less likely to happen with dense retrievers.