Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Snippet generation breaks with fuzzy fields #2576

Open
mrcnski opened this issue Jan 30, 2025 · 0 comments
Open

Snippet generation breaks with fuzzy fields #2576

mrcnski opened this issue Jan 30, 2025 · 0 comments

Comments

@mrcnski
Copy link

mrcnski commented Jan 30, 2025

Describe the bug

  • I added a fuzzy field to the query parser.
  • Snippet generation started returning "".
  • I would expect it to return snippets around the matched fuzzy terms.

Which version of tantivy are you using?
4aa8cd247

To Reproduce

I wrote the following test case in snippet/mod.rs (based on test_snippet_generator):

#[test]
fn test_snippet_generator_with_fuzzy_field() -> crate::Result<()> {
    let mut schema_builder = Schema::builder();
    let text_options = TextOptions::default().set_indexing_options(
        TextFieldIndexing::default()
            .set_tokenizer("en_stem")
            .set_index_option(IndexRecordOption::Basic),
    );
    let text_field = schema_builder.add_text_field("text", text_options);
    let schema = schema_builder.build();
    let index = Index::create_in_ram(schema);
    {
        // writing the segment
        let mut index_writer = index.writer_for_tests()?;
        let doc = doc!(text_field => TEST_TEXT);
        index_writer.add_document(doc)?;
        index_writer.commit()?;
    }
    let searcher = index.reader().unwrap().searcher();
    let mut query_parser = QueryParser::for_index(&index, vec![text_field]);
    query_parser.set_field_fuzzy(text_field, false, 1, true); // <--------------- THIS LINE ADDED <----
    let query = query_parser.parse_query("rust design").unwrap();
    let mut snippet_generator =
        SnippetGenerator::create(&searcher, &*query, text_field).unwrap();
    {
        let snippet = snippet_generator.snippet(TEST_TEXT);
        assert_eq!(
            snippet.to_html(),
            "imperative-procedural paradigms. <b>Rust</b> is syntactically similar to \
             C++[according to whom?],\nbut its <b>designers</b> intend it to provide better \
             memory safety"
        );
    }
    {
        snippet_generator.set_max_num_chars(90);
        let snippet = snippet_generator.snippet(TEST_TEXT);
        assert_eq!(
            snippet.to_html(),
            "<b>Rust</b> is syntactically similar to C++[according to whom?],\nbut its \
             <b>designers</b> intend it to"
        );
    }
    Ok(())
}

Output is:

---- snippet::tests::test_snippet_generator_with_fuzzy_field stdout ----
thread 'snippet::tests::test_snippet_generator_with_fuzzy_field' panicked at src/snippet/mod.rs:800:13:
assertion `left == right` failed
  left: ""
 right: "imperative-procedural paradigms. <b>Rust</b> is syntactically similar to C++[according to whom?],\nbut its <b>designers</b> intend it to provide better memory safety"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant