Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

change lunr config #168

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

change lunr config #168

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
58ce62c
change lunr config
koilebeit Oct 24, 2024
26bd503
format files
koilebeit Oct 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion _data/theme.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ timeline-child-objects: true # true / false - if true, and if child object has d
data-child-objects: true # true / false - if true, child objects will appear linked in table on data page
carousel-child-objects: true # true / false - if true, child objects will appear on homepage carousel
browse-child-objects: false # true / false - if true, child objects will appear on browse page and child objects' metadata will populate cloud pages like Subjects page and Locations page, as well as featured terms boxes on the home page
search-child-objects: true # true / false - if true, child objects will appear on on search page along with parent objects
search-child-objects: false # true / false - if true, child objects will appear on on search page along with parent objects


##########
Expand Down
32 changes: 23 additions & 9 deletions _includes/js/lunr-js.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,25 +14,39 @@
this.field({{ f.field | jsonify }})
{% endfor %}

//this.pipeline.remove(lunr.trimmer)
this.pipeline.remove(lunr.trimmer)
this.pipeline.remove(lunr.stemmer);
Comment on lines +17 to +18
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Reconsider removing lunr.trimmer and lunr.stemmer for optimal search performance

By removing lunr.trimmer and lunr.stemmer from the pipeline, you may reduce the effectiveness of search queries. The lunr.trimmer removes unnecessary characters from tokens, and lunr.stemmer reduces words to their root form, allowing for better matching of different word variations. Consider retaining these processes unless there's a specific reason for their removal.

Apply this diff to keep them in the pipeline:

-    this.pipeline.remove(lunr.trimmer)
-    this.pipeline.remove(lunr.stemmer);

Committable suggestion was skipped due to low confidence.


// Add each item to the index, processing fields with semicolons
for (var item in store) {
this.add({
{% for f in index_fields %}
{{ f.field | jsonify }}: store[item][{{ f.field | jsonify }}],
{% endfor %}
id: item
})
var indexedData = {};
{% for f in index_fields %}
var fieldValue = store[item][{{ f.field | jsonify }}];
if (fieldValue) {
// Split semicolon-separated values and join them with spaces
indexedData[{{ f.field | jsonify }}] = fieldValue.split(";").join(" ");
}
{% endfor %}

// Add the indexed data to the Lunr index
this.add(Object.assign(indexedData, { id: item }));
}

});

/* search function */
function lunr_search () {
var resultdiv = document.querySelector('#lunrResults');
var query = document.querySelector('#lunrSearchBox').value;//.toLowerCase();
var query = document.querySelector('#lunrSearchBox').value.toLowerCase();
/* basic search that supports operators */
var result = idx.search(query);
// var result = idx.search(query);

/* Enable trailing wildcard for partial matches */
var result = idx.query(function (q) {
query.split(lunr.tokenizer.separator).forEach(function (term) {
q.term(term, { wildcard: lunr.Query.wildcard.TRAILING });
});
});
Comment on lines +45 to +49
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Handle empty or whitespace query terms to prevent errors

When processing the search query, empty strings or terms with only whitespace can cause issues. It's advisable to trim each term and check for emptiness before adding it to the query.

Apply this diff to filter out empty terms:

      query.split(lunr.tokenizer.separator).forEach(function (term) {
+       term = term.trim();
+       if (term) {
          q.term(term, { wildcard: lunr.Query.wildcard.TRAILING });
+       }
      });
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
var result = idx.query(function (q) {
query.split(lunr.tokenizer.separator).forEach(function (term) {
q.term(term, { wildcard: lunr.Query.wildcard.TRAILING });
});
});
var result = idx.query(function (q) {
query.split(lunr.tokenizer.separator).forEach(function (term) {
term = term.trim();
if (term) {
q.term(term, { wildcard: lunr.Query.wildcard.TRAILING });
}
});
});

/* more fuzzy search, but doesn't support operators:
var result =
idx.query(function (q) {
Expand Down