-
Notifications
You must be signed in to change notification settings - Fork 421
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
start adding documentation on updating mapping #5290
Conversation
36528e1
to
02f2edb
Compare
quickwit/quickwit-doc-mapper/src/default_doc_mapper/mapping_tree.rs
Outdated
Show resolved
Hide resolved
quickwit/quickwit-doc-mapper/src/default_doc_mapper/mapping_tree.rs
Outdated
Show resolved
Hide resolved
docs/reference/updating-mapper.md
Outdated
|
||
## Indexing | ||
|
||
When you update a doc mapping for an index, Quickwit will restart indexing pipelines to take the changes into account. As both this operation and the document ingestion are asynchronous, you can't assume documents sent immediately after the update will necessarily use the new mapping nor that documents sent immediately before the update won't be indexed using the new doc mapping. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When you update a doc mapping for an index, Quickwit will restart indexing pipelines to take the changes into account. As both this operation and the document ingestion are asynchronous, you can't assume documents sent immediately after the update will necessarily use the new mapping nor that documents sent immediately before the update won't be indexed using the new doc mapping. | |
When you update a doc mapping for an index, Quickwit will restart indexing pipelines to take the changes into account. As both this operation and the document ingestion are asynchronous, you can't assume documents sent immediately after the update will necessarily use the new mapping or documents sent immediately before the update will be indexed using the new doc mapping. |
Removing the double negative.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i think won't was what i meant, if you send a document while there is a large queue in the wal, and then update the mapping, it is possible for the pipeline to be restarted before reaching that document, so it may be indexed with the newer mapping (or it may not, if the wal was short/empty).
Description
add documentation on the known limitations when updating a doc mapping
@rdettai i know you were surprised by a few things, please step in if you think I missed anything important
@fmassot you might have opinions on things to add too.
this is based on the expect stat of updates once all currently existing PR on the subject are merged