[META] Chunking and querying of long passages for vector search

[samuel-oci]

Is your feature request related to a problem?

This is meta issue that orchestrate all tasks related to chunking and search of long passages and include both ingest, storage and query paths.

RFC ingest path:
#548

RFC query path:
TBD

RFC storage path:
TBD

Issues:
opensearch-project/k-NN#1065
#482

[yuye-aws]

Thanks @samuel-oci ! This is the draft PR to implement fixed token length algorithm and ingest processor: #607.

[martin-gaievski]

should we link #794 and #751 to this meta issue?

[navneet1v]

@samuel-oci can we close this issue. As the chunking processor is launched.

[yuye-aws]

@samuel-oci You can check the existing text_chunking documents here: https://opensearch.org/docs/latest/search-plugins/text-chunking/. Can we close this issue?

[yuye-aws]

Closing this issue as text chunking processor has been released.