vecindex: support deleting vectors from C-SPANN index #135230
Conversation
|
This change is |
There was a problem hiding this comment.
Reviewed 7 of 7 files at r1, all commit messages.
Reviewable status:complete! 1 of 0 LGTMs obtained (waiting on @andy-kimball and @mw5h)
pkg/sql/vecindex/vector_index.go line 244 at r1 (raw file):
defer searchCtx.Workspace.FreeVector(tempRandomized) vi.quantizer.RandomizeVector(ctx, vector, tempRandomized, false /* invert */) searchCtx.Randomized = tempRandomized
[nit] Is it possible to move some of this logic above the loop?
pkg/sql/vecindex/vector_index.go line 255 at r1 (raw file):
// Retry search with significantly higher beam size. if baseBeamSize == vi.options.BaseBeamSize { baseBeamSize *= 8
If vi.options.BaseBeamSize was somehow zero, we'd have an infinite loop, right?
pkg/sql/vecindex/vecstore/search_set.go line 174 at r1 (raw file):
// MatchKey, if non-nil, filters out all search candidates that do not have // a matching primary key. MatchKey PrimaryKey
Think we could make this more general down the line? For example, simple filters on the primary key columns or stored columns.
There was a problem hiding this comment.
Reviewable status:
pkg/sql/vecindex/vector_index.go line 244 at r1 (raw file):
Previously, DrewKimball (Drew Kimball) wrote…
[nit] Is it possible to move some of this logic above the loop?
Done.
pkg/sql/vecindex/vector_index.go line 255 at r1 (raw file):
Previously, DrewKimball (Drew Kimball) wrote…
If
vi.options.BaseBeamSizewas somehow zero, we'd have an infinite loop, right?
Done.
pkg/sql/vecindex/vecstore/search_set.go line 174 at r1 (raw file):
Previously, DrewKimball (Drew Kimball) wrote…
Think we could make this more general down the line? For example, simple filters on the primary key columns or stored columns.
In other prototypes, I've had other kinds of filters here. Never anything so general, but I have found it useful to have the flexibility.
Add Delete method to the vector index, which attempts to remove a vector from the index, given its value and primary key. Delete may not be able to locate the vector in the index, leaving a "dangling vector" reference which other methods need to take care not to ignore. Epic: CRDB-42943 Release note: None
|
bors r=drewkimball |
135230: vecindex: support deleting vectors from C-SPANN index r=drewkimball a=andy-kimball Add Delete method to the vector index, which attempts to remove a vector from the index, given its value and primary key. Delete may not be able to locate the vector in the index, leaving a "dangling vector" reference which other methods need to take care not to ignore. Epic: CRDB-42943 Release note: None Co-authored-by: Andrew Kimball <[email protected]>
|
Build failed: |
|
bors r=drewkimball |
Add Delete method to the vector index, which attempts to remove a vector from the index, given its value and primary key. Delete may not be able to locate the vector in the index, leaving a "dangling vector" reference which other methods need to take care not to ignore.
Epic: CRDB-42943
Release note: None