Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proximity Map implementation with support for incremental edits. #8686

Merged
merged 42 commits into from
Jan 16, 2025
Merged

Proximity Map implementation with support for incremental edits. #8686

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
1defec9
Add Vector Index message type.
nicktobey Oct 1, 2024
28b7065
Add ProximityMap class.
nicktobey Oct 1, 2024
6b91635
Support getting the N closest vectors in a ProximityMap (Adds depende…
nicktobey Dec 17, 2024
b14736e
Propagate Vector properties when manipulating indexes.
nicktobey Oct 25, 2024
82b0b9f
Add vector field to schema flatbuffer.
nicktobey Oct 25, 2024
764a61c
Serialize and deserialize vector index info into table schemas on disk.
nicktobey Oct 25, 2024
41bb1e6
Correctly generate CREATE TABLE statements with vector indexes.
nicktobey Oct 25, 2024
a1977ee
Allow creating vector indexes.
nicktobey Oct 25, 2024
814fc6d
Implement durable.ProximityIndex and all building vector indexes in D…
nicktobey Oct 25, 2024
34061c7
Add support for vector indexes to common index code.
nicktobey Oct 30, 2024
ca2c862
Create MapIters for vector indexes.
nicktobey Oct 30, 2024
2d17c68
Add vector partition type for indexed dolt tables and expose it to in…
nicktobey Oct 31, 2024
8e62177
Implement `NewEmptyProximityIndex`
nicktobey Nov 26, 2024
f89f4c7
Propagate logchunksize
nicktobey Dec 31, 2024
5b3c467
Add vector index support to git show / noms show
nicktobey Nov 26, 2024
caadb11
Run GMS `TestVectorIndexes` and `TestVectorFunctions` with Dolt.
nicktobey Oct 26, 2024
149d13f
Add context to MapInterfaceFromValue
nicktobey Jan 1, 2025
fe16088
TODO: replace default values with values read from message.
nicktobey Jan 1, 2025
d1764fd
Expand ItemAccess fields to 32 bits
nicktobey Dec 16, 2024
3c2de74
Add vector-index bats tests.
nicktobey Dec 31, 2024
e0b41ce
Allow `dolt index cat` to work with vector indexes.
nicktobey Jan 3, 2025
826f51c
Update licenses.
nicktobey Jan 3, 2025
3f75e65
[ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/upda…
nicktobey Jan 3, 2025
a73a2ce
Add additional bats tests.
nicktobey Jan 6, 2025
d844f69
Propagate context into vector index `Convert` function instead of cap…
nicktobey Jan 6, 2025
84f0b04
Respond to PR feedback: fix typos and outline methods.
nicktobey Jan 6, 2025
c82e9d5
Add comment for vector index message size estimate (and update to acc…
nicktobey Jan 12, 2025
a5ec782
Respond to PR feedback.
nicktobey Jan 12, 2025
81d82e9
Update vector index tests.
nicktobey Jan 12, 2025
79a9339
Update copyright date in expected header.
nicktobey Jan 12, 2025
290609f
Verify after incremental vector index operations that tables have the…
nicktobey Jan 12, 2025
3fb884f
Refactor proximity map code to make it more readable, breaking up lar…
nicktobey Jan 12, 2025
ae76256
[ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/upda…
nicktobey Jan 12, 2025
94f6aeb
Store and propagate vector logChunkSize and distanceType
nicktobey Jan 13, 2025
e6c433e
Correctly handle incremental updates: there will always be one key fr…
nicktobey Jan 13, 2025
acbf25b
[ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/upda…
nicktobey Jan 13, 2025
d4041dd
Add docstring to ProximityFlusher.visitNode.
nicktobey Jan 14, 2025
513ab1a
Remove hardcoded use of vector.DistanceL2Squared{} in favor of previo…
nicktobey Jan 14, 2025
e86d7b0
Allow using proximityIter for non-covering indexes.
nicktobey Jan 15, 2025
c237c1c
[ga-bump-dep] Bump dependency in Dolt by nicktobey
nicktobey Jan 16, 2025
f19df88
Merge remote-tracking branch 'origin/main' into nicktobey/proximity-map2
nicktobey Jan 16, 2025
1ba5e0a
Run go mod tidy.
nicktobey Jan 16, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
128 changes: 128 additions & 0 deletions go/Godeps/LICENSES
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion go/cmd/dolt/commands/indexcmds/cat.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -151,7 +151,7 @@ func (cmd CatCmd) prettyPrintResults(ctx context.Context, doltSch schema.Schema,

sqlCtx := sql.NewEmptyContext()

rowItr, err := table.NewTableIterator(ctx, doltSch, idx, 0)
rowItr, err := table.NewTableIterator(ctx, doltSch, idx)
if err != nil {
return err
}
Expand Down
1 change: 1 addition & 0 deletions go/gen/fb/serial/fileidentifiers.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ const StashFileID = "STSH"
const StatisticFileID = "STAT"
const DoltgresRootValueFileID = "DGRV"
const TupleFileID = "TUPL"
const VectorIndexNodeFileID = "IVFF"

const MessageTypesKind int = 27

Expand Down
118 changes: 117 additions & 1 deletion go/gen/fb/serial/schema.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading