Code

Better search#858

Code

Merged

lukebrody merged 93 commits intomasterkavigupta/urbanstats:masterfrom better-searchkavigupta/urbanstats:better-searchCopy head branch name to clipboardFeb 3, 2025

+548-92

Commits

Commits on Jan 4, 2025

• Bitap impl!!

  lukebrody

  committed
  dc569a3

  Copy full SHA for dc569a3

• Crazy bitap algo

  lukebrody

  committed
  0d23a02

  Copy full SHA for 0d23a02

• Let’s try using a lib

  lukebrody

  committed
  f2cf5a7

  Copy full SHA for f2cf5a7

• Results are looking good

  lukebrody

  committed
  48ba580

  Copy full SHA for 48ba580

Commits on Jan 5, 2025

• Write down the new plan

  lukebrody

  committed
  9acf3d2

  Copy full SHA for 9acf3d2

• Redo search, seems to be working well enough

  lukebrody

  committed
  430a777

  Copy full SHA for 430a777

• Let’s cache token matches

  lukebrody

  committed
  5fe042e

  Copy full SHA for 5fe042e

• Speed up cache somewhat

  lukebrody

  committed
  3b1e0eb

  Copy full SHA for 3b1e0eb

• More optimization

  lukebrody

  committed
  c939eac

  Copy full SHA for c939eac

• Looking good!

  lukebrody

  committed
  2b94085

  Copy full SHA for 2b94085

• Merge branch 'master' of https://github.com/kavigupta/urbanstats into better-search

  lukebrody

  committed
  706a1d9

  Copy full SHA for 706a1d9

• handle historical cds

  lukebrody

  committed
  3abfc2e

  Copy full SHA for 3abfc2e

• clean up

  lukebrody

  committed
  9fb99b5

  Copy full SHA for 9fb99b5

• Add memory tests

  lukebrody

  committed
  f4e3b3b

  Copy full SHA for f4e3b3b

• Merge branch 'memory-tests' into better-search

  lukebrody

  committed
  78c41fd

  Copy full SHA for 78c41fd

• Use caching when running testcafe

  lukebrody

  committed
  3782ce9

  Copy full SHA for 3782ce9

• Merge branch 'master' of https://github.com/kavigupta/urbanstats into better-search

  lukebrody

  committed
  7ffb4a3

  Copy full SHA for 7ffb4a3

• Merge branch 'testcafe-cache' into better-search

  lukebrody

  committed
  e6bee34

  Copy full SHA for e6bee34

Commits on Jan 9, 2025

• Revert "Merge branch 'testcafe-cache' into better-search"

  Show description for 514512a
  lukebrody

  committed
  514512a

  Copy full SHA for 514512a

• Merge branch 'master' of github.com:kavigupta/urbanstats into origin/better-search

  lukebrody

  committed
  6cc5e07

  Copy full SHA for 6cc5e07

• Discard the decompressed search index when the user is not using the search field

  lukebrody

  committed
  856cd2a

  Copy full SHA for 856cd2a

• lint

  lukebrody

  committed
  a0bcb91

  Copy full SHA for a0bcb91

• split up memory tests

  lukebrody

  committed
  f88de97

  Copy full SHA for f88de97

• decrease article threshold

  lukebrody

  committed
  2d98d15

  Copy full SHA for 2d98d15

Commits on Jan 11, 2025

• going to add placement statistic

  lukebrody

  committed
  972f543

  Copy full SHA for 972f543

• Add position hueristic for results

  lukebrody

  committed
  7df1ddb

  Copy full SHA for 7df1ddb

• changes

  lukebrody

  committed
  8e8734f

  Copy full SHA for 8e8734f

• log result explanation

  lukebrody

  committed
  d428d7a

  Copy full SHA for d428d7a

Commits on Jan 13, 2025

• do tokenization

  lukebrody

  committed
  46b5bc5

  Copy full SHA for 46b5bc5

Commits on Jan 14, 2025

• going to introduce new search algorithm

  lukebrody

  committed
  416b526

  Copy full SHA for 416b526

• happy with results, now to work on performance

  lukebrody

  committed
  5b98cc5

  Copy full SHA for 5b98cc5

• improve performance

  lukebrody

  committed
  4c9c1c1

  Copy full SHA for 4c9c1c1

• High impact optimization

  lukebrody

  committed
  2492a65

  Copy full SHA for 2492a65

• remove many marginal optimizations that added complexity

  lukebrody

  committed
  de53a8d

  Copy full SHA for de53a8d

• signatures for tokens

  lukebrody

  committed
  db763f8

  Copy full SHA for db763f8

• let's see if we can encode the existence of two letters

  lukebrody

  committed
  7d50587

  Copy full SHA for 7d50587

• with bigints

  lukebrody

  committed
  bd68992

  Copy full SHA for bd68992

• quite happy with how search is working/performing

  lukebrody

  committed
  25afd95

  Copy full SHA for 25afd95

• nicer performance logging

  lukebrody

  committed
  d7e803c

  Copy full SHA for d7e803c

Commits on Jan 16, 2025

• Remove caching from protobufs

  lukebrody

  committed
  30a3066

  Copy full SHA for 30a3066

• Merge branch 'master' of https://github.com/kavigupta/urbanstats into better-search

  lukebrody

  committed
  dbf667b

  Copy full SHA for dbf667b

Commits on Jan 19, 2025

• Add unit test runner and example test

  lukebrody

  committed
  2066c32

  Copy full SHA for 2066c32

• Add run unit tests command

  lukebrody

  committed
  012bb77

  Copy full SHA for 012bb77

• Fix check dependency

  lukebrody

  committed
  5f2043f

  Copy full SHA for 5f2043f

• Make the test pass again

  lukebrody

  committed
  0c713a1

  Copy full SHA for 0c713a1

• Merge branch 'master' of https://github.com/kavigupta/urbanstats into better-search

  lukebrody

  committed
  0fd76cc

  Copy full SHA for 0fd76cc

• Merge branch 'unit-testing' of https://github.com/kavigupta/urbanstats into better-search

  lukebrody

  committed
  889a47b

  Copy full SHA for 889a47b

• Add search test

  lukebrody

  committed
  d2b8e4e

  Copy full SHA for d2b8e4e

• Merge branch 'master' of https://github.com/kavigupta/urbanstats into better-search

  Show description for 75c327a
  lukebrody

  committed
  75c327a

  Copy full SHA for 75c327a

• Doing weights, but need more information (e.g. incomplete tokens match)

  lukebrody

  committed
  0f726b1

  Copy full SHA for 0f726b1

Commits on Jan 20, 2025

• Add incomplete and swap matches

  lukebrody

  committed
  2f2350a

  Copy full SHA for 2f2350a

• Fix up tests

  lukebrody

  committed
  43fb97a

  Copy full SHA for 43fb97a

• Sum weights to 1

  lukebrody

  committed
  5601133

  Copy full SHA for 5601133

• Remove allowPartial in bitap search

  lukebrody

  committed
  182b7da

  Copy full SHA for 182b7da

• clean up performance logging

  lukebrody

  committed
  6b08587

  Copy full SHA for 6b08587

Commits on Jan 22, 2025

• Merge branch 'master' of https://github.com/kavigupta/urbanstats into better-search

  lukebrody

  committed
  883eac4

  Copy full SHA for 883eac4

• Merge branch 'master' of https://github.com/kavigupta/urbanstats into better-search

  lukebrody

  committed
  ea98e77

  Copy full SHA for ea98e77

Commits on Jan 27, 2025

• Count full search index load time

  lukebrody

  committed
  368bd4a

  Copy full SHA for 368bd4a

Commits on Feb 1, 2025

• Merge branch 'master' of https://github.com/kavigupta/urbanstats into better-search

  lukebrody

  committed
  15854ee

  Copy full SHA for 15854ee

• async search index

  lukebrody

  committed
  1875412

  Copy full SHA for 1875412

• Revert "async search index"

  Show description for d983686
  lukebrody

  committed
  d983686

  Copy full SHA for d983686

• Implement worker and clean up

  lukebrody

  committed
  ac11d31

  Copy full SHA for ac11d31

• Include iframes and workers in memory

  lukebrody

  committed
  dfdf670

  Copy full SHA for dfdf670

• Merge branch 'master' of https://github.com/kavigupta/urbanstats into betterMemoryTest

  lukebrody

  committed
  fbe0502

  Copy full SHA for fbe0502

• fix memory meter

  lukebrody

  committed
  76a6ff3

  Copy full SHA for 76a6ff3

• Redo memory util to include all targets

  lukebrody

  committed
  12e7fcd

  Copy full SHA for 12e7fcd

• Merge branch 'master' of https://github.com/kavigupta/urbanstats into better-search

  lukebrody

  committed
  306ef51

  Copy full SHA for 306ef51

• Use token pointers to save memory

  lukebrody

  committed
  cb75f24

  Copy full SHA for cb75f24

• Merge branch 'betterMemoryTest' into better-search

  lukebrody

  committed
  05a19f3

  Copy full SHA for 05a19f3

• use queue for worker so message are ordered

  lukebrody

  committed
  8642eaf

  Copy full SHA for 8642eaf

• Use new memory monitor

  lukebrody

  committed
  7bc4bb7

  Copy full SHA for 7bc4bb7

• Discard the raw index when we’re done with it

  lukebrody

  committed
  84eec68

  Copy full SHA for 84eec68

• Merge branch 'master' of https://github.com/kavigupta/urbanstats into better-search

  Show description for 8aa6675
  lukebrody

  committed
  8aa6675

  Copy full SHA for 8aa6675

• update memory tests

  lukebrody

  committed
  2601e14

  Copy full SHA for 2601e14

Commits on Feb 2, 2025

• Fix memory tests

  lukebrody

  committed
  e50f28e

  Copy full SHA for e50f28e

• Fix comment accuracy

  lukebrody

  committed
  08ea10d

  Copy full SHA for 08ea10d

• Minor fixes

  lukebrody

  committed
  641b4a5

  Copy full SHA for 641b4a5

• Refactor and fix tests involving search

  lukebrody

  committed
  18d5220

  Copy full SHA for 18d5220

• Merge branch 'betterSearchTests' into better-search

  lukebrody

  committed
  64e35b1

  Copy full SHA for 64e35b1

• update search screenshots

  lukebrody

  committed
  390210c

  Copy full SHA for 390210c

• Merge branch 'master' of https://github.com/kavigupta/urbanstats into better-search

  lukebrody

  committed
  55bdd06

  Copy full SHA for 55bdd06

• Merge branch 'master' of https://github.com/kavigupta/urbanstats into better-search

  lukebrody

  committed
  b5e9d00

  Copy full SHA for b5e9d00

• Simplify search worker

  lukebrody

  committed
  2c78727

  Copy full SHA for 2c78727

• Fix focus range

  lukebrody

  committed
  900c1f9

  Copy full SHA for 900c1f9

• Fix search

  lukebrody

  committed
  e372205

  Copy full SHA for e372205

• update screenshot

  lukebrody

  committed
  bd3ac63

  Copy full SHA for bd3ac63

• Increase errors and add test

  lukebrody

  committed
  8aa1c96

  Copy full SHA for 8aa1c96

Commits on Feb 3, 2025

• add some tests

  kavigupta

  committed
  442779f

  Copy full SHA for 442779f

• some tests were bad

  kavigupta

  committed
  54da5e6

  Copy full SHA for 54da5e6

• Reference token objects directly, rather than with indices

  lukebrody

  committed
  cffddd5

  Copy full SHA for cffddd5

• Merge branch 'better-search' of https://github.com/kavigupta/urbanstats into better-search

  lukebrody

  committed
  4e5f4dd

  Copy full SHA for 4e5f4dd

• bump version

  lukebrody

  committed
  94406cb

  Copy full SHA for 94406cb

• Merge branch 'master' of https://github.com/kavigupta/urbanstats into better-search

  Show description for 4f4b7db
  lukebrody

  committed
  4f4b7db

  Copy full SHA for 4f4b7db