Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Medicines: replace in-memory search with pg full text search with GIN index #1439

Closed
wants to merge 10 commits into from

Conversation

rithviknishad
Copy link
Member

@rithviknishad rithviknishad commented Jul 6, 2023

Proposed Changes

  • Removed in-memory table MedibaseMedicinesTable and support for in-memory search.
  • Adds GIN Index on the Medicines table (+ migration to install the Postgres extension)
  • Annotates and orders with search rank (priority: generic, name, company, cims_class, contents)
  • Updated API endpoint and views to reflect the changes and support for pagination.

Before: with the in-memory table (93ms ~ 128ms)

image

After: Search on Postgres with GIN Index and order by SearchRank (122ms ~ 170ms)

(Postgres is not on the same machine; on a Linux machine on the same local network)

image

Index Size (~64 MB)

image

Merge Checklist

  • Tests added/fixed
  • Update docs in /docs
  • Linting Complete
  • Any other necessary step

Only PR's with test cases included and passing lint and test pipelines will be reviewed

@coronasafe/care-backend-maintainers @coronasafe/care-backend-admins

@rithviknishad
Copy link
Member Author

will update tests

@sainak sainak added the reviewed label Jul 6, 2023
@rithviknishad rithviknishad requested a review from sainak July 6, 2023 18:27
@rithviknishad rithviknishad changed the title Medicines: Remove in-memory search; add Gin Index and search on postgres itself Medicines: Remove in-memory search; add GIN Index and search on postgres itself Jul 6, 2023
@rithviknishad rithviknishad added the P1 High priority; urgent label Jul 6, 2023
rithviknishad added a commit to ohcnetwork/care_fe that referenced this pull request Jul 6, 2023
@sainak sainak requested a review from Ashesh3 July 6, 2023 19:58
@sainak
Copy link
Member

sainak commented Jul 6, 2023

Migrations tested on latest staging snapshot

@sainak sainak changed the title Medicines: Remove in-memory search; add GIN Index and search on postgres itself Medicines: replace in-memory search with pg full text search with GIN index Jul 6, 2023
@rithviknishad
Copy link
Member Author

@sainak external_id was hidden too?

@sainak
Copy link
Member

sainak commented Jul 7, 2023

Yeah we are using the id field for it right

@rithviknishad
Copy link
Member Author

oh my bad; got confused

@nihal467
Copy link
Member

image

I tested in tunnel provided by @rithviknishad's and added multiple medicines to a patients but partial search is not giving expected results

@rithviknishad
Copy link
Member Author

Alternate approach used. Completed via #1443

@sainak sainak deleted the medibase-postgres-search branch July 21, 2023 16:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants