Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improvements for the search page #541

Merged
merged 50 commits into from
Jan 5, 2024

Conversation

karpov-sv
Copy link
Collaborator

@karpov-sv karpov-sv commented Dec 18, 2023

Redesign of the search page to use "smart search" like in Google or ADS. Like the latter, offer the palette of search terms above the input field.

Possible queries are like:

  • 10 20 300 - cone search at ra=10 dec=20 with 300 arcsec radius
  • Vega r=10 - cone search within 10 arcsec around Vega
  • 10 20 300 after="2023-01-01" window=1 - cone search within 300 arcsec from Vega in 24 hours after given moment
  • ZTF22abzanvj - exact search for objectId
  • ZTF23abshpmf r=100 - cone search within 100 arcsec around the alert
  • ZTF22abzan - partial match search for objectIds starting with given pattern
  • TRCK_20231212 - partial match search for tracklets (effectively, all tracklets from a given date)
  • class=Unknown - last 100 alerts of a given class
  • class=Unknown last=10 - last 10 alerts of a given class
  • class=Unknown before="2023-01-01 01:12:23" - last 100 alerts before given moment
  • class=Anomaly last=10 - last 10 anomalies
  • after="2022-12-29 13:36:52" before="2023-12-29 13:46:52" - date range search with exact specification of start and stop times

Everything is somehow working, except for SSO - this one is waiting for backend API improvements as discussed with Julien.
GET api (does anyone actually use it?..) is changed to reflect underlying query parser, so the urls that work now look like:

  • /?action=conesearch&ra=10&dec=20&r=300
  • /?action=class&class=Unknown&last=10

Requires installation of two additional packages from pip - regex (used for partial matches) and dash-autocomplete-input (used for dropdown menu for class search).
Also requires updating dash-mantine-components to latest version from pip.

Documentation (mostly the help popup inside the search bar itself) is not yet updated.

… is smaller than 400 px, to optimize the display on smaller landscape screens
@karpov-sv
Copy link
Collaborator Author

Card view added in latest commits actually quickly hits the limit of deploying with finite number of workers in Gunicorn (as suggested by README) due to its lazy loading of cutouts and lightcurves - as many component update requests actually perform one more request to /api/ under the hood, and block until it returned. So when there are many simultaneous ones the worker pool is exhausted and everything just hangs.

Solution might be to deploy a number of dedicated workers for /api/ that do not serve anything else. I did not find any built-in option in Gunicorn for that, so for now ended with deploying two instances on different ports, and pointing APIURL to the second one that is not visible from outside.

@karpov-sv
Copy link
Collaborator Author

With the last commit, the problem of exhaustion of web workers pool should be solved. The commit isolates all API access outside of /api/ into dedicated function that is able to either call requests.post(), or, if LOCALAPI=True, to use the endpoints directly, without doing unnecessary HTTP requests.

@karpov-sv
Copy link
Collaborator Author

At this point, I consider this PR feature-complete, and ready for review.

@JulienPeloton
Copy link
Member

Thanks @karpov-sv ! I started to test all the new features -- it looks super nice! I noticed one big thing though: the switch card/table does not work. If I trigger the button, I get an error from the fact that the endpoint /api/v1/columns does not have the 'POST' method. But even if I manually fix this, the switch does not operate (no error though -- just doing nothing).

@karpov-sv
Copy link
Collaborator Author

Thanks @JulienPeloton for reporting, early morning coding has its downsides :-) I fixed it, should now be working.

Copy link
Member

@JulienPeloton JulienPeloton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @karpov-sv -- I tried all options, and everything works on my side :-)
Let's merge this!

@JulienPeloton JulienPeloton merged commit 143a72f into astrolabsoftware:master Jan 5, 2024
1 check failed
@karpov-sv karpov-sv deleted the search branch January 22, 2024 23:14
@JulienPeloton JulienPeloton mentioned this pull request Feb 6, 2024
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants