Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scraper uses Raw Inmate Data to determine which Booking Ids to check for #444

Open
nwinklareth opened this issue Jun 8, 2014 · 0 comments
Labels

Comments

@nwinklareth
Copy link
Contributor

This is part of #395.

The first version of the Next NG version of the Scraper queries the V1 database to find out which booking_ids to scrape from the Sheriff's website. This dependency needs to be brokern. Currently the scraper needs to know the following set of bookings ids:

  • the set of active booking ids from the last scrape - this is the last known population
  • the set of potential booking ids for the days since the last scrape - this is the potential new booked inmates
  • the set of discharged booking ids for the last N days - this helps to find inmates that were marked as discharged, as no repsonse was received when they were scraped often due to network or system problems.
  • the set of not found booking ids for the last N days - this helps to find inmates that for one or reason or not were not found in earlier scrape, oftem due to network or system problems, howver may also be due to input issues at the Sheriff's site..

All of these booking ids can be determined from the raw Inmate data. For a discussion on how to do so is found on the Wiki page: [Notes On Working With Raw Inmate Data}(https://github.com/sc3/cookcountyjail/wiki/Notes-On-Working-With-Raw-Inmate-Data).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant