govinfo.gov/bulkdata reporting wrong last updated at time

[ryanoillataguerre]

Description

I noticed that the bulk data page on the website is not reflecting the most recently updated elements in that page. I think that what's going on is you're using the first element in the array's date, when it's sorting in ascending order.

[llaplant]

Thank you for the feedback!

[jonquandt]

We are looking at a fix for this issue, so that the higher-level folders incorporate updates from new content at lower levels. You can also get latest BILLS changes via the API -- https://api.govinfo.gov/collections/BILLS/2024-12-15T00:00:00Z?offsetMark=*&pageSize=100&api_key=DEMO_KEY will return all of the BILLS that have been added or updated since the specified startDate/time.

You can then follow each results to the summary to get additional metadata or choose to pull the xml directly by constructing a link like:

https://api.govinfo.gov/packages/{packageId from collections endpoint result}/xml?api_key=DEMO_KEY

[ryanoillataguerre]

Awesome, thanks a bunch for the extra context @jonquandt and that's SUPER helpful. I was trying to do exactly that, you may have just saved me from scraping thousands of bills and diffing their changes 🤩

[jonquandt]

Reopening issue, since we still plan on an update so that the higher-level directories get updated modified times as new content is added to the bulk data repository.