Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce filter handlers #244

Merged
merged 8 commits into from
Mar 10, 2024
Merged

Introduce filter handlers #244

merged 8 commits into from
Mar 10, 2024

Conversation

lonvia
Copy link
Member

@lonvia lonvia commented Mar 10, 2024

Filter handler can be used to drop irrelevant OSM objects before they are handed into Python callbacks, thus significantly speeding up processing. They can be either used with apply (order matters!):

osmium.apply("osm_file.pbf", osmium.filter.KeyFilter('amenity'), MyAmenityHandler())

or handed into the apply_file/buffer functions of the SimpleHandler:

MyAmenityHandler().apply_file('osm_file.pbf', filters=[osmium.filter.KeyFilter('amenity')])

The latter has the advantage that it will correctly work together with the location cache and area handling.

Filters are always implemented in C++. Allowing filters to be defined in Python would defy the purpose of avoiding callbacks into Python. There are two filters defined right now:

  • EmptyTagFilter filters out all objects without any tags.
  • KeyFilter only lets objects pass that have one of the given keys

Processing a Switzerland extract with the EmptyTagFilter reduces processing time from 40s to 6s.

Callbacks now return a boolean value indicating if processing should
be stopped after the handler.
Filters all objects without any tags.
Changes amenity_list example to use the new filter function.
It's too confusing to explain.
@lonvia lonvia merged commit 5e231e4 into osmcode:master Mar 10, 2024
20 checks passed
@lonvia lonvia deleted the filters branch March 10, 2024 20:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

1 participant