-
Notifications
You must be signed in to change notification settings - Fork 19
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add pgBouncer into Docker image #236
Comments
It's unlikely at this point that PgBouncer will be directly added into this project's Docker image. First challenge is that as of now, pgbouncer does not actually provide any benefit. See osm2pgsql-dev/osm2pgsql#1650 (reply in thread). Further, putting pgbouncer into the docker image means it needs to be maintained then forever, and I don't see the great benefit to adding that overhead to this project at this time. |
If osm2pgsql changes their connection handling behavior to not hold onto idle connections this can be reconsidered. Though, if they do that, the biggest portion of the problem this was trying to solve would be removed too... either way: closed |
It looks like using pgbouncer in transaction mode might be an option again. Updated discussion on osm2pgsql. The PR was merged in pgbouncer recently, released in |
Once again, closing... Recent work in osm2pgsql to reduce idle connections has removed my reasoning to work further on this. See #376 for details. The initial import w/ default layerset dropped from 43 connections down to 10. Replication updates dropped from 207 connections down to 17. |
Details
Using osm2pgsql for PgOSM Flex creates a relatively large number of connections to Postgres based on the number of tables created by the Flex output. This was discovered here (#226 (comment)) and answered on the osm2pgsql discussion here. Now documented on osm2pgsql.org: https://osm2pgsql.org/doc/manual.html#number-of-connections
It is unlikely this behavior will change in osm2pgsql so the plan is to bake pgBouncer into the Docker image, example standalone pgbouncer image here. Until this is in place, running
--append
mode will require upping themax_connections
to at least 210, preferably 300 to give headroom.The text was updated successfully, but these errors were encountered: