[Mapnik-de] [OpenStreetMap] #3891: implement pgsql_nodes_get_list
OpenStreetMap
trac at openstreetmap.org
Mi Jul 6 02:51:15 CEST 2011
#3891: implement pgsql_nodes_get_list
-------------------------+--------------------------------------------------
Reporter: amm | Owner: mapnik-de@…
Type: enhancement | Status: new
Priority: minor | Milestone:
Component: mapnik-de | Version:
Keywords: |
-------------------------+--------------------------------------------------
Currently pgsql_nodes_get_list is implemented in the naive way of running
a for loop through pgsql_nodes_get, resulting in an sql query per node.
This patch implements the function by sending only a single sql query and
matching the results back onto the list.
This should be more efficient, although the degree to which it enhances
performance will heavily depend on both hardware, the data imported and
the cache hit ratio.
Preliminary results indicate it can be a substantial win though in low
cache hit regimes.
E.g. importing a berlin extract (17 mb .osm.pbf) on a laptop with a single
slow disk reduced the import time from about 5 - 6 mintues down to ~ 3
minutes.
As for normal settings of the cache, one would get 100% hit ratio for such
a small extract, it was run with only 8Mb of cache in osm2pgsql (Although
with normal disk cache size for postgresql), resulting in cache hit ration
of about 20%.
Someone else (Komzpa?) has reported similar numbers on irc of up to 2
times performance improvement from an earlier independent patch that was
never submitted.
I have not committed the patch directly to svn, as I currently don't have
a full test rig to verify that it handles all the corner cases correctly.
--
Ticket URL: <https://trac.openstreetmap.org/ticket/3891>
OpenStreetMap <http://www.openstreetmap.org/>
OpenStreetMap is a free editable map of the whole world