Memory use of SRR for large networks

[sebhoerl]

I was already discussing this a bit with Marcel on the side, but I also create this issue to have some reminder for myself.

Currently, loading the (pt2matsim-generated) transit schedule for the Île-de-France region uses a lot of RAM, and the major share seems to be the transfer table that is kept in memory. Here is an example where we simply load the schedule, and we vary the maximum transfer distance in the configuration. The time for constructing the data goes up, but especially the memory use goes from ~30GB at 100m to ~60m at 800m. Especially when we run small 1% simulations, it is a pity that we have to keep 30GB of transfers in memory :)

We are wondering if, for such large instances, it may make sense to swap some computational time for memory use. Instead of performing lookups in the transfer table (of which a large share will probably never be used) we could perform queries in a quadtree or similar when identifying the possible transfers during routing. Maybe this could even be combined with some kind of intelligent cache.

PS: Just a comment, this does not concern the transfers that are explicitly indicated in the GTFS. This is rather the transfers that are possible "on top" just by evaluating the distances.