WIP: handling viterbi breaks as multiple sequences

CONTRIBUTORS.md

@@ -5,7 +5,8 @@
  * michaz, very important hidden markov improvement via hmm-lib, see #49
  * rory, support milisecond gpx timestamps, see #4 
  * stefanholder, Stefan Holder, BMW AG, creating and integrating the hmm-lib (#49, #66, #69) and
- penalizing inner-link U-turns (#70)
- * kodonnell, adding support for CH and other algorithms (#60) and penalizing inner-link U-turns (#70)
+ penalizing inner-link U-turns (#88, #91)
+ * kodonnell, adding support for CH and other algorithms (#60) and penalizing inner-link U-turns
+ (#88) and handling sequence breaks as separate sequences (#87).
 
 For GraphHopper contributors see [here](https://github.com/graphhopper/graphhopper/blob/master/CONTRIBUTORS.md).

README.md

@@ -1,125 +0,0 @@
-## Map Matching based on GraphHopper
-
-[![Build Status](https://secure.travis-ci.org/graphhopper/map-matching.png?branch=master)](http://travis-ci.org/graphhopper/map-matching)
-
-Map matching is the process to match a sequence of real world coordinates into a digital map.
-Read more at [Wikipedia](https://en.wikipedia.org/wiki/Map_matching). It can be used for tracking vehicles' GPS information, important for further digital analysis. Or e.g. attaching turn instructions for any recorded GPX route.
-
-Currently this project is under heavy development but produces already good results for various use cases. Let us know if not and create an issue!
-
-See the demo in action (black is GPS track, green is matched result):
-
-![map-matching-example](https://cloud.githubusercontent.com/assets/129644/14740686/188a181e-0891-11e6-820c-3bd0a975f8a5.png)
-
-### License
-
-Apache License 2.0
-
-### Discussion
-
-Discussion happens [here](https://discuss.graphhopper.com/c/graphhopper/map-matching).
-
-### Installation and Usage
-
-Java 8 and Maven >=3.3 are required. For the 'core' module Java 7 is sufficient.
-
-Then you need to import the area you want to do map-matching on:
-
-```bash
-git checkout [stable-branch] # optional
-./map-matching.sh action=import datasource=./some-dir/osm-file.pbf vehicle=car
-```
-
-As an example you use `datasource=./map-data/leipzig_germany.osm.pbf` as road network base or any other pbf or xml from [here](http://download.geofabrik.de/).
-
-The optional parameter `vehicle` defines the routing profile like `car`, `bike`, `motorcycle` or `foot`. 
-You can also provide a comma separated list. For all supported values see the variables in the [FlagEncoderFactory](https://github.com/graphhopper/graphhopper/blob/0.7/core/src/main/java/com/graphhopper/routing/util/FlagEncoderFactory.java) of GraphHopper. 
-
-If you have already imported a datasource with a specific profile, you need to remove the folder graph-cache in your map-matching root directory.
-
-Now you can do these matches:
-```bash
-./map-matching.sh action=match gpx=./some-dir/*.gpx
-```
-
-As example use `gpx=./matching-core/src/test/resources/*.gpx` or one specific gpx file.
-
-Possible arguments are:
-```bash
-instructions=de             # default=, type=String, if an country-iso-code (like en or de) is specified turn instructions are included in the output, leave empty or default to avoid this
-gps_accuracy=15              # default=15, type=int, unit=meter, the precision of the used device
-```
-
-This will produce gpx results similar named as the input files.
-
-### UI and matching Service
-
-Start via:
-```bash
-./map-matching.sh action=start-server
-```
-
-Access the simple UI via localhost:8989.
-
-You can post GPX files and get back snapped results as GPX or as compatible GraphHopper JSON. An example curl request is:
-```bash
-curl -XPOST -H "Content-Type: application/gpx+xml" -d @/path/to/gpx/file.gpx "localhost:8989/match?vehicle=car&type=json"
-```
-
-#### Development tools
-
-Determine the maximum bounds of one or more GPX file:
-```bash
-./map-matching.sh action=getbounds gpx=./track-data/.*gpx
-```
-
-#### Java usage
-
-Or use this Java snippet:
-
-```java
-// import OpenStreetMap data
-GraphHopper hopper = new GraphHopper();
-hopper.setOSMFile("./map-data/leipzig_germany.osm.pbf");
-hopper.setGraphHopperLocation("./target/mapmatchingtest");
-CarFlagEncoder encoder = new CarFlagEncoder();
-hopper.setEncodingManager(new EncodingManager(encoder));
-hopper.getCHFactoryDecorator().setEnabled(false);
-hopper.importOrLoad();
-
-// create MapMatching object, can and should be shared accross threads
-
-GraphHopperStorage graph = hopper.getGraphHopperStorage();
-LocationIndexMatch locationIndex = new LocationIndexMatch(graph,
-                (LocationIndexTree) hopper.getLocationIndex());
-MapMatching mapMatching = new MapMatching(graph, locationIndex, encoder);
-
-// do the actual matching, get the GPX entries from a file or via stream
-List<GPXEntry> inputGPXEntries = new GPXFile().doImport("nice.gpx").getEntries();
-MatchResult mr = mapMatching.doWork(inputGPXEntries);
-
-// return GraphHopper edges with all associated GPX entries
-List<EdgeMatch> matches = mr.getEdgeMatches();
-// now do something with the edges like storing the edgeIds or doing fetchWayGeometry etc
-matches.get(0).getEdgeState();
-```
-
-with this maven dependency:
-
-```xml
-<dependency>
-    <groupId>com.graphhopper</groupId>
-    <artifactId>map-matching</artifactId>
-    <!-- or 0.9-SNAPSHOT for the unstable -->
-    <version>0.8.2</version>
-</dependency>
-```
-
-### Note
-
-Note that the edge and node IDs from GraphHopper will change for different PBF files,
-like when updating the OSM data.
-
-### About
-
-See [this project](https://github.com/bmwcarit/hmm-lib) from [Stefan](https://github.com/stefanholder) which is used in combination with the GraphHopper routing engine and is used as the algorithmic approach now. Before it was [this faster but more heuristic approach](https://karussell.wordpress.com/2014/07/28/digitalizing-gpx-points-or-how-to-track-vehicles-with-graphhopper/).

matching-core/src/main/java/com/graphhopper/matching/MapMatching.java

@@ -42,7 +42,6 @@
 import java.util.*;
 import java.util.Map.Entry;
 
-
 /**
  * This class matches real world GPX entries to the digital road network stored in GraphHopper. The
  * Viterbi algorithm is used to compute the most likely sequence of map matching candidates. The
@@ -60,9 +59,9 @@
  * @author kodonnell
  */
 public class MapMatching {
-    
+
     private final Logger logger = LoggerFactory.getLogger(getClass());
-    
+
     // Penalty in m for each U-turn performed at the beginning or end of a path between two
     // subsequent candidates.
     private double uTurnDistancePenalty;
@@ -204,7 +203,7 @@ public MatchResult doWork(List<GPXEntry> gpxList) {
                         qr.getSnappedPoint().getLon(), qr.getQueryDistance());
             }
         }
-        
+
         logger.debug("=============== Time steps ===============");
         i = 1;
         for (ViterbiMatchEntry entry : viterbiMatchEntries) {
@@ -213,18 +212,18 @@ public MatchResult doWork(List<GPXEntry> gpxList) {
                 logger.debug(candidate.toString());
             }
         }
-        
+
         // compute the most likely sequences of map matching candidates:
         List<MatchSequence> sequences = computeViterbiSequence(viterbiMatchEntries, queryGraph);
-        
+
         // TODO: refactor this into a separate methods per PR87 discussion
         logger.debug("=============== Viterbi results =============== ");
         i = 1;
-        for (MatchSequence seq: sequences) {
+        for (MatchSequence seq : sequences) {
             int j = 1;
-            for (SequenceState<Candidate, MatchEntry, Path> ss: seq.matchedSequence) {
-                logger.debug("{}-{}: {}, path: {}", i, j, ss.state,
-                        ss.transitionDescriptor != null ? ss.transitionDescriptor.calcEdges() : null);
+            for (SequenceState<Candidate, MatchEntry, Path> ss : seq.matchedSequence) {
+                logger.debug("{}-{}: {}, path: {}", i, j, ss.state, ss.transitionDescriptor != null
+                        ? ss.transitionDescriptor.calcEdges() : null);
                 j++;
             }
             i++;
@@ -238,7 +237,7 @@ public MatchResult doWork(List<GPXEntry> gpxList) {
         final EdgeExplorer explorer = queryGraph.createEdgeExplorer(edgeFilter);
         MatchResult matchResult = computeMatchResult(contiguousSequences, viterbiMatchEntries,
                 matchEntries, allCandidateLocations, explorer);
-        
+
         // TODO: refactor this into a separate methods per PR87 discussion
         logger.debug("=============== Matched real edges =============== ");
         i = 1;
@@ -304,9 +303,26 @@ private void calculateCandidatesPerEvent(List<ViterbiMatchEntry> viterbiMatchEnt
         // virtual nodes/edges in the same queryGraph, and b) we can only call 'lookup' once.
         queryGraph.lookup(allCandidateLocations);
 
+        // Different QueryResult can have the same tower node as their closest node. Hence, we now
+        // dedupe the query results of each GPX entry by their closest node (#91). This must be done
+        // after calling queryGraph.lookup() since this replaces some of the QueryResult nodes with
+        // virtual nodes. Virtual nodes are not deduped since there is at most one QueryResult per
+        // edge and virtual nodes are inserted into the middle of an edge. Reducing the number of
+        // QueryResults improves performance since less shortest/fastest routes need to be computed.
+        final List<Collection<QueryResult>> dedupedCandidateLocationsPerEvent = new ArrayList<Collection<QueryResult>>(
+                candidateLocationsPerEvent.size());
+        for (List<QueryResult> candidateLocations: candidateLocationsPerEvent) {
+            final Map<Integer, QueryResult> dedupedCandidateLocations = new HashMap<>(
+                    candidateLocations.size());
+            for (QueryResult qr : candidateLocations) {
+                dedupedCandidateLocations.put(qr.getClosestNode(), qr);
+            }
+            dedupedCandidateLocationsPerEvent.add(dedupedCandidateLocations.values());
+        }
+
         // create the final candidate and viterbiMatchEntry per event:
         for (int i = 0; i < viterbiMatchEntries.size(); i++) {
-            viterbiMatchEntries.get(i).createCandidates(candidateLocationsPerEvent.get(i),
+            viterbiMatchEntries.get(i).createCandidates(dedupedCandidateLocationsPerEvent.get(i),
                     queryGraph);
         }
     }
@@ -548,7 +564,6 @@ private double penalizedPathDistance(Path path, Set<EdgeIteratorState> penalized
     private MatchResult computeMatchResult(List<MatchSequence> sequences,
             List<ViterbiMatchEntry> viterbiMatchEntries, List<MatchEntry> matchEntries,
             List<QueryResult> allCandidateLocations, EdgeExplorer explorer) {
-
         final Map<String, EdgeIteratorState> virtualEdgesMap = createVirtualEdgesMap(
                 allCandidateLocations, explorer);
 

matching-core/src/main/java/com/graphhopper/matching/MapMatchingMain.java

@@ -87,6 +87,8 @@ private void start(CmdArgs args) {
                     algorithm(Parameters.Algorithms.DIJKSTRA_BI).traversalMode(hopper.getTraversalMode()).
                     weighting(new FastestWeighting(firstEncoder)).
                     maxVisitedNodes(args.getInt("max_visited_nodes", 1000)).
+                    // Penalizing inner-link U-turns only works with fastest weighting, since
+                    // shortest weighting does not apply penalties to unfavored virtual edges.
                     hints(new HintsMap().put("weighting", "fastest").put("vehicle", firstEncoder.toString())).
                     build();
             MapMatching mapMatching = new MapMatching(hopper, opts);

matching-core/src/main/java/com/graphhopper/matching/ViterbiMatchEntry.java

@@ -99,7 +99,12 @@ public ViterbiMatchEntry(MatchEntry matchEntry) {
     }
 
     /**
-     * Find all (real) edges/nodes locations which are within the provided search radius.
+     * Find all (real) edges/nodes locations which are within the provided search radius. Note that
+     * this searches at most 9 index cells to avoid performance problems, and hence if the radius is
+     * larger than the cell width then not all edges might be returned.
+     * 
+     * TODO: throw an error if searchRadiusMeters > index.getMinResolutionInMeter() as not all edges
+     * may be found.
      * 
      * @param graph the base graph to search
      * @param index the base location index to search
@@ -118,7 +123,8 @@ public List<QueryResult> findCandidateLocations(final Graph graph,
         // implement a cheap priority queue via List, sublist and Collections.sort
         final List<QueryResult> candidateLocations = new ArrayList<QueryResult>();
         GHIntHashSet set = new GHIntHashSet();
-
+
+        // Doing 2 iterations means searching 9 tiles.
         for (int iteration = 0; iteration < 2; iteration++) {
             // should we use the return value of earlyFinish?
             index.findNetworkEntries(lat, lon, set, iteration);
@@ -140,6 +146,12 @@ protected double getQueryDistance() {
                         @Override
                         protected boolean check(int node, double normedDist, int wayIndex,
                                 EdgeIteratorState edge, QueryResult.Position pos) {
+                            // TODO: refactor below:
+                            //   - should only add edges within search radius (below allows the
+                            //     returning of a candidate outside search radius if it's the only
+                            //     one. Removing this test would simplify it a lot.
+                            //   - create QueryResult first and the add/set as required - clean up
+                            //     the index tracking business.
                             if (normedDist < returnAllResultsWithin
                                     || candidateLocations.isEmpty()
                                     || candidateLocations.get(0).getQueryDistance() > normedDist) {
@@ -148,7 +160,7 @@ protected boolean check(int node, double normedDist, int wayIndex,
                                 for (int qrIndex = 0; qrIndex < candidateLocations.size();
                                         qrIndex++) {
                                     QueryResult qr = candidateLocations.get(qrIndex);
-                                    // overwrite older queryResults which are potentially more far
+                                    // overwrite older queryResults which are potentially further
                                     // away than returnAllResultsWithin
                                     if (qr.getQueryDistance() > returnAllResultsWithin) {
                                         index = qrIndex;
@@ -204,13 +216,13 @@ protected boolean check(int node, double normedDist, int wayIndex,
     }
 
     /**
-     * Create the (directed) candidates based on the provided candidate locations. 
+     * Create the (directed) candidates based on the provided candidate locations.
      * 
      * @param candidateLocations list of candidate location (as provided by findCandidateLocations
      *        but already looked up in queryGraph)
      * @param queryGraph the queryGraph being used
      */
-    public void createCandidates(List<QueryResult> candidateLocations, QueryGraph queryGraph) {
+    public void createCandidates(Collection<QueryResult> candidateLocations, QueryGraph queryGraph) {
         candidates = new ArrayList<Candidate>();
         for (QueryResult qr: candidateLocations) {
             int closestNode = qr.getClosestNode();