Update changelog

Use a shared (!) reader writer lock to protect CURRENT_REGIONS
This fixes issue #3016.
2016-10-10 12:31:26 +02:00 · 2016-10-10 12:26:14 +02:00 · 2016-10-04 14:15:37 +02:00 · 2016-10-04 09:38:51 +02:00 · 2016-10-03 20:02:11 +02:00 · 2016-10-03 15:47:29 +02:00
98 changed files with 1771 additions and 1534 deletions
@@ -1,11 +1,10 @@
-# 5.4.0-rc.3
-  Changes from 5.4.0-rc.2
+# 5.4.1
+  - Changes from 5.4.0
    - Bugfixes
-      - BREAKING: Fixed a bug where some roads could be falsly identified as sliproadsi This change requires reprocessing datasets with osrm-extract and osrm-contract
-      - BREAKING: Fixed a bug that resulted in false names/ref/destination/pronunciation This change requires reprocessing datasets with osrm-extract and osrm-contract
+      - #3016: Fixes shared memory updates while queries are running

 # 5.4.0
-  Changes from 5.3.0
+  - Changes from 5.3.0
    - Profiles
      - includes library guidance.lua that offers preliminary configuration on guidance.
      - added left_hand_driving flag in global profile properties
@@ -31,8 +30,43 @@
    - Infrastructure:
      - Adds a feature to limit results in nearest service with a default of 100 in `osrm-routed`

+# 5.4.0-rc.7
+  - Chages from 5.4.0-rc.6
+    - Bugfixes re-introduce space between two entries in summaries
+
+# 5.4.0-rc.6
+  - Changes from 5.4.0-rc.5
+    - Bugfixes
+      - fixed a bug where polyline decoding on a defective polyline could end up in out-of-bound access on a vector
+    - Guidance
+      - Summaries have been improved to consider references as well
+
+# 5.4.0-rc.5
+  - Changes from 5.4.0-rc.4
+    - Guidance
+      - Improved detection of obvious name changes
+    - Profiles
+      - The default profile for car now excludes HOV-only routes in navigation by default
+    - Bugfixes
+      - Fixed a bug that could result in endless loops in combination with sliproads
+
+# 5.4.0-rc.4
+  - Changes from 5.4.0-rc.3
+    - Bugfixes
+      - Fixed a bug where roundabout intersections could result in breaking assertions when immediately exited
+
+# 5.4.0-rc.3
+  - Changes from 5.4.0-rc.2
+    - Bugfixes
+      - BREAKING: Fixed a bug where some roads could be falsly identified as sliproadsi This change requires reprocessing datasets with osrm-extract and osrm-contract
+      - BREAKING: Fixed a bug that resulted in false names/ref/destination/pronunciation This change requires reprocessing datasets with osrm-extract and osrm-contract
+      - `restrictions` is now used for namespaced restrictions and restriction exceptions (e.g. `restriction:motorcar=` as well as `except=motorcar`)
+      - replaced lhs/rhs profiles by using test defined profiles
+    - Trip Plugin
+      - changed internal behaviour to prefer the smallest lexicographic result over the largest one
+
 # 5.3.0
-  Changes from 5.3.0-rc.3
+  - Changes from 5.3.0-rc.3
    - Guidance
      - Only announce `use lane` on required turns (not using all lanes to go straight)
      - Moved `lanes` to the intersection objects. This is BREAKING in relation to other Release Candidates but not with respect to other releases.
@@ -40,7 +74,7 @@
      - Fix BREAKING: bug that could result in failure to load 'osrm.icd' files. This breaks the dataformat
      - Fix: bug that results in segfaults when `use lane` instructions are suppressed

-  Changes form 5.2.7
+  - Changes form 5.2.7
    - API
      - Introduces new `TurnType` in the form of `use lane`. The type indicates that you have to stick to a lane without turning
      - Introduces `lanes` to the `Intersection` object. The lane data contains both the markings at the intersection and a flag indicating if they can be chosen for the next turn
@@ -64,7 +98,7 @@
      - Fix devide by zero on updating speed data using osrm-contract

 # 5.3.0 RC3
-  Changes from 5.3.0-rc.2
+  - Changes from 5.3.0-rc.2
    - Guidance
      - Improved detection of obvious turns
      - Improved turn lane detection
@@ -127,8 +127,8 @@ if(ENABLE_GOLD_LINKER)

        # Issue 2785: check gold binutils version and don't use gc-sections for versions prior 2.25
        string(REGEX REPLACE ".*\\(GNU Binutils[^\\)0-9]+([0-9]+\\.[0-9]+)[^\\)]*\\).*" "\\1" GOLD_BINUTILS_VERSION "${LD_VERSION}")
-        if ("${GOLD_BINUTILS_VERSION}" VERSION_LESS "2.25")
-          message(STATUS "Disabling gc-sections on gold binutils < 2.25, see: https://sourceware.org/bugzilla/show_bug.cgi?id=17639")
+        if ("${GOLD_BINUTILS_VERSION}" VERSION_LESS "2.26")
+          message(STATUS "Disabling gc-sections on gold binutils < 2.26, see: https://sourceware.org/bugzilla/show_bug.cgi?id=17639")
          set(LD_AVOID_GC_SECTIONS TRUE)
        endif()
    else()
@@ -1,10 +1,8 @@
 module.exports = {
-    default: '--require features --tags ~@stress --tags ~@todo',
-    verify: '--require features --tags ~@todo --tags ~@bug --tags ~@stress -f progress',
-    jenkins: '--require features --tags ~@todo --tags ~@bug --tags ~@stress --tags ~@options -f progress',
-    bugs: '--require features --tags @bug',
-    todo: '--require features --tags @todo',
-    all: '--require features'
+    default: '--strict --tags ~@stress --tags ~@todo --require features/support --require features/step_definitions',
+    verify: '--strict --tags ~@stress --tags ~@todo -f progress --require features/support --require features/step_definitions',
+    todo: '--strict --tags @todo --require features/support --require features/step_definitions',
+    all: '--strict --require features/support --require features/step_definitions'
 }


@@ -34,7 +34,7 @@ int main(int argc, const char *argv[])
    config.use_shared_memory = false;

    // Routing machine with several services (such as Route, Table, Nearest, Trip, Match)
-    OSRM osrm{config};
+    const OSRM osrm{config};

    // The following shows how to use the Route service; configure this service
    RouteParameters params;
@@ -156,6 +156,25 @@ Feature: Car - Restricted access
            | primary | yes        | x     |
            | primary | no         | x     |

+    Scenario: Car - a way with all lanes HOV-designated is inaccessible by default (similar to hov=designated)
+        Then routability should be
+            | highway | hov:lanes:forward      | hov:lanes:backward     | hov:lanes              | oneway | forw | backw |
+            | primary | designated             | designated             |                        |        |      |       |
+            | primary |                        | designated             |                        |        | x    |       |
+            | primary | designated             |                        |                        |        |      | x     |
+            | primary | designated\|designated | designated\|designated |                        |        |      |       |
+            | primary | designated\|no         | designated\|no         |                        |        | x    | x     |
+            | primary | yes\|no                | yes\|no                |                        |        | x    | x     |
+            | primary |                        |                        |                        |        | x    | x     |
+            | primary | designated             |                        |                        | -1     |      |       |
+            | primary |                        | designated             |                        | -1     |      | x     |
+            | primary |                        |                        | designated             | yes    |      |       |
+            | primary |                        |                        | designated             | -1     |      |       |
+            | primary |                        |                        | designated\|designated | yes    |      |       |
+            | primary |                        |                        | designated\|designated | -1     |      |       |
+            | primary |                        |                        | designated\|yes        | yes    | x    |       |
+            | primary |                        |                        | designated\|no         | -1     |      | x     |
+
     Scenario: Car - these toll roads always work
        Then routability should be
            | highway | toll        | bothw |
@@ -0,0 +1,94 @@
+@routing @basic @car
+Feature: Basic Routing
+
+    Background:
+        Given the profile "car"
+        Given a grid size of 500 meters
+
+    @smallest
+    Scenario: Summaries when routing on a simple network
+        Given the node map
+            | b |   |   | f |
+            |   |   |   |   |
+            | c | d |   | g |
+            |   |   |   |   |
+            | a |   | e |   |
+
+        And the ways
+            | nodes | name   |
+            | acb   | road   |
+            | de    | 1 st   |
+            | cd    |        |
+            | dg    | blvd   |
+            | df    | street |
+
+        When I route I should get
+            | waypoints | route                               | summary                 |
+            | a,e       | road,,1 st,1 st                     | road, 1 st              |
+            | a,d,f     | road,,,street,street                | road;street             |
+            | a,e,f     | road,,1 st,1 st,1 st,street,street  | road, 1 st;1 st, street |
+
+     Scenario: Name Empty
+        Given the node map
+            | a |   | b |   |   | c |
+
+        And the ways
+            | nodes | name |
+            | ab    | road |
+            | bc    |      |
+
+        When I route I should get
+            | waypoints | route  | summary |
+            | a,c       | road,  | road    |
+
+     Scenario: Name Empty But Ref
+        Given the node map
+            | a |   | b |   |   | c |
+
+        And the ways
+            | nodes | name | ref |
+            | ab    | road |     |
+            | bc    |      | 101 |
+
+        When I route I should get
+            | waypoints | route | summary   |
+            | a,c       | road, | road, 101 |
+
+     Scenario: Only Refs
+        Given the node map
+            | a |   | b |   |   | c |
+
+        And the ways
+            | nodes | name | ref |
+            | ab    |      | 100 |
+            | bc    |      | 101 |
+
+        When I route I should get
+            | waypoints | route  | summary  |
+            | a,c       | ,      | 100, 101 |
+
+     Scenario: Single Ref
+        Given the node map
+            | a |   | b |   |   | c |
+
+        And the ways
+            | nodes | name | ref |
+            | ab    |      |     |
+            | bc    |      | 101 |
+
+        When I route I should get
+            | waypoints | route | summary |
+            | a,c       | ,,    | 101     |
+
+     Scenario: Nothing
+        Given the node map
+            | a |   | b |   |   | c |
+
+        And the ways
+            | nodes | name |
+            | ab    |      |
+            | bc    |      |
+
+        When I route I should get
+            | waypoints | route | summary |
+            | a,c       | ,     |         |
@@ -25,7 +25,7 @@ Feature: Traffic - speeds
            | fb    | primary |
        Given the profile "testbot"
        Given the extract extra arguments "--generate-edge-lookup"
-        Given the contract extra arguments "--segment-speed-file speeds.csv"
+        Given the contract extra arguments "--segment-speed-file {speeds_file}"
        Given the speed file
        """
        1,2,0
@@ -69,7 +69,7 @@ Feature: Traffic - speeds
            | fb    | primary |
        Given the profile "testbot"
        Given the extract extra arguments "--generate-edge-lookup"
-        Given the contract extra arguments "--segment-speed-file speeds.csv"
+        Given the contract extra arguments "--segment-speed-file {speeds_file}"
        Given the speed file
        """
        1,2,0
@@ -112,7 +112,6 @@ Feature: Traffic - speeds
            | fb    | primary |
        Given the profile "testbot"
        Given the extract extra arguments "--generate-edge-lookup"
-        Given the contract extra arguments "--segment-speed-file speeds.csv"
        Given the speed file
        """
        1,2,-10
@@ -123,6 +122,6 @@ Feature: Traffic - speeds
        4,1,-5
        """
        And the data has been extracted
-        When I run "osrm-contract --segment-speed-file speeds.csv {extracted_base}.osrm"
+        When I try to run "osrm-contract --segment-speed-file {speeds_file} {processed_file}"
        And stderr should contain "malformed"
-        And it should exit with code not 0
+        And it should exit with an error
@@ -58,7 +58,7 @@ Feature: Traffic - turn penalties
            8,11,12,23
            1,4,5,-0.2
            """
-        And the contract extra arguments "--turn-penalty-file penalties.csv"
+        And the contract extra arguments "--turn-penalty-file {penalties_file}"
        When I route I should get
            | from | to | route                 | speed   | time      |
            | a    | h  | ad,dhk,dhk            | 63 km/h | 11.5s +-1 |
@@ -81,7 +81,7 @@ Feature: Traffic - turn penalties
                                                                              # double left - hdc penalty ever so slightly higher than imn; forces all the way around

    Scenario: Too-negative penalty clamps, but does not fail
-        Given the contract extra arguments "--turn-penalty-file penalties.csv"
+        Given the contract extra arguments "--turn-penalty-file {penalties_file}"
        And the profile "testbot"
        And the turn penalty file
            """
@@ -103,7 +103,7 @@ Feature: Turn Lane Guidance
            | a,j       | in,turn,other,other  | depart,turn right,turn left,arrive  | ,,left:true right:false,               |
            | a,i       | in,turn,other,other  | depart,turn right,turn right,arrive | ,,left:false right:true,               |

-    @todo @bug @2654 @none
+    @todo @2654 @none
    #https://github.com/Project-OSRM/osrm-backend/issues/2645
    #http://www.openstreetmap.org/export#map=19/52.56054/13.32152
    Scenario: Kurt-Schuhmacher-Damm
@@ -131,7 +131,7 @@ Feature: Turn Lane Guidance
            | a,f       | ,ksd,ksd | depart,turn left,arrive  | ,left:true none:true right:false, |
            | a,i       | ,ksd,ksd | depart,turn right,arrive | ,left:false none:true right:true, |

-    @todo @bug @2650 @sliproads
+    @todo @2650 @sliproads
    #market and haight in SF, restricted turn
    #http://www.openstreetmap.org/#map=19/37.77308/-122.42238
    Scenario: Market/Haight without Through Street
@@ -177,7 +177,7 @@ Feature: Turn Lane Guidance
            | a,j       | ghough,market,market | depart,turn left,arrive            | ,none:true straight:false straight:false straight:false, |
            | a,f       | ghough,ghough,ghough | depart,continue slight left,arrive | ,none:true straight:true straight:false straight:false,  |

-    @todo @2650 @bug @sliproads
+    @todo @2650 @sliproads
    #market and haight in SF, unrestricted
    #http://www.openstreetmap.org/#map=19/37.77308/-122.42238
    Scenario: Market/Haight without Through Street
@@ -422,7 +422,7 @@ Feature: Turn Lane Guidance
               | waypoints | route               | turns                                          | lanes                                                                                                                                                             |
               | a,e       | main,main,main,main | depart,use lane straight,continue right,arrive | ,left:false straight:false straight:false straight:false straight:true straight:true right:false,straight:false straight:false right:false right:true right:true, |

-    @anticipate @todo @bug @2661
+    @anticipate @todo @2661
    Scenario: Anticipate with lanes in roundabout: roundabouts as the unit of anticipation
        Given the node map
            |   |   | e |   |   |
@@ -667,7 +667,7 @@ Feature: Turn Lane Guidance
            | a,f       | abc,bdeh,feg,feg | depart,turn right,turn right,arrive | ,none:false none:false right:false right:true,left:false none:false none:false right:true, |

    @anticipate
-    Scenario: Tripple Right keeping Left
+    Scenario: Triple Right keeping Left
        Given the node map
            | a |   |   |   | b |   | i |
            |   |   |   |   |   |   |   |
@@ -313,3 +313,39 @@ Feature: Slipways and Dedicated Turn Lanes
            | a,d       | new york,new york                       | depart,arrive                      | this is the sinatra route            |
            | a,j       | new york,1st street,1st street          | depart,turn left,arrive            |                                      |
            | a,1       | new york,m street,1st street,1st street | depart,turn right,turn left,arrive | this can false be seen as a sliproad |
+
+    # Merging into degree two loop on dedicated turn detection / 2927
+    Scenario: Turn Instead of Ramp
+        Given the node map
+            |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   | f |
+            |   |   |   |   | g |   |   |   |   |   | h |   |   |   |   |   |
+            |   |   |   |   |   |   |   |   |   |   |   |   | d |   |   | e |
+            | i |   |   |   | c |   |   |   |   |   | j |   |   |   |   |   |
+            |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |
+            |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |
+            |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |
+            |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |
+            |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |
+            |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |
+            |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |
+            |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |
+            |   |   | b |   |   |   |   |   |   |   |   |   |   |   |   |   |
+            |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |
+            |   |   | a |   |   |   |   |   |   |   |   |   |   |   |   |   |
+
+        And the ways
+            | nodes | highway | name | oneway |
+            | abi   | primary | road | yes    |
+            | bcjd  | primary | loop | yes    |
+            | dhgf  | primary | loop | yes    |
+            | fed   | primary | loop | yes    |
+
+        And the nodes
+            | node | highway         |
+            | g    | traffic_signals |
+            | c    | traffic_signals |
+
+        # We don't actually care about routes here, this is all about endless loops in turn discovery
+        When I route I should get
+            | waypoints | route          | turns                          |
+            | a,i       | road,road,road | depart,fork slight left,arrive |
@@ -164,3 +164,172 @@ Feature: New-Name Instructions
            | waypoints | route                    | turns                           |
            | a,e       | name,with-name,with-name | depart,new name straight,arrive |
            | b,e       | with-name,with-name      | depart,arrive                   |
+
+    Scenario: Both Name and Ref Empty
+        Given the node map
+            | a |  | b |  | c |
+
+        And the ways
+            | nodes | name | ref |
+            | ab    |      |     |
+            | bc    |      |     |
+
+        When I route I should get
+            | waypoints | route | turns         |
+            | a,c       | ,     | depart,arrive |
+
+    Scenario: Same Name, Ref Extended
+        Given the node map
+            | a |  | b |  | c |
+
+        And the ways
+            | nodes | name | ref   |
+            | ab    | A    | B1    |
+            | bc    | C    | B1;B2 |
+
+        When I route I should get
+            | waypoints | route | turns                           |
+            | a,c       | A,C,C | depart,new name straight,arrive |
+
+    Scenario: Same Name, Ref Removed
+        Given the node map
+            | a |  | b |  | c |
+
+        And the ways
+            | nodes | name | ref   |
+            | ab    | A    | B1;B2 |
+            | bc    | C    | B1    |
+
+        When I route I should get
+            | waypoints | route | turns                           |
+            | a,c       | A,C,C | depart,new name straight,arrive |
+
+    Scenario: Name Removed, Ref Extended
+        Given the node map
+            | a |  | b |  | c |
+
+        And the ways
+            | nodes | name | ref   |
+            | ab    | A    | B1    |
+            | bc    |      | B1;B2 |
+
+        When I route I should get
+            | waypoints | route | turns         |
+            | a,c       | A,    | depart,arrive |
+
+    Scenario: Name Added, Ref Removed
+        Given the node map
+            | a |  | b |  | c |
+
+        And the ways
+            | nodes | name | ref   |
+            | ab    |      | B1    |
+            | bc    | A    |       |
+
+        When I route I should get
+            | waypoints | route | turns                           |
+            | a,c       | ,A,A  | depart,new name straight,arrive |
+
+    Scenario: Prefix Change
+        Given the node map
+            | a |   |   |   | b |   |   |   | c |
+
+        And the ways
+            | nodes | name                     | ref   | highway  |
+            | ab    | North Central Expressway | US 75 | motorway |
+            | bc    | Central Expressway       | US 75 | motorway |
+
+        When I route I should get
+            | waypoints | route                                                          | turns                           |
+            | a,c       | North Central Expressway,Central Expressway,Central Expressway | depart,new name straight,arrive |
+
+    Scenario: Prefix Change
+        Given the node map
+            | a |   |   |   | b |   |   |   | c |
+
+        And the ways
+            | nodes | name                     | ref   | highway  |
+            | ba    | North Central Expressway | US 75 | motorway |
+            | cb    | Central Expressway       | US 75 | motorway |
+
+        When I route I should get
+            | waypoints | route                                                                | turns                           |
+            | c,a       | Central Expressway,North Central Expressway,North Central Expressway | depart,new name straight,arrive |
+
+    Scenario: No Name, Same Reference
+        Given the node map
+            | a |   |   |   | b |   |   |   | c |
+
+        And the ways
+            | nodes | name               | ref   | highway  |
+            | ab    | Central Expressway | US 75 | motorway |
+            | bc    |                    | US 75 | motorway |
+
+        When I route I should get
+            | waypoints | route               | turns         |
+            | a,c       | Central Expressway, | depart,arrive |
+
+    Scenario: No Name, Same Reference
+        Given the node map
+            | a |   |   |   | b |   |   |   | c |
+
+        And the ways
+            | nodes | name               | ref   | highway  |
+            | ab    |                    | US 75 | motorway |
+            | bc    | Central Expressway | US 75 | motorway |
+
+        When I route I should get
+            | waypoints | route               | turns         |
+            | a,c       | ,Central Expressway | depart,arrive |
+
+    Scenario: No Name, Same Reference
+        Given the node map
+            | a |   |   |   | b |   |   |   | c |
+
+        And the ways
+            | nodes | name | ref         | highway  |
+            | ab    |      | US 75;US 69 | motorway |
+            | bc    |      | US 75       | motorway |
+
+        When I route I should get
+            | waypoints | route | turns         |
+            | a,c       | ,     | depart,arrive |
+
+    Scenario: No Name, Same Reference
+        Given the node map
+            | a |   |   |   | b |   |   |   | c |
+
+        And the ways
+            | nodes | name | ref         | highway  |
+            | ab    |      | US 69;US 75 | motorway |
+            | bc    |      | US 75       | motorway |
+
+        When I route I should get
+            | waypoints | route | turns         |
+            | a,c       | ,     | depart,arrive |
+
+    Scenario: No Name, Same Reference
+        Given the node map
+            | a |   |   |   | b |   |   |   | c |
+
+        And the ways
+            | nodes | name | ref         | highway  |
+            | ab    |      | US 75       | motorway |
+            | bc    |      | US 75;US 69 | motorway |
+
+        When I route I should get
+            | waypoints | route | turns         |
+            | a,c       | ,     | depart,arrive |
+
+    Scenario: No Name, Same Reference
+        Given the node map
+            | a |   |   |   | b |   |   |   | c |
+
+        And the ways
+            | nodes | name | ref         | highway  |
+            | ab    |      | US 75       | motorway |
+            | bc    |      | US 69;US 75 | motorway |
+
+        When I route I should get
+            | waypoints | route | turns         |
+            | a,c       | ,     | depart,arrive |
@@ -491,3 +491,23 @@ Feature: Basic Roundabout
           | h,a       | gh,ab,ab | depart,roundabout turn left exit-3,arrive     |
           | h,d       | gh,cd,cd | depart,roundabout turn straight exit-2,arrive |
           | h,f       | gh,ef,ef | depart,roundabout turn right exit-1,arrive    |
+
+    #http://www.openstreetmap.org/#map=19/41.03275/-2.18990
+    #at some point we probably want to recognise these situations and don't mention the roundabout at all here
+    Scenario: Enter And Exit Throughabout
+        Given the node map
+            |   |   |   |   |   | h |   |   |   |   |
+            |   |   |   |   |   |   |   |   |   |   |
+            | c | b |   | d |   |   |   | e |   | f |
+            |   |   |   |   |   |   |   |   |   |   |
+            |   | a |   |   |   | g |   |   |   |   |
+
+        And the ways
+            | nodes | highway       | name    | junction   | oneway |
+            | dghd  | tertiary_link |         | roundabout |        |
+            | cbdef | trunk         | through |            | no     |
+            | ab    | residential   | in      |            |        |
+
+        When I route I should get
+            | waypoints | turns                                                    | route                      |
+            | a,f       | depart,turn right,roundabout turn straight exit-1,arrive | in,through,through,through |
@@ -36,6 +36,25 @@ Feature: Basic Roundabout
           | h,c       | gh,bcegb,bcegb | depart,roundabout-exit-undefined,arrive |
           | h,e       | gh,bcegb,bcegb | depart,roundabout-exit-undefined,arrive |

+    #2927
+    Scenario: Only Roundabout
+        Given the node map
+            |   |   | a |   |   |
+            |   |   |   |   |   |
+            |   |   |   |   |   |
+            | b |   |   |   | d |
+            |   |   |   |   |   |
+            |   |   |   |   |   |
+            |   |   | c |   |   |
+
+       And the ways
+            | nodes  | junction   |
+            | abcda  | roundabout |
+
+       When I route I should get
+           | waypoints | route       | turns         |
+           | a,c       | abcda,abcda | depart,arrive |
+
    Scenario: Only Exit
        Given the node map
            |   |   | a |   |   |
@@ -145,7 +145,6 @@ Feature: Turn Lane Guidance
            | a,c       | road,road      | depart,arrive           | ,                           |

    # This tests whether empty/invalid PSV tags cause osrm-extract to crash
-    @bug
    Scenario: Turn with Bus-Lane
        Given the node map
            | a |   | b |   | c |
@@ -181,7 +180,7 @@ Feature: Turn Lane Guidance
            | a,c       | road,road      | depart,arrive            |

    #turn lanes are often drawn at the incoming road, even though the actual turn requires crossing the intersection first
-    @todo @bug @collapse @partition-lanes
+    @todo @collapse @partition-lanes
    Scenario: Turn Lanes at Segregated Road
        Given the node map
            |   |   | i | l |   |   |
@@ -383,7 +382,7 @@ Feature: Turn Lane Guidance
            | a,f       | road,turn,turn   | depart,turn left,arrive  | ,left:true right:false, |
            | a,g       | road,right,right | depart,turn right,arrive | ,left:false right:true, |

-    @todo @bug @partition-lanes @previous-lanes
+    @todo @partition-lanes @previous-lanes
    Scenario: Narrowing Turn Lanes
        Given the node map
            |   |   |   |   | g |   |
@@ -428,7 +427,7 @@ Feature: Turn Lane Guidance
            | a,d       | road,road      | depart,arrive            | ,                           |
            | a,e       | road,turn,turn | depart,turn right,arrive | ,straight:false right:true, |

-    @bug @todo @roundabout
+    @todo @roundabout
    Scenario: Theodor Heuss Platz
        Given the node map
            |   |   |   | i | o |   |   | l |   |
@@ -555,7 +554,7 @@ Feature: Turn Lane Guidance
            | a,e       | road,road,road         | depart,turn uturn,arrive        | ,left:true left:false left:false straight:false straight:false, |
            | a,g       | road,straight,straight | depart,new name straight,arrive | ,left:false left:false left:false straight:true straight:true,  |

-    @bug @todo @roundabout
+    @todo @roundabout
    Scenario: Passing Through a Roundabout
        Given the node map
            |   |   | h |   | g |   |   |
@@ -620,7 +619,7 @@ Feature: Turn Lane Guidance
            | a,d       | hwy,hwy       | depart,arrive                       | ,                                                                            |
            | a,e       | hwy,ramp,ramp | depart,off ramp slight right,arrive | ,straight:false straight:false straight;slight right:true slight right:true, |

-    @bug @todo
+    @todo
    Scenario: Turning Off Ramp
        Given the node map
            |   | a |   |
@@ -864,7 +863,7 @@ Feature: Turn Lane Guidance
            | a,c       | ab,bc,bc  | depart,turn left,arrive | ,left:true right:false, |

    # http://www.openstreetmap.org/#map=19/47.97685/7.82933&layers=D
-    @bug @todo
+    @todo
    Scenario: Lane Parsing Issue #2706: None Assignments I
        Given the node map
            |   | f |   |   | j  |   |
@@ -904,7 +903,7 @@ Feature: Turn Lane Guidance
            # Note: at the moment we don't care about routes, we care about the extract process triggering assertions

    # https://www.openstreetmap.org/#map=19/47.99257/7.83276&layers=D
-    @bug @todo
+    @todo
    Scenario: Lane Parsing Issue #2706: None Assignments II
        Given the node map
            |   | k | l |   |
@@ -939,7 +938,7 @@ Feature: Turn Lane Guidance
            | i,e ||||
            # Note: at the moment we don't care about routes, we care about the extract process triggering assertions

-    @bug @todo
+    @todo
    Scenario: Lane Parsing Issue #2706: None Assignments III - Minimal reproduction recipe
        Given the node map
            |   |   | l |   |
@@ -0,0 +1,31 @@
+'use strict';
+
+const fs = require('fs');
+const crypto = require('crypto');
+const d3 = require('d3-queue');
+
+module.exports =  {
+    hashOfFiles: (paths, cb) => {
+        let queue = d3.queue();
+        for (let i = 0; i < paths.length; ++i) {
+            queue.defer(fs.readFile, paths[i]);
+        }
+        queue.awaitAll((err, results) => {
+            if (err) return cb(err);
+            let checksum = crypto.createHash('md5');
+            for (let i = 0; i < results.length; ++i) {
+                checksum.update(results[i]);
+            }
+            cb(null, checksum.digest('hex'));
+        });
+    },
+
+     hashOfFile: (path, cb) => {
+        fs.readFile(path, (err, result) => {
+            if (err) return cb(err);
+            let checksum = crypto.createHash('md5');
+            checksum.update(result);
+            cb(null, checksum.digest('hex'));
+        });
+    }
+};
@@ -1,11 +1,7 @@
 'use strict';

-var builder = require('xmlbuilder');
-
-var ensureDecimal = (i) => {
-    if (parseInt(i) === i) return i.toFixed(1);
-    else return i;
-};
+const builder = require('xmlbuilder');
+const ensureDecimal = require('./utils').ensureDecimal;

 class DB {
    constructor () {
@@ -0,0 +1,169 @@
+'use strict';
+
+const fs = require('fs');
+const util = require('util');
+const Timeout = require('node-timeout');
+const tryConnect = require('../lib/try_connect');
+const errorReason = require('./utils').errorReason;
+
+class OSRMBaseLoader{
+    constructor (scope) {
+        this.scope = scope;
+        this.child = null;
+    }
+
+    launch (callback) {
+        var limit = Timeout(this.scope.TIMEOUT, { err: new Error('*** Launching osrm-routed timed out.') });
+
+        var runLaunch = (cb) => {
+            this.osrmUp(() => { this.waitForConnection(cb); });
+        };
+
+        runLaunch(limit((e) => { if (e) callback(e); else callback(); }));
+    }
+
+    shutdown (callback) {
+        if (!this.osrmIsRunning()) return callback();
+
+        var limit = Timeout(this.scope.TIMEOUT, { err: new Error('*** Shutting down osrm-routed timed out.')});
+
+        this.osrmDown(limit(callback));
+    }
+
+    osrmIsRunning () {
+        return this.child && !this.child.killed;
+    }
+
+    osrmDown (callback) {
+        if (this.osrmIsRunning()) {
+            this.child.on('exit', (code, signal) => {callback();});
+            this.child.kill();
+        } else callback();
+    }
+
+    waitForConnection (callback) {
+        var retryCount = 0;
+        let retry = (err) => {
+          if (err) {
+            if (retryCount < 10) {
+              retryCount++;
+              setTimeout(() => { tryConnect(this.scope.OSRM_PORT, retry); }, 10);
+            } else {
+              callback(new Error("Could not connect to osrm-routed after ten retries."));
+            }
+          }
+          else
+          {
+            callback();
+          }
+        };
+
+        tryConnect(this.scope.OSRM_PORT, retry);
+    }
+};
+
+class OSRMDirectLoader extends OSRMBaseLoader {
+    constructor (scope) {
+        super(scope);
+    }
+
+    load (inputFile, callback) {
+        this.inputFile = inputFile;
+        this.shutdown(() => {
+            this.launch(callback);
+        });
+    }
+
+    osrmUp (callback) {
+        if (this.osrmIsRunning()) return callback(new Error("osrm-routed already running!"));
+
+        this.child = this.scope.runBin('osrm-routed', util.format("%s -p %d", this.inputFile, this.scope.OSRM_PORT), this.scope.environment, (err) => {
+          if (err) {
+              throw new Error(util.format('osrm-routed %s: %s', errorReason(err), err.cmd));
+          }
+        });
+        callback();
+    }
+};
+
+class OSRMDatastoreLoader extends OSRMBaseLoader {
+    constructor (scope) {
+        super(scope);
+    }
+
+    load (inputFile, callback) {
+        this.inputFile = inputFile;
+
+        this.loadData((err) => {
+            if (err) return callback(err);
+            if (!this.osrmIsRunning()) this.launch(callback);
+            else {
+                this.scope.setupOutputLog(this.child, fs.createWriteStream(this.scope.scenarioLogFile, {'flags': 'a'}));
+                callback();
+            }
+        });
+    }
+
+    loadData (callback) {
+        this.scope.runBin('osrm-datastore', this.inputFile, this.scope.environment, (err) => {
+            if (err) return callback(new Error('*** osrm-datastore exited with ' + err.code + ': ' + err));
+            callback();
+        });
+    }
+
+    osrmUp (callback) {
+        if (this.osrmIsRunning()) return callback();
+
+        this.child = this.scope.runBin('osrm-routed', util.format('--shared-memory=1 -p %d', this.scope.OSRM_PORT), this.scope.environment, (err) => {
+            if (err) {
+                throw new Error(util.format('osrm-routed %s: %s', errorReason(err), err.cmd));
+            }
+        });
+
+        // we call the callback here, becuase we don't want to wait for the child process to finish
+        callback();
+    }
+};
+
+class OSRMLoader {
+    constructor (scope) {
+        this.scope = scope;
+        this.sharedLoader = new OSRMDatastoreLoader(this.scope);
+        this.directLoader = new OSRMDirectLoader(this.scope);
+        this.method = scope.DEFAULT_LOAD_METHOD;
+    }
+
+    load (inputFile, callback) {
+        if (this.method === 'datastore') {
+            this.directLoader.shutdown((err) => {
+              if (err) return callback(err);
+              this.loader = this.sharedLoader;
+              this.sharedLoader.load(inputFile, callback);
+            });
+        } else if (this.method === 'directly') {
+            this.sharedLoader.shutdown((err) => {
+              if (err) return callback(err);
+              this.loader = this.directLoader;
+              this.directLoader.load(inputFile, callback);
+            });
+        } else {
+            callback(new Error('*** Unknown load method ' + method));
+        }
+    }
+
+    setLoadMethod (method) {
+        this.method = method;
+    }
+
+    shutdown (callback) {
+        if (!this.loader) return callback();
+
+        this.loader.shutdown(callback);
+    }
+
+    up () {
+        return this.loader ? this.loader.osrmIsRunning() : false;
+    }
+};
+
+module.exports = OSRMLoader;
@@ -0,0 +1,54 @@
+'use strict';
+
+var util = require('util');
+var path = require('path');
+var fs = require('fs');
+var chalk = require('chalk');
+
+var unescapeStr = (str) => str.replace(/\\\|/g, '\|').replace(/\\\\/g, '\\');
+
+module.exports = function (expected, actual) {
+    let headers = expected.raw()[0];
+    let expected_keys = expected.hashes();
+    let diff = [];
+    let hasErrors = false;
+
+    var good = 0, bad = 0;
+
+    expected_keys.forEach((row, i) => {
+        var rowError = false;
+
+        for (var j in row) {
+            if (unescapeStr(row[j]) != actual[i][j]) {
+                rowError = true;
+                hasErrors = true;
+                break;
+            }
+        }
+
+        if (rowError) {
+            bad++;
+            diff.push(Object.assign({}, row, {c_status: 'undefined'}));
+            diff.push(Object.assign({}, actual[i], {c_status: 'comment'}));
+        } else {
+            good++;
+            diff.push(row);
+        }
+    });
+
+    if (!hasErrors) return null;
+
+    var s = ['Tables were not identical:'];
+    s.push(headers.map(key => '    ' + key).join(' | '));
+    diff.forEach((row) => {
+        var rowString = '| ';
+        headers.forEach((header) => {
+            if (!row.c_status) rowString += chalk.green('    ' + row[header] + ' | ');
+            else if (row.c_status === 'undefined') rowString += chalk.yellow('(-) ' + row[header] + ' | ');
+            else rowString += chalk.red('(+) ' + row[header] + ' | ');
+        });
+        s.push(rowString);
+    });
+
+    return s.join('\n') + '\nTODO this is a temp workaround waiting for https://github.com/cucumber/cucumber-js/issues/534';
+};
@@ -0,0 +1,13 @@
+'use strict';
+
+const net = require('net');
+const Timeout = require('node-timeout');
+
+module.exports = function tryConnect(port, callback) {
+  net.connect({ port: port, host: '127.0.0.1' })
+    .on('connect', () => { callback(); })
+    .on('error', () => {
+        callback(new Error('Could not connect.'));
+    });
+}
+
@@ -0,0 +1,17 @@
+'use strict';
+
+const util = require('util');
+
+module.exports = {
+
+    ensureDecimal: (i) => {
+        if (parseInt(i) === i) return i.toFixed(1);
+        else return i;
+    },
+
+    errorReason: (err) => {
+        return err.signal ?
+            util.format('killed by signal %s', err.signal) :
+            util.format('exited with code %d', err.code);
+    }
+};
@@ -1,8 +1,7 @@
@prepare @options @files
 Feature: osrm-contract command line options: datasources
 # expansions:
-# {extracted_base} => path to current extracted input file
-# {profile} => path to current profile script
+# {processed_file} => path to .osrm file

    Background:
        Given the profile "testbot"
@@ -24,7 +23,6 @@ Feature: osrm-contract command line options: datasources
        And the data has been extracted

    Scenario: osrm-contract - Passing base file
-        When I run "osrm-contract --segment-speed-file speeds.csv {extracted_base}.osrm"
-        Then stderr should be empty
-        And datasource names should contain "lua profile,speeds"
-        And it should exit with code 0
+        When I run "osrm-contract --segment-speed-file {speeds_file} {processed_file}"
+        Then datasource names should contain "lua profile,25_osrmcontract_passing_base_file_speeds"
+        And it should exit successfully
@@ -1,9 +1,5 @@
@prepare @options @files
 Feature: osrm-contract command line options: files
-# expansions:
-# {extracted_base} => path to current extracted input file
-# {profile} => path to current profile script
-
    Background:
        Given the profile "testbot"
        And the node map
@@ -14,12 +10,11 @@ Feature: osrm-contract command line options: files
        And the data has been extracted

    Scenario: osrm-contract - Passing base file
-        When I run "osrm-contract {extracted_base}.osrm"
-        Then stderr should be empty
-        And it should exit with code 0
+        When I run "osrm-contract {processed_file}"
+        Then it should exit successfully

    Scenario: osrm-contract - Missing input file
-        When I run "osrm-contract over-the-rainbow.osrm"
+        When I try to run "osrm-contract over-the-rainbow.osrm"
        And stderr should contain "over-the-rainbow.osrm"
        And stderr should contain "not found"
-        And it should exit with code 1
+        And it should exit with an error
@@ -2,7 +2,7 @@
 Feature: osrm-contract command line options: help

    Scenario: osrm-contract - Help should be shown when no options are passed
-        When I run "osrm-contract"
+        When I try to run "osrm-contract"
        Then stderr should be empty
        And stdout should contain "osrm-contract <input.osrm> [options]:"
        And stdout should contain "Options:"
@@ -13,7 +13,7 @@ Feature: osrm-contract command line options: help
        And stdout should contain "--core"
        And stdout should contain "--level-cache"
        And stdout should contain "--segment-speed-file"
-        And it should exit with code 1
+        And it should exit with an error

    Scenario: osrm-contract - Help, short
        When I run "osrm-contract -h"
@@ -27,7 +27,7 @@ Feature: osrm-contract command line options: help
        And stdout should contain "--core"
        And stdout should contain "--level-cache"
        And stdout should contain "--segment-speed-file"
-        And it should exit with code 0
+        And it should exit successfully

    Scenario: osrm-contract - Help, long
        When I run "osrm-contract --help"
@@ -41,4 +41,4 @@ Feature: osrm-contract command line options: help
        And stdout should contain "--core"
        And stdout should contain "--level-cache"
        And stdout should contain "--segment-speed-file"
-        And it should exit with code 0
+        And it should exit successfully
@@ -5,8 +5,8 @@ Feature: osrm-contract command line options: invalid options
        Given the profile "testbot"

    Scenario: osrm-contract - Non-existing option
-        When I run "osrm-contract --fly-me-to-the-moon"
+        When I try to run "osrm-contract --fly-me-to-the-moon"
        Then stdout should be empty
        And stderr should contain "option"
        And stderr should contain "fly-me-to-the-moon"
-        And it should exit with code 1
+        And it should exit with an error
@@ -12,11 +12,11 @@ Feature: osrm-contract command line options: version
        Then stderr should be empty
        And stdout should contain 1 line
        And stdout should contain /(v\d{1,2}\.\d{1,2}\.\d{1,2}|\w*-\d+-\w+)/
-        And it should exit with code 0
+        And it should exit successfully

    Scenario: osrm-contract - Version, long
        When I run "osrm-contract --version"
        Then stderr should be empty
        And stdout should contain 1 line
        And stdout should contain /(v\d{1,2}\.\d{1,2}\.\d{1,2}|\w*-\d+-\w+)/
-        And it should exit with code 0
+        And it should exit successfully
@@ -14,17 +14,15 @@ Feature: osrm-extract command line options: files
        And the data has been saved to disk

    Scenario: osrm-extract - Passing base file
-        When I run "osrm-extract {osm_base}.osm --profile {profile}"
-        Then stderr should be empty
-        And it should exit with code 0
+        When I run "osrm-extract {osm_file} --profile {profile_file}"
+        Then it should exit successfully

    Scenario: osrm-extract - Order of options should not matter
-        When I run "osrm-extract --profile {profile} {osm_base}.osm"
-        Then stderr should be empty
-        And it should exit with code 0
+        When I run "osrm-extract --profile {profile_file} {osm_file}"
+        Then it should exit successfully

    Scenario: osrm-extract - Missing input file
-        When I run "osrm-extract over-the-rainbow.osrm --profile {profile}"
+        When I try to run "osrm-extract over-the-rainbow.osrm --profile {profile_file}"
        And stderr should contain "over-the-rainbow.osrm"
        And stderr should contain "not found"
-        And it should exit with code 1
+        And it should exit with an error
@@ -16,7 +16,7 @@ Feature: osrm-extract command line options: help
        And stdout should contain "--threads"
        And stdout should contain "--generate-edge-lookup"
        And stdout should contain "--small-component-size"
-        And it should exit with code 0
+        And it should exit successfully

    Scenario: osrm-extract - Help, short
        When I run "osrm-extract -h"
@@ -30,7 +30,7 @@ Feature: osrm-extract command line options: help
        And stdout should contain "--threads"
        And stdout should contain "--generate-edge-lookup"
        And stdout should contain "--small-component-size"
-        And it should exit with code 0
+        And it should exit successfully

    Scenario: osrm-extract - Help, long
        When I run "osrm-extract --help"
@@ -44,4 +44,4 @@ Feature: osrm-extract command line options: help
        And stdout should contain "--threads"
        And stdout should contain "--generate-edge-lookup"
        And stdout should contain "--small-component-size"
-        And it should exit with code 0
+        And it should exit successfully
@@ -5,8 +5,8 @@ Feature: osrm-extract command line options: invalid options
        Given the profile "testbot"

    Scenario: osrm-extract - Non-existing option
-        When I run "osrm-extract --fly-me-to-the-moon"
+        When I try to run "osrm-extract --fly-me-to-the-moon"
        Then stdout should be empty
        And stderr should contain "option"
        And stderr should contain "fly-me-to-the-moon"
-        And it should exit with code 1
+        And it should exit with an error
@@ -12,11 +12,11 @@ Feature: osrm-extract command line options: version
        Then stderr should be empty
        And stdout should contain 1 line
        And stdout should contain /(v\d{1,2}\.\d{1,2}\.\d{1,2}|\w*-\d+-\w+)/
-        And it should exit with code 0
+        And it should exit successfully

    Scenario: osrm-extract - Version, long
        When I run "osrm-extract --version"
        Then stderr should be empty
        And stdout should contain 1 line
        And stdout should contain /(v\d{1,2}\.\d{1,2}\.\d{1,2}|\w*-\d+-\w+)/
-        And it should exit with code 0
+        And it should exit successfully
@@ -29,4 +29,4 @@ Feature: osrm-routed command line options: files
        And stdout should contain /^\[info\] loaded plugin: viaroute/
        And stdout should contain /^\[info\] trial run/
        And stdout should contain /^\[info\] shutdown completed/
-        And it should exit with code 0
+        And it should exit successfully
@@ -21,7 +21,7 @@ Feature: osrm-routed command line options: help
        And stdout should contain "--max-trip-size"
        And stdout should contain "--max-table-size"
        And stdout should contain "--max-matching-size"
-        And it should exit with code 0
+        And it should exit successfully

    Scenario: osrm-routed - Help, short
        When I run "osrm-routed -h"
@@ -40,7 +40,7 @@ Feature: osrm-routed command line options: help
        And stdout should contain "--max-trip-size"
        And stdout should contain "--max-table-size"
        And stdout should contain "--max-matching-size"
-        And it should exit with code 0
+        And it should exit successfully

    Scenario: osrm-routed - Help, long
        When I run "osrm-routed --help"
@@ -59,4 +59,4 @@ Feature: osrm-routed command line options: help
        And stdout should contain "--max-table-size"
        And stdout should contain "--max-table-size"
        And stdout should contain "--max-matching-size"
-        And it should exit with code 0
+        And it should exit successfully
@@ -5,14 +5,14 @@ Feature: osrm-routed command line options: invalid options
        Given the profile "testbot"

    Scenario: osrm-routed - Non-existing option
-        When I run "osrm-routed --fly-me-to-the-moon"
+        When I try to run "osrm-routed --fly-me-to-the-moon"
        Then stdout should be empty
        And stderr should contain "unrecognised"
        And stderr should contain "fly-me-to-the-moon"
-        And it should exit with code 1
+        And it should exit with an error

    Scenario: osrm-routed - Missing file
-        When I run "osrm-routed over-the-rainbow.osrm"
+        When I try to run "osrm-routed over-the-rainbow.osrm"
        Then stderr should contain "over-the-rainbow.osrm"
        And stderr should contain "not found"
-        And it should exit with code 1
+        And it should exit with an error
@@ -12,11 +12,11 @@ Feature: osrm-routed command line options: version
        Then stderr should be empty
        And stdout should contain 1 line
        And stdout should contain /(v\d{1,2}\.\d{1,2}\.\d{1,2}|\w*-\d+-\w+)/
-        And it should exit with code 0
+        And it should exit successfully

    Scenario: osrm-routed - Version, long
        When I run "osrm-routed --version"
        Then stderr should be empty
        And stdout should contain 1 line
        And stdout should contain /(v\d{1,2}\.\d{1,2}\.\d{1,2}|\w*-\d+-\w+)/
-        And it should exit with code 0
+        And it should exit successfully
@@ -1,9 +1,5 @@
@raster @extract
 Feature: osrm-extract with a profile containing raster source
-# expansions:
-# {osm_base} => path to current input file
-# {profile} => path to current profile script
-
    Scenario: osrm-extract on a valid profile
        Given the profile "rasterbot"
        And the node map
@@ -11,8 +7,15 @@ Feature: osrm-extract with a profile containing raster source
        And the ways
            | nodes |
            | ab    |
+        And the raster source
+            """
+            0  0  0   0
+            0  0  0   250
+            0  0  250 500
+            0  0  0   250
+            0  0  0   0
+            """
        And the data has been saved to disk
-        When I run "osrm-extract {osm_base}.osm -p {profile}"
-        Then stderr should be empty
-        And stdout should contain "source loader"
-        And it should exit with code 0
+        When I run "osrm-extract {osm_file} -p {profile_file}"
+        Then stdout should contain "source loader"
+        And it should exit successfully
@@ -32,8 +32,8 @@ Feature: Raster - weights

    Scenario: Weighting not based on raster sources
        Given the profile "testbot"
-        When I run "osrm-extract {osm_base}.osm -p {profile}"
-        And I run "osrm-contract {osm_base}.osm"
+        When I run "osrm-extract {osm_file} -p {profile_file}"
+        And I run "osrm-contract {processed_file}"
        And I route I should get
            | from | to | route    | speed   |
            | a    | b  | ab,ab    | 36 km/h |
@@ -44,9 +44,9 @@ Feature: Raster - weights

    Scenario: Weighting based on raster sources
        Given the profile "rasterbot"
-        When I run "osrm-extract {osm_base}.osm -p {profile}"
+        When I run "osrm-extract {osm_file} -p {profile_file}"
        Then stdout should contain "evaluating segment"
-        And I run "osrm-contract {osm_base}.osm"
+        And I run "osrm-contract {processed_file}"
        And I route I should get
            | from | to | route    | speed   |
            | a    | b  | ab,ab    | 8 km/h  |
@@ -62,9 +62,9 @@ Feature: Raster - weights

    Scenario: Weighting based on raster sources
        Given the profile "rasterbotinterp"
-        When I run "osrm-extract {osm_base}.osm -p {profile}"
+        When I run "osrm-extract {osm_file} -p {profile_file}"
        Then stdout should contain "evaluating segment"
-        And I run "osrm-contract {osm_base}.osm"
+        And I run "osrm-contract {processed_file}"
        And I route I should get
            | from | to | route    | speed   |
            | a    | b  | ab,ab    | 8 km/h  |
@@ -2,19 +2,23 @@ var util = require('util');
 var path = require('path');
 var fs = require('fs');
 var d3 = require('d3-queue');
-var OSM = require('../support/build_osm');
+var OSM = require('../lib/osm');

 module.exports = function () {
    this.Given(/^the profile "([^"]*)"$/, (profile, callback) => {
-        this.setProfile(profile, callback);
+        this.profile = profile;
+        this.profileFile = path.join(this.PROFILES_PATH, this.profile + '.lua');
+        callback();
    });

    this.Given(/^the extract extra arguments "(.*?)"$/, (args, callback) => {
-        this.setExtractArgs(args, callback);
+        this.extractArgs = this.expandOptions(args);
+        callback();
    });

    this.Given(/^the contract extra arguments "(.*?)"$/, (args, callback) => {
-        this.setContractArgs(args, callback);
+        this.contractArgs = this.expandOptions(args);
+        callback();
    });

    this.Given(/^a grid size of ([0-9.]+) meters$/, (meters, callback) => {
@@ -228,58 +232,46 @@ module.exports = function () {
    });

    this.Given(/^the raster source$/, (data, callback) => {
-        this.updateFingerprintExtract(data);
-        fs.writeFile(path.resolve(this.TEST_FOLDER, 'rastersource.asc'), data, callback);
+        // TODO: Don't overwrite if it exists
+        fs.writeFile(this.rasterCacheFile, data, callback);
+        // we need this to pass it to the profiles
+        this.environment = Object.assign({OSRM_RASTER_SOURCE: this.rasterCacheFile}, this.environment);
    });

    this.Given(/^the speed file$/, (data, callback) => {
-        this.updateFingerprintContract(data);
-        fs.writeFile(path.resolve(this.TEST_FOLDER, 'speeds.csv'), data, callback);
+        // TODO: Don't overwrite if it exists
+        fs.writeFile(this.speedsCacheFile, data, callback);
    });

    this.Given(/^the turn penalty file$/, (data, callback) => {
-        this.updateFingerprintContract(data);
-        fs.writeFile(path.resolve(this.TEST_FOLDER, 'penalties.csv'), data, callback);
+        // TODO: Don't overwrite if it exists
+        fs.writeFile(this.penaltiesCacheFile, data, callback);
    });

    this.Given(/^the data has been saved to disk$/, (callback) => {
-        try {
-            this.reprocess(callback);
-        } catch(e) {
-            this.processError = e;
-            callback(e);
-        }
+        this.reprocess(callback);
    });

    this.Given(/^the data has been extracted$/, (callback) => {
-        this.osmData.populate(() => {
-            this.writeAndExtract((err) => {
-                if (err) this.processError = err;
-                callback();
-            });
-        });
+        this.reprocess(callback);
    });

    this.Given(/^the data has been contracted$/, (callback) => {
-        this.reprocess((err) => {
-            if (err) this.processError = err;
-            callback();
-        });
+        this.reprocess(callback);
    });

    this.Given(/^osrm\-routed is stopped$/, (callback) => {
-        this.OSRMLoader.shutdown((err) => {
-            if (err) this.processError = err;
-            callback();
-        });
+        this.OSRMLoader.shutdown(callback);
    });

-    this.Given(/^data is loaded directly/, () => {
-        this.loadMethod = 'directly';
+    this.Given(/^data is loaded directly/, (callback) => {
+        this.osrmLoader.setLoadMethod('directly');
+        callback();
    });

-    this.Given(/^data is loaded with datastore$/, () => {
-        this.loadMethod = 'datastore';
+    this.Given(/^data is loaded with datastore$/, (callback) => {
+        this.osrmLoader.setLoadMethod('datastore');
+        callback();
    });

    this.Given(/^the HTTP method "([^"]*)"$/, (method, callback) => {
@@ -53,8 +53,6 @@ module.exports = function () {
                });

                var testRow = (row, ri, cb) => {
-                    var ok = true;
-
                    for (var k in result[ri]) {
                        if (this.FuzzyMatch.match(result[ri][k], row[k])) {
                            result[ri][k] = row[k];
@@ -62,15 +60,9 @@ module.exports = function () {
                            result[ri][k] = '';
                        } else {
                            result[ri][k] = result[ri][k].toString();
-                            ok = false;
                        }
                    }

-                    if (!ok) {
-                        var failed = { attempt: 'distance_matrix', query: this.query, response: response };
-                        this.logFail(row, result[ri], [failed]);
-                    }
-
                    result[ri][''] = row[''];
                    cb(null, result[ri]);
                };
@@ -1,18 +0,0 @@
-var util = require('util');
-
-module.exports = function () {
-    this.Before((scenario, callback) => {
-        this.scenarioTitle = scenario.getName();
-
-        this.loadMethod = this.DEFAULT_LOAD_METHOD;
-        this.queryParams = {};
-        var d = new Date();
-        this.scenarioTime = util.format('%d-%d-%dT%s:%s:%sZ', d.getFullYear(), d.getMonth()+1, d.getDate(), d.getHours(), d.getMinutes(), d.getSeconds());
-        this.resetData();
-        this.hasLoggedPreprocessInfo = false;
-        this.hasLoggedScenarioInfo = false;
-        this.setGridSize(this.DEFAULT_GRID_SIZE);
-        this.setOrigin(this.DEFAULT_ORIGIN);
-        callback();
-    });
-};
@@ -157,7 +157,6 @@ module.exports = function () {
                        } else {
                            got.matchings = encodedResult;
                            row.matchings = extendedTarget;
-                            this.logFail(row, got, { matching: { query: this.query, response: res } });
                        }

                        cb(null, got);
@@ -22,24 +22,16 @@ module.exports = function () {

                        var got = { in: row.in, out: row.out };

-                        var ok = true;
-
                        Object.keys(row).forEach((key) => {
                            if (key === 'out') {
                                if (this.FuzzyMatch.matchLocation(coord, outNode)) {
                                    got[key] = row[key];
                                } else {
                                    row[key] = util.format('%s [%d,%d]', row[key], outNode.lat, outNode.lon);
-                                    ok = false;
                                }
                            }
                        });

-                        if (!ok) {
-                            var failed = { attempt: 'nearest', query: this.query, response: response };
-                            this.logFail(row, got, [failed]);
-                        }
-
                        cb(null, got);
                    }
                    else {
@@ -2,36 +2,58 @@ var assert = require('assert');
 var fs = require('fs');

 module.exports = function () {
-    this.When(/^I run "osrm\-routed\s?(.*?)"$/, { timeout: this.TIMEOUT }, (options, callback) => {
-        this.runBin('osrm-routed', options, () => {
-            callback();
+    this.resetOptionsOutput = () => {
+        this.stdout = null;
+        this.stderr = null;
+        this.exitCode = null;
+        this.termSignal = null;
+    };
+
+    this.runAndSafeOutput = (binary, options, callback) => {
+        this.runBin(binary, this.expandOptions(options), this.environment, (err, stdout, stderr) => {
+            this.stdout = stdout;
+            this.stderr = stderr;
+            this.exitCode = err && err.code || 0;
+            this.termSignal = err && err.signal || '';
+            callback(err);
        });
+    };
+
+    this.When(/^I run "osrm\-routed\s?(.*?)"$/, { timeout: this.TIMEOUT }, (options, callback) => {
+        this.runAndSafeOutput('osrm-routed', options, callback);
    });

    this.When(/^I run "osrm\-extract\s?(.*?)"$/, (options, callback) => {
-        this.runBin('osrm-extract', options, () => {
-            callback();
-        });
+        this.runAndSafeOutput('osrm-extract', options, callback);
    });

    this.When(/^I run "osrm\-contract\s?(.*?)"$/, (options, callback) => {
-        this.runBin('osrm-contract', options, () => {
-            callback();
-        });
+        this.runAndSafeOutput('osrm-contract', options, callback);
+    });
+
+    this.When(/^I try to run "osrm\-routed\s?(.*?)"$/, (options, callback) => {
+        this.runAndSafeOutput('osrm-routed', options, () => { callback(); });
+    });
+
+    this.When(/^I try to run "osrm\-extract\s?(.*?)"$/, (options, callback) => {
+        this.runAndSafeOutput('osrm-extract', options, () => { callback(); });
+    });
+
+    this.When(/^I try to run "osrm\-contract\s?(.*?)"$/, (options, callback) => {
+        this.runAndSafeOutput('osrm-contract', options, () => { callback(); });
    });

    this.When(/^I run "osrm\-datastore\s?(.*?)"$/, (options, callback) => {
-        this.runBin('osrm-datastore', options, () => {
-            callback();
-        });
+        this.runAndSafeOutput('osrm-datastore', options, callback);
    });

-    this.Then(/^it should exit with code (\d+)$/, (code) => {
-        assert.equal(this.exitCode, parseInt(code));
+    this.Then(/^it should exit successfully$/, () => {
+        assert.equal(this.exitCode, 0);
+        assert.equal(this.termSignal, '');
    });

-    this.Then(/^it should exit with code not (\d+)$/, (code) => {
-        assert.notEqual(this.exitCode, parseInt(code));
+    this.Then(/^it should exit with an error$/, () => {
+        assert.ok(this.exitCode !== 0 || this.termSignal);
    });

    this.Then(/^stdout should contain "(.*?)"$/, (str) => {
@@ -65,7 +87,7 @@ module.exports = function () {
    });

    this.Then(/^datasource names should contain "(.+)"$/, (expectedData) => {
-        var actualData = fs.readFileSync(this.osmData.extractedFile + '.osrm.datasource_names', {encoding:'UTF-8'}).trim().split('\n').join(',');
+        var actualData = fs.readFileSync(this.processedCacheFile + '.datasource_names', {encoding:'UTF-8'}).trim().split('\n').join(',');
        assert.equal(actualData, expectedData);
    });

@@ -51,7 +51,7 @@ module.exports = function () {
    });

    this.Then(/^"([^"]*)" should return code (\d+)$/, (binary, code) => {
-        assert.ok(this.processError instanceof this.OSRMError);
+        assert.ok(this.processError instanceof Error);
        assert.equal(this.processError.process, binary);
        assert.equal(parseInt(this.processError.code), parseInt(code));
    });
@@ -13,7 +13,7 @@ module.exports = function () {
            }

            this.reprocessAndLoadData((e) => {
-                if (e) callback(e);
+                if (e) return callback(e);
                var testRow = (row, i, cb) => {
                    var outputRow = row;

@@ -41,10 +41,6 @@ module.exports = function () {
                            }
                        });

-                        if (outputRow != row) {
-                            this.logFail(row, outputRow, result);
-                        }
-
                        cb(null, outputRow);
                    });
                };
@@ -116,7 +112,7 @@ module.exports = function () {
                    sq.defer(parseRes, key);
                });

-                sq.awaitAll(() => { cb(null, result); });
+                sq.awaitAll((err) => { cb(err, result); });
            });
    };
 };
@@ -85,23 +85,14 @@ module.exports = function () {
                    } else {
                        got.trips = encodedResult;
                        got.trips = extendedTarget;
-                        this.logFail(row, got, { trip: { query: this.query, response: res }});
                    }

-                    ok = true;
-
                    for (var key in row) {
                        if (this.FuzzyMatch.match(got[key], row[key])) {
                            got[key] = row[key];
-                        } else {
-                            ok = false;
                        }
                    }

-                    if (!ok) {
-                        this.logFail(row, got, { trip: { query: this.query, response: res }});
-                    }
-
                    cb(null, got);
                };

@@ -0,0 +1,184 @@
+'use strict';
+
+const d3 = require('d3-queue');
+const fs = require('fs');
+const util = require('util');
+const path = require('path');
+const mkdirp = require('mkdirp');
+const hash = require('../lib/hash');
+const rimraf = require('rimraf');
+
+module.exports = function() {
+    this.initializeCache = (callback) => {
+        this.getOSRMHash((err, osrmHash) => {
+            if (err) return callback(err);
+            this.osrmHash = osrmHash;
+            callback();
+        });
+    };
+
+    // computes all paths for every feature
+    this.setupFeatures = (features, callback) => {
+        this.featureIDs = {};
+        this.featureCacheDirectories = {};
+        this.featureProcessedCacheDirectories = {};
+        let queue = d3.queue();
+
+        function initializeFeature(feature, callback) {
+            let uri = feature.getUri();
+
+            // setup cache for feature data
+            hash.hashOfFile(uri, (err, hash) => {
+                if (err) return callback(err);
+
+                // shorten uri to be realtive to 'features/'
+                let featurePath = path.relative(path.resolve('./features'), uri);
+                // bicycle/bollards/{HASH}/
+                let featureID = path.join(featurePath, hash);
+                let featureCacheDirectory = this.getFeatureCacheDirectory(featureID);
+                let featureProcessedCacheDirectory = this.getFeatureProcessedCacheDirectory(featureCacheDirectory, this.osrmHash);
+                this.featureIDs[uri] = featureID;
+                this.featureCacheDirectories[uri] = featureCacheDirectory;
+                this.featureProcessedCacheDirectories[uri] = featureProcessedCacheDirectory;
+
+                d3.queue(1)
+                  .defer(mkdirp, featureProcessedCacheDirectory)
+                  .defer(this.cleanupFeatureCache.bind(this), featureCacheDirectory, hash)
+                  .defer(this.cleanupProcessedFeatureCache.bind(this), featureProcessedCacheDirectory, this.osrmHash)
+                  .awaitAll(callback);
+            });
+        }
+
+        for (let i = 0; i < features.length; ++i) {
+            queue.defer(initializeFeature.bind(this), features[i]);
+        }
+        queue.awaitAll(callback);
+    };
+
+    this.cleanupProcessedFeatureCache = (directory, osrmHash, callback) => {
+        let parentPath = path.resolve(path.join(directory, '..'));
+        fs.readdir(parentPath, (err, files) => {
+            let q = d3.queue();
+            function runStats(path, callback) {
+                fs.stat(path, (err, stat) => {
+                    if (err) return callback(err);
+                    callback(null, {file: path, stat: stat});
+                });
+            }
+            files.map(f => { q.defer(runStats, path.join(parentPath, f)); });
+            q.awaitAll((err, results) => {
+                if (err) return callback(err);
+                let q = d3.queue();
+                results.forEach(r => {
+                    if (r.stat.isDirectory() && r.file.search(osrmHash) < 0) {
+                        q.defer(rimraf, r.file);
+                    }
+                });
+                q.awaitAll(callback);
+            });
+        });
+    };
+
+    this.cleanupFeatureCache = (directory, featureHash, callback) => {
+        let parentPath = path.resolve(path.join(directory, '..'));
+        fs.readdir(parentPath, (err, files) => {
+            let q = d3.queue();
+            files.filter(name => { return name !== featureHash;})
+                 .map((f) => { q.defer(rimraf, path.join(parentPath, f)); });
+            q.awaitAll(callback);
+        });
+    };
+
+    this.setupFeatureCache = (feature) => {
+        let uri = feature.getUri();
+        this.featureID = this.featureIDs[uri];
+        this.featureCacheDirectory = this.featureCacheDirectories[uri];
+        this.featureProcessedCacheDirectory = this.featureProcessedCacheDirectories[uri];
+    };
+
+    this.setupScenarioCache = (scenarioID) => {
+        this.scenarioCacheFile = this.getScenarioCacheFile(this.featureCacheDirectory, scenarioID);
+        this.processedCacheFile = this.getProcessedCacheFile(this.featureProcessedCacheDirectory, scenarioID);
+        this.inputCacheFile = this.getInputCacheFile(this.featureProcessedCacheDirectory, scenarioID);
+        this.rasterCacheFile = this.getRasterCacheFile(this.featureProcessedCacheDirectory, scenarioID);
+        this.speedsCacheFile = this.getSpeedsCacheFile(this.featureProcessedCacheDirectory, scenarioID);
+        this.penaltiesCacheFile = this.getPenaltiesCacheFile(this.featureProcessedCacheDirectory, scenarioID);
+    };
+
+    // returns a hash of all OSRM code side dependencies
+    this.getOSRMHash = (callback) => {
+        let dependencies = [
+            this.OSRM_EXTRACT_PATH,
+            this.OSRM_CONTRACT_PATH,
+            this.LIB_OSRM_EXTRACT_PATH,
+            this.LIB_OSRM_CONTRACT_PATH
+        ];
+
+        var addLuaFiles = (directory, callback) => {
+            fs.readdir(path.normalize(directory), (err, files) => {
+                if (err) return callback(err);
+
+                var luaFiles = files.filter(f => !!f.match(/\.lua$/)).map(f => path.normalize(directory + '/' + f));
+                Array.prototype.push.apply(dependencies, luaFiles);
+
+                callback();
+            });
+        };
+
+        // Note: we need a serialized queue here to ensure that the order of the files
+        // passed is stable. Otherwise the hash will not be stable
+        d3.queue(1)
+            .defer(addLuaFiles, this.PROFILES_PATH)
+            .defer(addLuaFiles, this.PROFILES_PATH + '/lib')
+            .awaitAll(hash.hashOfFiles.bind(hash, dependencies, callback));
+    };
+
+    // test/cache/bicycle/bollards/{HASH}/
+    this.getFeatureCacheDirectory = (featureID) => {
+        return path.join(this.CACHE_PATH, featureID);
+    };
+
+    // converts the scenario titles in file prefixes
+    this.getScenarioID = (scenario) => {
+        let name = scenario.getName().toLowerCase().replace(/[\/\-'=,\(\)]/g, '').replace(/\s/g, '_').replace(/__/g, '_').replace(/\.\./g, '.');
+        return util.format('%d_%s', scenario.getLine(), name);
+    };
+
+    // test/cache/{feature_path}/{feature_hash}/{scenario}_raster.asc
+    this.getRasterCacheFile = (featureCacheDirectory, scenarioID) => {
+        return path.join(featureCacheDirectory, scenarioID) + '_raster.asc';
+    };
+
+    // test/cache/{feature_path}/{feature_hash}/{scenario}_speeds.csv
+    this.getSpeedsCacheFile = (featureCacheDirectory, scenarioID) => {
+        return path.join(featureCacheDirectory, scenarioID) + '_speeds.csv';
+    };
+
+    // test/cache/{feature_path}/{feature_hash}/{scenario}_penalties.csv
+    this.getPenaltiesCacheFile = (featureCacheDirectory, scenarioID) => {
+        return path.join(featureCacheDirectory, scenarioID) + '_penalties.csv';
+    };
+
+    // test/cache/{feature_path}/{feature_hash}/{scenario}.osm
+    this.getScenarioCacheFile = (featureCacheDirectory, scenarioID) => {
+        return path.join(featureCacheDirectory, scenarioID) + '.osm';
+    };
+
+    // test/cache/{feature_path}/{feature_hash}/{osrm_hash}/
+    this.getFeatureProcessedCacheDirectory = (featureCacheDirectory, osrmHash) => {
+        return path.join(featureCacheDirectory, osrmHash);
+    };
+
+    // test/cache/{feature_path}/{feature_hash}/{osrm_hash}/{scenario}.osrm
+    this.getProcessedCacheFile = (featureProcessedCacheDirectory, scenarioID) => {
+        return path.join(featureProcessedCacheDirectory, scenarioID) + '.osrm';
+    };
+
+    // test/cache/{feature_path}/{feature_hash}/{osrm_hash}/{scenario}.osm
+    this.getInputCacheFile = (featureProcessedCacheDirectory, scenarioID) => {
+        return path.join(featureProcessedCacheDirectory, scenarioID) + '.osm';
+    };
+
+
+    return this;
+};
@@ -1,127 +0,0 @@
-var fs = require('fs');
-var path = require('path');
-var util = require('util');
-var d3 = require('d3-queue');
-var OSM = require('./build_osm');
-var classes = require('./data_classes');
-
-module.exports = function () {
-    this.initializeOptions = (callback) => {
-        this.profile = this.profile || this.DEFAULT_SPEEDPROFILE;
-
-        this.OSMDB = this.OSMDB || new OSM.DB();
-
-        this.nameNodeHash = this.nameNodeHash || {};
-
-        this.locationHash = this.locationHash || {};
-
-        this.nameWayHash = this.nameWayHash || {};
-
-        this.osmData = new classes.osmData(this);
-
-        this.OSRMLoader = this._OSRMLoader();
-
-        this.PREPROCESS_LOG_FILE = path.resolve(this.TEST_FOLDER, 'preprocessing.log');
-
-        this.LOG_FILE = path.resolve(this.TEST_FOLDER, 'fail.log');
-
-        this.HOST = 'http://127.0.0.1:' + this.OSRM_PORT;
-
-        this.DESTINATION_REACHED = 15;  // OSRM instruction code
-
-        this.shortcutsHash = this.shortcutsHash || {};
-
-        var hashLuaLib = (cb) => {
-            fs.readdir(path.normalize(this.PROFILES_PATH + '/lib/'), (err, files) => {
-                if (err) cb(err);
-                var luaFiles = files.filter(f => !!f.match(/\.lua$/)).map(f => path.normalize(this.PROFILES_PATH + '/lib/' + f));
-                this.hashOfFiles(luaFiles, hash => {
-                    this.luaLibHash = hash;
-                    cb();
-                });
-            });
-        };
-
-        var hashProfile = (cb) => {
-            this.hashProfile((hash) => {
-                this.profileHash = hash;
-                cb();
-            });
-        };
-
-        var hashExtract = (cb) => {
-            var files = [ util.format('%s/osrm-extract%s', this.BIN_PATH, this.EXE),
-                          util.format('%s/libosrm_extract%s', this.BIN_PATH, this.LIB) ];
-            this.hashOfFiles(files, (hash) => {
-                this.binExtractHash = hash;
-                cb();
-            });
-        };
-
-        var hashContract = (cb) => {
-            var files = [ util.format('%s/osrm-contract%s', this.BIN_PATH, this.EXE),
-                          util.format('%s/libosrm_contract%s', this.BIN_PATH, this.LIB) ];
-            this.hashOfFiles(files, (hash) => {
-                this.binContractHash = hash;
-                cb();
-            });
-        };
-
-        var hashRouted = (cb) => {
-            var files = [ util.format('%s/osrm-routed%s', this.BIN_PATH, this.EXE),
-                          util.format('%s/libosrm%s', this.BIN_PATH, this.LIB) ];
-            this.hashOfFiles(files, (hash) => {
-                this.binRoutedHash = hash;
-                this.fingerprintRoute = this.hashString(this.binRoutedHash);
-                cb();
-            });
-        };
-
-        d3.queue()
-            .defer(hashLuaLib)
-            .defer(hashProfile)
-            .defer(hashExtract)
-            .defer(hashContract)
-            .defer(hashRouted)
-            .awaitAll(() => {
-                this.AfterConfiguration(() => {
-                    callback();
-                });
-            });
-    };
-
-    this.updateFingerprintExtract = (str) => {
-        this.fingerprintExtract = this.hashString([this.fingerprintExtract, str].join('-'));
-    };
-
-    this.updateFingerprintContract = (str) => {
-        this.fingerprintContract = this.hashString([this.fingerprintContract, str].join('-'));
-    };
-
-    this.setProfile = (profile, cb) => {
-        var lastProfile = this.profile;
-        if (profile !== lastProfile) {
-            this.profile = profile;
-            this.hashProfile((hash) => {
-                this.profileHash = hash;
-                this.updateFingerprintExtract(this.profileHash);
-                cb();
-            });
-        } else {
-            this.updateFingerprintExtract(this.profileHash);
-            cb();
-        }
-    };
-
-    this.setExtractArgs = (args, callback) => {
-        this.extractArgs = args;
-        this.updateFingerprintExtract(args);
-        callback();
-    };
-
-    this.setContractArgs = (args, callback) => {
-        this.contractArgs = args;
-        this.updateFingerprintContract(args);
-        callback();
-    };
-};
@@ -1,11 +1,14 @@
-var fs = require('fs');
-var path = require('path');
-var util = require('util');
-var exec = require('child_process').exec;
-var d3 = require('d3-queue');
+'use strict';

-var OSM = require('./build_osm');
-var classes = require('./data_classes');
+const fs = require('fs');
+const util = require('util');
+const d3 = require('d3-queue');
+
+const OSM = require('../lib/osm');
+const classes = require('./data_classes');
+const tableDiff = require('../lib/table_diff');
+const ensureDecimal = require('../lib/utils').ensureDecimal;
+const errorReason = require('../lib/utils').errorReason;

 module.exports = function () {
    this.setGridSize = (meters) => {
@@ -94,13 +97,8 @@ module.exports = function () {
        q.awaitAll(callback);
    };

-    this.ensureDecimal = (i) => {
-        if (parseInt(i) === i) return i.toFixed(1);
-        else return i;
-    };
-
    this.tableCoordToLonLat = (ci, ri) => {
-        return [this.origin[0] + ci * this.zoom, this.origin[1] - ri * this.zoom].map(this.ensureDecimal);
+        return [this.origin[0] + ci * this.zoom, this.origin[1] - ri * this.zoom].map(ensureDecimal);
    };

    this.addOSMNode = (name, lon, lat, id) => {
@@ -132,10 +130,6 @@ module.exports = function () {
        return this.nameWayHash[s.toString()] || this.nameWayHash[s.toString().split('').reverse().join('')];
    };

-    this.resetData = () => {
-        this.resetOSM();
-    };
-
    this.makeOSMId = () => {
        this.osmID = this.osmID + 1;
        return this.osmID;
@@ -143,206 +137,88 @@ module.exports = function () {

    this.resetOSM = () => {
        this.OSMDB.clear();
-        this.osmData.reset();
        this.nameNodeHash = {};
        this.locationHash = {};
+        this.shortcutsHash = {};
        this.nameWayHash = {};
        this.osmID = 0;
    };

    this.writeOSM = (callback) => {
-        fs.exists(this.DATA_FOLDER, (exists) => {
-            var mkDirFn = exists ? (cb) => { cb(); } : fs.mkdir.bind(fs.mkdir, this.DATA_FOLDER);
-            mkDirFn((err) => {
-                if (err) return callback(err);
-                var osmPath = path.resolve(this.DATA_FOLDER, util.format('%s.osm', this.osmData.osmFile));
-                fs.exists(osmPath, (exists) => {
-                    if (!exists) fs.writeFile(osmPath, this.osmData.str, callback);
-                    else callback();
+        fs.exists(this.scenarioCacheFile, (exists) => {
+            if (exists) callback();
+            else {
+                this.OSMDB.toXML((xml) => {
+                    fs.writeFile(this.scenarioCacheFile, xml, callback);
                });
-            });
-        });
-    };
-
-    this.isExtracted = (callback) => {
-        fs.exists(util.format('%s.osrm', this.osmData.extractedFile), (core) => {
-            if (!core) return callback(false);
-            fs.exists(util.format('%s.osrm.names', this.osmData.extractedFile), (names) => {
-                if (!names) return callback(false);
-                fs.exists(util.format('%s.osrm.restrictions', this.osmData.extractedFile), (restrictions) => {
-                    return callback(restrictions);
-                });
-            });
-        });
-    };
-
-    this.isContracted = (callback) => {
-        fs.exists(util.format('%s.osrm.hsgr', this.osmData.contractedFile), callback);
-    };
-
-    this.writeTimestamp = (callback) => {
-        fs.writeFile(util.format('%s.osrm.timestamp', this.osmData.contractedFile), this.OSM_TIMESTAMP, callback);
-    };
-
-    this.writeInputData = (callback) => {
-        this.writeOSM((err) => {
-            if (err) return callback(err);
-            this.writeTimestamp(callback);
-        });
-    };
-
-    this.extractData = (callback) => {
-        this.logPreprocessInfo();
-        this.log(util.format('== Extracting %s.osm...', this.osmData.osmFile), 'preprocess');
-        var cmd = util.format('%s/osrm-extract %s.osm %s --profile %s/%s.lua >>%s 2>&1',
-            this.BIN_PATH, this.osmData.osmFile, this.extractArgs || '', this.PROFILES_PATH, this.profile, this.PREPROCESS_LOG_FILE);
-        this.log(cmd);
-        process.chdir(this.TEST_FOLDER);
-        exec(cmd, (err) => {
-            if (err) {
-                this.log(util.format('*** Exited with code %d', err.code), 'preprocess');
-                process.chdir('../');
-                return callback(this.ExtractError(err.code, util.format('osrm-extract exited with code %d', err.code)));
            }
-
-            var q = d3.queue();
-
-            var rename = (file, cb) => {
-                this.log(util.format('Renaming %s.%s to %s.%s', this.osmData.osmFile, file, this.osmData.extractedFile, file), 'preprocess');
-                fs.rename([this.osmData.osmFile, file].join('.'), [this.osmData.extractedFile, file].join('.'), (err) => {
-                    if (err) return cb(this.FileError(null, 'failed to rename data file after extracting'));
-                    cb();
-                });
-            };
-
-            var renameIfExists = (file, cb) => {
-                fs.stat([this.osmData.osmFile, file].join('.'), (doesNotExistErr, exists) => {
-                    if (exists) rename(file, cb);
-                    else cb();
-                });
-            };
-
-            ['osrm', 'osrm.ebg', 'osrm.edges', 'osrm.enw', 'osrm.fileIndex', 'osrm.geometry', 'osrm.icd',
-             'osrm.names', 'osrm.nodes', 'osrm.properties', 'osrm.ramIndex', 'osrm.restrictions', 'osrm.tld', 'osrm.tls'].forEach(file => {
-                 q.defer(rename, file);
-             });
-
-            ['osrm.edge_penalties', 'osrm.edge_segment_lookup'].forEach(file => {
-                q.defer(renameIfExists, file);
-            });
-
-            q.awaitAll((err) => {
-                this.log('Finished extracting ' + this.osmData.extractedFile, 'preprocess');
-                process.chdir('../');
-                callback(err);
-            });
        });
    };

-    this.contractData = (callback) => {
-        this.logPreprocessInfo();
-        this.log(util.format('== Contracting %s.osm...', this.osmData.extractedFile), 'preprocess');
-        var cmd = util.format('%s/osrm-contract %s %s.osrm >>%s 2>&1',
-            this.BIN_PATH, this.contractArgs || '', this.osmData.extractedFile, this.PREPROCESS_LOG_FILE);
-        this.log(cmd);
-        process.chdir(this.TEST_FOLDER);
-        exec(cmd, (err) => {
-            if (err) {
-                this.log(util.format('*** Exited with code %d', err.code), 'preprocess');
-                process.chdir('../');
-                return callback(this.ContractError(err.code, util.format('osrm-contract exited with code %d', err.code)));
+    this.linkOSM = (callback) => {
+        fs.exists(this.inputCacheFile, (exists) => {
+            if (exists) callback();
+            else {
+                fs.link(this.scenarioCacheFile, this.inputCacheFile, callback);
            }
+        });
+    };

-            var rename = (file, cb) => {
-                this.log(util.format('Renaming %s.%s to %s.%s', this.osmData.extractedFile, file, this.osmData.contractedFile, file), 'preprocess');
-                fs.rename([this.osmData.extractedFile, file].join('.'), [this.osmData.contractedFile, file].join('.'), (err) => {
-                    if (err) return cb(this.FileError(null, 'failed to rename data file after contracting.'));
-                    cb();
-                });
-            };
+    this.extractData = (p, callback) => {
+        let stamp = p.processedCacheFile + '.extract';
+        fs.exists(stamp, (exists) => {
+            if (exists) return callback();

-            var renameIfExists = (file, cb) => {
-                fs.stat([this.osmData.extractedFile, file].join('.'), (doesNotExistErr, exists) => {
-                    if (exists) rename(file, cb);
-                    else cb();
-                });
-            };
-
-            var copy = (file, cb) => {
-                this.log(util.format('Copying %s.%s to %s.%s', this.osmData.extractedFile, file, this.osmData.contractedFile, file), 'preprocess');
-                fs.createReadStream([this.osmData.extractedFile, file].join('.'))
-                    .pipe(fs.createWriteStream([this.osmData.contractedFile, file].join('.'))
-                            .on('finish', cb)
-                        )
-                    .on('error', () => {
-                        return cb(this.FileError(null, 'failed to copy data after contracting.'));
-                    });
-            };
-
-            var q = d3.queue();
-
-            ['osrm', 'osrm.core', 'osrm.datasource_indexes', 'osrm.datasource_names', 'osrm.ebg','osrm.edges',
-             'osrm.enw', 'osrm.fileIndex', 'osrm.geometry', 'osrm.hsgr', 'osrm.icd','osrm.level', 'osrm.names',
-             'osrm.nodes', 'osrm.properties', 'osrm.ramIndex', 'osrm.restrictions', 'osrm.tld', 'osrm.tls'].forEach((file) => {
-                 q.defer(rename, file);
-             });
-
-            ['osrm.edge_penalties', 'osrm.edge_segment_lookup'].forEach(file => {
-                q.defer(renameIfExists, file);
-            });
-
-            [].forEach((file) => {
-                q.defer(copy, file);
-            });
-
-            q.awaitAll((err) => {
-                this.log('Finished contracting ' + this.osmData.contractedFile, 'preprocess');
-                process.chdir('../');
-                callback(err);
+            this.runBin('osrm-extract', util.format('%s --profile %s %s', p.extractArgs, p.profileFile, p.inputCacheFile), p.environment, (err) => {
+                if (err) {
+                    return callback(new Error(util.format('osrm-extract %s: %s', errorReason(err), err.cmd)));
+                }
+                fs.writeFile(stamp, 'ok', callback);
            });
        });
    };

-    var noop = (cb) => cb();
+    this.contractData = (p, callback) => {
+        let stamp = p.processedCacheFile + '.contract';
+        fs.exists(stamp, (exists) => {
+            if (exists) return callback();
+
+            this.runBin('osrm-contract', util.format('%s %s', p.contractArgs, p.processedCacheFile), p.environment, (err) => {
+                if (err) {
+                    return callback(new Error(util.format('osrm-contract %s: %s', errorReason(err), err)));
+                }
+                fs.writeFile(stamp, 'ok', callback);
+            });
+        });
+    };
+
+    this.extractAndContract = (callback) => {
+        // a shallow copy of scenario parameters to avoid data inconsistency
+        // if a cucumber timeout occurs during deferred jobs
+        let p = {extractArgs: this.extractArgs, contractArgs: this.contractArgs,
+                 profileFile: this.profileFile, inputCacheFile: this.inputCacheFile,
+                 processedCacheFile: this.processedCacheFile, environment: this.environment};
+        let queue = d3.queue(1);
+        queue.defer(this.extractData.bind(this), p);
+        queue.defer(this.contractData.bind(this), p);
+        queue.awaitAll(callback);
+    };

    this.reprocess = (callback) => {
-        this.osmData.populate(() => {
-            this.isContracted((isContracted) => {
-                if (!isContracted) {
-                    this.writeAndExtract((e) => {
-                        if (e) return callback(e);
-                        this.contractData((e) => {
-                            if (e) return callback(e);
-                            this.logPreprocessDone();
-                            callback();
-                        });
-                    });
-                } else {
-                    this.log('Already contracted ' + this.osmData.contractedFile, 'preprocess');
-                    callback();
-                }
-            });
-        });
-    };
-
-    this.writeAndExtract = (callback) => {
-        this.writeInputData((e) => {
-            if (e) return callback(e);
-            this.isExtracted((isExtracted) => {
-                var extractFn = isExtracted ? noop : this.extractData;
-                if (isExtracted) this.log('Already extracted ' + this.osmData.extractedFile, 'preprocess');
-                extractFn((e) => {
-                    callback(e);
-                });
-            });
-        });
+        let queue = d3.queue(1);
+        queue.defer(this.writeOSM.bind(this));
+        queue.defer(this.linkOSM.bind(this));
+        queue.defer(this.extractAndContract.bind(this));
+        queue.awaitAll(callback);
    };

    this.reprocessAndLoadData = (callback) => {
-        this.reprocess((e) => {
-            if (e) return callback(e);
-            this.OSRMLoader.load(util.format('%s.osrm', this.osmData.contractedFile), callback);
-        });
+        let queue = d3.queue(1);
+        queue.defer(this.writeOSM.bind(this));
+        queue.defer(this.linkOSM.bind(this));
+        queue.defer(this.extractAndContract.bind(this));
+        queue.defer(this.osrmLoader.load.bind(this.osrmLoader), this.processedCacheFile);
+        queue.awaitAll(callback);
    };

    this.processRowsAndDiff = (table, fn, callback) => {
@@ -352,7 +228,9 @@ module.exports = function () {

        q.awaitAll((err, actual) => {
            if (err) return callback(err);
-            this.diffTables(table, actual, {}, callback);
+            let diff = tableDiff(table, actual);
+            if (diff) callback(new Error(diff));
+            else callback();
        });
    };
 };
@@ -1,7 +1,6 @@
 'use strict';

-var util = require('util');
-var path = require('path');
+const util = require('util');

 module.exports = {
    Location: class {
@@ -11,43 +10,6 @@ module.exports = {
        }
    },

-    osmData: class {
-        constructor (scope) {
-            this.scope          = scope;
-            this.str            = null;
-            this.hash           = null;
-            this.fingerprintOSM = null;
-            this.osmFile        = null;
-            this.extractedFile  = null;
-            this.contractedFile   = null;
-        }
-
-        populate (callback) {
-            this.scope.OSMDB.toXML((str) => {
-                this.str = str;
-
-                this.hash = this.scope.hashString(str);
-                this.fingerprintOSM = this.scope.hashString(this.hash);
-
-                this.osmFile        = path.resolve(this.scope.DATA_FOLDER, this.fingerprintOSM);
-
-                this.extractedFile  = path.resolve([this.osmFile, this.scope.fingerprintExtract].join('_'));
-                this.contractedFile   = path.resolve([this.osmFile, this.scope.fingerprintExtract, this.scope.fingerprintContract].join('_'));
-
-                callback();
-            });
-        }
-
-        reset () {
-            this.str            = null;
-            this.hash           = null;
-            this.fingerprintOSM = null;
-            this.osmFile        = null;
-            this.extractedFile  = null;
-            this.contractedFile   = null;
-        }
-    },
-
    FuzzyMatch: class {
        match (got, want) {
            var matchPercent = want.match(/(.*)\s+~(.+)%$/),
@@ -1,32 +1,46 @@
-var path = require('path');
-var util = require('util');
-var fs = require('fs');
-var exec = require('child_process').exec;
-var d3 = require('d3-queue');
+'use strict';

+const path = require('path');
+const util = require('util');
+const fs = require('fs');
+const d3 = require('d3-queue');
+const child_process = require('child_process');
+const tryConnect = require('../lib/try_connect');
+
+// Sets up all constants that are valid for all features
 module.exports = function () {
    this.initializeEnv = (callback) => {
-        this.OSRM_PORT = process.env.OSRM_PORT && parseInt(process.env.OSRM_PORT) || 5000;
        this.TIMEOUT = process.env.CUCUMBER_TIMEOUT && parseInt(process.env.CUCUMBER_TIMEOUT) || 5000;
+        // set cucumber default timeout
        this.setDefaultTimeout(this.TIMEOUT);
-        this.ROOT_FOLDER = process.cwd();
+        this.ROOT_PATH = process.cwd();
+
+        this.TEST_PATH = path.resolve(this.ROOT_PATH, 'test');
+        this.CACHE_PATH = path.resolve(this.TEST_PATH, 'cache');
+        this.LOGS_PATH = path.resolve(this.TEST_PATH, 'logs');
+
+        this.PROFILES_PATH = path.resolve(this.ROOT_PATH, 'profiles');
+        this.FIXTURES_PATH = path.resolve(this.ROOT_PATH, 'unit_tests/fixtures');
+        this.BIN_PATH = process.env.OSRM_BUILD_DIR && process.env.OSRM_BUILD_DIR || path.resolve(this.ROOT_PATH, 'build');
+        var stxxl_config = path.resolve(this.ROOT_PATH, 'test/.stxxl');
+        if (!fs.existsSync(stxxl_config)) {
+            return callback(new Error('*** '+stxxl_config+ 'does not exist'));
+        }
+
+        this.DEFAULT_ENVIRONMENT = Object.assign({STXXLCFG: stxxl_config}, process.env);
+        this.DEFAULT_PROFILE = 'bicycle';
+        this.DEFAULT_INPUT_FORMAT = 'osm';
+        this.DEFAULT_LOAD_METHOD = 'datastore';
+        this.DEFAULT_ORIGIN = [1,1];
        this.OSM_USER = 'osrm';
        this.OSM_GENERATOR = 'osrm-test';
        this.OSM_UID = 1;
-        this.TEST_FOLDER = path.resolve(this.ROOT_FOLDER, 'test');
-        this.DATA_FOLDER = path.resolve(this.TEST_FOLDER, 'cache');
        this.OSM_TIMESTAMP = '2000-01-01T00:00:00Z';
-        this.DEFAULT_SPEEDPROFILE = 'bicycle';
        this.WAY_SPACING = 100;
-        this.DEFAULT_GRID_SIZE = 100;    // meters
-        this.PROFILES_PATH = path.resolve(this.ROOT_FOLDER, 'profiles');
-        this.FIXTURES_PATH = path.resolve(this.ROOT_FOLDER, 'unit_tests/fixtures');
-        this.BIN_PATH = process.env.OSRM_BUILD_DIR && process.env.OSRM_BUILD_DIR || path.resolve(this.ROOT_FOLDER, 'build');
-        this.DEFAULT_INPUT_FORMAT = 'osm';
-        this.DEFAULT_ORIGIN = [1,1];
-        this.DEFAULT_LOAD_METHOD = 'datastore';
-        this.OSRM_ROUTED_LOG_FILE = path.resolve(this.TEST_FOLDER, 'osrm-routed.log');
-        this.ERROR_LOG_FILE = path.resolve(this.TEST_FOLDER, 'error.log');
+        this.DEFAULT_GRID_SIZE = 100; // meters
+
+        this.OSRM_PORT = process.env.OSRM_PORT && parseInt(process.env.OSRM_PORT) || 5000;
+        this.HOST = 'http://127.0.0.1:' + this.OSRM_PORT;

        // TODO make sure this works on win
        if (process.platform.match(/indows.*/)) {
@@ -37,36 +51,67 @@ module.exports = function () {
        } else {
            this.TERMSIGNAL = 'SIGTERM';
            this.EXE = '';
-            this.LIB = '.so';
+
+            // heuristically detect .so/.a suffix
+            this.LIB = null;
+
+            try {
+                const dot_a = util.format('%s/libosrm%s', this.BIN_PATH, '.a');
+                fs.accessSync(dot_a, fs.F_OK);
+                this.LIB = '.a';
+            } catch(e) { /*nop*/ }
+
+            try {
+                const dot_so = util.format('%s/libosrm%s', this.BIN_PATH, '.so');
+                fs.accessSync(dot_so, fs.F_OK);
+                this.LIB = '.so';
+            } catch(e) { /*nop*/ }
+
+            if (!this.LIB) {
+                throw new Error('*** Unable to detect dynamic or static libosrm libraries');
+            }
+
            this.QQ = '';
        }

+        this.OSRM_EXTRACT_PATH = path.resolve(util.format('%s/%s%s', this.BIN_PATH, 'osrm-extract', this.EXE));
+        this.OSRM_CONTRACT_PATH = path.resolve(util.format('%s/%s%s', this.BIN_PATH, 'osrm-contract', this.EXE));
+        this.OSRM_ROUTED_PATH = path.resolve(util.format('%s/%s%s', this.BIN_PATH, 'osrm-routed', this.EXE));
+        this.LIB_OSRM_EXTRACT_PATH = util.format('%s/libosrm_extract%s', this.BIN_PATH, this.LIB),
+        this.LIB_OSRM_CONTRACT_PATH = util.format('%s/libosrm_contract%s', this.BIN_PATH, this.LIB),
+        this.LIB_OSRM_PATH = util.format('%s/libosrm%s', this.BIN_PATH, this.LIB);
+
        // eslint-disable-next-line no-console
        console.info(util.format('Node Version', process.version));
-        if (parseInt(process.version.match(/v(\d)/)[1]) < 4) throw new Error('*** PLease upgrade to Node 4.+ to run OSRM cucumber tests');
+        if (parseInt(process.version.match(/v(\d)/)[1]) < 4) throw new Error('*** Please upgrade to Node 4.+ to run OSRM cucumber tests');

-        fs.exists(this.TEST_FOLDER, (exists) => {
-            if (!exists) throw new Error(util.format('*** Test folder %s doesn\'t exist.', this.TEST_FOLDER));
-            callback();
+        fs.exists(this.TEST_PATH, (exists) => {
+            if (exists)
+                return callback();
+            else
+                return callback(new Error('*** Test folder doesn\'t exist.'));
        });
    };

-    this.verifyOSRMIsNotRunning = () => {
-        if (this.OSRMLoader.up()) {
-            throw new Error('*** osrm-routed is already running.');
-        }
+    this.getProfilePath = (profile) => {
+        return path.resolve(this.PROFILES_PATH, profile + '.lua');
+    };
+
+    this.verifyOSRMIsNotRunning = (callback) => {
+        tryConnect(this.OSRM_PORT, (err) => {
+            if (!err) return callback(new Error('*** osrm-routed is already running.'));
+            else callback();
+        });
    };

    this.verifyExistenceOfBinaries = (callback) => {
-        var verify = (bin, cb) => {
-            var binPath = path.resolve(util.format('%s/%s%s', this.BIN_PATH, bin, this.EXE));
+        var verify = (binPath, cb) => {
            fs.exists(binPath, (exists) => {
-                if (!exists) throw new Error(util.format('%s is missing. Build failed?', binPath));
+                if (!exists) return cb(new Error(util.format('%s is missing. Build failed?', binPath)));
                var helpPath = util.format('%s --help > /dev/null 2>&1', binPath);
-                exec(helpPath, (err) => {
+                child_process.exec(helpPath, (err) => {
                    if (err) {
-                        this.log(util.format('*** Exited with code %d', err.code), 'preprocess');
-                        throw new Error(util.format('*** %s exited with code %d', helpPath, err.code));
+                        return cb(new Error(util.format('*** %s exited with code %d', helpPath, err.code)));
                    }
                    cb();
                });
@@ -74,23 +119,12 @@ module.exports = function () {
        };

        var q = d3.queue();
-        ['osrm-extract', 'osrm-contract', 'osrm-routed'].forEach(bin => { q.defer(verify, bin); });
-        q.awaitAll(() => {
-            callback();
-        });
-    };
-
-    this.AfterConfiguration = (callback) => {
-        this.clearLogFiles(() => {
-            this.verifyOSRMIsNotRunning();
-            this.verifyExistenceOfBinaries(() => {
-                callback();
-            });
-        });
+        [this.OSRM_EXTRACT_PATH, this.OSRM_CONTRACT_PATH, this.OSRM_ROUTED_PATH].forEach(bin => { q.defer(verify, bin); });
+        q.awaitAll(callback);
    };

    process.on('exit', () => {
-        if (this.OSRMLoader.loader) this.OSRMLoader.shutdown(() => {});
+        this.osrmLoader.shutdown(() => {});
    });

    process.on('SIGINT', () => {
@@ -1,132 +0,0 @@
-'use strict';
-
-var util = require('util');
-var path = require('path');
-var fs = require('fs');
-var chalk = require('chalk');
-
-var OSRMError = class extends Error {
-    constructor (process, code, msg, log, lines) {
-        super(msg);
-        this.process = process;
-        this.code = code;
-        this.msg = msg;
-        this.lines = lines;
-        this.log = log;
-    }
-
-    extract (callback) {
-        this.logTail(this.log, this.lines, callback);
-    }
-
-    // toString (callback) {
-    //     this.extract((tail) => {
-    //         callback(util.format('*** %s\nLast %s from %s:\n%s\n', this.msg, this.lines, this.log, tail));
-    //     });
-    // }
-
-    logTail (logPath, n, callback) {
-        var expanded = path.resolve(this.TEST_FOLDER, logPath);
-        fs.exists(expanded, (exists) => {
-            if (exists) {
-                fs.readFile(expanded, (err, data) => {
-                    var lines = data.toString().trim().split('\n');
-                    callback(lines
-                        .slice(lines.length - n)
-                        .map(line => util.format('    %s', line))
-                        .join('\n'));
-                });
-            } else {
-                callback(util.format('File %s does not exist!', expanded));
-            }
-        });
-    }
-};
-
-var unescapeStr = (str) => str.replace(/\\\|/g, '\|').replace(/\\\\/g, '\\');
-
-module.exports = {
-    OSRMError: OSRMError,
-
-    FileError: class extends OSRMError {
-        constructor (logFile, code, msg) {
-            super ('fileutil', code, msg, logFile, 5);
-        }
-    },
-
-    LaunchError: class extends OSRMError {
-        constructor (logFile, launchProcess, code, msg) {
-            super (launchProcess, code, msg, logFile, 5);
-        }
-    },
-
-    ExtractError: class extends OSRMError {
-        constructor (logFile, code, msg) {
-            super('osrm-extract', code, msg, logFile, 3);
-        }
-    },
-
-    ContractError:  class extends OSRMError {
-        constructor (logFile, code, msg) {
-            super('osrm-contract', code, msg, logFile, 3);
-        }
-    },
-
-    RoutedError: class extends OSRMError {
-        constructor (logFile, msg) {
-            super('osrm-routed', null, msg, logFile, 3);
-        }
-    },
-
-    TableDiffError: class extends Error {
-        constructor (expected, actual) {
-            super();
-            this.headers = expected.raw()[0];
-            this.expected = expected.hashes();
-            this.actual = actual;
-            this.diff = [];
-            this.hasErrors = false;
-
-            var good = 0, bad = 0;
-
-            this.expected.forEach((row, i) => {
-                var rowError = false;
-
-                for (var j in row) {
-                    if (unescapeStr(row[j]) != actual[i][j]) {
-                        rowError = true;
-                        this.hasErrors = true;
-                        break;
-                    }
-                }
-
-                if (rowError) {
-                    bad++;
-                    this.diff.push(Object.assign({}, row, {c_status: 'undefined'}));
-                    this.diff.push(Object.assign({}, actual[i], {c_status: 'comment'}));
-                } else {
-                    good++;
-                    this.diff.push(row);
-                }
-            });
-        }
-
-        get string () {
-            if (!this.hasErrors) return null;
-
-            var s = ['Tables were not identical:'];
-            s.push(this.headers.map(key => '    ' + key).join(' | '));
-            this.diff.forEach((row) => {
-                var rowString = '| ';
-                this.headers.forEach((header) => {
-                    if (!row.c_status) rowString += chalk.green('    ' + row[header] + ' | ');
-                    else if (row.c_status === 'undefined') rowString += chalk.yellow('(-) ' + row[header] + ' | ');
-                    else rowString += chalk.red('(+) ' + row[header] + ' | ');
-                });
-                s.push(rowString);
-            });
-
-            return s.join('\n') + '\nTODO this is a temp workaround waiting for https://github.com/cucumber/cucumber-js/issues/534';
-        }
-    }
-};
@@ -1,15 +0,0 @@
-var exceptions = require('./exception_classes');
-
-module.exports = function () {
-    this.OSRMError = exceptions.OSRMError,
-
-    this.FileError = (code, msg) => new (exceptions.FileError.bind(exceptions.FileError, this.PREPROCESS_LOG_FILE))(code, msg);
-
-    this.LaunchError = (code, launchProcess, msg) => new (exceptions.LaunchError.bind(exceptions.LaunchError, this.ERROR_LOG_FILE))(code, launchProcess, msg);
-
-    this.ExtractError = (code, msg) => new (exceptions.ExtractError.bind(exceptions.ExtractError, this.PREPROCESS_LOG_FILE))(code, msg);
-
-    this.ContractError = (code, msg) => new (exceptions.ContractError.bind(exceptions.ContractError, this.PREPROCESS_LOG_FILE))(code, msg);
-
-    this.RoutedError = (msg) => new (exceptions.RoutedError.bind(exceptions.RoutedError, this.OSRM_ROUTED_LOG_FILE))(msg);
-};
@@ -1,43 +0,0 @@
-var fs = require('fs');
-var path = require('path');
-var crypto = require('crypto');
-var d3 = require('d3-queue');
-
-module.exports = function () {
-    this.hashOfFiles = (paths, cb) => {
-        paths = Array.isArray(paths) ? paths : [paths];
-        var shasum = crypto.createHash('sha1'), hashedFiles = false;
-
-        var q = d3.queue(1);
-
-        var addFile = (path, cb) => {
-            fs.readFile(path, (err, data) => {
-                if (err && err.code === 'ENOENT') cb(); // ignore non-existing files
-                else if (err) cb(err);
-                else {
-                    shasum.update(data);
-                    hashedFiles = true;
-                    cb();
-                }
-            });
-        };
-
-        paths.forEach(path => { q.defer(addFile, path); });
-
-        q.awaitAll(err => {
-            if (err) throw new Error('*** Error reading files:', err);
-            if (!hashedFiles) throw new Error('*** No files found: [' + paths.join(', ') + ']');
-            cb(shasum.digest('hex'));
-        });
-    };
-
-    this.hashProfile = (cb) => {
-        this.hashOfFiles(path.resolve(this.PROFILES_PATH, this.profile + '.lua'), cb);
-    };
-
-    this.hashString = (str) => {
-        return crypto.createHash('sha1').update(str).digest('hex');
-    };
-
-    return this;
-};
@@ -1,36 +1,61 @@
-var util = require('util');
+'use strict';
+
+var d3 = require('d3-queue');
+var path = require('path');
+var mkdirp = require('mkdirp');
+var rimraf = require('rimraf');
+var OSM = require('../lib/osm');
+var OSRMLoader = require('../lib/osrm_loader');

 module.exports = function () {
-    this.BeforeFeatures((features, callback) => {
-        this.pid = null;
-        this.initializeEnv(() => {
-            this.initializeOptions(callback);
-        });
+    this.registerHandler('BeforeFeatures', {timeout: 30000},  (features, callback) => {
+        this.osrmLoader = new OSRMLoader(this);
+        this.OSMDB = new OSM.DB();
+
+        let queue = d3.queue(1);
+        queue.defer(this.initializeEnv.bind(this));
+        queue.defer(this.verifyOSRMIsNotRunning.bind(this));
+        queue.defer(this.verifyExistenceOfBinaries.bind(this));
+        queue.defer(this.initializeCache.bind(this));
+        queue.defer(this.setupFeatures.bind(this, features));
+        queue.awaitAll(callback);
    });

-    this.Before((scenario, callback) => {
-        this.scenarioTitle = scenario.getName();
-
-        this.loadMethod = this.DEFAULT_LOAD_METHOD;
-        this.queryParams = {};
-        var d = new Date();
-        this.scenarioTime = util.format('%d-%d-%dT%s:%s:%sZ', d.getFullYear(), d.getMonth()+1, d.getDate(), d.getHours(), d.getMinutes(), d.getSeconds());
-        this.resetData();
-        this.hasLoggedPreprocessInfo = false;
-        this.hasLoggedScenarioInfo = false;
-        this.setGridSize(this.DEFAULT_GRID_SIZE);
-        this.setOrigin(this.DEFAULT_ORIGIN);
-        this.fingerprintExtract = this.hashString([this.luaLibHash, this.binExtractHash].join('-'));
-        this.fingerprintContract = this.hashString(this.binContractHash);
+    this.BeforeFeature((feature, callback) => {
+        this.profile = this.DEFAULT_PROFILE;
+        this.profileFile = path.join(this.PROFILES_PATH, this.profile + '.lua');
+        this.setupFeatureCache(feature);
        callback();
    });

+    this.Before((scenario, callback) => {
+        this.osrmLoader.setLoadMethod(this.DEFAULT_LOAD_METHOD);
+        this.setGridSize(this.DEFAULT_GRID_SIZE);
+        this.setOrigin(this.DEFAULT_ORIGIN);
+        this.queryParams = {};
+        this.extractArgs = '';
+        this.contractArgs = '';
+        this.environment = Object.assign(this.DEFAULT_ENVIRONMENT);
+        this.resetOSM();
+
+        this.scenarioID = this.getScenarioID(scenario);
+        this.setupScenarioCache(this.scenarioID);
+
+        // setup output logging
+        let logDir = path.join(this.LOGS_PATH, this.featureID);
+        this.scenarioLogFile = path.join(logDir, this.scenarioID) + '.log';
+        d3.queue(1)
+            .defer(mkdirp, logDir)
+            .defer(rimraf, this.scenarioLogFile)
+            .awaitAll(callback);
+    });
+
    this.After((scenario, callback) => {
-        this.setExtractArgs('', () => {
-            this.setContractArgs('', () => {
-                if (this.loadMethod === 'directly' && !!this.OSRMLoader.loader) this.OSRMLoader.shutdown(callback);
-                else callback();
-            });
-        });
+        this.resetOptionsOutput();
+        callback();
+    });
+
+    this.AfterFeatures((features, callback) => {
+        callback();
    });
 };
@@ -19,6 +19,9 @@ module.exports = function () {
        return paramString;
    };

+    // FIXME this needs to be simplified!
+    // - remove usage of node-timeout
+    // - replace with node's native timout mechanism
    this.sendRequest = (baseUri, parameters, callback) => {
        var limit = Timeout(this.TIMEOUT, { err: { statusCode: 408 } });

@@ -28,9 +31,9 @@ module.exports = function () {

            request(this.query, (err, res, body) => {
                if (err && err.code === 'ECONNREFUSED') {
-                    throw new Error('*** osrm-routed is not running.');
+                    return cb(new Error('*** osrm-routed is not running.'));
                } else if (err && err.statusCode === 408) {
-                    throw new Error();
+                    return cb(new Error());
                }

                return cb(err, res, body);
@@ -40,11 +43,10 @@ module.exports = function () {
        runRequest(limit((err, res, body) => {
            if (err) {
                if (err.statusCode === 408)
-                    return callback(this.RoutedError('*** osrm-routed did not respond'));
+                    return callback(new Error('*** osrm-routed did not respond'));
                else if (err.code === 'ECONNREFUSED')
-                    return callback(this.RoutedError('*** osrm-routed is not running'));
+                    return callback(new Error('*** osrm-routed is not running'));
            }
-            //console.log(body+"\n");
            return callback(err, res, body);
        }));
    };
@@ -1,5 +0,0 @@
-var launchClasses = require('./launch_classes');
-
-module.exports = function () {
-    this._OSRMLoader = () => new (launchClasses._OSRMLoader.bind(launchClasses._OSRMLoader, this))();
-};
@@ -1,164 +0,0 @@
-'use strict';
-
-var fs = require('fs');
-var spawn = require('child_process').spawn;
-var util = require('util');
-var net = require('net');
-var Timeout = require('node-timeout');
-
-var OSRMBaseLoader = class {
-    constructor (scope) {
-        this.scope = scope;
-    }
-
-    launch (callback) {
-        var limit = Timeout(this.scope.TIMEOUT, { err: this.scope.RoutedError('Launching osrm-routed timed out.') });
-
-        var runLaunch = (cb) => {
-            this.osrmUp(() => { this.waitForConnection(cb); });
-        };
-
-        runLaunch(limit((e) => { if (e) callback(e); else callback(); }));
-    }
-
-    shutdown (callback) {
-        var limit = Timeout(this.scope.TIMEOUT, { err: this.scope.RoutedError('Shutting down osrm-routed timed out.')});
-
-        var runShutdown = (cb) => {
-            this.osrmDown(cb);
-        };
-
-        runShutdown(limit((e) => { if (e) callback(e); else callback(); }));
-    }
-
-    osrmIsRunning () {
-        return !!this.scope.pid && this.child && !this.child.killed;
-    }
-
-    osrmDown (callback) {
-        if (this.scope.pid) {
-            process.kill(this.scope.pid, this.scope.TERMSIGNAL);
-            this.waitForShutdown(callback);
-            this.scope.pid = null;
-        } else callback(true);
-    }
-
-    waitForConnection (callback) {
-        var retryCount = 0;
-        var connectWithRetry = () => {
-            net.connect({ port: this.scope.OSRM_PORT, host: '127.0.0.1' })
-               .on('connect', () => { callback(); })
-               .on('error', () => {
-                   if (retryCount < 2) {
-                       retryCount++;
-                       setTimeout(connectWithRetry, 100);
-                   } else {
-                       callback(new Error('Could not connect to osrm-routed after three retires'));
-                   }
-               });
-        };
-
-        connectWithRetry();
-    }
-
-    waitForShutdown (callback) {
-        var check = () => {
-            if (!this.osrmIsRunning()) return callback();
-        };
-        setTimeout(check, 100);
-    }
-};
-
-var OSRMDirectLoader = class extends OSRMBaseLoader {
-    constructor (scope) {
-        super(scope);
-    }
-
-    load (inputFile, callback) {
-        this.inputFile = inputFile;
-        this.shutdown(() => {
-            this.launch(callback);
-        });
-    }
-
-    osrmUp (callback) {
-        if (this.scope.pid) return callback();
-        var writeToLog = (data) => {
-            fs.appendFile(this.scope.OSRM_ROUTED_LOG_FILE, data, (err) => { if (err) throw err; });
-        };
-
-        var child = spawn(util.format('%s/osrm-routed', this.scope.BIN_PATH), [this.inputFile, util.format('-p%d', this.scope.OSRM_PORT)]);
-        this.scope.pid = child.pid;
-        child.stdout.on('data', writeToLog);
-        child.stderr.on('data', writeToLog);
-
-        callback();
-    }
-};
-
-var OSRMDatastoreLoader = class extends OSRMBaseLoader {
-    constructor (scope) {
-        super(scope);
-    }
-
-    load (inputFile, callback) {
-        this.inputFile = inputFile;
-        this.loadData((err) => {
-            if (err) return callback(err);
-            if (!this.scope.pid) return this.launch(callback);
-            else callback();
-        });
-    }
-
-    loadData (callback) {
-        this.scope.runBin('osrm-datastore', this.inputFile, (err) => {
-            if (err) return callback(this.scope.LaunchError(this.exitCode, 'datastore', err));
-            callback();
-        });
-    }
-
-    osrmUp (callback) {
-        if (this.scope.pid) return callback();
-        var writeToLog = (data) => {
-            fs.appendFile(this.scope.OSRM_ROUTED_LOG_FILE, data, (err) => { if (err) throw err; });
-        };
-
-        var child = spawn(util.format('%s/osrm-routed', this.scope.BIN_PATH), ['--shared-memory=1', util.format('-p%d', this.scope.OSRM_PORT)]);
-        this.child = child;
-        this.scope.pid = child.pid;
-        child.stdout.on('data', writeToLog);
-        child.stderr.on('data', writeToLog);
-
-        callback();
-    }
-};
-
-module.exports = {
-    _OSRMLoader: class {
-        constructor (scope) {
-            this.scope = scope;
-            this.loader = null;
-        }
-
-        load (inputFile, callback) {
-            var method = this.scope.loadMethod;
-            if (method === 'datastore') {
-                this.loader = new OSRMDatastoreLoader(this.scope);
-                this.loader.load(inputFile, callback);
-            } else if (method === 'directly') {
-                this.loader = new OSRMDirectLoader(this.scope);
-                this.loader.load(inputFile, callback);
-            } else {
-                callback(new Error('*** Unknown load method ' + method));
-            }
-        }
-
-        shutdown (callback) {
-            this.loader.shutdown(callback);
-        }
-
-        up () {
-            return this.loader ? this.loader.osrmIsRunning() : false;
-        }
-    }
-};
@@ -1,90 +0,0 @@
-var fs = require('fs');
-
-module.exports = function () {
-    this.clearLogFiles = (callback) => {
-        // emptying existing files, rather than deleting and writing new ones makes it
-        // easier to use tail -f from the command line
-        fs.writeFile(this.OSRM_ROUTED_LOG_FILE, '', err => {
-            if (err) throw err;
-            fs.writeFile(this.PREPROCESS_LOG_FILE, '', err => {
-                if (err) throw err;
-                fs.writeFile(this.LOG_FILE, '', err => {
-                    if (err) throw err;
-                    callback();
-                });
-            });
-        });
-    };
-
-    var log = this.log = (s, type) => {
-        s = s || '';
-        type = type || null;
-        var file = type === 'preprocess' ? this.PREPROCESS_LOG_FILE : this.LOG_FILE;
-        fs.appendFile(file, s + '\n', err => {
-            if (err) throw err;
-        });
-    };
-
-    this.logScenarioFailInfo = () => {
-        if (this.hasLoggedScenarioInfo) return;
-
-        log('=========================================');
-        log('Failed scenario: ' + this.scenarioTitle);
-        log('Time: ' + this.scenarioTime);
-        log('Fingerprint osm stage: ' + this.osmData.fingerprintOSM);
-        log('Fingerprint extract stage: ' + this.fingerprintExtract);
-        log('Fingerprint contract stage: ' + this.fingerprintContract);
-        log('Fingerprint route stage: ' + this.fingerprintRoute);
-        log('Profile: ' + this.profile);
-        log();
-        log('```xml');               // so output can be posted directly to github comment fields
-        log(this.osmData.str.trim());
-        log('```');
-        log();
-        log();
-
-        this.hasLoggedScenarioInfo = true;
-    };
-
-    this.logFail = (expected, got, attempts) => {
-        this.logScenarioFailInfo();
-        log('== ');
-        log('Expected: ' + JSON.stringify(expected));
-        log('Got:      ' + JSON.stringify(got));
-        log();
-        ['route','forw','backw'].forEach((direction) => {
-            if (attempts[direction]) {
-                log('Direction: ' + direction);
-                log('Query: ' + attempts[direction].query);
-                log('Response: ' + attempts[direction].response.body);
-                log();
-            }
-        });
-    };
-
-    this.logPreprocessInfo = () => {
-        if (this.hasLoggedPreprocessInfo) return;
-        log('=========================================', 'preprocess');
-        log('Preprocessing data for scenario: ' + this.scenarioTitle, 'preprocess');
-        log('Time: ' + this.scenarioTime, 'preprocess');
-        log('', 'preprocess');
-        log('== OSM data:', 'preprocess');
-        log('```xml', 'preprocess');            // so output can be posted directly to github comment fields
-        log(this.osmData.str, 'preprocess');
-        log('```', 'preprocess');
-        log('', 'preprocess');
-        log('== Profile:', 'preprocess');
-        log(this.profile, 'preprocess');
-        log('', 'preprocess');
-        this.hasLoggedPreprocessInfo = true;
-    };
-
-    this.logPreprocess = (str) => {
-        this.logPreprocessInfo();
-        log(str, 'preprocess');
-    };
-
-    this.logPreprocessDone = () => {
-        log('Done with preprocessing at ' + new Date(), 'preprocess');
-    };
-};
@@ -1,7 +1,8 @@
 'use strict';

-var Timeout = require('node-timeout');
-var request = require('request');
+const Timeout = require('node-timeout');
+const request = require('request');
+const ensureDecimal = require('../lib/utils').ensureDecimal;

 module.exports = function () {
    this.requestPath = (service, params, callback) => {
@@ -42,7 +43,7 @@ module.exports = function () {
    };

    var encodeWaypoints = (waypoints) => {
-        return waypoints.map(w => [w.lon, w.lat].map(this.ensureDecimal).join(','));
+        return waypoints.map(w => [w.lon, w.lat].map(ensureDecimal).join(','));
    };

    this.requestRoute = (waypoints, bearings, userParams, callback) => {
@@ -131,7 +132,7 @@ module.exports = function () {

    this.summary = (instructions) => {
        if (instructions) {
-            return instructions.legs.map(l => l.summary).join(',');
+            return instructions.legs.map(l => l.summary).join(';');
        }
    };

@@ -1,40 +1,52 @@
-var fs = require('fs');
-var util = require('util');
-var exec = require('child_process').exec;
+'use strict';
+
+const fs = require('fs');
+const util = require('util');
+const child_process = require('child_process');

 module.exports = function () {
-    this.runBin = (bin, options, callback) => {
-        var opts = options.slice();
+    // replaces placeholders for in user supplied commands
+    this.expandOptions = (options) => {
+        let opts = options.slice();
+        let table = {
+            '{osm_file}': this.inputCacheFile,
+            '{processed_file}': this.processedCacheFile,
+            '{profile_file}': this.profileFile,
+            '{rastersource_file}': this.rasterCacheFile,
+            '{speeds_file}': this.speedsCacheFile,
+            '{penalties_file}': this.penaltiesCacheFile
+        };

-        if (opts.match('{osm_base}')) {
-            if (!this.osmData.osmFile) throw new Error('*** {osm_base} is missing');
-            opts = opts.replace('{osm_base}', this.osmData.osmFile);
+        for (let k in table) {
+            opts = opts.replace(k, table[k]);
        }

-        if (opts.match('{extracted_base}')) {
-            if (!this.osmData.extractedFile) throw new Error('*** {extracted_base} is missing');
-            opts = opts.replace('{extracted_base}', this.osmData.extractedFile);
+        return opts;
+    };
+
+    this.setupOutputLog = (process, log) => {
+        if (process.logFunc) {
+            process.stdout.removeListener('data', process.logFunc);
+            process.stderr.removeListener('data', process.logFunc);
        }

-        if (opts.match('{contracted_base}')) {
-            if (!this.osmData.contractedFile) throw new Error('*** {contracted_base} is missing');
-            opts = opts.replace('{contracted_base}', this.osmData.contractedFile);
-        }
+        process.logFunc = (message) => { log.write(message); };
+        process.stdout.on('data', process.logFunc);
+        process.stderr.on('data', process.logFunc);
+    };

-        if (opts.match('{profile}')) {
-            opts = opts.replace('{profile}', [this.PROFILES_PATH, this.profile + '.lua'].join('/'));
-        }
-
-        var cmd = util.format('%s%s/%s%s%s %s 2>%s', this.QQ, this.BIN_PATH, bin, this.EXE, this.QQ, opts, this.ERROR_LOG_FILE);
-        process.chdir(this.TEST_FOLDER);
-        exec(cmd, (err, stdout, stderr) => {
-            this.stdout = stdout.toString();
-            fs.readFile(this.ERROR_LOG_FILE, (e, data) => {
-                this.stderr = data ? data.toString() : '';
-                this.exitCode = err && err.code || 0;
-                process.chdir('../');
-                callback(err, stdout, stderr);
-            });
-        });
+    this.runBin = (bin, options, env, callback) => {
+        let cmd = util.format('%s%s/%s%s%s', this.QQ, this.BIN_PATH, bin, this.EXE, this.QQ);
+        let opts = options.split(' ').filter((x) => { return x && x.length > 0; });
+        let log = fs.createWriteStream(this.scenarioLogFile, {'flags': 'a'});
+        log.write(util.format('*** running %s %s\n', cmd, options));
+        // we need to set a large maxbuffer here because we have long running processes like osrm-routed
+        // with lots of log output
+        let child = child_process.execFile(cmd, opts, {maxBuffer: 1024 * 1024 * 1000, env: env}, callback);
+        child.on('exit', function(code) {
+            log.write(util.format('*** %s exited with code %d\n', bin, code));
+        }.bind(this));
+        this.setupOutputLog(child, log);
+        return child;
    };
 };
@@ -93,7 +93,7 @@ module.exports = function () {
                            if (headers.has('distance')) {
                                if (row.distance.length) {
                                    if (!row.distance.match(/\d+m/))
-                                        throw new Error('*** Distance must be specified in meters. (ex: 250m)');
+                                        return cb(new Error('*** Distance must be specified in meters. (ex: 250m)'));
                                    got.distance = instructions ? util.format('%dm', distance) : '';
                                } else {
                                    got.distance = '';
@@ -102,7 +102,7 @@ module.exports = function () {

                            if (headers.has('time')) {
                                if (!row.time.match(/\d+s/))
-                                    throw new Error('*** Time must be specied in seconds. (ex: 60s)');
+                                    return cb(new Error('*** Time must be specied in seconds. (ex: 60s)'));
                                got.time = instructions ? util.format('%ds', time) : '';
                            }

@@ -113,7 +113,7 @@ module.exports = function () {
                            if (headers.has('speed')) {
                                if (row.speed !== '' && instructions) {
                                    if (!row.speed.match(/\d+ km\/h/))
-                                        throw new Error('*** Speed must be specied in km/h. (ex: 50 km/h)');
+                                        cb(new Error('*** Speed must be specied in km/h. (ex: 50 km/h)'));
                                    var speed = time > 0 ? Math.round(3.6*distance/time) : null;
                                    got.speed = util.format('%d km/h', speed);
                                } else {
@@ -139,20 +139,12 @@ module.exports = function () {
                            putValue('destinations', destinations);
                        }

-                        var ok = true;
-
                        for (var key in row) {
                            if (this.FuzzyMatch.match(got[key], row[key])) {
                                got[key] = row[key];
-                            } else {
-                                ok = false;
                            }
                        }

-                        if (!ok) {
-                            this.logFail(row, got, { route: { query: this.query, response: res }});
-                        }
-
                        cb(null, got);
                    } else {
                        cb(new Error('request failed to return valid body'));
@@ -189,11 +181,11 @@ module.exports = function () {

                    if (row.from && row.to) {
                        var fromNode = this.findNodeByName(row.from);
-                        if (!fromNode) throw new Error(util.format('*** unknown from-node "%s"'), row.from);
+                        if (!fromNode) return cb(new Error(util.format('*** unknown from-node "%s"'), row.from));
                        waypoints.push(fromNode);

                        var toNode = this.findNodeByName(row.to);
-                        if (!toNode) throw new Error(util.format('*** unknown to-node "%s"'), row.to);
+                        if (!toNode) return cb(new Error(util.format('*** unknown to-node "%s"'), row.to));
                        waypoints.push(toNode);

                        got.from = row.from;
@@ -202,13 +194,13 @@ module.exports = function () {
                    } else if (row.waypoints) {
                        row.waypoints.split(',').forEach((n) => {
                            var node = this.findNodeByName(n.trim());
-                            if (!node) throw new Error('*** unknown waypoint node "%s"', n.trim());
+                            if (!node) return cb(new Error('*** unknown waypoint node "%s"', n.trim()));
                            waypoints.push(node);
                        });
                        got.waypoints = row.waypoints;
                        this.requestRoute(waypoints, bearings, params, afterRequest);
                    } else {
-                        throw new Error('*** no waypoints');
+                        return cb(new Error('*** no waypoints'));
                    }
                }
            };
@@ -1,11 +0,0 @@
-var DifferentError = require('./exception_classes').TableDiffError;
-
-module.exports = function () {
-    this.diffTables = (expected, actual, options, callback) => {
-        // this is a temp workaround while waiting for https://github.com/cucumber/cucumber-js/issues/534
-
-        var error = new DifferentError(expected, actual);
-
-        return callback(error.string);
-    };
-};
@@ -11,8 +11,8 @@ Feature: Handle bad data in a graceful manner
        Given the ways
            | nodes |

-        When the data has been contracted
-        Then "osrm-extract" should return code 1
+        When I try to run "osrm-extract {osm_file} --profile {profile_file}"
+        Then it should exit with an error

    Scenario: Only dead-end oneways
        Given the node map
@@ -1,5 +0,0 @@
-@routing @testbot @bug
-Feature: Known bugs
-
-    Background:
-        Given the profile "testbot"
@@ -124,7 +124,7 @@ Feature: Basic Map Matching
        1,2,36
        """

-        And the contract extra arguments "--segment-speed-file speeds.csv"
+        And the contract extra arguments "--segment-speed-file {speeds_file}"

        When I match I should get
            | trace | matchings | annotation                                                                                     |
@@ -53,9 +53,9 @@ Feature: Basic Routing
            | ab    |

        When I route I should get
-            | from | to | route | summary  |
-            | a    | b  | ab,ab | ab       |
-            | b    | a  | ab,ab | ab       |
+            | from | to | route | summary |
+            | a    | b  | ab,ab | ab      |
+            | b    | a  | ab,ab | ab      |

    @repeated
    Scenario: Check handling empty values
@@ -26,7 +26,7 @@ Feature: Traffic - turn penalties applied to turn onto which a phantom node snap
            1,2,5,0,comment
            3,4,7,-20
            """
-        And the contract extra arguments "--turn-penalty-file penalties.csv"
+        And the contract extra arguments "--turn-penalty-file {penalties_file}"
        When I route I should get
            | from | to | route    | speed   | time    |
            | a    | e  | ab,be,be | 36 km/h | 40s +-1 |
@@ -173,7 +173,7 @@ Feature: Via points
            | c,d,a     | abc,bd,bd,bd,abc,abc |

    # See issue #2349
-    @bug @todo
+    @todo
    Scenario: Via point at a dead end with oneway
        Given the node map
            | a | b | c |
@@ -69,6 +69,21 @@ struct RouteParameters : public BaseParameters

    RouteParameters() = default;

+    template <typename... Args>
+    RouteParameters(const bool steps_,
+                    const bool alternatives_,
+                    const GeometriesType geometries_,
+                    const OverviewType overview_,
+                    const boost::optional<bool> continue_straight_,
+                    Args... args_)
+        : BaseParameters{std::forward<Args>(args_)...}, steps{steps_}, alternatives{alternatives_},
+          annotations{false}, geometries{geometries_}, overview{overview_},
+          continue_straight{continue_straight_}
+    // Once we perfectly-forward `args` (see #2990) this constructor can delegate to the one below.
+    {
+    }
+
+    // RouteParameters constructor adding the `annotations` setting in a API-compatible way.
    template <typename... Args>
    RouteParameters(const bool steps_,
                    const bool alternatives_,
@@ -1,7 +1,6 @@
 #ifndef ENGINE_HPP
 #define ENGINE_HPP

-#include "storage/shared_barriers.hpp"
 #include "engine/status.hpp"
 #include "util/json_container.hpp"

@@ -20,6 +19,11 @@ struct Object;
 }
 }

+namespace storage
+{
+struct SharedBarriers;
+}
+
 // Fwd decls
 namespace engine
 {
@@ -52,10 +56,7 @@ class BaseDataFacade;
 class Engine final
 {
  public:
-    // Needs to be public
-    struct EngineLock;
-
-    explicit Engine(EngineConfig &config);
+    explicit Engine(const EngineConfig &config);

    Engine(Engine &&) noexcept;
    Engine &operator=(Engine &&) noexcept;
@@ -63,15 +64,15 @@ class Engine final
    // Impl. in cpp since for unique_ptr of incomplete types
    ~Engine();

-    Status Route(const api::RouteParameters &parameters, util::json::Object &result);
-    Status Table(const api::TableParameters &parameters, util::json::Object &result);
-    Status Nearest(const api::NearestParameters &parameters, util::json::Object &result);
-    Status Trip(const api::TripParameters &parameters, util::json::Object &result);
-    Status Match(const api::MatchParameters &parameters, util::json::Object &result);
-    Status Tile(const api::TileParameters &parameters, std::string &result);
+    Status Route(const api::RouteParameters &parameters, util::json::Object &result) const;
+    Status Table(const api::TableParameters &parameters, util::json::Object &result) const;
+    Status Nearest(const api::NearestParameters &parameters, util::json::Object &result) const;
+    Status Trip(const api::TripParameters &parameters, util::json::Object &result) const;
+    Status Match(const api::MatchParameters &parameters, util::json::Object &result) const;
+    Status Tile(const api::TileParameters &parameters, std::string &result) const;

  private:
-    std::unique_ptr<EngineLock> lock;
+    std::unique_ptr<storage::SharedBarriers> lock;

    std::unique_ptr<plugins::ViaRoutePlugin> route_plugin;
    std::unique_ptr<plugins::TablePlugin> table_plugin;
@@ -6,6 +6,11 @@
 #include "engine/guidance/route_leg.hpp"
 #include "engine/guidance/route_step.hpp"
 #include "engine/internal_route_result.hpp"
+#include "util/typedefs.hpp"
+
+#include <boost/algorithm/string/join.hpp>
+#include <boost/range/adaptor/filtered.hpp>
+#include <boost/range/adaptor/transformed.hpp>

 #include <cstddef>
 #include <cstdint>
@@ -107,7 +112,7 @@ std::array<std::uint32_t, SegmentNumber> summarizeRoute(const std::vector<PathDa
        });

    std::array<std::uint32_t, SegmentNumber> summary;
-    std::fill(summary.begin(), summary.end(), 0);
+    std::fill(summary.begin(), summary.end(), EMPTY_NAMEID);
    std::transform(segments.begin(),
                   segments.end(),
                   summary.begin(),
@@ -172,21 +177,28 @@ inline RouteLeg assembleLeg(const datafacade::BaseDataFacade &facade,
    {
        auto summary_array = detail::summarizeRoute<detail::MAX_USED_SEGMENTS>(
            route_data, target_node, target_traversed_in_reverse);
-        if (route_data.empty())
-            summary_array[0] = source_node.name_id;

        BOOST_ASSERT(detail::MAX_USED_SEGMENTS > 0);
        BOOST_ASSERT(summary_array.begin() != summary_array.end());
-        summary = std::accumulate(std::next(summary_array.begin()),
-                                  summary_array.end(),
-                                  facade.GetNameForID(summary_array.front()),
-                                  [&facade](std::string previous, const std::uint32_t name_id) {
-                                      if (name_id != 0)
-                                      {
-                                          previous += ", " + facade.GetNameForID(name_id);
-                                      }
-                                      return previous;
-                                  });
+
+        // transform a name_id into a string containing either the name, or -if the name is empty-
+        // the reference.
+        const auto name_id_to_string = [&](const NameID name_id) {
+            const auto name = facade.GetNameForID(name_id);
+            if (!name.empty())
+                return name;
+            else
+            {
+                const auto ref = facade.GetRefForID(name_id);
+                return ref;
+            }
+        };
+
+        const auto not_empty = [&](const std::string &name) { return !name.empty(); };
+
+        const auto summary_names = summary_array | boost::adaptors::transformed(name_id_to_string) |
+                                   boost::adaptors::filtered(not_empty);
+        summary = boost::algorithm::join(summary_names, ", ");
    }

    return RouteLeg{duration, distance, summary, {}};
@@ -1,7 +1,6 @@
 #ifndef ENGINE_GUIDANCE_POST_PROCESSING_HPP
 #define ENGINE_GUIDANCE_POST_PROCESSING_HPP

-#include "engine/datafacade/datafacade_base.hpp"
 #include "engine/guidance/leg_geometry.hpp"
 #include "engine/guidance/route_step.hpp"
 #include "engine/phantom_node.hpp"
@@ -39,10 +39,6 @@ class ExtractionContainers
    void WriteNodes(std::ofstream &file_out_stream) const;
    void WriteRestrictions(const std::string &restrictions_file_name) const;
    void WriteEdges(std::ofstream &file_out_stream) const;
-    void
-    WriteTurnLaneMasks(const std::string &file_name,
-                       const stxxl::vector<std::uint32_t> &turn_lane_offsets,
-                       const stxxl::vector<guidance::TurnLaneType::Mask> &turn_lane_masks) const;
    void WriteCharData(const std::string &file_name);

  public:
@@ -11,7 +11,6 @@

 #include "extractor/compressed_edge_container.hpp"
 #include "extractor/query_node.hpp"
-#include "extractor/suffix_table.hpp"

 #include "extractor/guidance/discrete_angle.hpp"
 #include "extractor/guidance/intersection.hpp"
@@ -26,7 +25,6 @@
 #include <utility>

 #include <boost/algorithm/string.hpp>
-#include <boost/algorithm/string/predicate.hpp>
 #include <boost/functional/hash.hpp>
 #include <boost/tokenizer.hpp>

@@ -151,122 +149,6 @@ getRepresentativeCoordinate(const NodeID from_node,
    }
 }

-inline std::pair<std::string, std::string> getPrefixAndSuffix(const std::string &data)
-{
-    const auto suffix_pos = data.find_last_of(' ');
-    if (suffix_pos == std::string::npos)
-        return {};
-
-    const auto prefix_pos = data.find_first_of(' ');
-    auto result = std::make_pair(data.substr(0, prefix_pos), data.substr(suffix_pos + 1));
-    boost::to_lower(result.first);
-    boost::to_lower(result.second);
-    return result;
-}
-
-inline bool requiresNameAnnounced(const std::string &from,
-                                  const std::string &to,
-                                  const SuffixTable &suffix_table)
-{
-    // first is empty and the second is not
-    if (from.empty() && !to.empty())
-        return true;
-
-    // FIXME, handle in profile to begin with?
-    // this uses the encoding of references in the profile, which is very BAD
-    // Input for this function should be a struct separating streetname, suffix (e.g. road,
-    // boulevard, North, West ...), and a list of references
-    std::string from_name;
-    std::string from_ref;
-    std::string to_name;
-    std::string to_ref;
-
-    // Split from the format "{name} ({ref})" -> name, ref
-    auto split = [](const std::string &name, std::string &out_name, std::string &out_ref) {
-        const auto ref_begin = name.find_first_of('(');
-        if (ref_begin != std::string::npos)
-        {
-            if (ref_begin != 0)
-                out_name = name.substr(0, ref_begin - 1);
-            const auto ref_end = name.find_first_of(')');
-            out_ref = name.substr(ref_begin + 1, ref_end - ref_begin - 1);
-        }
-        else
-        {
-            out_name = name;
-        }
-    };
-
-    split(from, from_name, from_ref);
-    split(to, to_name, to_ref);
-
-    // check similarity of names
-    const auto names_are_empty = from_name.empty() && to_name.empty();
-    const auto name_is_contained =
-        boost::starts_with(from_name, to_name) || boost::starts_with(to_name, from_name);
-
-    const auto checkForPrefixOrSuffixChange = [](
-        const std::string &first, const std::string &second, const SuffixTable &suffix_table) {
-
-        const auto first_prefix_and_suffixes = getPrefixAndSuffix(first);
-        const auto second_prefix_and_suffixes = getPrefixAndSuffix(second);
-        // reverse strings, get suffices and reverse them to get prefixes
-        const auto checkTable = [&](const std::string &str) {
-            return str.empty() || suffix_table.isSuffix(str);
-        };
-
-        const auto getOffset = [](const std::string &str) -> std::size_t {
-            if (str.empty())
-                return 0;
-            else
-                return str.length() + 1;
-        };
-
-        const bool is_prefix_change = [&]() -> bool {
-            if (!checkTable(first_prefix_and_suffixes.first))
-                return false;
-            if (!checkTable(second_prefix_and_suffixes.first))
-                return false;
-            return !first.compare(getOffset(first_prefix_and_suffixes.first),
-                                  std::string::npos,
-                                  second,
-                                  getOffset(second_prefix_and_suffixes.first),
-                                  std::string::npos);
-        }();
-
-        const bool is_suffix_change = [&]() -> bool {
-            if (!checkTable(first_prefix_and_suffixes.second))
-                return false;
-            if (!checkTable(second_prefix_and_suffixes.second))
-                return false;
-            return !first.compare(0,
-                                  first.length() - getOffset(first_prefix_and_suffixes.second),
-                                  second,
-                                  0,
-                                  second.length() - getOffset(second_prefix_and_suffixes.second));
-        }();
-
-        return is_prefix_change || is_suffix_change;
-    };
-
-    const auto is_suffix_change = checkForPrefixOrSuffixChange(from_name, to_name, suffix_table);
-    const auto names_are_equal = from_name == to_name || name_is_contained || is_suffix_change;
-    const auto name_is_removed = !from_name.empty() && to_name.empty();
-    // references are contained in one another
-    const auto refs_are_empty = from_ref.empty() && to_ref.empty();
-    const auto ref_is_contained =
-        from_ref.empty() || to_ref.empty() ||
-        (from_ref.find(to_ref) != std::string::npos || to_ref.find(from_ref) != std::string::npos);
-    const auto ref_is_removed = !from_ref.empty() && to_ref.empty();
-
-    const auto obvious_change =
-        (names_are_empty && refs_are_empty) || (names_are_equal && ref_is_contained) ||
-        (names_are_equal && refs_are_empty) || (ref_is_contained && name_is_removed) ||
-        (names_are_equal && ref_is_removed) || is_suffix_change;
-
-    return !obvious_change;
-}
-
 // To simplify handling of Left/Right hand turns, we can mirror turns and write an intersection
 // handler only for one side. The mirror function turns a left-hand turn in a equivalent right-hand
 // turn and vice versa.
@@ -25,6 +25,7 @@ class SuffixTable final
  private:
    std::unordered_set<std::string> suffix_set;
 };
+
 } /* namespace extractor */
 } /* namespace osrm */

@@ -83,7 +83,7 @@ class OSRM final
     * \return Status indicating success for the query or failure
     * \see Status, RouteParameters and json::Object
     */
-    Status Route(const RouteParameters &parameters, json::Object &result);
+    Status Route(const RouteParameters &parameters, json::Object &result) const;

    /**
     * Distance tables for coordinates.
@@ -92,7 +92,7 @@ class OSRM final
     * \return Status indicating success for the query or failure
     * \see Status, TableParameters and json::Object
     */
-    Status Table(const TableParameters &parameters, json::Object &result);
+    Status Table(const TableParameters &parameters, json::Object &result) const;

    /**
     * Nearest street segment for coordinate.
@@ -101,7 +101,7 @@ class OSRM final
     * \return Status indicating success for the query or failure
     * \see Status, NearestParameters and json::Object
     */
-    Status Nearest(const NearestParameters &parameters, json::Object &result);
+    Status Nearest(const NearestParameters &parameters, json::Object &result) const;

    /**
     * Trip: shortest round trip between coordinates.
@@ -110,7 +110,7 @@ class OSRM final
     * \return Status indicating success for the query or failure
     * \see Status, TripParameters and json::Object
     */
-    Status Trip(const TripParameters &parameters, json::Object &result);
+    Status Trip(const TripParameters &parameters, json::Object &result) const;

    /**
     * Match: snaps noisy coordinate traces to the road network
@@ -119,7 +119,7 @@ class OSRM final
     * \return Status indicating success for the query or failure
     * \see Status, MatchParameters and json::Object
     */
-    Status Match(const MatchParameters &parameters, json::Object &result);
+    Status Match(const MatchParameters &parameters, json::Object &result) const;

    /**
     * Tile: vector tiles with internal graph representation
@@ -128,7 +128,7 @@ class OSRM final
     * \return Status indicating success for the query or failure
     * \see Status, TileParameters and json::Object
     */
-    Status Tile(const TileParameters &parameters, std::string &result);
+    Status Tile(const TileParameters &parameters, std::string &result) const;

  private:
    std::unique_ptr<engine::Engine> engine_;
@@ -3,6 +3,7 @@

 #include <boost/interprocess/sync/named_condition.hpp>
 #include <boost/interprocess/sync/named_mutex.hpp>
+#include <boost/interprocess/sync/named_sharable_mutex.hpp>

 namespace osrm
 {
@@ -13,25 +14,13 @@ struct SharedBarriers

    SharedBarriers()
        : pending_update_mutex(boost::interprocess::open_or_create, "pending_update"),
-          update_mutex(boost::interprocess::open_or_create, "update"),
-          query_mutex(boost::interprocess::open_or_create, "query"),
-          no_running_queries_condition(boost::interprocess::open_or_create, "no_running_queries"),
-          update_ongoing(false), number_of_queries(0)
+          query_mutex(boost::interprocess::open_or_create, "query")
    {
    }

    // Mutex to protect access to the boolean variable
    boost::interprocess::named_mutex pending_update_mutex;
-    boost::interprocess::named_mutex update_mutex;
-    boost::interprocess::named_mutex query_mutex;
-
-    // Condition that no update is running
-    boost::interprocess::named_condition no_running_queries_condition;
-
-    // Is there an ongoing update?
-    bool update_ongoing;
-    // Is there any query?
-    int number_of_queries;
+    boost::interprocess::named_sharable_mutex query_mutex;
 };
 }
 }
@@ -12,8 +12,13 @@
 #include "util/simple_logger.hpp"

 #include <algorithm>
+#include <string>
 #include <vector>

+#include <boost/algorithm/string.hpp>
+#include <boost/algorithm/string/predicate.hpp>
+#include <boost/functional/hash.hpp>
+
 namespace osrm
 {
 namespace util
@@ -113,7 +118,7 @@ inline bool entersRoundabout(const extractor::guidance::TurnInstruction instruct
            instruction.type == extractor::guidance::TurnType::EnterRoundaboutIntersectionAtExit ||
            instruction.type == extractor::guidance::TurnType::EnterAndExitRoundabout ||
            instruction.type == extractor::guidance::TurnType::EnterAndExitRotary ||
-            instruction.type == extractor::guidance::TurnType::EnterAndExitRotary);
+            instruction.type == extractor::guidance::TurnType::EnterAndExitRoundaboutIntersection);
 }

 inline bool leavesRoundabout(const extractor::guidance::TurnInstruction instruction)
@@ -131,6 +136,128 @@ inline bool staysOnRoundabout(const extractor::guidance::TurnInstruction instruc
    return instruction.type == extractor::guidance::TurnType::StayOnRoundabout;
 }

+// Name Change Logic
+// Used both during Extraction as well as during Post-Processing
+
+inline std::pair<std::string, std::string> getPrefixAndSuffix(const std::string &data)
+{
+    const auto suffix_pos = data.find_last_of(' ');
+    if (suffix_pos == std::string::npos)
+        return {};
+
+    const auto prefix_pos = data.find_first_of(' ');
+    auto result = std::make_pair(data.substr(0, prefix_pos), data.substr(suffix_pos + 1));
+    boost::to_lower(result.first);
+    boost::to_lower(result.second);
+    return result;
+}
+
+// Note: there is an overload without suffix checking below.
+// (that's the reason we template the suffix table here)
+template <typename SuffixTable>
+inline bool requiresNameAnnounced(const std::string &from_name,
+                                  const std::string &from_ref,
+                                  const std::string &to_name,
+                                  const std::string &to_ref,
+                                  const SuffixTable &suffix_table)
+{
+    // first is empty and the second is not
+    if ((from_name.empty() && from_ref.empty()) && !(to_name.empty() && to_ref.empty()))
+        return true;
+
+    // FIXME, handle in profile to begin with?
+    // Input for this function should be a struct separating streetname, suffix (e.g. road,
+    // boulevard, North, West ...), and a list of references
+
+    // check similarity of names
+    const auto names_are_empty = from_name.empty() && to_name.empty();
+    const auto name_is_contained =
+        boost::starts_with(from_name, to_name) || boost::starts_with(to_name, from_name);
+
+    const auto checkForPrefixOrSuffixChange = [](
+        const std::string &first, const std::string &second, const SuffixTable &suffix_table) {
+
+        const auto first_prefix_and_suffixes = getPrefixAndSuffix(first);
+        const auto second_prefix_and_suffixes = getPrefixAndSuffix(second);
+
+        // reverse strings, get suffices and reverse them to get prefixes
+        const auto checkTable = [&](const std::string &str) {
+            return str.empty() || suffix_table.isSuffix(str);
+        };
+
+        const auto getOffset = [](const std::string &str) -> std::size_t {
+            if (str.empty())
+                return 0;
+            else
+                return str.length() + 1;
+        };
+
+        const bool is_prefix_change = [&]() -> bool {
+            if (!checkTable(first_prefix_and_suffixes.first))
+                return false;
+            if (!checkTable(second_prefix_and_suffixes.first))
+                return false;
+            return !first.compare(getOffset(first_prefix_and_suffixes.first),
+                                  std::string::npos,
+                                  second,
+                                  getOffset(second_prefix_and_suffixes.first),
+                                  std::string::npos);
+        }();
+
+        const bool is_suffix_change = [&]() -> bool {
+            if (!checkTable(first_prefix_and_suffixes.second))
+                return false;
+            if (!checkTable(second_prefix_and_suffixes.second))
+                return false;
+            return !first.compare(0,
+                                  first.length() - getOffset(first_prefix_and_suffixes.second),
+                                  second,
+                                  0,
+                                  second.length() - getOffset(second_prefix_and_suffixes.second));
+        }();
+
+        return is_prefix_change || is_suffix_change;
+    };
+
+    const auto is_suffix_change = checkForPrefixOrSuffixChange(from_name, to_name, suffix_table);
+    const auto names_are_equal = from_name == to_name || name_is_contained || is_suffix_change;
+    const auto name_is_removed = !from_name.empty() && to_name.empty();
+    // references are contained in one another
+    const auto refs_are_empty = from_ref.empty() && to_ref.empty();
+    const auto ref_is_contained =
+        from_ref.empty() || to_ref.empty() ||
+        (from_ref.find(to_ref) != std::string::npos || to_ref.find(from_ref) != std::string::npos);
+    const auto ref_is_removed = !from_ref.empty() && to_ref.empty();
+
+    const auto obvious_change =
+        (names_are_empty && refs_are_empty) || (names_are_equal && ref_is_contained) ||
+        (names_are_equal && refs_are_empty) || (ref_is_contained && name_is_removed) ||
+        (names_are_equal && ref_is_removed) || is_suffix_change;
+
+    const auto needs_announce =
+        // " (Ref)" -> "Name "
+        (from_name.empty() && !from_ref.empty() && !to_name.empty() && to_ref.empty());
+
+    return !obvious_change || needs_announce;
+}
+
+// Overload without suffix checking
+inline bool requiresNameAnnounced(const std::string &from_name,
+                                  const std::string &from_ref,
+                                  const std::string &to_name,
+                                  const std::string &to_ref)
+{
+    // Dummy since we need to provide a SuffixTable but do not have the data for it.
+    // (Guidance Post-Processing does not keep the suffix table around at the moment)
+    struct NopSuffixTable final
+    {
+        NopSuffixTable(){}
+        bool isSuffix(const std::string &) const { return false; }
+    } static const table;
+
+    return requiresNameAnnounced(from_name, from_ref, to_name, to_ref, table);
+}
+
 } // namespace guidance
 } // namespace util
 } // namespace osrm
@@ -23,7 +23,13 @@ class NameTable

  public:
    NameTable(const std::string &filename);
+
+    // This class provides a limited view over all the string data we serialize out.
+    // The following functions are a subset of what is available.
+    // See the data facades for they provide full access to this serialized string data.
+    // (at time of writing this: get{Name,Ref,Pronunciation,Destinations}ForID(name_id);)
    std::string GetNameForID(const unsigned name_id) const;
+    std::string GetRefForID(const unsigned name_id) const;
 };
 } // namespace util
 } // namespace osrm
@@ -7,9 +7,11 @@
    "chalk": "^1.1.3",
    "cucumber": "^1.2.1",
    "d3-queue": "^2.0.3",
+    "mkdirp": "^0.5.1",
    "node-timeout": "0.0.4",
    "polyline": "^0.2.0",
    "request": "^2.69.0",
+    "rimraf": "^2.5.4",
    "xmlbuilder": "^4.2.1"
  },
  "bin": {
@@ -238,6 +238,10 @@ function way_function (way, result)
    return
  end

+  -- default to driving mode, may get overwritten below
+  result.forward_mode = mode.driving
+  result.backward_mode = mode.driving
+
  -- we dont route over areas
  local area = way:get_value_by_key("area")
  if ignore_areas and area and "yes" == area then
@@ -245,10 +249,58 @@ function way_function (way, result)
  end

  -- respect user-preference for HOV-only ways
-  local hov = way:get_value_by_key("hov")
-  if ignore_hov_ways and hov and "designated" == hov then
-    return
-  end
+  if ignore_hov_ways then
+    local hov = way:get_value_by_key("hov")
+    if hov and "designated" == hov then
+      return
+    end
+
+    -- also respect user-preference for HOV-only ways when all lanes are HOV-designated
+    local function has_all_designated_hov_lanes(lanes)
+      local all = true
+      for lane in lanes:gmatch("(%w+)") do
+        if lane and lane ~= "designated" then
+          all = false
+          break
+        end
+      end
+      return all
+    end
+
+    local hov_lanes = way:get_value_by_key("hov:lanes")
+    local hov_lanes_forward = way:get_value_by_key("hov:lanes:forward")
+    local hov_lanes_backward = way:get_value_by_key("hov:lanes:backward")
+
+    local hov_all_designated = hov_lanes and hov_lanes ~= ""
+                               and has_all_designated_hov_lanes(hov_lanes)
+
+    local hov_all_designated_forward = hov_lanes_forward and hov_lanes_forward ~= ""
+                                       and has_all_designated_hov_lanes(hov_lanes_forward)
+
+    local hov_all_designated_backward = hov_lanes_backward and hov_lanes_backward ~= ""
+                                        and has_all_designated_hov_lanes(hov_lanes_backward)
+
+    -- forward/backward lane depend on a way's direction
+    local oneway = way:get_value_by_key("oneway")
+    local reverse = oneway and oneway == "-1"
+
+    if hov_all_designated or hov_all_designated_forward then
+      if reverse then
+        result.backward_mode = mode.inaccessible
+      else
+        result.forward_mode = mode.inaccessible
+      end
+    end
+
+    if hov_all_designated_backward then
+      if reverse then
+        result.forward_mode = mode.inaccessible
+      else
+        result.backward_mode = mode.inaccessible
+      end
+    end
+
+  end -- hov handling

  -- respect user-preference for toll=yes ways
  local toll = way:get_value_by_key("toll")
@@ -278,9 +330,6 @@ function way_function (way, result)
    return
  end

-  result.forward_mode = mode.driving
-  result.backward_mode = mode.driving
-
  -- handling ferries and piers
  local route_speed = speed_profile[route]
  if (route_speed and route_speed > 0) then
@@ -21,8 +21,12 @@ function way_function (way, result)
 end

 function source_function ()
+  local path = os.getenv('OSRM_RASTER_SOURCE')
+  if not path then
+    path = "rastersource.asc"
+  end
  raster_source = sources:load(
-    "../test/rastersource.asc",
+    path,
    0,    -- lon_min
    0.1,  -- lon_max
    0,    -- lat_min
@@ -21,8 +21,12 @@ function way_function (way, result)
 end

 function source_function ()
+  local path = os.getenv('OSRM_RASTER_SOURCE')
+  if not path then
+    path = "rastersource.asc"
+  end
  raster_source = sources:load(
-    "../test/rastersource.asc",
+    path,
    0,    -- lon_min
    0.1,  -- lon_max
    0,    -- lat_min
@@ -1,5 +1,5 @@
-#include "engine/engine.hpp"
 #include "engine/api/route_parameters.hpp"
+#include "engine/engine.hpp"
 #include "engine/engine_config.hpp"
 #include "engine/status.hpp"

@@ -21,6 +21,7 @@
 #include <boost/assert.hpp>
 #include <boost/interprocess/sync/named_condition.hpp>
 #include <boost/interprocess/sync/scoped_lock.hpp>
+#include <boost/interprocess/sync/sharable_lock.hpp>
 #include <boost/thread/lock_types.hpp>

 #include <algorithm>
@@ -28,68 +29,17 @@
 #include <utility>
 #include <vector>

-namespace osrm
-{
-namespace engine
-{
-struct Engine::EngineLock
-{
-    // will only be initialized if shared memory is used
-    storage::SharedBarriers barrier;
-    // decrease number of concurrent queries
-    void DecreaseQueryCount();
-    // increase number of concurrent queries
-    void IncreaseQueryCount();
-};
-
-// decrease number of concurrent queries
-void Engine::EngineLock::DecreaseQueryCount()
-{
-    // lock query
-    boost::interprocess::scoped_lock<boost::interprocess::named_mutex> query_lock(
-        barrier.query_mutex);
-
-    // decrement query count
-    --(barrier.number_of_queries);
-    BOOST_ASSERT_MSG(0 <= barrier.number_of_queries, "invalid number of queries");
-
-    // notify all processes that were waiting for this condition
-    if (0 == barrier.number_of_queries)
-    {
-        barrier.no_running_queries_condition.notify_all();
-    }
-}
-
-// increase number of concurrent queries
-void Engine::EngineLock::IncreaseQueryCount()
-{
-    // lock update pending
-    boost::interprocess::scoped_lock<boost::interprocess::named_mutex> pending_lock(
-        barrier.pending_update_mutex);
-
-    // lock query
-    boost::interprocess::scoped_lock<boost::interprocess::named_mutex> query_lock(
-        barrier.query_mutex);
-
-    // unlock update pending
-    pending_lock.unlock();
-
-    // increment query count
-    ++(barrier.number_of_queries);
-}
-} // ns engine
-} // ns osrm
-
 namespace
 {
 // Abstracted away the query locking into a template function
 // Works the same for every plugin.
 template <typename ParameterT, typename PluginT, typename ResultT>
-osrm::engine::Status RunQuery(const std::unique_ptr<osrm::engine::Engine::EngineLock> &lock,
-                              osrm::engine::datafacade::BaseDataFacade &facade,
-                              const ParameterT &parameters,
-                              PluginT &plugin,
-                              ResultT &result)
+osrm::engine::Status
+RunQuery(const std::unique_ptr<osrm::storage::SharedBarriers> &lock,
+         osrm::engine::datafacade::BaseDataFacade &facade,
+         const ParameterT &parameters,
+         PluginT &plugin,
+         ResultT &result)
 {
    if (!lock)
    {
@@ -97,7 +47,10 @@ osrm::engine::Status RunQuery(const std::unique_ptr<osrm::engine::Engine::Engine
    }

    BOOST_ASSERT(lock);
-    lock->IncreaseQueryCount();
+    // this locks aquires shared ownership of the query mutex: other requets are allowed
+    // to run, but data updates need to wait for all queries to finish until they can aquire an exclusive lock
+    boost::interprocess::sharable_lock<boost::interprocess::named_sharable_mutex> query_lock(
+        lock->query_mutex);

    auto &shared_facade = static_cast<osrm::engine::datafacade::SharedDataFacade &>(facade);
    shared_facade.CheckAndReloadFacade();
@@ -107,7 +60,6 @@ osrm::engine::Status RunQuery(const std::unique_ptr<osrm::engine::Engine::Engine

    osrm::engine::Status status = plugin.HandleRequest(parameters, result);

-    lock->DecreaseQueryCount();
    return status;
 }

@@ -124,11 +76,12 @@ namespace osrm
 namespace engine
 {

-Engine::Engine(EngineConfig &config)
+Engine::Engine(const EngineConfig &config)
+    : lock(config.use_shared_memory ? std::make_unique<storage::SharedBarriers>()
+                                    : std::unique_ptr<storage::SharedBarriers>())
 {
    if (config.use_shared_memory)
    {
-        lock = util::make_unique<EngineLock>();
        query_data_facade = util::make_unique<datafacade::SharedDataFacade>();
    }
    else
@@ -157,32 +110,32 @@ Engine::~Engine() = default;
 Engine::Engine(Engine &&) noexcept = default;
 Engine &Engine::operator=(Engine &&) noexcept = default;

-Status Engine::Route(const api::RouteParameters &params, util::json::Object &result)
+Status Engine::Route(const api::RouteParameters &params, util::json::Object &result) const
 {
    return RunQuery(lock, *query_data_facade, params, *route_plugin, result);
 }

-Status Engine::Table(const api::TableParameters &params, util::json::Object &result)
+Status Engine::Table(const api::TableParameters &params, util::json::Object &result) const
 {
    return RunQuery(lock, *query_data_facade, params, *table_plugin, result);
 }

-Status Engine::Nearest(const api::NearestParameters &params, util::json::Object &result)
+Status Engine::Nearest(const api::NearestParameters &params, util::json::Object &result) const
 {
    return RunQuery(lock, *query_data_facade, params, *nearest_plugin, result);
 }

-Status Engine::Trip(const api::TripParameters &params, util::json::Object &result)
+Status Engine::Trip(const api::TripParameters &params, util::json::Object &result) const
 {
    return RunQuery(lock, *query_data_facade, params, *trip_plugin, result);
 }

-Status Engine::Match(const api::MatchParameters &params, util::json::Object &result)
+Status Engine::Match(const api::MatchParameters &params, util::json::Object &result) const
 {
    return RunQuery(lock, *query_data_facade, params, *match_plugin, result);
 }

-Status Engine::Tile(const api::TileParameters &params, std::string &result)
+Status Engine::Tile(const api::TileParameters &params, std::string &result) const
 {
    return RunQuery(lock, *query_data_facade, params, *tile_plugin, result);
 }
@@ -83,18 +83,6 @@ bool isCollapsableInstruction(const TurnInstruction instruction)

 bool compatible(const RouteStep &lhs, const RouteStep &rhs) { return lhs.mode == rhs.mode; }

-double nameSegmentLength(std::size_t at, const std::vector<RouteStep> &steps)
-{
-    double result = steps[at].distance;
-    while (at + 1 < steps.size() && steps[at + 1].name_id == steps[at].name_id)
-    {
-        ++at;
-        result += steps[at].distance;
-    }
-
-    return result;
-}
-
 // invalidate a step and set its content to nothing
 void invalidateStep(RouteStep &step) { step = getInvalidRouteStep(); }

@@ -129,6 +117,28 @@ double turn_angle(const double entry_bearing, const double exit_bearing)
    return angle > 360 ? angle - 360 : angle;
 }

+// Checks if name change happens the user wants to know about.
+// Treats e.g. "Name (Ref)" -> "Name" changes still as same name.
+bool isNoticeableNameChange(const RouteStep &lhs, const RouteStep &rhs)
+{
+    // TODO: at some point we might want to think about pronunciation here.
+    // Also rotary_name is not handled at the moment.
+    return util::guidance::requiresNameAnnounced(lhs.name, lhs.ref, rhs.name, rhs.ref);
+}
+
+double nameSegmentLength(std::size_t at, const std::vector<RouteStep> &steps)
+{
+    BOOST_ASSERT(at < steps.size());
+
+    double result = steps[at].distance;
+    while (at + 1 < steps.size() && !isNoticeableNameChange(steps[at], steps[at + 1]))
+    {
+        at += 1;
+        result += steps[at].distance;
+    }
+    return result;
+}
+
 OSRM_ATTR_WARN_UNUSED
 RouteStep forwardInto(RouteStep destination, const RouteStep &source)
 {
@@ -203,6 +213,8 @@ bool setUpRoundabout(RouteStep &step)
        instruction.type == TurnType::EnterRoundaboutAtExit ||
        instruction.type == TurnType::EnterRoundaboutIntersectionAtExit)
    {
+        // Here we consider an actual entry, not an exit. We simply have to count the additional
+        // exit
        step.maneuver.exit = 1;
        // prevent futher special case handling of these two.
        if (instruction.type == TurnType::EnterRotaryAtExit)
@@ -215,6 +227,7 @@ bool setUpRoundabout(RouteStep &step)

    if (leavesRoundabout(instruction))
    {
+        // This set-up, even though it looks the same, is actually looking at entering AND exiting
        step.maneuver.exit = 1; // count the otherwise missing exit

        // prevent futher special case handling of these two.
@@ -406,12 +419,16 @@ void collapseTurnAt(std::vector<RouteStep> &steps,
        if (compatible(one_back_step, current_step))
        {
            steps[one_back_index] = elongate(std::move(steps[one_back_index]), steps[step_index]);
+
            if ((TurnType::Continue == one_back_step.maneuver.instruction.type ||
                 TurnType::Suppressed == one_back_step.maneuver.instruction.type) &&
-                current_step.name_id != steps[two_back_index].name_id)
+                isNoticeableNameChange(steps[two_back_index], current_step))
+            {
+
                steps[one_back_index].maneuver.instruction.type = TurnType::Turn;
+            }
            else if (TurnType::Turn == one_back_step.maneuver.instruction.type &&
-                     current_step.name_id == steps[two_back_index].name_id)
+                     !isNoticeableNameChange(steps[two_back_index], current_step))
            {
                steps[one_back_index].maneuver.instruction.type = TurnType::Continue;

@@ -457,7 +474,7 @@ void collapseTurnAt(std::vector<RouteStep> &steps,
    {
        BOOST_ASSERT(two_back_index < steps.size());
        // the simple case is a u-turn that changes directly into the in-name again
-        const bool direct_u_turn = steps[two_back_index].name_id == current_step.name_id;
+        const bool direct_u_turn = !isNoticeableNameChange(steps[two_back_index], current_step);

        // however, we might also deal with a dual-collapse scenario in which we have to
        // additionall collapse a name-change as welll
@@ -468,7 +485,7 @@ void collapseTurnAt(std::vector<RouteStep> &steps,
             isCollapsableInstruction(steps[next_step_index].maneuver.instruction));
        const bool u_turn_with_name_change =
            continues_with_name_change &&
-            steps[next_step_index].name_id == steps[two_back_index].name_id;
+            !isNoticeableNameChange(steps[two_back_index], steps[next_step_index]);

        if (direct_u_turn || u_turn_with_name_change)
        {
@@ -792,9 +809,8 @@ std::vector<RouteStep> collapseTurns(std::vector<RouteStep> steps)
                    // Turn Types in the response depend on whether we find the same road name
                    // (sliproad indcating a u-turn) or if we are turning onto a different road, in
                    // which case we use a turn.
-                    if (steps[getPreviousIndex(one_back_index)].name_id ==
-                            steps[step_index].name_id &&
-                        steps[step_index].name_id != EMPTY_NAMEID)
+                    if (!isNoticeableNameChange(steps[getPreviousIndex(one_back_index)],
+                                                steps[step_index]))
                        steps[one_back_index].maneuver.instruction.type = TurnType::Continue;
                    else
                        steps[one_back_index].maneuver.instruction.type = TurnType::Turn;
@@ -828,7 +844,7 @@ std::vector<RouteStep> collapseTurns(std::vector<RouteStep> steps)
        // These have to be handled in post-processing
        else if (isCollapsableInstruction(current_step.maneuver.instruction) &&
                 current_step.maneuver.instruction.type != TurnType::Suppressed &&
-                 steps[getPreviousNameIndex(step_index)].name_id == current_step.name_id &&
+                 !isNoticeableNameChange(steps[getPreviousNameIndex(step_index)], current_step) &&
                 // canCollapseAll is also checking for compatible(step,step+1) for all indices
                 canCollapseAll(getPreviousNameIndex(step_index) + 1, next_step_index))
        {
@@ -853,8 +869,7 @@ std::vector<RouteStep> collapseTurns(std::vector<RouteStep> steps)
            const auto two_back_index = getPreviousIndex(one_back_index);
            BOOST_ASSERT(two_back_index < steps.size());
            // valid, since one_back is collapsable or a turn and therefore not depart:
-            const auto &coming_from_name_id = steps[two_back_index].name_id;
-            if (current_step.name_id == coming_from_name_id)
+            if (!isNoticeableNameChange(steps[two_back_index], current_step))
            {
                if (compatible(one_back_step, steps[two_back_index]))
                {
@@ -886,7 +901,7 @@ std::vector<RouteStep> collapseTurns(std::vector<RouteStep> steps)
            else if (step_index + 2 < steps.size() &&
                     current_step.maneuver.instruction.type == TurnType::NewName &&
                     steps[next_step_index].maneuver.instruction.type == TurnType::NewName &&
-                     one_back_step.name_id == steps[next_step_index].name_id)
+                     !isNoticeableNameChange(one_back_step, steps[next_step_index]))
            {
                if (compatible(steps[step_index], steps[next_step_index]))
                {
@@ -8,6 +8,7 @@
 #include <string>

 #include <boost/assert.hpp>
+#include <boost/numeric/conversion/cast.hpp>

 namespace osrm
 {
@@ -26,7 +27,8 @@ Status NearestPlugin::HandleRequest(const api::NearestParameters &params,
 {
    BOOST_ASSERT(params.IsValid());

-    if (params.number_of_results > max_results)
+    if (max_results > 0 &&
+        (boost::numeric_cast<std::int64_t>(params.number_of_results) > max_results))
    {
        return Error("TooBig",
                     "Number of results " + std::to_string(params.number_of_results) +
@@ -100,7 +100,7 @@ std::vector<util::Coordinate> decodePolyline(const std::string &geometry_string)
            b = geometry_string.at(index++) - 63;
            result |= (b & 0x1f) << shift;
            shift += 5;
-        } while (b >= 0x20);
+        } while (b >= 0x20 && index < len);
        int dlat = ((result & 1) != 0 ? ~(result >> 1) : (result >> 1));
        lat += dlat;

@@ -111,7 +111,7 @@ std::vector<util::Coordinate> decodePolyline(const std::string &geometry_string)
            b = geometry_string.at(index++) - 63;
            result |= (b & 0x1f) << shift;
            shift += 5;
-        } while (b >= 0x20);
+        } while (b >= 0x20 && index < len);
        int dlng = ((result & 1) != 0 ? ~(result >> 1) : (result >> 1));
        lng += dlng;

@@ -142,7 +142,7 @@ void ExtractorCallbacks::ProcessWay(const osmium::Way &input_way, const Extracti

    // FIXME this need to be moved into the profiles
    const guidance::RoadClassification road_classification = parsed_way.road_classification;
-    const auto laneStringToDescription = [](std::string lane_string) -> TurnLaneDescription {
+    const auto laneStringToDescription = [](const std::string &lane_string) -> TurnLaneDescription {
        if (lane_string.empty())
            return {};

@@ -214,7 +214,7 @@ void ExtractorCallbacks::ProcessWay(const osmium::Way &input_way, const Extracti

    // convert the lane description into an ID and, if necessary, remembr the description in the
    // description_map
-    const auto requestId = [&](std::string lane_string) {
+    const auto requestId = [&](const std::string &lane_string) {
        if (lane_string.empty())
            return INVALID_LANE_DESCRIPTIONID;
        TurnLaneDescription lane_description = laneStringToDescription(std::move(lane_string));
@@ -1,11 +1,12 @@
-#include "extractor/guidance/intersection_generator.hpp"
 #include "extractor/guidance/constants.hpp"
+#include "extractor/guidance/intersection_generator.hpp"
 #include "extractor/guidance/toolkit.hpp"

 #include <algorithm>
 #include <iomanip>
 #include <iterator>
 #include <limits>
+#include <unordered_set>
 #include <utility>

 #include <boost/range/algorithm/count_if.hpp>
@@ -268,8 +269,11 @@ bool IntersectionGenerator::CanMerge(const NodeID node_at_intersection,
        return becomes_narrower;
    };

+    const bool is_y_arm_first = isValidYArm(first_index, second_index);
+    const bool is_y_arm_second = isValidYArm(second_index, first_index);
+
    // Only merge valid y-arms
-    if (!isValidYArm(first_index, second_index) || !isValidYArm(second_index, first_index))
+    if (!is_y_arm_first || !is_y_arm_second)
        return false;

    if (angle_between < 60)
@@ -577,10 +581,14 @@ IntersectionGenerator::GetActualNextIntersection(const NodeID starting_node,
    // to prevent endless loops
    const auto termination_node = node_based_graph.GetTarget(via_edge);

-    while (result.size() == 2 &&
-           node_based_graph.GetEdgeData(via_edge).IsCompatibleTo(
-               node_based_graph.GetEdgeData(result[1].turn.eid)))
+    // using a maximum lookahead, we make sure not to end up in some form of loop
+    std::unordered_set<NodeID> visited_nodes;
+    while (visited_nodes.count(node_at_intersection) == 0 &&
+           (result.size() == 2 &&
+            node_based_graph.GetEdgeData(via_edge).IsCompatibleTo(
+                node_based_graph.GetEdgeData(result[1].turn.eid))))
    {
+        visited_nodes.insert(node_at_intersection);
        node_at_intersection = node_based_graph.GetTarget(incoming_edge);
        incoming_edge = result[1].turn.eid;
        result = GetConnectedRoads(node_at_intersection, incoming_edge);
@@ -4,6 +4,7 @@

 #include "util/coordinate_calculation.hpp"
 #include "util/guidance/toolkit.hpp"
+#include "util/guidance/toolkit.hpp"
 #include "util/simple_logger.hpp"

 #include <algorithm>
@@ -90,9 +91,11 @@ TurnInstruction IntersectionHandler::getInstructionForObvious(const std::size_t
        const auto &in_data = node_based_graph.GetEdgeData(via_edge);
        const auto &out_data = node_based_graph.GetEdgeData(road.turn.eid);
        if (in_data.name_id != out_data.name_id &&
-            requiresNameAnnounced(name_table.GetNameForID(in_data.name_id),
-                                  name_table.GetNameForID(out_data.name_id),
-                                  street_name_suffix_table))
+            util::guidance::requiresNameAnnounced(name_table.GetNameForID(in_data.name_id),
+                                                  name_table.GetRefForID(in_data.name_id),
+                                                  name_table.GetNameForID(out_data.name_id),
+                                                  name_table.GetRefForID(out_data.name_id),
+                                                  street_name_suffix_table))
        {
            // obvious turn onto a through street is a merge
            if (through_street)
@@ -597,17 +600,18 @@ std::size_t IntersectionHandler::findObviousTurn(const EdgeID via_edge,
            if (deviation_ratio < DISTINCTION_RATIO / 1.5)
                return 0;

-            // in comparison to another continuing road, we need a better distinction. This prevents
-            // situations where the turn is probably less obvious. An example are places that have a
-            // road with the same name entering/exiting:
-            //
-            //         d
-            //        /
-            //       /
-            // a -- b
-            //       \
-            //        \
-            //         c
+            /* in comparison to another continuing road, we need a better distinction. This prevents
+               situations where the turn is probably less obvious. An example are places that have a
+               road with the same name entering/exiting:
+
+                       d
+                      /
+                     /
+               a -- b
+                     \
+                      \
+                       c
+            */

            if (turn_data.name_id == continue_data.name_id &&
                deviation_ratio < 1.5 * DISTINCTION_RATIO)
@@ -222,48 +222,46 @@ RoundaboutType RoundaboutHandler::getRoundaboutType(const NodeID nid) const
    std::unordered_set<unsigned> roundabout_name_ids;
    std::unordered_set<unsigned> connected_names;

-    const auto getNextOnRoundabout =
-        [this, &roundabout_name_ids, &connected_names](const NodeID node) {
-            EdgeID continue_edge = SPECIAL_EDGEID;
-            for (const auto edge : node_based_graph.GetAdjacentEdgeRange(node))
+    const auto getNextOnRoundabout = [this, &roundabout_name_ids, &connected_names](
+        const NodeID node) {
+        EdgeID continue_edge = SPECIAL_EDGEID;
+        for (const auto edge : node_based_graph.GetAdjacentEdgeRange(node))
+        {
+            const auto &edge_data = node_based_graph.GetEdgeData(edge);
+            if (!edge_data.reversed && edge_data.roundabout)
            {
-                const auto &edge_data = node_based_graph.GetEdgeData(edge);
-                if (!edge_data.reversed && edge_data.roundabout)
+                if (SPECIAL_EDGEID != continue_edge)
                {
-                    if (SPECIAL_EDGEID != continue_edge)
-                    {
-                        // fork in roundabout
-                        return SPECIAL_EDGEID;
-                    }
-
-                    if (EMPTY_NAMEID != edge_data.name_id)
-                    {
-                        bool add = true;
-                        for (auto name_id : roundabout_name_ids)
-                        {
-
-                            if (!requiresNameAnnounced(name_table.GetNameForID(name_id),
-                                                       name_table.GetNameForID(edge_data.name_id),
-                                                       street_name_suffix_table))
-                            {
-                                add = false;
-                                break;
-                            }
-                        }
-                        if (add)
-                            roundabout_name_ids.insert(edge_data.name_id);
-                    }
-
-                    continue_edge = edge;
+                    // fork in roundabout
+                    return SPECIAL_EDGEID;
                }
-                else if (!edge_data.roundabout)
+
+                if (EMPTY_NAMEID != edge_data.name_id)
                {
-                    // remember all connected road names
-                    connected_names.insert(edge_data.name_id);
+
+                    const auto announce = [&](unsigned id) {
+                        return util::guidance::requiresNameAnnounced(
+                            name_table.GetNameForID(id),
+                            name_table.GetRefForID(id),
+                            name_table.GetNameForID(edge_data.name_id),
+                            name_table.GetRefForID(edge_data.name_id),
+                            street_name_suffix_table);
+                    };
+
+                    if (std::all_of(begin(roundabout_name_ids), end(roundabout_name_ids), announce))
+                        roundabout_name_ids.insert(edge_data.name_id);
                }
+
+                continue_edge = edge;
            }
-            return continue_edge;
-        };
+            else if (!edge_data.roundabout)
+            {
+                // remember all connected road names
+                connected_names.insert(edge_data.name_id);
+            }
+        }
+        return continue_edge;
+    };
    // the roundabout radius has to be the same for all locations we look at it from
    // to guarantee this, we search the full roundabout for its vertices
    // and select the three smallest ids
@@ -1,6 +1,6 @@
-#include "extractor/guidance/sliproad_handler.hpp"
 #include "extractor/guidance/constants.hpp"
 #include "extractor/guidance/intersection_scenario_three_way.hpp"
+#include "extractor/guidance/sliproad_handler.hpp"
 #include "extractor/guidance/toolkit.hpp"

 #include "util/guidance/toolkit.hpp"
@@ -56,9 +56,14 @@ operator()(const NodeID, const EdgeID source_edge_id, Intersection intersection)
            auto intersection = intersection_generator(at_node, road.turn.eid);
            auto in_edge = road.turn.eid;
            // skip over traffic lights
-            while (intersection.size() == 2)
+            // to prevent ending up in an endless loop, we remember all visited nodes. This is
+            // necessary, since merging of roads can actually create enterable loops of degree two
+            std::unordered_set<NodeID> visited_nodes;
+            auto node = at_node;
+            while (intersection.size() == 2 && visited_nodes.count(node) == 0)
            {
-                const auto node = node_based_graph.GetTarget(in_edge);
+                visited_nodes.insert(node);
+                node = node_based_graph.GetTarget(in_edge);
                if (node == at_node)
                {
                    // we ended up in a loop without exit
@@ -70,6 +75,11 @@ operator()(const NodeID, const EdgeID source_edge_id, Intersection intersection)
                output_node = node_based_graph.GetTarget(in_edge);
                intersection = intersection_generator(node, in_edge);
            }
+            if (intersection.size() <= 2)
+            {
+                output_node = SPECIAL_NODEID;
+                intersection.clear();
+            }
            return intersection;
        };

@@ -84,7 +94,7 @@ operator()(const NodeID, const EdgeID source_edge_id, Intersection intersection)
            // a one-sided sliproad, however, the non-sliproad can be considered `obvious`. Here we
            // assume that this could be the case and check for a potential sliproad/non-sliproad
            // situation.
-            NodeID intersection_node_one, intersection_node_two;
+            NodeID intersection_node_one = SPECIAL_NODEID, intersection_node_two = SPECIAL_NODEID;
            const auto intersection_following_index_one = findNextIntersectionForRoad(
                intersection_node_id, intersection[1], intersection_node_one);
            const auto intersection_following_index_two = findNextIntersectionForRoad(
@@ -162,6 +172,9 @@ operator()(const NodeID, const EdgeID source_edge_id, Intersection intersection)
    const auto next_road_next_intersection =
        findNextIntersectionForRoad(intersection_node_id, next_road, next_intersection_node);

+    if (next_road_next_intersection.empty())
+        return intersection;
+
    // If we are at a traffic loop at the end of a road, don't consider it a sliproad
    if (intersection_node_id == next_intersection_node)
        return intersection;
@@ -194,13 +207,14 @@ operator()(const NodeID, const EdgeID source_edge_id, Intersection intersection)
            const auto link_data = node_based_graph.GetEdgeData(road.turn.eid);
            // Check if the road continues here
            const bool is_through_street =
+                !target_intersection.empty() &&
                target_intersection.end() !=
-                std::find_if(target_intersection.begin() + 1,
-                             target_intersection.end(),
-                             [this, &link_data](const ConnectedRoad &road) {
-                                 return node_based_graph.GetEdgeData(road.turn.eid).name_id ==
-                                        link_data.name_id;
-                             });
+                    std::find_if(target_intersection.begin() + 1,
+                                 target_intersection.end(),
+                                 [this, &link_data](const ConnectedRoad &road) {
+                                     return node_based_graph.GetEdgeData(road.turn.eid).name_id ==
+                                            link_data.name_id;
+                                 });

            // if the sliproad candidate is a through street, we cannot handle it as a sliproad
            if (is_through_street)
@@ -1,5 +1,3 @@
-#include "util/debug.hpp"
-
 #include "extractor/guidance/turn_lane_augmentation.hpp"
 #include "extractor/guidance/turn_lane_types.hpp"
 #include "util/simple_logger.hpp"
@@ -21,32 +21,32 @@ OSRM &OSRM::operator=(OSRM &&) noexcept = default;

 // Forward to implementation

-engine::Status OSRM::Route(const engine::api::RouteParameters &params, util::json::Object &result)
+engine::Status OSRM::Route(const engine::api::RouteParameters &params, util::json::Object &result) const
 {
    return engine_->Route(params, result);
 }

-engine::Status OSRM::Table(const engine::api::TableParameters &params, json::Object &result)
+engine::Status OSRM::Table(const engine::api::TableParameters &params, json::Object &result) const
 {
    return engine_->Table(params, result);
 }

-engine::Status OSRM::Nearest(const engine::api::NearestParameters &params, json::Object &result)
+engine::Status OSRM::Nearest(const engine::api::NearestParameters &params, json::Object &result) const
 {
    return engine_->Nearest(params, result);
 }

-engine::Status OSRM::Trip(const engine::api::TripParameters &params, json::Object &result)
+engine::Status OSRM::Trip(const engine::api::TripParameters &params, json::Object &result) const
 {
    return engine_->Trip(params, result);
 }

-engine::Status OSRM::Match(const engine::api::MatchParameters &params, json::Object &result)
+engine::Status OSRM::Match(const engine::api::MatchParameters &params, json::Object &result) const
 {
    return engine_->Match(params, result);
 }

-engine::Status OSRM::Tile(const engine::api::TileParameters &params, std::string &result)
+engine::Status OSRM::Tile(const engine::api::TileParameters &params, std::string &result) const
 {
    return engine_->Tile(params, result);
 }
@@ -42,11 +42,6 @@ namespace osrm
 namespace storage
 {

-using RTreeLeaf = engine::datafacade::BaseDataFacade::RTreeLeaf;
-using RTreeNode =
-    util::StaticRTree<RTreeLeaf, util::ShM<util::Coordinate, true>::vector, true>::TreeNode;
-using QueryGraph = util::StaticGraph<contractor::QueryEdge::EdgeData>;
-
 // delete a shared memory region. report warning if it could not be deleted
 void deleteRegion(const SharedDataType region)
 {
@@ -76,6 +71,11 @@ void deleteRegion(const SharedDataType region)
    }
 }

+using RTreeLeaf = engine::datafacade::BaseDataFacade::RTreeLeaf;
+using RTreeNode =
+    util::StaticRTree<RTreeLeaf, util::ShM<util::Coordinate, true>::vector, true>::TreeNode;
+using QueryGraph = util::StaticGraph<contractor::QueryEdge::EdgeData>;
+
 Storage::Storage(StorageConfig config_) : config(std::move(config_)) {}

 int Storage::Run()
@@ -738,20 +738,15 @@ int Storage::Run()
    SharedDataTimestamp *data_timestamp_ptr =
        static_cast<SharedDataTimestamp *>(data_type_memory->Ptr());

-    boost::interprocess::scoped_lock<boost::interprocess::named_mutex> query_lock(
-        barrier.query_mutex);
-
-    // notify all processes that were waiting for this condition
-    if (0 < barrier.number_of_queries)
    {
-        barrier.no_running_queries_condition.wait(query_lock);
-    }
+        boost::interprocess::scoped_lock<boost::interprocess::named_sharable_mutex> query_lock(barrier.query_mutex);

-    data_timestamp_ptr->layout = layout_region;
-    data_timestamp_ptr->data = data_region;
-    data_timestamp_ptr->timestamp += 1;
-    deleteRegion(previous_data_region);
-    deleteRegion(previous_layout_region);
+        data_timestamp_ptr->layout = layout_region;
+        data_timestamp_ptr->data = data_region;
+        data_timestamp_ptr->timestamp += 1;
+        deleteRegion(previous_data_region);
+        deleteRegion(previous_layout_region);
+    }
    util::SimpleLogger().Write() << "all data loaded";

    return EXIT_SUCCESS;
@@ -10,6 +10,5 @@ int main()
    osrm::storage::SharedBarriers barrier;
    barrier.pending_update_mutex.unlock();
    barrier.query_mutex.unlock();
-    barrier.update_mutex.unlock();
    return 0;
 }
@@ -22,6 +22,9 @@ NameTable::NameTable(const std::string &filename)

    name_stream >> m_name_table;

+    if (!name_stream)
+        throw exception("Unable to deserialize RangeTable for NameTable");
+
    unsigned number_of_chars = 0;
    name_stream.read(reinterpret_cast<char *>(&number_of_chars), sizeof(number_of_chars));
    if (!name_stream)
@@ -64,5 +67,23 @@ std::string NameTable::GetNameForID(const unsigned name_id) const
    }
    return result;
 }
+
+std::string NameTable::GetRefForID(const unsigned name_id) const
+{
+    // Way string data is stored in blocks based on `name_id` as follows:
+    //
+    // | name | destination | pronunciation | ref |
+    //                                      ^     ^
+    //                                      [range)
+    //                                       ^ name_id + 3
+    //
+    // `name_id + offset` gives us the range of chars.
+    //
+    // Offset 0 is name, 1 is destination, 2 is pronunciation, 3 is ref.
+    // See datafacades and extractor callbacks for details.
+    const constexpr auto OFFSET_REF = 3u;
+    return GetNameForID(name_id + OFFSET_REF);
+}
+
 } // namespace util
 } // namespace osrm
@@ -62,6 +62,24 @@
            "object_types": [ "way" ],
            "description": "Roads with hov=designated are ignored by default."
        },
+        {
+            "key": "hov:lanes",
+            "value": "designated",
+            "object_types": [ "way" ],
+            "description": "Roads with hov:lanes all-designated are ignored by default."
+        },
+        {
+            "key": "hov:lanes:forward",
+            "value": "designated",
+            "object_types": [ "way" ],
+            "description": "Roads with hov:lanes:forward all-designated are ignored by default."
+        },
+        {
+            "key": "hov:lanes:backward",
+            "value": "designated",
+            "object_types": [ "way" ],
+            "description": "Roads with hov:lanes:backward all-designated are ignored by default."
+        },
        {
            "key": "toll",
            "value": "yes",
Author	SHA1	Message	Date
Patrick Niklaus	f04377abff	Update changelog	2016-10-10 12:31:26 +02:00
Patrick Niklaus	e82f678665	Use a shared (!) reader writer lock to protect CURRENT_REGIONS This fixes issue #3016.	2016-10-10 12:26:14 +02:00
Daniel J. Hofmann	4db1b7bea5	Implements Cucumber shared vs. static library detection, fixes #2999	2016-10-04 14:15:37 +02:00
Daniel J. Hofmann	32c5f14ed3	Makes the OSRM interface threadsafe. Technically speaking we're changing the `libosrm` API. But since we're only lifting restrictions by marking the API threadsafe, we should be fine here.	2016-10-04 09:38:51 +02:00
Daniel J. Hofmann	cbd88c63b9	Re-introduces the old `RouteParameters` ctor for API compatibility, see #2978	2016-10-03 20:02:11 +02:00
Daniel J. Hofmann	621e302a38	Disables ld.gold on binutils < 2.26, resolves #2984	2016-10-03 15:47:29 +02:00
Moritz Kobitzsch	1db794b2cb	prepare 5.4.0-rc.7	2016-09-30 14:48:32 +02:00
Moritz Kobitzsch	a1ccedb5bf	re-introduce space into summaries	2016-09-30 14:46:34 +02:00
Moritz Kobitzsch	189f8c3265	move summaries to car profile, test for references, use boost adaptors	2016-09-30 11:07:46 +02:00
karenzshea	12238ebb52	handle empty names in summaries	2016-09-30 11:06:39 +02:00
Moritz Kobitzsch	b033ac9f0b	initialize 5.4.0-rc.6 changelog	2016-09-29 17:44:28 +02:00
Moritz Kobitzsch	a411589092	fix polyline decoding	2016-09-29 17:42:29 +02:00
Johan Uhle	45673581ea	Changelog: Fix typo for 5.4.0-rc.5	2016-09-26 14:47:21 +02:00
Moritz Kobitzsch	f9ecdca5d5	add changelog	2016-09-21 12:57:34 +02:00
Daniel J. Hofmann	accdbe92f0	Reduce NewName Instructructions / Name Changes With @karenzshea's name / ref split (ref. #2857) in master we want to make use of it and reduce `NewName` instructions when ever possible. This is a first step towards #2744 by using the already existing name change heuristic from the extractor now in post-processing as well. Limitations: at the moment we don't have the `SuffixTable` in post-processing; this would require us serializing and subsequently deserializing the table, passing it through from the profiles to the API.	2016-09-21 12:56:38 +02:00
Moritz Kobitzsch	644d082cb9	add changelog entry for profile change	2016-09-21 11:48:09 +02:00
Daniel J. Hofmann	f948fb081b	Handle HOV designated-only Lanes, resolves #2929 In https://github.com/Project-OSRM/osrm-backend/issues/2711 we made `hov=designated` routability configurable. We want to handle designated-only lanes in the same way. Example: hov:lanes:forward=designated\|designated hov:lanes:backward=designated should have more or less the same effects as hov=designated In contrast hov:lanes:forward=designated\|no\|yes hov:lanes:backward=yes should not be handled. See https://github.com/Project-OSRM/osrm-backend/issues/2711 for explanation wrt. tag semantics with the difference that backward/forward only set the backward/forward mode to inaccessible. References: - http://wiki.openstreetmap.org/wiki/Key:hov#hov:lanes.3D.2A	2016-09-21 11:47:03 +02:00
Michael Krasnyk	42445dc10c	Fix "Error: write after end" and removed @bug tags https://nodejs.org/api/child_process.html#child_process_event_exit Note that when the 'exit' event is triggered, child process stdio streams might still be open.	2016-09-21 10:54:11 +02:00
Patrick Niklaus	c1041e5a64	Close GH-2795: Rewrite cucumber test caching (and support logic). Fixes #2745	2016-09-20 08:36:20 +02:00
Moritz Kobitzsch	b2bc169547	add changelog entry	2016-09-19 14:08:54 +02:00
Moritz Kobitzsch	5713460331	Prevent loops in extraction based on merge	2016-09-19 14:06:38 +02:00
Daniel J. Hofmann	cc2e26fd52	Fixes sign mismatch in Nearest limit comparison	2016-09-16 15:43:21 +02:00
Moritz Kobitzsch	c0bd9da645	remove dead code	2016-09-16 07:30:22 +02:00
Moritz Kobitzsch	7348e7ca55	fix may be uninitialised warnings	2016-09-15 13:24:58 +02:00
Daniel J. Hofmann	519dc986f9	Do Not Copy Lane Strings Into Fn	2016-09-15 08:22:39 +02:00
Moritz Kobitzsch	1ae2964b0f	fix roundabout-intersection on immediate exit	2016-09-13 19:28:20 +02:00