Update changelog

Flip source and target coordinates for segments in only backward direction
Fix unnecessary duplication of backward segments for non-split edges
2017-03-29 22:08:39 +00:00 · 2017-03-29 22:06:34 +00:00 · 2017-03-29 22:06:22 +00:00 · 2017-03-21 13:43:48 -07:00 · 2017-03-09 12:05:01 +00:00 · 2017-03-09 11:49:10 +00:00
22 changed files with 598 additions and 103 deletions
@@ -1,3 +1,29 @@
+# 5.6.5
+  - Changes from 5.6.4
+    - Bugfixes
+      - Fix #3873 excessive usage of disk and memory during osrm-extract because edge deduplication was broken
+
+# 5.6.4
+  - Changes from 5.6.3
+    - Bugfixes
+      - Fix #3838 throws errors if write streams have failed
+
+# 5.6.3
+  - Changes from 5.6.2
+    - Bugfixes
+      - #3790 Fix incorrect speed values in tile plugin
+
+# 5.6.2
+  - Changes from 5.6.1
+    - Bugfixes
+      - Fix incorrect forward datasources getter in facade
+      - Fix include `access=private` non-car roads in the car profile
+
+# 5.6.1
+  - Changes from 5.6.0
+    - Bugfixes
+      - Fix #3754 restricted access roads not penalized if restriction begins non at an intersection
+
 # 5.6.0
  - Changes from 5.5
    - Bugfixes
@@ -165,7 +165,7 @@ Feature: Car - Restricted access
            | motorway    | yes          | permissive |               | private     | x    |       | implied oneway  |
            | trunk       | agricultural | designated | permissive    | no          |      |       |   |
            | pedestrian  |              |            |               |             |      |       |   |
-            | pedestrian  |              |            |               | destination | x    | x     |   |
+            | pedestrian  |              |            |               | destination |      |       | temporary disabled #3773 |

    Scenario: Car - Ignore access tags for other modes
        Then routability should be
@@ -202,8 +202,9 @@ Feature: Car - Restricted access
        Then routability should be
            | highway | hov:lanes:forward      | hov:lanes:backward     | hov:lanes              | oneway | forw | backw | forw_rate | backw_rate |
            | primary | designated             | designated             |                        |        | x    | x     | 18        | 18         |
-            | primary |                        | designated             |                        |        | x    | x     | 18        | 18         |
-            | primary | designated             |                        |                        |        | x    | x     | 18        | 18         |
+            # This test is flaky because non-deterministic turn generation sometimes emits a NoTurn here that is marked as restricted. #3769
+            #| primary |                        | designated             |                        |        | x    | x     | 18        | 18         |
+            #| primary | designated             |                        |                        |        | x    | x     | 18        | 18         |
            | primary | designated\|designated | designated\|designated |                        |        | x    | x     | 18        | 18         |
            | primary | designated\|no         | designated\|no         |                        |        | x    | x     | 18        | 18         |
            | primary | yes\|no                | yes\|no                |                        |        | x    | x     | 18        | 18         |
@@ -261,9 +262,11 @@ Feature: Car - Restricted access

    Scenario: Car - a way with a list of tags
        Then routability should be
-            | highway | motor_vehicle            | motor_vehicle:forward | motor_vehicle:backward | forw | backw |
-            | footway |                          |                       | destination            |      | x     |
-            | track   | destination;agricultural | destination           |                        | x    | x     |
+            | highway | motor_vehicle            | motor_vehicle:forward | motor_vehicle:backward | forw | backw | # |
+            | primary |                          | no                    | destination            |      | x     |   |
+            | primary | destination;agricultural | destination           |                        | x    | x     |   |
+            | footway |                          |                       | destination            |      |       | temporary #3373 |
+            | track   | destination;agricultural | destination           |                        |      | x     | temporary #3373 |

    Scenario: Car - Don't route over steps even if marked as accessible
        Then routability should be
@@ -279,3 +282,24 @@ Feature: Car - Restricted access
            | steps      | permissive |       |
            | footway    | permissive | x     |
            | garbagetag | permissive | x     |
+
+    Scenario: Car - Access private blacklist
+        Then routability should be
+            | highway    | access     | bothw |
+            | footway    | yes        |   x   |
+            | pedestrian | private    |       |
+            | footway    | private    |       |
+            | service    | private    |       |
+            | cycleway   | private    |       |
+            | track      | private    |       |
+
+    Scenario: Car - Access blacklist
+        Then routability should be
+            | highway    | access       | bothw |
+            | primary    |              |   x   |
+            | primary    | customer     |       |
+            | primary    | emergency    |       |
+            | primary    | forestry     |       |
+            | primary    | agricultural |       |
+            | primary    | psv          |       |
+            | primary    | no           |       |
@@ -81,3 +81,53 @@ Feature: Car - Destination only, no passing through
            | e    | a  | de,cd,bc,ab,ab |
            | b    | d  | bc,cd,cd       |
            | d    | b  | cd,bc,bc       |
+
+    Scenario: Car - Routing around a way that becomes destination only
+        Given the node map
+            """
+            b
+             \
+              |
+              e++d++++++c--i
+              |             \
+               \             h--a
+                \            |
+                 \___________g
+            """
+
+        And the ways
+            | nodes | access      | oneway |
+            | ah    |             | no     |
+            | ihg   |             | no     |
+            | eg    |             | no     |
+            | icde  |             | no     |
+            | cde   | destination | no     |
+            | eb    |             | no     |
+
+        When I route I should get
+            | from | to | route           | # |
+            | i    | b  | ihg,eg,eb,eb    | # goes around access=destination, though restricted way starts at two node intersection |
+            | b    | d  | eb,cde,cde      | # ends in restricted way correctly     |
+            | b    | i  | eb,eg,ihg,ihg   | # goes around restricted way correctly |
+
+    Scenario: Car - Routing around a way that becomes destination only
+        Given the node map
+            """
+               a---c---b
+                   +    \
+                   +    |
+                   d    |
+                    \___e
+            """
+
+        And the ways
+            | nodes | access      | oneway |
+            | acbe  |             | no     |
+            | cd    | destination | no     |
+            | de    |             | no     |
+
+        When I route I should get
+            | from | to | route         |
+            | e    | a  | acbe,acbe     |
+            | d    | a  | de,acbe,acbe  |
+            | c    | d  | cd,cd         |
@@ -28,13 +28,14 @@ module.exports = function () {
                        directions.filter(d => headers.has(d + '_rate')).forEach((direction) => {
                            var rate = direction + '_rate';
                            var want = row[rate];
+
                            switch (true) {
                            case '' === want:
                                outputRow[rate] = result[direction].status ?
                                    result[direction].status.toString() : '';
                                break;
                            case /^\d+$/.test(want):
-                                if (result[direction].rate && !isNaN(result[direction].rate)) {
+                                if (result[direction].rate !== undefined && !isNaN(result[direction].rate)) {
                                    outputRow[rate] = result[direction].rate.toString();
                                } else {
                                    outputRow[rate] = '';
@@ -36,15 +36,18 @@ Feature: Traffic - speeds
        1,4,27
        4,1,27
        """
-        And I route I should get
-          | from | to | route       | speed   | weights              |
-          | a    | b  | ad,de,eb,eb | 30 km/h | 1275.7,400.4,378.2,0 |
-          | a    | c  | ad,dc,dc    | 31 km/h | 1275.7,956.8,0       |
-          | b    | c  | bc,bc       | 27 km/h | 741.5,0              |
-          | a    | d  | ad,ad       | 27 km/h | 1275.7,0             |
-          | d    | c  | dc,dc       | 36 km/h | 956.8,0              |
-          | g    | b  | fb,fb       | 36 km/h | 164.7,0              |
-          | a    | g  | ad,df,fb,fb | 30 km/h | 1275.7,487.5,304.7,0 |
+        And the query options
+          | annotations | datasources |
+
+        When I route I should get
+          | from | to | route       | speed   | weights              | a:datasources |
+          | a    | b  | ad,de,eb,eb | 30 km/h | 1275.7,400.4,378.2,0 | 1:0:0:0       |
+          | a    | c  | ad,dc,dc    | 31 km/h | 1275.7,956.8,0       | 1:0           |
+          | b    | c  | bc,bc       | 27 km/h | 741.5,0              | 1:0           |
+          | a    | d  | ad,ad       | 27 km/h | 1275.7,0             | 1:0           |
+          | d    | c  | dc,dc       | 36 km/h | 956.8,0              | 0             |
+          | g    | b  | fb,fb       | 36 km/h | 164.7,0              | 0             |
+          | a    | g  | ad,df,fb,fb | 30 km/h | 1275.7,487.5,304.7,0 | 1:0:0         |


    Scenario: Weighting based on speed file weights, ETA based on file durations
@@ -58,15 +61,18 @@ Feature: Traffic - speeds
        1,4,27,1275.7
        4,1,27,1275.7
        """
-        And I route I should get
-          | from | to | route       | speed   | weights              |
-          | a    | b  | ad,de,eb,eb | 30 km/h | 1275.7,400.4,378.2,0 |
-          | a    | c  | ad,dc,dc    | 31 km/h | 1275.7,956.8,0       |
-          | b    | c  | bc,bc       | 27 km/h | 741.5,0              |
-          | a    | d  | ad,ad       | 27 km/h | 1275.7,0             |
-          | d    | c  | dc,dc       | 36 km/h | 956.8,0              |
-          | g    | b  | ab,ab       | 1 km/h  | 10010.4,0            |
-          | a    | g  | ab,ab       | 1 km/h  | 10010.3,0            |
+        And the query options
+          | annotations | datasources |
+
+        When I route I should get
+          | from | to | route       | speed   | weights              | a:datasources |
+          | a    | b  | ad,de,eb,eb | 30 km/h | 1275.7,400.4,378.2,0 | 1:0:0:0       |
+          | a    | c  | ad,dc,dc    | 31 km/h | 1275.7,956.8,0       | 1:0           |
+          | b    | c  | bc,bc       | 27 km/h | 741.5,0              | 1:0           |
+          | a    | d  | ad,ad       | 27 km/h | 1275.7,0             | 1:0           |
+          | d    | c  | dc,dc       | 36 km/h | 956.8,0              | 0             |
+          | g    | b  | ab,ab       | 1 km/h  | 10010.4,0            | 1:0           |
+          | a    | g  | ab,ab       | 1 km/h  | 10010.3,0            | 1             |


    Scenario: Weighting based on speed file weights, ETA based on file durations
@@ -87,16 +93,19 @@ Feature: Traffic - speeds
        1,4,1,34445.12
        4,1,1,34445.3
        """
-        And I route I should get
-          | from | to | route       | speed   | weights                     |
-          | a    | b  | ab,ab       | 1 km/h  | 20020.789,0                 |
-          | a    | c  | ab,bc,bc    | 2 km/h  | 20020.789,741.568,0         |
-          | b    | c  | bc,bc       | 27 km/h | 741.568,0                   |
-          | a    | d  | ab,eb,de,de | 2 km/h  | 20020.789,378.169,400.415,0 |
-          | d    | c  | dc,dc       | 36 km/h | 956.805,0                   |
-          | g    | b  | ab,ab       | 1 km/h  | 10010.392,0                 |
-          | a    | g  | ab,ab       | 1 km/h  | 10010.397,0                 |
-          | g    | a  | ab,ab       | 1 km/h  | 10010.064,0                 |
+        And the query options
+          | annotations | datasources |
+
+        When I route I should get
+          | from | to | route       | speed   | weights                     | a:datasources |
+          | a    | b  | ab,ab       | 1 km/h  | 20020.789,0                 | 1:0           |
+          | a    | c  | ab,bc,bc    | 2 km/h  | 20020.789,741.568,0         | 1:1:0         |
+          | b    | c  | bc,bc       | 27 km/h | 741.568,0                   | 1:0           |
+          | a    | d  | ab,eb,de,de | 2 km/h  | 20020.789,378.169,400.415,0 | 1:0:0         |
+          | d    | c  | dc,dc       | 36 km/h | 956.805,0                   | 0             |
+          | g    | b  | ab,ab       | 1 km/h  | 10010.392,0                 | 1:0           |
+          | a    | g  | ab,ab       | 1 km/h  | 10010.397,0                 | 1             |
+          | g    | a  | ab,ab       | 1 km/h  | 10010.064,0                 | 1:1           |


    Scenario: Speeds that isolate a single node (a)
@@ -113,15 +122,18 @@ Feature: Traffic - speeds
        1,4,0
        4,1,0
        """
-        And I route I should get
-          | from | to | route    | speed   | weights       |
-          | a    | b  | fb,fb    | 36 km/h | 329.4,0       |
-          | a    | c  | fb,bc,bc | 30 km/h | 329.4,741.5,0 |
-          | b    | c  | bc,bc    | 27 km/h | 741.5,0       |
-          | a    | d  | fb,df,df | 36 km/h | 140,487.5,0   |
-          | d    | c  | dc,dc    | 36 km/h | 956.8,0       |
-          | g    | b  | fb,fb    | 36 km/h | 164.7,0       |
-          | a    | g  | fb,fb    | 36 km/h | 164.7,0       |
+        And the query options
+          | annotations | true |
+
+        When I route I should get
+          | from | to | route    | speed   | weights       | a:datasources |
+          | a    | b  | fb,fb    | 36 km/h | 329.4,0       | 0             |
+          | a    | c  | fb,bc,bc | 30 km/h | 329.4,741.5,0 | 0:1:0         |
+          | b    | c  | bc,bc    | 27 km/h | 741.5,0       | 1:0           |
+          | a    | d  | fb,df,df | 36 km/h | 140,487.5,0   | 0:0:0         |
+          | d    | c  | dc,dc    | 36 km/h | 956.8,0       | 0             |
+          | g    | b  | fb,fb    | 36 km/h | 164.7,0       | 0             |
+          | a    | g  | fb,fb    | 36 km/h | 164.7,0       | 0             |


    Scenario: Verify that negative values cause an error, they're not valid at all
@@ -865,7 +865,6 @@ class ContiguousInternalMemoryDataFacade : public BaseDataFacade
        }
        else
        {
-            result_datasources.resize(end - begin);
            std::copy(m_datasource_list.begin() + begin,
                      m_datasource_list.begin() + end,
                      std::back_inserter(result_datasources));
@@ -2,6 +2,7 @@
 #define OSRM_INDEXED_DATA_HPP

 #include "util/exception.hpp"
+#include "util/exception_utils.hpp"
 #include "util/string_view.hpp"

 #include <boost/assert.hpp>
@@ -107,6 +108,11 @@ template <int N, typename T = std::string> struct VariableGroupBlock

        out.write((const char *)&refernce, sizeof(refernce));

+        if (!out)
+        {
+            throw util::exception(std::string("Error writing block reference: ") + SOURCE_REF);
+        }
+
        return prefix_length;
    }

@@ -128,6 +134,11 @@ template <int N, typename T = std::string> struct VariableGroupBlock
                continue;

            out.write((const char *)&data_length, byte_length);
+
+            if (!out)
+            {
+                throw util::exception(std::string("Error writing block prefix: ") + SOURCE_REF);
+            }
        }
    }

@@ -183,6 +194,12 @@ template <int N, typename T = std::string> struct FixedGroupBlock
        BlockReference refernce{static_cast<decltype(BlockReference::offset)>(data_offset)};
        out.write((const char *)&refernce, sizeof(refernce));

+        if (!out)
+        {
+            throw util::exception(std::string("Error writing group block reference: ") +
+                                  SOURCE_REF);
+        }
+
        return BLOCK_SIZE;
    }

@@ -201,6 +218,12 @@ template <int N, typename T = std::string> struct FixedGroupBlock
            block_prefix[index++] = static_cast<ValueType>(data_length);
        }
        out.write((const char *)block_prefix.data(), block_prefix.size());
+
+        if (!out)
+        {
+            throw util::exception(std::string("Error writing a fixed length block prefix: ") +
+                                  SOURCE_REF);
+        }
    }

    /// Advances the range to an item stored in the referenced block.
@@ -261,6 +284,11 @@ template <typename GroupBlock> struct IndexedData
                                    : 1 + (std::distance(first, sentinel) - 1) / (BLOCK_SIZE + 1);
        out.write((const char *)&number_of_blocks, sizeof(number_of_blocks));

+        if (!out)
+        {
+            throw util::exception(std::string("Error writing indexed data: ") + SOURCE_REF);
+        }
+
        // Write block references and compute the total data size that includes prefix and data
        const GroupBlock block;
        DataSizeType data_size = 0;
@@ -275,6 +303,12 @@ template <typename GroupBlock> struct IndexedData
        // Write the total data size
        out.write((const char *)&data_size, sizeof(data_size));

+        if (!out)
+        {
+            throw util::exception(std::string("Error writing the total indexed data size: ") +
+                                  SOURCE_REF);
+        }
+
        // Write data blocks that are (prefix, data)
        for (OffsetIterator curr = first, next = first; next != sentinel; curr = next)
        {
@@ -15,6 +15,7 @@
 #include <vector>

 #include "storage/io.hpp"
+#include "util/exception_utils.hpp"
 #include "util/fingerprint.hpp"

 namespace osrm
@@ -26,6 +27,10 @@ inline bool writeFingerprint(std::ostream &stream)
 {
    const auto fingerprint = FingerPrint::GetValid();
    stream.write(reinterpret_cast<const char *>(&fingerprint), sizeof(fingerprint));
+    if (!stream)
+    {
+        throw util::exception(std::string("Error writing fingerprint: ") + SOURCE_REF);
+    }
    return static_cast<bool>(stream);
 }

@@ -36,6 +41,10 @@ bool serializeVector(std::ostream &stream, const std::vector<simple_type> &data)
    stream.write(reinterpret_cast<const char *>(&count), sizeof(count));
    if (!data.empty())
        stream.write(reinterpret_cast<const char *>(&data[0]), sizeof(simple_type) * count);
+    if (!stream)
+    {
+        throw util::exception(std::string("Error serializing vector: ") + SOURCE_REF);
+    }
    return static_cast<bool>(stream);
 }

@@ -82,6 +91,11 @@ bool serializeVector(std::ofstream &out_stream, const stxxl::vector<simple_type>
 {
    const std::uint64_t size = data.size();
    out_stream.write(reinterpret_cast<const char *>(&size), sizeof(size));
+    if (!out_stream)
+    {
+        throw util::exception(std::string("Error serializing vector. Writing size step: ") +
+                              SOURCE_REF);
+    }

    simple_type write_buffer[WRITE_BLOCK_BUFFER_SIZE];
    std::size_t buffer_len = 0;
@@ -102,6 +116,11 @@ bool serializeVector(std::ofstream &out_stream, const stxxl::vector<simple_type>
    if (buffer_len > 0)
        out_stream.write(reinterpret_cast<const char *>(write_buffer),
                         buffer_len * sizeof(simple_type));
+    if (!out_stream)
+    {
+        throw util::exception(std::string("Error serializing vector. Writing data step: ") +
+                              SOURCE_REF);
+    }

    return static_cast<bool>(out_stream);
 }
@@ -133,6 +152,11 @@ inline bool serializeFlags(const boost::filesystem::path &path, const std::vecto

    std::uint32_t number_of_bits = flags.size();
    flag_stream.write(reinterpret_cast<const char *>(&number_of_bits), sizeof(number_of_bits));
+    if (!flag_stream)
+    {
+        throw util::exception(std::string("Error on  serializing flags. Write size step: ") +
+                              SOURCE_REF);
+    }
    // putting bits in ints
    std::uint32_t chunk = 0;
    std::size_t chunk_count = 0;
@@ -147,6 +171,11 @@ inline bool serializeFlags(const boost::filesystem::path &path, const std::vecto
        ++chunk_count;
        flag_stream.write(reinterpret_cast<const char *>(&chunk), sizeof(chunk));
    }
+    if (!flag_stream)
+    {
+        throw util::exception(std::string("Error on  serializing flags. Write data step: ") +
+                              SOURCE_REF);
+    }
    Log() << "Wrote " << number_of_bits << " bits in " << chunk_count << " chunks (Flags).";
    return static_cast<bool>(flag_stream);
 }
@@ -2,6 +2,7 @@
 #define RANGE_TABLE_HPP

 #include "storage/io.hpp"
+#include "util/exception_utils.hpp"
 #include "util/integer_range.hpp"
 #include "util/shared_memory_vector_wrapper.hpp"

@@ -221,12 +222,34 @@ std::ostream &operator<<(std::ostream &out, const RangeTable<BLOCK_SIZE, USE_SHA
    // write number of block
    const unsigned number_of_blocks = table.diff_blocks.size();
    out.write((char *)&number_of_blocks, sizeof(unsigned));
+    if (!out)
+    {
+        throw util::exception(
+            std::string("Error writing block size of range table to shared memory: ") + SOURCE_REF);
+    }
    // write total length
    out.write((char *)&table.sum_lengths, sizeof(unsigned));
+    if (!out)
+    {
+        throw util::exception(
+            std::string("Error writing total length of range table to shared memory: ") +
+            SOURCE_REF);
+    }
    // write block offsets
    out.write((char *)table.block_offsets.data(), sizeof(unsigned) * table.block_offsets.size());
+    if (!out)
+    {
+        throw util::exception(
+            std::string("Error writing block offsets of range table to shared memory: ") +
+            SOURCE_REF);
+    }
    // write blocks
    out.write((char *)table.diff_blocks.data(), BLOCK_SIZE * table.diff_blocks.size());
+    if (!out)
+    {
+        throw util::exception(
+            std::string("Error writing blocks of range table to shared memory: ") + SOURCE_REF);
+    }

    return out;
 }
@@ -263,6 +263,12 @@ class StaticRTree

                // write leaf_node to leaf node file
                leaf_node_file.write((char *)&current_leaf, sizeof(current_leaf));
+                if (!leaf_node_file)
+                {
+                    throw util::exception(std::string("Error in static_rtree, writing leaf_node to "
+                                                      "leaf node file (`.fileIndex`): ") +
+                                          SOURCE_REF);
+                }
            }

            tree_nodes_in_level.emplace_back(current_node);
@@ -336,7 +342,19 @@ class StaticRTree
        std::uint64_t size_of_tree = m_search_tree.size();
        BOOST_ASSERT_MSG(0 < size_of_tree, "tree empty");
        tree_node_file.write((char *)&size_of_tree, sizeof(size_of_tree));
+        if (!tree_node_file)
+        {
+            throw util::exception(
+                std::string("Error in static_rtree, writing size of tree in `.fileIndex` file: ") +
+                SOURCE_REF);
+        }
        tree_node_file.write((char *)&m_search_tree[0], sizeof(TreeNode) * size_of_tree);
+        if (!tree_node_file)
+        {
+            throw util::exception(
+                std::string("Error in static_rtree, writing tree data in `.fileIndex` file: ") +
+                SOURCE_REF);
+        }

        MapLeafNodesFile(leaf_node_filename);
    }
@@ -67,6 +67,8 @@ local profile = {

  restricted_access_tag_list = Set { },

+  restricted_highway_whitelist = Set { },
+
  access_tags_hierarchy = Sequence {
  	'bicycle',
  	'vehicle',
@@ -74,7 +74,11 @@ local profile = {
    'agricultural',
    'forestry',
    'emergency',
-    'psv'
+    'psv',
+    'customer',
+    'private',
+    'delivery',
+    'destination'
  },

  restricted_access_tag_list = Set {
@@ -137,6 +141,21 @@ local profile = {
    ["drive-thru"] = 0.5
  },

+ restricted_highway_whitelist = Set {
+      'motorway',
+      'motorway_link',
+      'trunk',
+      'trunk_link',
+      'primary',
+      'primary_link',
+      'secondary',
+      'secondary_link',
+      'tertiary',
+      'tertiary_link',
+      'residential',
+      'living_street',
+  },
+
  route_speeds = {
    ferry = 5,
    shuttle_train = 10
@@ -259,7 +278,7 @@ function node_function (node, result)
  -- parse access and barrier tags
  local access = find_access_tag(node, profile.access_tags_hierarchy)
  if access then
-    if profile.access_tag_blacklist[access] then
+    if profile.access_tag_blacklist[access] and not profile.restricted_access_tag_list[access] then
      result.barrier = true
    end
  else
@@ -392,11 +411,11 @@ function turn_function (turn)
    else
       turn.weight = turn.duration
    end
-    if properties.weight_name == 'routability' then
-        -- penalize turns from non-local access only segments onto local access only tags
-        if not turn.source_restricted and turn.target_restricted then
-            turn.weight = turn.weight + profile.restricted_penalty
-        end
-    end
+  end
+  if properties.weight_name == 'routability' then
+      -- penalize turns from non-local access only segments onto local access only tags
+      if not turn.source_restricted and turn.target_restricted then
+          turn.weight = turn.weight + profile.restricted_penalty
+      end
  end
 end
@@ -53,6 +53,8 @@ local profile = {

  restricted_access_tag_list = Set { },

+  restricted_highway_whitelist = Set { },
+
  access_tags_hierarchy = Sequence {
    'foot',
    'access'
@@ -212,25 +212,28 @@ function Handlers.handle_access(way,result,data,profile)
  data.forward_access, data.backward_access =
    Tags.get_forward_backward_by_set(way,data,profile.access_tags_hierarchy)

-  if profile.access_tag_blacklist[data.forward_access] then
+  -- only allow a subset of roads that are marked as restricted
+  if profile.restricted_highway_whitelist[data.highway] then
+      if profile.restricted_access_tag_list[data.forward_access] then
+          result.forward_restricted = true
+      end
+
+      if profile.restricted_access_tag_list[data.backward_access] then
+          result.backward_restricted = true
+      end
+  end
+
+  if profile.access_tag_blacklist[data.forward_access] and not result.forward_restricted then
    result.forward_mode = mode.inaccessible
  end

-  if profile.access_tag_blacklist[data.backward_access] then
+  if profile.access_tag_blacklist[data.backward_access] and not result.backward_restricted then
    result.backward_mode = mode.inaccessible
  end

  if result.forward_mode == mode.inaccessible and result.backward_mode == mode.inaccessible then
    return false
  end
-
-  if profile.restricted_access_tag_list[data.forward_access] then
-      result.forward_restricted = true
-  end
-
-  if profile.restricted_access_tag_list[data.backward_access] then
-      result.backward_restricted = true
-  end
 end

 -- handle speed (excluding maxspeed)
@@ -736,6 +736,14 @@ Contractor::LoadEdgeExpandedGraph(const ContractorConfig &config,
                              number_of_compressed_geometries * sizeof(EdgeWeight));
        geometry_stream.write(reinterpret_cast<char *>(&(geometry_rev_duration_list[0])),
                              number_of_compressed_geometries * sizeof(EdgeWeight));
+        if (!geometry_stream)
+        {
+            throw util::exception(
+                std::string(
+                    "Error in contractor, updating compressed geometries in `.geometries` file: ") +
+                config.geometry_path);
+        }
+
    };

    const auto save_datasource_indexes = [&] {
@@ -754,6 +762,12 @@ Contractor::LoadEdgeExpandedGraph(const ContractorConfig &config,
            datasource_stream.write(reinterpret_cast<char *>(&(geometry_datasource[0])),
                                    number_of_datasource_entries * sizeof(uint8_t));
        }
+        if (!datasource_stream)
+        {
+            throw util::exception(std::string("Error in contractor, saving datasource indices in "
+                                              "`.datasource_indexes` file: ") +
+                                  SOURCE_REF);
+        }
    };

    const auto save_datastore_names = [&] {
@@ -956,6 +970,13 @@ void Contractor::WriteNodeLevels(std::vector<float> &&in_node_levels) const
    unsigned level_size = node_levels.size();
    order_output_stream.write((char *)&level_size, sizeof(unsigned));
    order_output_stream.write((char *)node_levels.data(), sizeof(float) * node_levels.size());
+
+    if (!order_output_stream)
+    {
+        throw util::exception(
+            std::string("Error in contractor, writing node levels to `.level` file: ") +
+            SOURCE_REF);
+    }
 }

 void Contractor::WriteCoreNodeMarker(std::vector<bool> &&in_is_core_node) const
@@ -973,6 +994,13 @@ void Contractor::WriteCoreNodeMarker(std::vector<bool> &&in_is_core_node) const
    core_marker_output_stream.write((char *)&size, sizeof(unsigned));
    core_marker_output_stream.write((char *)unpacked_bool_flags.data(),
                                    sizeof(char) * unpacked_bool_flags.size());
+
+    if (!core_marker_output_stream)
+    {
+        throw util::exception(
+            std::string("Error in contractor, writing code node markers to `.core` file: ") +
+            SOURCE_REF);
+    }
 }

 std::size_t
@@ -987,6 +1015,12 @@ Contractor::WriteContractedGraph(unsigned max_node_id,
    const util::FingerPrint fingerprint = util::FingerPrint::GetValid();
    boost::filesystem::ofstream hsgr_output_stream(config.graph_output_path, std::ios::binary);
    hsgr_output_stream.write((char *)&fingerprint, sizeof(util::FingerPrint));
+    if (!hsgr_output_stream)
+    {
+        throw util::exception(
+            std::string("Error in contractor, writing contracted graph to `.hsgr` file: ") +
+            SOURCE_REF);
+    }
    const NodeID max_used_node_id = [&contracted_edge_list] {
        NodeID tmp_max = 0;
        for (const QueryEdge &edge : contracted_edge_list)
@@ -1050,6 +1084,12 @@ Contractor::WriteContractedGraph(unsigned max_node_id,
                                 sizeof(util::StaticGraph<EdgeData>::NodeArrayEntry) *
                                     node_array_size);
    }
+    if (!hsgr_output_stream)
+    {
+        throw util::exception(
+            std::string("Error in contractor, serializing graph node array in `.hsgr` file: ") +
+            SOURCE_REF);
+    }

    // serialize all edges
    util::Log() << "Building edge array";
@@ -1058,7 +1098,7 @@ Contractor::WriteContractedGraph(unsigned max_node_id,
    util::StaticGraph<EdgeData>::EdgeArrayEntry current_edge;
    for (const auto edge : util::irange<std::size_t>(0UL, contracted_edge_list.size()))
    {
-        // some self-loops are required for oneway handling. Need to assertthat we only keep these
+        // some self-loops are required for oneway handling. Need to assert that we only keep these
        // (TODO)
        // no eigen loops
        // BOOST_ASSERT(contracted_edge_list[edge].source != contracted_edge_list[edge].target ||
@@ -1084,6 +1124,12 @@ Contractor::WriteContractedGraph(unsigned max_node_id,
 #endif
        hsgr_output_stream.write((char *)&current_edge,
                                 sizeof(util::StaticGraph<EdgeData>::EdgeArrayEntry));
+        if (!hsgr_output_stream)
+        {
+            throw util::exception(
+                std::string("Error in contractor, serializing graph edge array in `.hsgr` file: ") +
+                SOURCE_REF);
+        }

        ++number_of_used_edges;
    }
@@ -79,9 +79,10 @@ struct TurnData final
    TurnData(const util::Coordinate coordinate_,
             const std::size_t _in,
             const std::size_t _out,
-             const std::size_t _weight)
+             const std::size_t _weight,
+             const std::size_t _duration)
        : coordinate(std::move(coordinate_)), in_angle_offset(_in), turn_angle_offset(_out),
-          weight_offset(_weight)
+          weight_offset(_weight), duration_offset(_duration)
    {
    }

@@ -89,6 +90,7 @@ struct TurnData final
    const std::size_t in_angle_offset;
    const std::size_t turn_angle_offset;
    const std::size_t weight_offset;
+    const std::size_t duration_offset;
 };

 using FixedPoint = Point<std::int32_t>;
@@ -457,6 +459,7 @@ Status TilePlugin::HandleRequest(const std::shared_ptr<const datafacade::BaseDat
        //  vw is the "exit"
        std::vector<contractor::QueryEdge::EdgeData> unpacked_shortcut;
        std::vector<EdgeWeight> approach_weight_vector;
+        std::vector<EdgeWeight> approach_duration_vector;

        // Make sure we traverse the startnodes in a consistent order
        // to ensure identical PBF encoding on all platforms.
@@ -539,16 +542,26 @@ Status TilePlugin::HandleRequest(const std::shared_ptr<const datafacade::BaseDat
                            approach_weight_vector = facade->GetUncompressedForwardWeights(
                                edge_based_node_info[approachedge.edge_based_node_id]
                                    .packed_geometry_id);
+                            approach_duration_vector = facade->GetUncompressedForwardDurations(
+                                edge_based_node_info[approachedge.edge_based_node_id]
+                                    .packed_geometry_id);
                        }
                        else
                        {
                            approach_weight_vector = facade->GetUncompressedReverseWeights(
                                edge_based_node_info[approachedge.edge_based_node_id]
                                    .packed_geometry_id);
+                            approach_duration_vector = facade->GetUncompressedReverseDurations(
+                                edge_based_node_info[approachedge.edge_based_node_id]
+                                    .packed_geometry_id);
                        }
                        const auto sum_node_weight = std::accumulate(approach_weight_vector.begin(),
                                                                     approach_weight_vector.end(),
                                                                     EdgeWeight{0});
+                        const auto sum_node_duration =
+                            std::accumulate(approach_duration_vector.begin(),
+                                            approach_duration_vector.end(),
+                                            EdgeWeight{0});

                        // The edge.weight is the whole edge weight, which includes the turn
                        // cost.
@@ -557,7 +570,8 @@ Status TilePlugin::HandleRequest(const std::shared_ptr<const datafacade::BaseDat
                        // intersections include stop signs, traffic signals and other
                        // penalties, but at this stage, we can't divide those out, so we just
                        // treat the whole lot as the "turn cost" that we'll stick on the map.
-                        const auto turn_cost = data.weight - sum_node_weight;
+                        const auto turn_weight = data.weight - sum_node_weight;
+                        const auto turn_duration = data.duration - sum_node_duration;

                        // Find the three nodes that make up the turn movement)
                        const auto node_from = startnode;
@@ -598,14 +612,19 @@ Status TilePlugin::HandleRequest(const std::shared_ptr<const datafacade::BaseDat
                        // And, same for the actual turn cost value - it goes in the lookup
                        // table,
                        // not directly on the feature itself.
-                        const auto turn_cost_index = use_point_float_value(
-                            turn_cost / 10.0); // Note conversion to float here
+                        const auto turn_weight_index = use_point_float_value(
+                            turn_weight / 10.0); // Note conversion to float here
+                        const auto turn_duration_index = use_point_float_value(
+                            turn_duration / 10.0); // Note conversion to float here

                        // Save everything we need to later add all the points to the tile.
                        // We need the coordinate of the intersection, the angle in, the turn
                        // angle and the turn cost.
-                        all_turn_data.emplace_back(
-                            coord_via, angle_in_index, turn_angle_index, turn_cost_index);
+                        all_turn_data.emplace_back(coord_via,
+                                                   angle_in_index,
+                                                   turn_angle_index,
+                                                   turn_weight_index,
+                                                   turn_duration_index);
                    }
                }
            }
@@ -672,6 +691,8 @@ Status TilePlugin::HandleRequest(const std::shared_ptr<const datafacade::BaseDat
            for (const auto &edge_index : sorted_edge_indexes)
            {
                const auto &edge = edges[edge_index];
+
+                // Weight values
                const auto forward_weight_vector =
                    facade->GetUncompressedForwardWeights(edge.packed_geometry_id);
                const auto reverse_weight_vector =
@@ -679,8 +700,20 @@ Status TilePlugin::HandleRequest(const std::shared_ptr<const datafacade::BaseDat
                const auto forward_weight = forward_weight_vector[edge.fwd_segment_position];
                const auto reverse_weight = reverse_weight_vector[reverse_weight_vector.size() -
                                                                  edge.fwd_segment_position - 1];
-                use_line_value(reverse_weight);
                use_line_value(forward_weight);
+                use_line_value(reverse_weight);
+
+                // Duration values
+                const auto forward_duration_vector =
+                    facade->GetUncompressedForwardDurations(edge.packed_geometry_id);
+                const auto reverse_duration_vector =
+                    facade->GetUncompressedReverseDurations(edge.packed_geometry_id);
+                const auto forward_duration = forward_duration_vector[edge.fwd_segment_position];
+                const auto reverse_duration =
+                    reverse_duration_vector[reverse_duration_vector.size() -
+                                            edge.fwd_segment_position - 1];
+                use_line_value(forward_duration);
+                use_line_value(reverse_duration);
            }

            // Begin the layer features block
@@ -701,6 +734,10 @@ Status TilePlugin::HandleRequest(const std::shared_ptr<const datafacade::BaseDat
                        facade->GetUncompressedForwardWeights(edge.packed_geometry_id);
                    const auto reverse_weight_vector =
                        facade->GetUncompressedReverseWeights(edge.packed_geometry_id);
+                    const auto forward_duration_vector =
+                        facade->GetUncompressedForwardDurations(edge.packed_geometry_id);
+                    const auto reverse_duration_vector =
+                        facade->GetUncompressedReverseDurations(edge.packed_geometry_id);
                    const auto forward_datasource_vector =
                        facade->GetUncompressedForwardDatasources(edge.packed_geometry_id);
                    const auto reverse_datasource_vector =
@@ -709,6 +746,11 @@ Status TilePlugin::HandleRequest(const std::shared_ptr<const datafacade::BaseDat
                    const auto reverse_weight =
                        reverse_weight_vector[reverse_weight_vector.size() -
                                              edge.fwd_segment_position - 1];
+                    const auto forward_duration =
+                        forward_duration_vector[edge.fwd_segment_position];
+                    const auto reverse_duration =
+                        reverse_duration_vector[reverse_duration_vector.size() -
+                                                edge.fwd_segment_position - 1];
                    const auto forward_datasource =
                        forward_datasource_vector[edge.fwd_segment_position];
                    const auto reverse_datasource =
@@ -735,6 +777,7 @@ Status TilePlugin::HandleRequest(const std::shared_ptr<const datafacade::BaseDat
                                                   &max_datasource_id,
                                                   &used_line_ints](const FixedLine &tile_line,
                                                                    const std::uint32_t speed_kmh,
+                                                                    const std::size_t weight,
                                                                    const std::size_t duration,
                                                                    const DatasourceID datasource,
                                                                    const std::size_t name_idx,
@@ -770,10 +813,13 @@ Status TilePlugin::HandleRequest(const std::shared_ptr<const datafacade::BaseDat
                                              (edge.component.is_tiny ? 0 : 1)); // is_small feature
                            field.add_element(2);                // "datasource" tag key offset
                            field.add_element(130 + datasource); // datasource value offset
-                            field.add_element(3);                // "duration" tag key offset
+                            field.add_element(3);                // "weight" tag key offset
+                            field.add_element(130 + max_datasource_id + 1 +
+                                              weight); // weight value offset
+                            field.add_element(4);      // "duration" tag key offset
                            field.add_element(130 + max_datasource_id + 1 +
                                              duration); // duration value offset
-                            field.add_element(4);        // "name" tag key offset
+                            field.add_element(5);        // "name" tag key offset

                            field.add_element(130 + max_datasource_id + 1 + used_line_ints.size() +
                                              name_idx); // name value offset
@@ -788,14 +834,14 @@ Status TilePlugin::HandleRequest(const std::shared_ptr<const datafacade::BaseDat
                    };

                    // If this is a valid forward edge, go ahead and add it to the tile
-                    if (forward_weight != 0 && edge.forward_segment_id.enabled)
+                    if (forward_duration != 0 && edge.forward_segment_id.enabled)
                    {
                        std::int32_t start_x = 0;
                        std::int32_t start_y = 0;

                        // Calculate the speed for this line
                        std::uint32_t speed_kmh =
-                            static_cast<std::uint32_t>(round(length / forward_weight * 10 * 3.6));
+                            static_cast<std::uint32_t>(round(length / forward_duration * 10 * 3.6));

                        auto tile_line = coordinatesToTileLine(a, b, tile_bbox);
                        if (!tile_line.empty())
@@ -803,6 +849,7 @@ Status TilePlugin::HandleRequest(const std::shared_ptr<const datafacade::BaseDat
                            encode_tile_line(tile_line,
                                             speed_kmh,
                                             line_int_offsets[forward_weight],
+                                             line_int_offsets[forward_duration],
                                             forward_datasource,
                                             name_offset,
                                             start_x,
@@ -812,14 +859,14 @@ Status TilePlugin::HandleRequest(const std::shared_ptr<const datafacade::BaseDat

                    // Repeat the above for the coordinates reversed and using the `reverse`
                    // properties
-                    if (reverse_weight != 0 && edge.reverse_segment_id.enabled)
+                    if (reverse_duration != 0 && edge.reverse_segment_id.enabled)
                    {
                        std::int32_t start_x = 0;
                        std::int32_t start_y = 0;

                        // Calculate the speed for this line
                        std::uint32_t speed_kmh =
-                            static_cast<std::uint32_t>(round(length / reverse_weight * 10 * 3.6));
+                            static_cast<std::uint32_t>(round(length / reverse_duration * 10 * 3.6));

                        auto tile_line = coordinatesToTileLine(b, a, tile_bbox);
                        if (!tile_line.empty())
@@ -827,6 +874,7 @@ Status TilePlugin::HandleRequest(const std::shared_ptr<const datafacade::BaseDat
                            encode_tile_line(tile_line,
                                             speed_kmh,
                                             line_int_offsets[reverse_weight],
+                                             line_int_offsets[reverse_duration],
                                             reverse_datasource,
                                             name_offset,
                                             start_x,
@@ -842,6 +890,7 @@ Status TilePlugin::HandleRequest(const std::shared_ptr<const datafacade::BaseDat
            line_layer_writer.add_string(util::vector_tile::KEY_TAG, "speed");
            line_layer_writer.add_string(util::vector_tile::KEY_TAG, "is_small");
            line_layer_writer.add_string(util::vector_tile::KEY_TAG, "datasource");
+            line_layer_writer.add_string(util::vector_tile::KEY_TAG, "weight");
            line_layer_writer.add_string(util::vector_tile::KEY_TAG, "duration");
            line_layer_writer.add_string(util::vector_tile::KEY_TAG, "name");

@@ -938,6 +987,8 @@ Status TilePlugin::HandleRequest(const std::shared_ptr<const datafacade::BaseDat
                        field.add_element(1); // "turn_angle" tag key offset
                        field.add_element(point_turn_data.turn_angle_offset);
                        field.add_element(2); // "cost" tag key offset
+                        field.add_element(used_point_ints.size() + point_turn_data.duration_offset);
+                        field.add_element(3); // "cost" tag key offset
                        field.add_element(used_point_ints.size() + point_turn_data.weight_offset);
                    }
                    {
@@ -965,6 +1016,7 @@ Status TilePlugin::HandleRequest(const std::shared_ptr<const datafacade::BaseDat
            point_layer_writer.add_string(util::vector_tile::KEY_TAG, "bearing_in");
            point_layer_writer.add_string(util::vector_tile::KEY_TAG, "turn_angle");
            point_layer_writer.add_string(util::vector_tile::KEY_TAG, "cost");
+            point_layer_writer.add_string(util::vector_tile::KEY_TAG, "weight");

            // Now, save the lists of integers and floats that our features refer to.
            for (const auto &value : used_point_ints)
@@ -1,4 +1,6 @@
 #include "extractor/compressed_edge_container.hpp"
+#include "util/exception.hpp"
+#include "util/exception_utils.hpp"
 #include "util/log.hpp"

 #include <boost/assert.hpp>
@@ -109,6 +111,13 @@ void CompressedEdgeContainer::SerializeInternalVector(const std::string &path) c
    // write compressed geometry reverse durations
    geometry_out_stream.write((char *)(m_compressed_geometry_rev_durations.data()),
                              sizeof(EdgeWeight) * m_compressed_geometry_rev_durations.size());
+
+    if (!geometry_out_stream)
+    {
+        throw util::exception(std::string("Error in CompressedEdgeContainer, writing compressed "
+                                          "geometry to `.geometry` file: ") +
+                              SOURCE_REF);
+    }
 }

 // Adds info for a compressed edge to the container.   edge_id_2
@@ -4,6 +4,7 @@
 #include "util/coordinate.hpp"
 #include "util/coordinate_calculation.hpp"
 #include "util/exception.hpp"
+#include "util/exception_utils.hpp"
 #include "util/guidance/turn_bearing.hpp"
 #include "util/integer_range.hpp"
 #include "util/log.hpp"
@@ -183,6 +184,15 @@ void EdgeBasedGraphFactory::FlushVectorToStream(
    }
    edge_data_file.write((char *)&(original_edge_data_vector[0]),
                         original_edge_data_vector.size() * sizeof(OriginalEdgeData));
+
+    if (!edge_data_file)
+    {
+        throw util::exception(
+            std::string(
+                "Error in EdgeBasedGraphFactory, flushing vector to stream for `.ebg` file: ") +
+            SOURCE_REF);
+    }
+
    original_edge_data_vector.clear();
 }

@@ -339,6 +349,13 @@ void EdgeBasedGraphFactory::GenerateEdgeExpandedEdges(
    edge_data_file.write(reinterpret_cast<const char *>(&length_prefix_empty_space),
                         sizeof(length_prefix_empty_space));

+    if (!edge_data_file)
+    {
+        throw util::exception(
+            std::string("Error writing dummy value to edge based graph in `.ebg` file: ") +
+            SOURCE_REF);
+    }
+
    std::vector<OriginalEdgeData> original_edge_data_vector;
    original_edge_data_vector.reserve(1024 * 1024);

@@ -608,9 +625,18 @@ void EdgeBasedGraphFactory::GenerateEdgeExpandedEdges(

                            edge_segment_file.write(reinterpret_cast<const char *>(&nodeblock),
                                                    sizeof(nodeblock));
+
                            previous = target_node.node_id;
                        }

+                        if (!edge_segment_file)
+                        {
+                            throw util::exception(
+                                std::string("Error in generating edge lookups and writing to "
+                                            "`.edge_segment_lookup` file: ") +
+                                SOURCE_REF);
+                        }
+
                        // We also now write out the mapping between the edge-expanded edges and the
                        // original nodes. Since each edge represents a possible maneuver, external
                        // programs can use this to quickly perform updates to edge weights in order
@@ -644,6 +670,14 @@ void EdgeBasedGraphFactory::GenerateEdgeExpandedEdges(
                        turn_penalties_index_file.write(
                            reinterpret_cast<const char *>(&turn_index_block),
                            sizeof(turn_index_block));
+
+                        if (!turn_penalties_index_file)
+                        {
+                            throw util::exception(
+                                std::string("Error writing data for turn penalties to "
+                                            "`.turn_penalties_index` file ") +
+                                SOURCE_REF);
+                        }
                    }
                }
            }
@@ -659,6 +693,12 @@ void EdgeBasedGraphFactory::GenerateEdgeExpandedEdges(
    turn_weight_penalties_file.write(reinterpret_cast<const char *>(turn_weight_penalties.data()),
                                     sizeof(decltype(turn_weight_penalties)::value_type) *
                                         turn_weight_penalties.size());
+    if (!turn_weight_penalties_file)
+    {
+        throw util::exception(
+            std::string("Error writing turn weight penalties to `.turn_weight_penalties` file ") +
+            SOURCE_REF);
+    }

    // write duration penalties per turn if we need them
    BOOST_ASSERT(!profile_properties.fallback_to_duration || turn_duration_penalties.size() == 0);
@@ -668,12 +708,20 @@ void EdgeBasedGraphFactory::GenerateEdgeExpandedEdges(
    turn_duration_penalties_file.write(
        reinterpret_cast<const char *>(&turn_duration_penalties_header),
        sizeof(turn_duration_penalties_header));
+
    if (!profile_properties.fallback_to_duration)
    {
        BOOST_ASSERT(turn_weight_penalties.size() == turn_duration_penalties.size());
        turn_duration_penalties_file.write(
            reinterpret_cast<const char *>(turn_duration_penalties.data()),
            sizeof(decltype(turn_duration_penalties)::value_type) * turn_duration_penalties.size());
+        if (!turn_duration_penalties_file)
+        {
+            throw util::exception(
+                std::string(
+                    "Error writing turn duration penalties to `.turn_duration_penalties` file ") +
+                SOURCE_REF);
+        }
    }

    util::Log() << "Created " << entry_class_hash.size() << " entry classes and "
@@ -693,6 +741,11 @@ void EdgeBasedGraphFactory::GenerateEdgeExpandedEdges(
        turn_lane_data_file.write(reinterpret_cast<const char *>(&lane_data[0]),
                                  sizeof(util::guidance::LaneTupleIdPair) * lane_data.size());

+    if (!turn_lane_data_file)
+    {
+        throw util::exception(std::string("Error writing turn lane data file: ") + SOURCE_REF);
+    }
+
    util::Log() << "done.";

    FlushVectorToStream(edge_data_file, original_edge_data_vector);
@@ -704,6 +757,10 @@ void EdgeBasedGraphFactory::GenerateEdgeExpandedEdges(
    static_assert(sizeof(length_prefix_empty_space) == sizeof(length_prefix), "type mismatch");

    edge_data_file.write(reinterpret_cast<const char *>(&length_prefix), sizeof(length_prefix));
+    if (!edge_data_file)
+    {
+        throw util::exception(std::string("Error writing data to `.ebg`") + SOURCE_REF);
+    }

    util::Log() << "Generated " << m_edge_based_node_list.size() << " edge based nodes";
    util::Log() << "Node-based graph contains " << node_based_edge_counter << " edges";
@@ -158,6 +158,13 @@ void ExtractionContainers::PrepareData(ScriptingEnvironment &scripting_environme
    file_out_stream.open(output_file_name.c_str(), std::ios::binary);
    const util::FingerPrint fingerprint = util::FingerPrint::GetValid();
    file_out_stream.write((char *)&fingerprint, sizeof(util::FingerPrint));
+    if (!file_out_stream)
+    {
+        throw util::exception(
+            std::string(
+                "Error writing fingerprint during extraction containers data preparation: ") +
+            SOURCE_REF);
+    }

    FlushVectors();

@@ -390,21 +397,24 @@ void ExtractionContainers::PrepareEdges(ScriptingEnvironment &scripting_environm
            BOOST_ASSERT(edge_iterator->source_coordinate.lon !=
                         util::FixedLongitude{std::numeric_limits<std::int32_t>::min()});

-            const util::Coordinate target_coord{node_iterator->lon, node_iterator->lat};
-            const double distance = util::coordinate_calculation::greatCircleDistance(
-                edge_iterator->source_coordinate, target_coord);
+            util::Coordinate source_coord(edge_iterator->source_coordinate);
+            util::Coordinate target_coord{node_iterator->lon, node_iterator->lat};

-            auto weight = edge_iterator->weight_data(distance);
-            auto duration = edge_iterator->duration_data(distance);
+            // flip source and target coordinates if segment is in backward direction only
+            if (!edge_iterator->result.forward && edge_iterator->result.backward)
+                std::swap(source_coord, target_coord);

-            ExtractionSegment extracted_segment(
-                edge_iterator->source_coordinate, target_coord, distance, weight, duration);
-            scripting_environment.ProcessSegment(extracted_segment);
+            const auto distance =
+                util::coordinate_calculation::greatCircleDistance(source_coord, target_coord);
+            const auto weight = edge_iterator->weight_data(distance);
+            const auto duration = edge_iterator->duration_data(distance);
+
+            ExtractionSegment segment(source_coord, target_coord, distance, weight, duration);
+            scripting_environment.ProcessSegment(segment);

            auto &edge = edge_iterator->result;
-            edge.weight =
-                std::max<EdgeWeight>(1, std::round(extracted_segment.weight * weight_multiplier));
-            edge.duration = std::max<EdgeWeight>(1, std::round(extracted_segment.duration * 10.));
+            edge.weight = std::max<EdgeWeight>(1, std::round(segment.weight * weight_multiplier));
+            edge.duration = std::max<EdgeWeight>(1, std::round(segment.duration * 10.));

            // assign new node id
            auto id_iter = external_to_internal_node_id_map.find(node_iterator->node_id);
@@ -571,9 +581,16 @@ void ExtractionContainers::WriteEdges(std::ofstream &file_out_stream) const
            // class of NodeBasedEdgeWithOSM
            NodeBasedEdge tmp = edge.result;
            file_out_stream.write((char *)&tmp, sizeof(NodeBasedEdge));
+
            used_edges_counter++;
        }

+        if (!file_out_stream)
+        {
+            throw util::exception(std::string("Error writing edges in extraction containers: ") +
+                                  SOURCE_REF);
+        }
+
        if (used_edges_counter > std::numeric_limits<unsigned>::max())
        {
            throw util::exception("There are too many edges, OSRM only supports 2^32" + SOURCE_REF);
@@ -591,6 +608,13 @@ void ExtractionContainers::WriteEdges(std::ofstream &file_out_stream) const
        file_out_stream.seekp(start_position);
        file_out_stream.write((char *)&used_edges_counter_buffer,
                              sizeof(used_edges_counter_buffer));
+
+        if (!file_out_stream)
+        {
+            throw util::exception(
+                std::string("Error writing number of used edges in extraction containers: ") +
+                SOURCE_REF);
+        }
        log << "ok";
    }

@@ -632,6 +656,12 @@ void ExtractionContainers::WriteNodes(std::ofstream &file_out_stream) const
            BOOST_ASSERT(*node_id_iterator == node_iterator->node_id);

            file_out_stream.write((char *)&(*node_iterator), sizeof(ExternalMemoryNode));
+            if (!file_out_stream)
+            {
+                throw util::exception(
+                    std::string("Error writing size of edges buffer in extraction containers: ") +
+                    SOURCE_REF);
+            }

            ++node_id_iterator;
            ++node_iterator;
@@ -651,6 +681,7 @@ void ExtractionContainers::WriteRestrictions(const std::string &path) const
    restrictions_out_stream.open(path.c_str(), std::ios::binary);
    const util::FingerPrint fingerprint = util::FingerPrint::GetValid();
    restrictions_out_stream.write((char *)&fingerprint, sizeof(util::FingerPrint));
+
    const auto count_position = restrictions_out_stream.tellp();
    restrictions_out_stream.write((char *)&written_restriction_count, sizeof(unsigned));

@@ -665,8 +696,14 @@ void ExtractionContainers::WriteRestrictions(const std::string &path) const
            ++written_restriction_count;
        }
    }
+
    restrictions_out_stream.seekp(count_position);
    restrictions_out_stream.write((char *)&written_restriction_count, sizeof(unsigned));
+    if (!restrictions_out_stream)
+    {
+        throw util::exception(std::string("Error writing restrictions in extraction containers: ") +
+                              SOURCE_REF);
+    }
    util::Log() << "usable restrictions: " << written_restriction_count;
 }

@@ -166,6 +166,12 @@ int Extractor::run(ScriptingEnvironment &scripting_environment)
        boost::filesystem::ofstream timestamp_out(config.timestamp_file_name);
        timestamp_out.write(timestamp.c_str(), timestamp.length());

+        if (!timestamp_out)
+        {
+            throw util::exception(std::string("Error writing timestamp in extractor: ") +
+                                  SOURCE_REF);
+        }
+
        // initialize vectors holding parsed objects
        tbb::concurrent_vector<std::pair<std::size_t, ExtractionNode>> resulting_nodes;
        tbb::concurrent_vector<std::pair<std::size_t, ExtractionWay>> resulting_ways;
@@ -317,6 +323,12 @@ void Extractor::WriteProfileProperties(const std::string &output_path,
    }

    out_stream.write(reinterpret_cast<const char *>(&properties), sizeof(properties));
+
+    if (!out_stream)
+    {
+        throw util::exception(std::string("Error writing profile properties in extractor: ") +
+                              SOURCE_REF);
+    }
 }

 void Extractor::FindComponents(unsigned max_edge_id,
@@ -536,6 +548,11 @@ void Extractor::WriteNodeMapping(const std::vector<QueryNode> &internal_to_exter
        node_stream.write((char *)internal_to_external_node_map.data(),
                          size_of_mapping * sizeof(QueryNode));
    }
+    if (!node_stream)
+    {
+        throw util::exception(std::string("Error writing node mapping in extractor: ") +
+                              SOURCE_REF);
+    }
 }

 /**
@@ -608,6 +625,12 @@ void Extractor::WriteEdgeBasedGraph(
        file_out_stream.write((char *)&edge, sizeof(EdgeBasedEdge));
    }

+    if (!file_out_stream)
+    {
+        throw util::exception(std::string("Error writing edge based graph in extractor: ") +
+                              SOURCE_REF);
+    }
+
    TIMER_STOP(write_edges);
    util::Log() << "ok, after " << TIMER_SEC(write_edges) << "s";

@@ -647,6 +670,14 @@ void Extractor::WriteIntersectionClassificationData(
    file_out_stream << bearing_class_range_table;

    file_out_stream.write(reinterpret_cast<const char *>(&total_bearings), sizeof(total_bearings));
+
+    if (!file_out_stream)
+    {
+        throw util::exception(
+            std::string("Error in writing interestction classification data in extractor: ") +
+            SOURCE_REF);
+    }
+
    for (const auto &bearing_class : bearing_classes)
    {
        const auto &bearings = bearing_class.getAvailableBearings();
@@ -330,7 +330,7 @@ void ExtractorCallbacks::ProcessWay(const osmium::Way &input_way, const Extracti
                             (turn_lane_id_forward != turn_lane_id_backward));

    if (in_forward_direction)
-    {
+    { // add (forward) segments or (forward,backward) for non-split edges in backward direction
        util::for_each_pair(
            nodes.cbegin(),
            nodes.cend(),
@@ -355,8 +355,8 @@ void ExtractorCallbacks::ProcessWay(const osmium::Way &input_way, const Extracti
            });
    }

-    if (in_backward_direction || split_edge)
-    {
+    if (in_backward_direction && (!in_forward_direction || split_edge))
+    { // add (backward) segments for split edges or not in forward direction
        util::for_each_pair(
            nodes.cbegin(),
            nodes.cend(),
@@ -32,7 +32,7 @@ BOOST_AUTO_TEST_CASE(test_tile)
    const auto rc = osrm.Tile(params, result);
    BOOST_CHECK(rc == Status::Ok);

-    BOOST_CHECK(result.size() > 115000);
+    BOOST_CHECK(result.size() > 114000);

    protozero::pbf_reader tile_message(result);
    tile_message.next();
@@ -54,7 +54,7 @@ BOOST_AUTO_TEST_CASE(test_tile)
        auto property_iter_pair = feature_message.get_packed_uint32();
        auto value_begin = property_iter_pair.begin();
        auto value_end = property_iter_pair.end();
-        BOOST_CHECK_EQUAL(std::distance(value_begin, value_end), 10);
+        BOOST_CHECK_EQUAL(std::distance(value_begin, value_end), 12);
        auto iter = value_begin;
        BOOST_CHECK_EQUAL(*iter++, 0); // speed key
        BOOST_CHECK_LT(*iter++, 128);  // speed value
@@ -65,10 +65,12 @@ BOOST_AUTO_TEST_CASE(test_tile)
        iter++;
        BOOST_CHECK_EQUAL(*iter++, 2); // data source key
        *iter++;                       // skip value check, can be valud uint32
-        BOOST_CHECK_EQUAL(*iter++, 3); // duration key
+        BOOST_CHECK_EQUAL(*iter++, 3); // weight key
+        BOOST_CHECK_GT(*iter++, 130);  // weight value
+        BOOST_CHECK_EQUAL(*iter++, 4); // duration key
        BOOST_CHECK_GT(*iter++, 130);  // duration value
        // name
-        BOOST_CHECK_EQUAL(*iter++, 4);
+        BOOST_CHECK_EQUAL(*iter++, 5);
        BOOST_CHECK_GT(*iter++, 130);
        BOOST_CHECK(iter == value_end);
        // geometry
@@ -137,7 +139,7 @@ BOOST_AUTO_TEST_CASE(test_tile)
        }
    }

-    BOOST_CHECK_EQUAL(number_of_speed_keys, 5);
+    BOOST_CHECK_EQUAL(number_of_speed_keys, 6);
    BOOST_CHECK_GT(number_of_speed_values, 128); // speed value resolution

    tile_message.next();
@@ -156,7 +158,7 @@ BOOST_AUTO_TEST_CASE(test_tile)
        BOOST_CHECK_EQUAL(feature_message.tag(), util::vector_tile::FEATURE_ATTRIBUTES_TAG);
        // properties
        auto feature_iter_pair = feature_message.get_packed_uint32();
-        BOOST_CHECK_EQUAL(std::distance(feature_iter_pair.begin(), feature_iter_pair.end()), 6);
+        BOOST_CHECK_EQUAL(std::distance(feature_iter_pair.begin(), feature_iter_pair.end()), 8);
        auto iter = feature_iter_pair.begin();
        BOOST_CHECK_EQUAL(*iter++, 0); // bearing_in key
        *iter++;
@@ -164,6 +166,8 @@ BOOST_AUTO_TEST_CASE(test_tile)
        *iter++;
        BOOST_CHECK_EQUAL(*iter++, 2); // cost key
        *iter++;                       // skip value check, can be valud uint32
+        BOOST_CHECK_EQUAL(*iter++, 3); // weight key
+        *iter++;                       // skip value check, can be valud uint32
        BOOST_CHECK(iter == feature_iter_pair.end());
        // geometry
        feature_message.next();
@@ -204,7 +208,7 @@ BOOST_AUTO_TEST_CASE(test_tile)
        }
    }

-    BOOST_CHECK_EQUAL(number_of_turn_keys, 3);
+    BOOST_CHECK_EQUAL(number_of_turn_keys, 4);
    BOOST_CHECK(number_of_turns_found > 700);
 }

@@ -236,6 +240,7 @@ BOOST_AUTO_TEST_CASE(test_tile_turns)
    std::vector<int> found_bearing_in_indexes;
    std::vector<int> found_turn_angles_indexes;
    std::vector<int> found_penalties_indexes;
+    std::vector<int> found_weight_indexes;

    const auto check_turn_feature = [&](protozero::pbf_reader feature_message) {
        feature_message.next(); // advance parser to first entry
@@ -250,7 +255,7 @@ BOOST_AUTO_TEST_CASE(test_tile_turns)
        BOOST_CHECK_EQUAL(feature_message.tag(), util::vector_tile::FEATURE_ATTRIBUTES_TAG);
        // properties
        auto feature_iter_pair = feature_message.get_packed_uint32();
-        BOOST_CHECK_EQUAL(std::distance(feature_iter_pair.begin(), feature_iter_pair.end()), 6);
+        BOOST_CHECK_EQUAL(std::distance(feature_iter_pair.begin(), feature_iter_pair.end()), 8);
        auto iter = feature_iter_pair.begin();
        BOOST_CHECK_EQUAL(*iter++, 0); // bearing_in key
        found_bearing_in_indexes.push_back(*iter++);
@@ -258,6 +263,8 @@ BOOST_AUTO_TEST_CASE(test_tile_turns)
        found_turn_angles_indexes.push_back(*iter++);
        BOOST_CHECK_EQUAL(*iter++, 2);              // cost key
        found_penalties_indexes.push_back(*iter++); // skip value check, can be valud uint32
+        BOOST_CHECK_EQUAL(*iter++, 3);              // weight key
+        found_weight_indexes.push_back(*iter++);    // skip value check, can be valud uint32
        BOOST_CHECK(iter == feature_iter_pair.end());
        // geometry
        feature_message.next();
@@ -333,6 +340,18 @@ BOOST_AUTO_TEST_CASE(test_tile_turns)
        0, 0, 0, 0, 0, 0, .1f, .1f, .3f, .4f, 1.2f, 1.9f, 5.3f, 5.5f, 5.8f, 7.1f, 7.2f, 7.2f};
    BOOST_CHECK(actual_turn_penalties == expected_turn_penalties);

+    // Verify that we got the expected turn penalties
+    std::vector<float> actual_turn_weights;
+    for (const auto &i : found_weight_indexes)
+    {
+        BOOST_CHECK(float_vals.count(i) == 1);
+        actual_turn_weights.push_back(float_vals[i]);
+    }
+    std::sort(actual_turn_weights.begin(), actual_turn_weights.end());
+    const std::vector<float> expected_turn_weights = {
+        0, 0, 0, 0, 0, 0, .1f, .1f, .3f, .4f, 1.2f, 1.9f, 5.3f, 5.5f, 5.8f, 7.1f, 7.2f, 7.2f};
+    BOOST_CHECK(actual_turn_weights == expected_turn_weights);
+
    // Verify the expected turn angles
    std::vector<std::int64_t> actual_turn_angles;
    for (const auto &i : found_turn_angles_indexes)
@@ -401,7 +420,7 @@ BOOST_AUTO_TEST_CASE(test_tile_speeds)
        auto property_iter_pair = feature_message.get_packed_uint32();
        auto value_begin = property_iter_pair.begin();
        auto value_end = property_iter_pair.end();
-        BOOST_CHECK_EQUAL(std::distance(value_begin, value_end), 10);
+        BOOST_CHECK_EQUAL(std::distance(value_begin, value_end), 12);
        auto iter = value_begin;
        BOOST_CHECK_EQUAL(*iter++, 0); // speed key
        found_speed_indexes.push_back(*iter++);
@@ -410,10 +429,12 @@ BOOST_AUTO_TEST_CASE(test_tile_speeds)
        found_component_indexes.push_back(*iter++);
        BOOST_CHECK_EQUAL(*iter++, 2); // data source key
        found_datasource_indexes.push_back(*iter++);
-        BOOST_CHECK_EQUAL(*iter++, 3); // duration key
+        BOOST_CHECK_EQUAL(*iter++, 3); // weight key
+        found_duration_indexes.push_back(*iter++);
+        BOOST_CHECK_EQUAL(*iter++, 4); // duration key
        found_duration_indexes.push_back(*iter++);
        // name
-        BOOST_CHECK_EQUAL(*iter++, 4);
+        BOOST_CHECK_EQUAL(*iter++, 5);
        found_name_indexes.push_back(*iter++);
        BOOST_CHECK(iter == value_end);
        // geometry
Author	SHA1	Message	Date
Patrick Niklaus	dc26ea2672	Update changelog	2017-03-29 22:08:39 +00:00
Michael Krasnyk	af94b3aebf	Flip source and target coordinates for segments in only backward direction	2017-03-29 22:06:34 +00:00
Michael Krasnyk	fd1d8052ac	Fix unnecessary duplication of backward segments for non-split edges for edges with the split flag false and in_backward_direction true if in_forward_direction is also true backward segments can be incorrectly duplicated	2017-03-29 22:06:22 +00:00
Kajari Ghosh	3b0b896501	throw errors after every write stream (#3843 )	2017-03-21 13:43:48 -07:00
Patrick Niklaus	cc0c28f366	Fix formating for cherry-picked commit	2017-03-09 12:05:01 +00:00
Michael Krasnyk	66cc99703c	Compute correct speed values in tile plugin	2017-03-09 11:49:10 +00:00
Patrick Niklaus	f7fbca3e5e	Update changelog	2017-03-03 17:27:28 +00:00
Patrick Niklaus	9695eaa28a	Fix tile datasize check	2017-03-03 16:03:47 +00:00
Patrick Niklaus	0d7546a510	Only allow restricted access for road of certain highway type	2017-03-03 16:03:36 +00:00
Michael Krasnyk	9dc8136240	Fix incorrect forward datasources getter in facade	2017-03-03 10:52:53 +00:00
Patrick Niklaus	a02a83f8bd	Other flakey test	2017-03-02 15:24:38 +00:00
Patrick Niklaus	9b18f55d29	Update changelog	2017-03-02 15:00:49 +00:00
Patrick Niklaus	a3434e7ae9	Mark test as flaky and track in own issue.	2017-03-02 14:59:10 +00:00
Patrick Niklaus	64b15028e4	Print 0 rate	2017-03-02 14:58:14 +00:00
karenzshea	d8e466fdaa	simplify test, add intersection turn test	2017-03-02 14:58:01 +00:00
karenzshea	43bbe8f2ae	add restricted penalty on NoTurn turns	2017-03-02 14:57:49 +00:00