add a geojson debugger that allows creating features for easy inspection

docs/developing.md

@@ -0,0 +1,61 @@
+# Developing / Debugging guidance code
+
+When changing guidance code, it is easy to introduce problems somewhere in the network.
+To get a better feeling of how your changes impact the OSRM experience, we offer ways of generating geojson output to inspect (e.g. with Mapbox Studio).
+When you do changes, make sure to inspect a few areas for the impact of the changes.
+
+## How to use GeoJson-Debugging
+
+This is a short guide to describe usage of our GeoJson debug logging mechanism. It is synchronized to guarantee thread-safe logging.
+
+## Outputting into a single file
+To use it, the inclusion of `geojson_debug_logger.hpp`    `geojson_debug_policies.hpp` from the `util` directory is required.
+
+Geojson debugging requires a few simple steps to output data into a feature collection.
+
+- Create a Scoped Guard that lives through the process and provide it with all required datastructures (it needs to span the lifetime of all your logging efforts)
+- At the location of the output, simply call Write with your own parameters.
+
+A guard (ScopedGeojsonLoggerGuard) requires a logging policy. Per default we provide a way of printing out node-ids as coordinates.
+
+The initialisation to do so looks like this:
+`util::ScopedGeojsonLoggerGuard<util::NodeIdVectorToLineString> geojson_guard( "debug.geojson", data-for-conversion);`
+
+The field `data-for-conversion` can be an arbitrary long set of features and needs to match the parameters used for constructing our policy (in this case `util::NodeIdVectorToLineString`).
+
+The policy itself offers a `operator()` accepting a `vector` of `NodeID`.
+
+For outputting data into our file (debug.geojson), we simply need to call the matching logging routine of the guard: `util::ScioedGeojsonLoggerGuard<util::NodeIdVectorToLineString>::Write(list_of_node_ids);`
+(or `guard.Write(list_of_node_ids)` if you created an instance).
+
+### Possible Scopeguard Location
+Think of the scopeguard as you would do of any reference. If you wan't to access to logging during a call, the guard object must be alive and valid.
+
+As an example: a good location to create the a scopeguard to log decisions in the edge-based-graph-factory would be right before we run it ([here](https://github.com/Project-OSRM/osrm-backend/blob/a933b5d94943bf3edaf42c84a614a99650d23cba/src/extractor/extractor.cpp#L497)). If you put `util::ScopedGeojsonLoggerGuard<util::NodeIdVectorToLineString> geojson_guard( "debug.geojson", node_coordinate_vector);` at that location, you can then print `util::ScopedGeojsonLoggerGuard<util::NodeIdVectorToLineString>::Write(list_of_node_ids);` anywhere within the `edge-based-graph-factory`.
+
+This location would enable call for all guidance related pre-processing which is called in the edge-based-graph-factory.
+Logging any turn-handler decisions, for example, would now be possible.
+
+## Limitations
+GeoJson debugging requires a single GeoJsonGuard (ScopedGeojsonLoggerGuard) for each desired output file.
+For each set of template parameters, only the most recent guard will actually produce output.
+
+`util::ScopedGeojsonLoggerGuard<util::NodeIdVectorToLineString> geojson_guard( "debug.geojson", data-for-conversion);`
+
+`util::ScopedGeojsonLoggerGuard<util::NodeIdVectorToLineString> geojson_guard( "debug-2.geojson", data-for-conversion);`
+
+Will not provide a way to write into two files, but only `debug-2` will actually contain features.
+
+We cannot nest-these calls.
+
+If we want to use the same policy for multiple files, we need to use different template parameters both for the logger and the guard.
+
+`util::ScopedGeojsonLoggerGuard<util::NodeIdVectorToLineString,0> geojson_guard( "debug.geojson", data-for-conversion);`
+
+`util::ScopedGeojsonLoggerGuard<util::NodeIdVectorToLineString,1> geojson_guard( "debug-2.geojson", data-for-conversion);`
+
+as well as,
+ 
+`util::ScopedGeojsonLoggerGuardr<util::NodeIdVectorToLineString,0>::Write(list_of_node_ids);`
+
+`util::ScopedGeojsonLoggerGuardr<util::NodeIdVectorToLineString,1>::Write(list_of_node_ids);`

include/extractor/geojson_debug_policies.hpp

@@ -0,0 +1,42 @@
+#ifndef OSRM_EXTRACTOR_GEOJSON_DEBUG_POLICIES
+#define OSRM_EXTRACTOR_GEOJSON_DEBUG_POLICIES
+
+#include <vector>
+
+#include "extractor/query_node.hpp"
+#include "util/coordinate.hpp"
+#include "util/json_container.hpp"
+#include "util/node_based_graph.hpp"
+#include "util/typedefs.hpp"
+
+#include "extractor/guidance/coordinate_extractor.hpp"
+#include "extractor/guidance/intersection.hpp"
+#include "extractor/guidance/toolkit.hpp"
+
+namespace osrm
+{
+namespace extractor
+{
+// generate a visualisation of an intersection, printing the coordinates used for angle calculation
+struct IntersectionPrinter
+{
+    IntersectionPrinter(const util::NodeBasedDynamicGraph &node_based_graph,
+                        const std::vector<extractor::QueryNode> &node_coordinates,
+                        const extractor::guidance::CoordinateExtractor &coordinate_extractor);
+
+    // renders the used coordinate locations for all entries/as well as the resulting
+    // intersection-classification
+    util::json::Array operator()(const NodeID intersection_node,
+                                 const extractor::guidance::Intersection &intersection,
+                                 const boost::optional<util::json::Object> &node_style = {},
+                                 const boost::optional<util::json::Object> &way_style = {}) const;
+
+    const util::NodeBasedDynamicGraph &node_based_graph;
+    const std::vector<extractor::QueryNode> &node_coordinates;
+    const extractor::guidance::CoordinateExtractor &coordinate_extractor;
+};
+
+} /* namespace extractor */
+} /* namespace osrm */
+
+#endif /* OSRM_EXTRACTOR_GEOJSON_DEBUG_POLICIES */

include/util/geojson_debug_logger.hpp

@@ -0,0 +1,185 @@
+#ifndef OSRM_GEOJSON_DEBUG_LOGGER_HPP
+#define OSRM_GEOJSON_DEBUG_LOGGER_HPP
+
+#include <fstream>
+#include <mutex>
+#include <string>
+
+#include "util/json_container.hpp"
+#include "util/json_renderer.hpp"
+#include "util/simple_logger.hpp"
+
+namespace osrm
+{
+namespace util
+{
+
+// in case we want to do scenario-based logging, we can specify a dedicated logging scenario to be
+// able to use multiple files for the same converter
+enum class LoggingScenario
+{
+    eDefault = 0
+};
+
+// forward declaration to become friends
+template <class geojson_conversion_policy, LoggingScenario scenario> class ScopedGeojsonLoggerGuard;
+
+// a geojson logger requires a conversion policy to transfer arbitrary input into viable geojson.
+// features of the same kind are stored in the same geojson file
+template <class geojson_conversion_policy, LoggingScenario scenario = LoggingScenario::eDefault>
+class GeojsonLogger
+{
+    // become friends with the guard
+    friend class ScopedGeojsonLoggerGuard<geojson_conversion_policy, scenario>;
+
+    // having these private enforces the guard to be used to initialise/close/write
+  private:
+    // cannot lock, is tooling for locked function
+    static void output(bool first, const util::json::Object &object)
+    {
+        if (!first)
+            ofs << ",\n\t\t";
+        // objects are simply forwarded
+        util::json::render(ofs, object);
+    }
+
+    // cannot lock, is tooling for locked function
+    static void output(bool first, const util::json::Array &array)
+    {
+        for (const auto object : array.values)
+        {
+            if (!first)
+                ofs << ",\n\t\t";
+
+            util::json::render(ofs, object.get<util::json::Object>());
+
+            first = false;
+        }
+    }
+
+    // writes a single feature into the Geojson file
+    template <typename... Args> static bool Write(Args &&... args)
+    {
+        // make sure to syncronize logging output, our writing should be sequential
+        std::lock_guard<std::mutex> guard(lock);
+
+        // if there is no logfile, we cannot write
+        if (!ofs.is_open() || (nullptr == policy))
+        {
+            // this can only happend between two guards when concurrent writing occurs
+            return false;
+        }
+
+        // use our policy to convert the arguments into geojson, this can be done in parallel
+        const auto json_object = (*policy)(std::forward<Args>(args)...);
+
+        // different features are separated by `,`
+        // since we are not building a full json collection of features, we have to do some of the
+        // bookeeping ourselves. This prevens us from doing one huge json object, if we are
+        // processing larger results
+        output(first, json_object);
+        first = false;
+
+        return static_cast<bool>(ofs);
+    }
+
+    // Opening a logger, we initialize a geojson feature collection
+    // to be later filled with additional data
+    static bool Open(const std::string &logfile)
+    {
+        // if a file is open, close it off. When this function returns, there should be an open
+        // logfile. However, there is a brief period between closing and openen where we could miss
+        // out on log output. Such a sad life
+        if (ofs.is_open())
+        {
+            util::SimpleLogger().Write(logWARNING)
+                << "Overwriting " << logfile
+                << ". Is this desired behaviour? If this message occurs more than once rethink the "
+                   "location of your Logger Guard.";
+            Close();
+        }
+
+        // make sure to syncronize logging output, cannot be locked earlier, since Close also locks
+        // and we don't want deadlocks
+        std::lock_guard<std::mutex> guard(lock);
+        ofs.open(logfile, std::ios::binary);
+
+        // set up a feature collection
+        ofs << "{\n\t\"type\": \"FeatureCollection\",\n\t\"features\": [\n\t";
+        // remember whether we need to output a colon
+        first = true;
+
+        return static_cast<bool>(ofs);
+    }
+
+    // finalising touches on the GeoJson
+    static bool Close()
+    {
+        // make sure to syncronize logging output
+        std::lock_guard<std::mutex> guard(lock);
+
+        // finishe the geojson feature collection and close it all off
+        if (ofs.is_open())
+        {
+            ofs << "\n\t]\n}";
+            ofs.close();
+        }
+
+        return static_cast<bool>(ofs);
+    }
+
+    static void SetPolicy(geojson_conversion_policy *new_policy) { policy = new_policy; }
+
+  private:
+    static bool first;
+    static std::mutex lock;
+    static std::ofstream ofs;
+
+    static geojson_conversion_policy *policy;
+};
+
+// make sure to do opening and closing of our guard
+template <class geojson_conversion_policy, LoggingScenario scenario = LoggingScenario::eDefault>
+class ScopedGeojsonLoggerGuard
+{
+  public:
+    template <typename... Args>
+    ScopedGeojsonLoggerGuard(const std::string &logfile, Args &&... args)
+        : policy(std::forward<Args>(args)...)
+    {
+        GeojsonLogger<geojson_conversion_policy, scenario>::Open(logfile);
+        GeojsonLogger<geojson_conversion_policy, scenario>::SetPolicy(&policy);
+    }
+
+    ~ScopedGeojsonLoggerGuard()
+    {
+        GeojsonLogger<geojson_conversion_policy, scenario>::Close();
+        GeojsonLogger<geojson_conversion_policy, scenario>::SetPolicy(nullptr);
+    }
+
+    template <typename... Args> static bool Write(Args &&... args)
+    {
+        return GeojsonLogger<geojson_conversion_policy, scenario>::Write(
+            std::forward<Args>(args)...);
+    }
+
+  private:
+    geojson_conversion_policy policy;
+};
+
+template <class geojson_conversion_policy, LoggingScenario scenario>
+bool GeojsonLogger<geojson_conversion_policy, scenario>::first;
+
+template <class geojson_conversion_policy, LoggingScenario scenario>
+std::mutex GeojsonLogger<geojson_conversion_policy, scenario>::lock;
+
+template <class geojson_conversion_policy, LoggingScenario scenario>
+std::ofstream GeojsonLogger<geojson_conversion_policy, scenario>::ofs;
+
+template <class geojson_conversion_policy, LoggingScenario scenario>
+geojson_conversion_policy *GeojsonLogger<geojson_conversion_policy, scenario>::policy;
+
+} // namespace util
+} // namespace osrm
+
+#endif /* OSRM_GEOJSON_DEBUG_LOGGER_HPP */

include/util/geojson_debug_policies.hpp

@@ -0,0 +1,58 @@
+#ifndef OSRM_GEOJSON_DEBUG_POLICIES
+#define OSRM_GEOJSON_DEBUG_POLICIES
+
+#include <vector>
+
+#include "extractor/query_node.hpp"
+#include "util/coordinate.hpp"
+#include "util/json_container.hpp"
+#include "util/node_based_graph.hpp"
+#include "util/typedefs.hpp"
+
+#include <boost/optional.hpp>
+
+namespace osrm
+{
+namespace util
+{
+
+struct NodeIdVectorToLineString
+{
+    NodeIdVectorToLineString(const std::vector<extractor::QueryNode> &node_coordinates);
+
+    // converts a vector of node ids into a linestring geojson feature
+    util::json::Object operator()(const std::vector<NodeID> &node_ids,
+                                  const boost::optional<json::Object> &properties = {}) const;
+
+    const std::vector<extractor::QueryNode> &node_coordinates;
+};
+
+struct CoordinateVectorToLineString
+{
+    // converts a vector of node ids into a linestring geojson feature
+    util::json::Object operator()(const std::vector<util::Coordinate> &coordinates,
+                                  const boost::optional<json::Object> &properties = {}) const;
+};
+
+struct NodeIdVectorToMultiPoint
+{
+    NodeIdVectorToMultiPoint(const std::vector<extractor::QueryNode> &node_coordinates);
+
+    // converts a vector of node ids into a linestring geojson feature
+    util::json::Object operator()(const std::vector<NodeID> &node_ids,
+                                  const boost::optional<json::Object> &properties = {}) const;
+
+    const std::vector<extractor::QueryNode> &node_coordinates;
+};
+
+struct CoordinateVectorToMultiPoint
+{
+    // converts a vector of node ids into a linestring geojson feature
+    util::json::Object operator()(const std::vector<util::Coordinate> &coordinates,
+                                  const boost::optional<json::Object> &properties = {}) const;
+};
+
+} /* namespace util */
+} /* namespace osrm */
+
+#endif /* OSRM_GEOJSON_DEBUG_POLICIES */

include/util/geojson_debug_policy_toolkit.hpp

@@ -0,0 +1,112 @@
+#ifndef OSRM_GEOJSON_DEBUG_POLICY_TOOLKIT_HPP
+#define OSRM_GEOJSON_DEBUG_POLICY_TOOLKIT_HPP
+
+#include "util/json_container.hpp"
+
+#include <algorithm>
+
+#include <boost/optional.hpp>
+
+namespace osrm
+{
+namespace util
+{
+
+enum GeojsonStyleSize
+{
+    tiny,
+    small,
+    medium,
+    large,
+    extra_large,
+    num_styles
+};
+
+enum GeojsonStyleColors
+{
+    red,
+    purple,
+    blue,
+    green,
+    yellow,
+    cyan,
+    brown,
+    pink,
+    num_colors
+};
+
+const constexpr char *geojson_debug_predifined_colors[GeojsonStyleColors::num_colors] = {
+    "#FF4848", "#800080", "#5757FF", "#1FCB4A", "#FFE920", "#29AFD6", "#B05F3C", "#FE67EB"};
+
+const constexpr double geojson_predefined_sizes[GeojsonStyleSize::num_styles] = {
+    2.0, 3.5, 5.0, 6.5, 8};
+
+inline util::json::Object makeStyle(const GeojsonStyleSize size_type,
+                                    const GeojsonStyleColors predefined_color)
+{
+    util::json::Object style;
+    // style everything, since we don't know the feature type
+    style.values["stroke"] = geojson_debug_predifined_colors[predefined_color];
+    style.values["circle-color"] = geojson_debug_predifined_colors[predefined_color];
+    style.values["line-width"] = geojson_predefined_sizes[size_type];
+    style.values["circle-radius"] = geojson_predefined_sizes[size_type];
+    return style;
+}
+
+struct CoordinateToJsonArray
+{
+    util::json::Array operator()(const util::Coordinate coordinate)
+    {
+        util::json::Array json_coordinate;
+        json_coordinate.values.push_back(static_cast<double>(toFloating(coordinate.lon)));
+        json_coordinate.values.push_back(static_cast<double>(toFloating(coordinate.lat)));
+        return json_coordinate;
+    }
+};
+
+struct NodeIdToCoordinate
+{
+    NodeIdToCoordinate(const std::vector<extractor::QueryNode> &node_coordinates)
+        : node_coordinates(node_coordinates)
+    {
+    }
+
+    const std::vector<extractor::QueryNode> &node_coordinates;
+
+    util::json::Array operator()(const NodeID nid)
+    {
+        auto coordinate = node_coordinates[nid];
+        CoordinateToJsonArray converter;
+        return converter(coordinate);
+    }
+};
+
+inline util::json::Object makeFeature(std::string type,
+                                      util::json::Array coordinates,
+                                      const boost::optional<util::json::Object> &properties = {})
+{
+    util::json::Object result;
+    result.values["type"] = "Feature";
+    result.values["properties"] = properties ? *properties : util::json::Object();
+    util::json::Object geometry;
+    geometry.values["type"] = std::move(type);
+    geometry.values["properties"] = util::json::Object();
+    geometry.values["coordinates"] = std::move(coordinates);
+    result.values["geometry"] = geometry;
+
+    return result;
+}
+
+inline util::json::Array makeJsonArray(const std::vector<util::Coordinate> &input_coordinates)
+{
+    util::json::Array coordinates;
+    std::transform(input_coordinates.begin(),
+                   input_coordinates.end(),
+                   std::back_inserter(coordinates.values),
+                   CoordinateToJsonArray());
+    return coordinates;
+}
+} // namespace util
+} // namespace osrm
+
+#endif /* OSRM_GEOJSON_DEBUG_POLICY_TOOLKIT_HPP */

src/extractor/geojson_debug_policies.cpp

@@ -0,0 +1,68 @@
+#include "extractor/geojson_debug_policies.hpp"
+#include "util/coordinate.hpp"
+#include "util/geojson_debug_policy_toolkit.hpp"
+
+#include <algorithm>
+
+namespace osrm
+{
+namespace extractor
+{
+
+IntersectionPrinter::IntersectionPrinter(
+    const util::NodeBasedDynamicGraph &node_based_graph,
+    const std::vector<extractor::QueryNode> &node_coordinates,
+    const extractor::guidance::CoordinateExtractor &coordinate_extractor)
+    : node_based_graph(node_based_graph), node_coordinates(node_coordinates),
+      coordinate_extractor(coordinate_extractor){};
+
+util::json::Array IntersectionPrinter::
+operator()(const NodeID intersection_node,
+           const extractor::guidance::Intersection &intersection,
+           const boost::optional<util::json::Object> &node_style,
+           const boost::optional<util::json::Object> &way_style) const
+{
+    // request the number of lanes. This process needs to be in sync with what happens over at
+    // intersection_generator
+    const auto intersection_lanes =
+        extractor::guidance::getLaneCountAtIntersection(intersection_node, node_based_graph);
+
+    std::vector<util::Coordinate> coordinates;
+    coordinates.reserve(intersection.size());
+    coordinates.push_back(node_coordinates[intersection_node]);
+
+    const auto road_to_coordinate = [&](const extractor::guidance::ConnectedRoad &connected_road) {
+        const constexpr auto FORWARD = false;
+        const auto to_node = node_based_graph.GetTarget(connected_road.turn.eid);
+        return coordinate_extractor.GetCoordinateAlongRoad(
+            intersection_node, connected_road.turn.eid, FORWARD, to_node, intersection_lanes);
+    };
+
+    std::transform(intersection.begin(),
+                   intersection.end(),
+                   std::back_inserter(coordinates),
+                   road_to_coordinate);
+
+    util::json::Array features;
+    features.values.push_back(
+        util::makeFeature("MultiPoint", makeJsonArray(coordinates), node_style));
+
+    if (coordinates.size() > 1)
+    {
+        std::vector<util::Coordinate> line_coordinates(2);
+        line_coordinates[0] = coordinates.front();
+        const auto coordinate_to_line = [&](const util::Coordinate coordinate) {
+            line_coordinates[1] = coordinate;
+            return util::makeFeature("LineString", makeJsonArray(line_coordinates), way_style);
+        };
+
+        std::transform(std::next(coordinates.begin()),
+                       coordinates.end(),
+                       std::back_inserter(features.values),
+                       coordinate_to_line);
+    }
+    return features;
+}
+
+} /* namespace extractor */
+} /* namespace osrm */

src/util/geojson_debug_policies.cpp

@@ -0,0 +1,71 @@
+#include "util/geojson_debug_policies.hpp"
+#include "util/coordinate.hpp"
+#include "util/geojson_debug_policy_toolkit.hpp"
+
+#include <algorithm>
+
+namespace osrm
+{
+namespace util
+{
+
+//----------------------------------------------------------------
+NodeIdVectorToLineString::NodeIdVectorToLineString(
+    const std::vector<extractor::QueryNode> &node_coordinates)
+    : node_coordinates(node_coordinates)
+{
+}
+
+// converts a vector of node ids into a linestring geojson feature
+util::json::Object NodeIdVectorToLineString::
+operator()(const std::vector<NodeID> &node_ids,
+           const boost::optional<json::Object> &properties) const
+{
+    util::json::Array coordinates;
+    std::transform(node_ids.begin(),
+                   node_ids.end(),
+                   std::back_inserter(coordinates.values),
+                   NodeIdToCoordinate(node_coordinates));
+
+    return makeFeature("LineString", std::move(coordinates), properties);
+}
+
+//----------------------------------------------------------------
+NodeIdVectorToMultiPoint::NodeIdVectorToMultiPoint(
+    const std::vector<extractor::QueryNode> &node_coordinates)
+    : node_coordinates(node_coordinates)
+{
+}
+util::json::Object NodeIdVectorToMultiPoint::
+operator()(const std::vector<NodeID> &node_ids,
+           const boost::optional<json::Object> &properties) const
+{
+    util::json::Array coordinates;
+    std::transform(node_ids.begin(),
+                   node_ids.end(),
+                   std::back_inserter(coordinates.values),
+                   NodeIdToCoordinate(node_coordinates));
+
+    return makeFeature("MultiPoint", std::move(coordinates), properties);
+}
+
+//----------------------------------------------------------------
+util::json::Object CoordinateVectorToMultiPoint::
+operator()(const std::vector<util::Coordinate> &input_coordinates,
+           const boost::optional<json::Object> &properties) const
+{
+    const auto coordinates = makeJsonArray(input_coordinates);
+    return makeFeature("MultiPoint", std::move(coordinates), properties);
+}
+
+//----------------------------------------------------------------
+util::json::Object CoordinateVectorToLineString::
+operator()(const std::vector<util::Coordinate> &input_coordinates,
+           const boost::optional<json::Object> &properties) const
+{
+    const auto coordinates = makeJsonArray(input_coordinates);
+    return makeFeature("LineString", std::move(coordinates), properties);
+}
+
+} /* namespace util */
+} /* namespace osrm */