Currently route results are annotated with additional path information,
such as geometries, turn-by-turn steps and other metadata.
These annotations are generated if they are not requested or returned
in the response.
Datasets needed to generate these annotations are loaded and available
to the OSRM process even when unused.
This commit is a first step towards making the loading of these datasets
optional. We refactor the code so that route annotations are only
generated if explicitly requested and needed in the response.
Specifically, we change the following annotations to be lazily generated:
- Turn-by-turn steps
- Route Overview geometry
- Route segment metadata
For example. a /route/v1 request with
steps=false&overview=false&annotations=false
would no longer call the following data facade methods:
- GetOSMNodeIDOfNode
- GetTurnInstructionForEdgeID
- GetNameIndex
- GetNameForID
- GetRefForID
- GetTurnInstructionForEdgeID
- GetClassData
- IsLeftHandDriving
- GetTravelMode
- IsSegregated
- PreTurnBearing
- PostTurnBearing
- HasLaneData
- GetLaneData
- GetEntryClass
Requests that include segment metadata and/or overview geometry
but not turn-by-turn instructions will also benefit from this,
although there is some interdependency with the step instructions
- a call to GetTurnInstructionForEdgeID is still required.
Requests for OSM annotations will understandably still need to
call GetOSMNodeIDOfNode.
Making these changes unlocks the optional loading of data contained in
the following OSRM files:
- osrm.names
- osrm.icd
- osrm.nbg_nodes (partial)
- osrm.ebg_nodes (partial)
- osrm.edges
The internal representation of turn restrictions expects only one
`from` way and only one `to` way.
`no_entry` and `no_exit` turn restrictions can have multiple `from` and
`to` ways respectively. This means they are not fully supported by
OSRM's restriction parser.
We complete support for these turn restriction types by parsing all
ways and converting a valid restriction with multiple `from`/`to` members
into multiple internal restrictions.
The data facade interface contains numerous methods for looking up
datapoints by identifiers.
Many of the parameters use the NodeID or EdgeID types. However, these two
identifier types are used for representing three different contexts:
1. Node-based graph edges and nodes
2. Edge-based graph edges and nodes
3. Packed geometries
Consider the use of identifier parameters in these examples:
---
GetWeightPenaltyForEdgeID(const EdgeID id) <- edge-based edge
GetUncompressedForwardWeights(const EdgeID id) <- packed geometry
IsLeftHandDriving(const NodeID id) <- edge-based node
GetBearingClass(const NodeID node) <- node-based node
---
This mixing of contexts within the same interface makes it
difficult to understand the relationships and dependencies between
the OSRM datasets.
For 1. and 2. we continue to use the NodeID and EdgeID types, but
change the interface parameter names to identify them as
edge-based or node-based graph properties.
For 3. we define a new type definition, PackedGeometryID.
These changes are to aid with readability. A next step would be
to strongly type these definitions, leveraging the Alias template
already used for OSM identifiers.
- Fix typo in util function name for_each_indexed.
- Use the overloaded functions for_each_indexed and for_each_pair
with a container argument where possible to improve readability.
* Add missing profile name to library extract test.
* Support both tzid and TZID properties on timezone geometry. Improve validation of timezone polygons.
* Missing tzid property wasn't a geojson validation issue, shouldn't have been tested there.
* Use filesystem glob to loop over all test executables so we don't miss any in the future.
Co-authored-by: Michael Bell <michael@mjjbell.com>
A request to osrm-routed can be assigned to a thread which
is currently busy processing another request, even when there
are other threads/cores available. This unnecessarily delays
the response, and can make requests appear to hang when
awaiting CPU intensive requests to finish.
The issue looks like a bug in Boost.Asio multithreaded
networking stack.
osrm-routed server implementation is
heavily influenced by the HTTP server 3 example in the
Boost.Asio docs. By upgrading to Boost 1.70 and updating the
server connections to match the example provided in the 1.70
release, the problem is resolved.
The diff of the changes to the Boost.Asio stack are
vast, so it's difficult to identify the exact cause. However
the implementation change is to push the strand of execution
into the socket (and timer) objects, which suggests it could
fix the type of threading issue we are observing.
Each MLD cell has source and destination nodes.
MLD is keeping a |source| x |destination| sized table
for various metrics (distances, durations, etc) from each
source to all destinations in a cell.
It stores all of the values for a metric in one large array, with
an offset for each cell to find its values. The offset is currently
limited to 32 bit values, which overflows on very large graphs
(e.g. Planet OSM).
We fix this by changing the offsets to be uint64_t types.
The generation of level masks for compactly storing partition cells
supports sizes that can be stored in 64 bits.
The current implementation fails if the total bit sum is 64 bits
exactly. A bit shift mechanism is used that is undefined when the
shift size is equal to the bit size of the underlying type. This
generates an incorrect mask value.
We fix this by adding a special case for a 64 bit offset. Given this
code is called at most |level| times, there will be no effect on
performance. We also update the assertions to reflect 64 bit masks
are now supported.
osrm-routed does not immediately clean up a keep-alive connection
when the client closes it. Instead it waits for five seconds
of inactivity before removing.
Given a setup with low file limits and clients opening and
closing a lot of keep-alive connections, it's possible for
osrm-routed to run out of file descriptors whilst it waits for
the clean-up to trigger.
Furthermore, this causes the connection acceptor loop to exit.
Even after the old connections are cleaned up, new ones
will not be created. Any new requests will block until the
server is restarted.
This commit improves the situation by:
- Immediately closing connections on error. This includes EOF errors
indicating that the client has closed the connection. This releases
resources early (including the open file) and doesn't wait for the
timer.
- Log when the acceptor loop exits. Whilst this means the behaviour
can still occur for reasons other than too many open files,
we will at least have visibility of the cause and can investigate further.
OSM node 2^33 was created in early April 2021. This and all
subsequently created IDs will be overflowing OSRM node storage
which only support 33 bit IDs.
Bump the number of bits to 34 to double node ID capacity. This
is a breaking change to the data format as it alters the layout
of .osrm.nbg_nodes.
Removes the breaking libosrm API change by adding the old interface to
the new. This does not introduce any new breaks.
The downside of this is that it allows for multiple ways to
return JSON responses.
When using non-default constructors for the API parameter classes,
vector arguments like coordinates and hints are copied at least once
(twice when passed as lvalue arguments).
Enable perfect forwarding of BaseParameter arguments and
pass-by-value in the constructor that uses the argument. This
ensures we copy at most once (zero for rvalue arguments).
Currently OSRM only supports turn restrictions with a single via-node or one
via-way. OSM allows for multiple via-ways to represent longer and more
complex restrictions.
This PR extends the use of duplicate nodes for representng via-way turn
restrictions to also support multi via-way restrictions. Effectively, this
increases the edge-based graph size by the number of edges in multi via-way
restrictions. However, given the low number of these restrictions it
has little effect on total graph size.
In addition, we add a new step in the extraction phase that constructs
a restriction graph to support more complex relationships between restrictions,
such as nested restrictions and overlapping restrictions.
* Use Nan:: getters and setters for node 12/14 compatibility
* Drop support for publishing node 8 bindings, add publishing support for node 12, 14
Co-authored-by: Daniel Patterson <danpat@danpat.net>
As part of graph contraction, node renumbering leads to
in-place permuting of graph state, including boolean vector elements.
std::vector<bool> returns proxy objects when referencing individual
bits. To correctly swap bool elements using MSVC, we need to explicitly
apply std::vector<bool>::swap.
Making this change fixes osrm-contract on Windows.
We also correct failing tests and other undefined behaviours
(mainly iterator access outside boundaries) highlighted by MSVC.
For very large graphs, generation of MLD level masks fail on Windows
due to bit shift overflow of unsigned long values.
Correct by using unsigned long long literals, which are 64 bit on
all major systems.
In cases where we are unable to find a phantom node for an input
coordinate, we return an error indicating which coordinate failed.
This would always refer to the coordinate with index equal to the
number of valid phantom nodes found.
We fix this by instead returning the first index for which a
phantom node could not be found.
It actually makes no sence, as the only output on the 'Nearest' service
is a list of wypoints. On the other hand it can be used now as a simple
health check.
* Replaced Response union with nullable fields
* "Ok" code replaced with a boolean and a separate structure for a error code and message.
* Inlined geometry, thus removing one layer of indirection.
Same as for Route service, you can now specify either json or flatbuffers
as output format for the other services (except Tile service, where it
makes no sense).
In file included from /home/travis/build/Project-OSRM/osrm-backend/include/storage/shared_data_index.hpp:4,
from /home/travis/build/Project-OSRM/osrm-backend/include/engine/datafacade/contiguous_block_allocator.hpp:4,
from /home/travis/build/Project-OSRM/osrm-backend/include/engine/datafacade/contiguous_internalmem_datafacade.hpp:5,
from /home/travis/build/Project-OSRM/osrm-backend/include/engine/plugins/nearest.hpp:5,
from /home/travis/build/Project-OSRM/osrm-backend/src/engine/plugins/nearest.cpp:1:
/home/travis/build/Project-OSRM/osrm-backend/include/storage/shared_datatype.hpp:102:26: warning: inline function ‘virtual void* osrm::storage::BaseDataLayout::GetBlockPtr(void*, const string&) const’ used but never defined
virtual inline void *GetBlockPtr(void *base_ptr, const std::string &name) const = 0;
Includes all edges in the rtree, but adds an `is_startpoint` flag to each. Most plugin behaviour remains unchanged (non-startpoint edges aren't used as snapping candidates), but for map matching, we allow snapping to any edge. This fixes map-matching across previously non-is_startpoint edges, like ferries, private service roads, and a few others.
* Revert "Remove estimated_cells value in the response."
This reverts commit 364e35af06.
* Update changelog.
* fix linting
* adjust fallback_speed check
* change [].includes to [].indexOf !== -1 for compatibility with node 4
* change param name
* more cuke tests
* fix formatting
* add a multiplier to the matrix
* add rounding
* remove scale_factor restrictions
* clamp for overflow error
* update check to match error message
* enforce clamping on < 0 and increase test coverage
* add an invalid scale_factor value to node tests
* increase test coverage
* changelog
* Revert "Update changelog"
This reverts commit 9b779c704f.
* Revert "Fix formating"
This reverts commit 5bd7d04fe3.
* Revert "Fix bug in computation of distance offset for phantom node"
This reverts commit 0f78f7b2cc.
* Revert "Adjust text cases for flightly different matching due to rounding"
This reverts commit 8473be69d2.
* Revert "Round network distance to deci-meter to retain previous behavior"
This reverts commit c0124f7d77.
* Revert "Preserve heap state in map matching"
This reverts commit b630b4e32a.
* Revert "Use distance functions from many to many"
This reverts commit 89fabc1b9c.
* Revert "Use FCC algorithm for map matching distance calculation"
This reverts commit a649a8a5cf.
