Moves Linux CI builds to the latest Ubuntu LTS.
- Bumps the GCC matrix to versions {7,8,9,10,11}, making 9 the default
for testing non-standard builds.
- Bump Node matrix to {12,14,16,LTS,latest} now that 10 is EOL.
- Fixes to CI builds due to library changes on newer distro.
A request to osrm-routed can be assigned to a thread which
is currently busy processing another request, even when there
are other threads/cores available. This unnecessarily delays
the response, and can make requests appear to hang when
awaiting CPU intensive requests to finish.
The issue looks like a bug in Boost.Asio multithreaded
networking stack.
osrm-routed server implementation is
heavily influenced by the HTTP server 3 example in the
Boost.Asio docs. By upgrading to Boost 1.70 and updating the
server connections to match the example provided in the 1.70
release, the problem is resolved.
The diff of the changes to the Boost.Asio stack are
vast, so it's difficult to identify the exact cause. However
the implementation change is to push the strand of execution
into the socket (and timer) objects, which suggests it could
fix the type of threading issue we are observing.
* Fix test cache to consider MLD executable changes
Currently the test cache is not invalidated if changes to
osrm-partition or osrm-customize executables. This is problematic
when these executables make breaking changes to the data format.
We fix this by including both in the list of executables that
can invalidate the test cache.
* Executables use hyphens
Each MLD cell has source and destination nodes.
MLD is keeping a |source| x |destination| sized table
for various metrics (distances, durations, etc) from each
source to all destinations in a cell.
It stores all of the values for a metric in one large array, with
an offset for each cell to find its values. The offset is currently
limited to 32 bit values, which overflows on very large graphs
(e.g. Planet OSM).
We fix this by changing the offsets to be uint64_t types.
The generation of level masks for compactly storing partition cells
supports sizes that can be stored in 64 bits.
The current implementation fails if the total bit sum is 64 bits
exactly. A bit shift mechanism is used that is undefined when the
shift size is equal to the bit size of the underlying type. This
generates an incorrect mask value.
We fix this by adding a special case for a 64 bit offset. Given this
code is called at most |level| times, there will be no effect on
performance. We also update the assertions to reflect 64 bit masks
are now supported.
osrm-routed does not immediately clean up a keep-alive connection
when the client closes it. Instead it waits for five seconds
of inactivity before removing.
Given a setup with low file limits and clients opening and
closing a lot of keep-alive connections, it's possible for
osrm-routed to run out of file descriptors whilst it waits for
the clean-up to trigger.
Furthermore, this causes the connection acceptor loop to exit.
Even after the old connections are cleaned up, new ones
will not be created. Any new requests will block until the
server is restarted.
This commit improves the situation by:
- Immediately closing connections on error. This includes EOF errors
indicating that the client has closed the connection. This releases
resources early (including the open file) and doesn't wait for the
timer.
- Log when the acceptor loop exits. Whilst this means the behaviour
can still occur for reasons other than too many open files,
we will at least have visibility of the cause and can investigate further.
In newer versions of cmake, FindBoost uses Imported Targets for library
component variables, rather than file paths to the Boost libraries.
cmake uses these targets when linking (e.g. target_link_library) and
knows how to correctly substitute the values. However, the OSRM
pkg-config file that we generate doesn't do this, and ends up writing
the actual target symbols, hence the errors trying to link Boost::<component>.
To fix this for newer cmake versions, we create an intermediate configure step
that references the linker files for the imported targets. This is followed
by a generate step that performs the correct substitution.
See this thread for more details: https://cmake.org/pipermail/cmake/2018-December/068812.html
This is backwards compatible to the existing min cmake version (3.1).
However, building using cmake 3.1 fails with a package.json parsing error,
so this commit also bumps the min version to 3.2.
Duplicate restriction nodes in the edge-based-graph are currently
not in included in a mapping (.osrm.cnbg_to_ebg) from
node-based-graph edges to edge-based-graph nodes.
This mapping is used by the MLD partitioner to assign EBG nodes
to partitions.
The omission from the mapping means all restriction nodes are
included in a special 'invalid' partition. This special partition
will break the geolocation properties of the multi-level hierarchy.
The partition and its super levels will have a large number of
border nodes and very few internal paths between them.
Given the partitioner is the only consumer of the mapping, we fix
the issue by including the duplicate restriction nodes in the mapping,
so that they are correctly assigned to a partition.
This has measurable improvement on MLD routing.
For a country-sized routing network, the fix reduces routing and table
request computation time by ~2% and ~6% respectively.
Replace Travis for continuous integration with Github Actions.
The Github Actions pipeline is functionally equivalent, with
all the same build permutations supported.
Whilst the Github Actions offering is broadly equivalent to
Travis, a few changes have been made as part of the migration.
- The 'core' and 'optional' Travis stages have been consolidated
into one build matrix. This is due to the current inability in
Github Actions to share build steps between jobs, so this avoids
having to duplicate the steps.
Optional stage jobs will now run in parallel with core jobs,
but they still remain optional in the sense that they don't fail
the build.
- A number of existing Github Action plugins are used to replace
functionality provided by Travis or other tools:
Node setup, caching, Codecov, publishing release artifacts.
- Linux builds are updated to build on Ubuntu 18.04.
MacOS builds are updated to run on 10.15. Similar to the
Travis Xenial upgrade attempt, some changes are required due
to underlying platform and compiler upgrades. This means some
Node 10 toolchains will no longer be supported.
Whilst there is opportunity to upgrade some dependencies and
make the CI steps more idiomatic, I've left this for future changes
and just focussed on functional replication.
* Use Github Releases for hosting node binaries
Replaces S3 hosting of node binaries with Github Releases.
`node-pre-gyp publish` works exclusively with S3, so upload step
is now performed by the Travis deployment provider.
The behaviour for the package user should not change.
When building a new version tag, Travis will create a release for the
tag if it does not already exist.
* Switch to the osrm-release-automation machine account rather than personal credentials.
Co-authored-by: Daniel Patterson <danpat@danpat.net>
OSM node 2^33 was created in early April 2021. This and all
subsequently created IDs will be overflowing OSRM node storage
which only support 33 bit IDs.
Bump the number of bits to 34 to double node ID capacity. This
is a breaking change to the data format as it alters the layout
of .osrm.nbg_nodes.
When using process memory, MLD cell metrics are loaded twice from
.osrm.cell_metrics - once when loading static data, and again when
loading updatable data. The former appears to be the mistake,
as .osrm.cell_metrics is only listed in `GetUpdatableFiles`.
OSM data contains many mistakes that tag kerbs as highway barriers
when instead they are only describing highway crossings.
This PR updates the default car profile to handle these mistakes
and unblock routing on the affected highways.
Removes the breaking libosrm API change by adding the old interface to
the new. This does not introduce any new breaks.
The downside of this is that it allows for multiple ways to
return JSON responses.
When using non-default constructors for the API parameter classes,
vector arguments like coordinates and hints are copied at least once
(twice when passed as lvalue arguments).
Enable perfect forwarding of BaseParameter arguments and
pass-by-value in the constructor that uses the argument. This
ensures we copy at most once (zero for rvalue arguments).
Regardless of any copy elision on the returned pair value, the
duration and distance results are always copied.
Fix this by passing rvalue references to std::make_pair.
Currently OSRM only supports turn restrictions with a single via-node or one
via-way. OSM allows for multiple via-ways to represent longer and more
complex restrictions.
This PR extends the use of duplicate nodes for representng via-way turn
restrictions to also support multi via-way restrictions. Effectively, this
increases the edge-based graph size by the number of edges in multi via-way
restrictions. However, given the low number of these restrictions it
has little effect on total graph size.
In addition, we add a new step in the extraction phase that constructs
a restriction graph to support more complex relationships between restrictions,
such as nested restrictions and overlapping restrictions.
* Use Nan:: getters and setters for node 12/14 compatibility
* Drop support for publishing node 8 bindings, add publishing support for node 12, 14
Co-authored-by: Daniel Patterson <danpat@danpat.net>
As part of graph contraction, node renumbering leads to
in-place permuting of graph state, including boolean vector elements.
std::vector<bool> returns proxy objects when referencing individual
bits. To correctly swap bool elements using MSVC, we need to explicitly
apply std::vector<bool>::swap.
Making this change fixes osrm-contract on Windows.
We also correct failing tests and other undefined behaviours
(mainly iterator access outside boundaries) highlighted by MSVC.
For very large graphs, generation of MLD level masks fail on Windows
due to bit shift overflow of unsigned long values.
Correct by using unsigned long long literals, which are 64 bit on
all major systems.
In situations where there is not a valid source or target phantom
node (e.g. when snapping to an edge with a zero weight), a
heap assertion will fail in the MLD alternative search code.
We fix this by checking for empty heaps before proceeding with
the search.
Fixes#5788
Table queries where source and destination are phantom nodes
on the same one-way segment can fail to find valid routes.
This is due to a bug in the MLD table generation for the
special case where the query can be simplified to a
one-to-many search.
If the destination is before the source on the one-way segment,
it will fail to find a route.
We fix this case by not marking the node as visited at the start,
so that valid paths to this node can be found later in the search.
We also remove redundant initialization for the source
node as the same actions are performed by a search step.
In cases where we are unable to find a phantom node for an input
coordinate, we return an error indicating which coordinate failed.
This would always refer to the coordinate with index equal to the
number of valid phantom nodes found.
We fix this by instead returning the first index for which a
phantom node could not be found.
In the case of a shared object compilation, the resulting binaries need
that library. It was not installed along the others, leading to failures
when running osrm-extract.
Fixes#5603
* In Belgium the maximum speed in rural areas is 70 in the region Flanders
* parse maxspeed using source:maxspeed and maxspeed:type tags
* add changelog
* make maxspeed:advisory more important than maxspeed
* add test for source:maxspeed
Includes all edges in the rtree, but adds an `is_startpoint` flag to each. Most plugin behaviour remains unchanged (non-startpoint edges aren't used as snapping candidates), but for map matching, we allow snapping to any edge. This fixes map-matching across previously non-is_startpoint edges, like ferries, private service roads, and a few others.
* Revert "Remove estimated_cells value in the response."
This reverts commit 364e35af06.
* Update changelog.
* fix linting
* adjust fallback_speed check
* change [].includes to [].indexOf !== -1 for compatibility with node 4
* change param name
* more cuke tests
* fix formatting
* add a multiplier to the matrix
* add rounding
* remove scale_factor restrictions
* clamp for overflow error
* update check to match error message
* enforce clamping on < 0 and increase test coverage
* add an invalid scale_factor value to node tests
* increase test coverage
* changelog
* Prepare RC.1
* Bump version.
* remove destination/sources length <= coordinates length check (#5289)
* Add node 10 builds to travis (#5246)
* Add node 10 builds to travis
* Add changelog
* bump version to rc5
* Fix fallback_speed vector access (#5291)
* add failing cuke test
* correctly access durations vector
* changelog
* one more cuke test
* bump rc version
* 5.20.0
* remove line from changelog about commit that isn't actually in here
* update CHANGELOG and osrm version in package.json for v5.20.0
* bump to restart appveyor
* Revert "Update changelog"
This reverts commit 9b779c704f.
* Revert "Fix formating"
This reverts commit 5bd7d04fe3.
* Revert "Fix bug in computation of distance offset for phantom node"
This reverts commit 0f78f7b2cc.
* Revert "Adjust text cases for flightly different matching due to rounding"
This reverts commit 8473be69d2.
* Revert "Round network distance to deci-meter to retain previous behavior"
This reverts commit c0124f7d77.
* Revert "Preserve heap state in map matching"
This reverts commit b630b4e32a.
* Revert "Use distance functions from many to many"
This reverts commit 89fabc1b9c.
* Revert "Use FCC algorithm for map matching distance calculation"
This reverts commit a649a8a5cf.
As I mentioned in the issue #5156, I met below issue on my Win10+WSL(Ubuntu) env:
The remote debugger (VSCode on Win10, gdb on Ubuntu 18.04 LTS) works well from the beginning of the main() function. But when I step over the code pthread_sigmask(SIG_BLOCK, &new_mask, &old_mask); (src/tools/routed.cpp(289)), below breakpoints can not work and displayed unverified breakpoint.
Then I found that gdb breakpoint need at least SIGTRAP, SIGSTOP to work (Please refer to [how debugger works](http://www.alexonlinux.com/how-debugger-works) for more details), but all signals are blocked in the source code until server initialized done.
In my understanding, block all signals DO NOT make sense for this osrm-routed process. Only several signals (SIGINT, SIGQUIT, SIGTERM) are expected to wait. So I made the change and it works well for me then.
* Added post process logic to collapse segregated turn instructions
* format updates
* Fixed coordinates to reflect reality
Updated left turn road name
* fixed coordinates to fix test
* Skip last step when processing segregated steps
* updated segregated turn test
* Updated segregated test
* Updated test: Segregated Intersection, Cross Belonging to Correct Street - features/guidance/collapse.feature:79
* Fixed all but one for features/guidance/collapse.feature:124
* Fixed Scenario: Partly Segregated Intersection, Two Segregated Roads, Intersection belongs to Second - features/guidance/collapse.feature:219
* Fixed 7 of th 9 failures for Scenario: Partly Segregated Intersection, Two Segregated Roads, Intersection belongs to Second - features/guidance/collapse.feature:219
* Fixed 7 of the 9 failures for Scenario: Segregated Intersection, Cross Belonging to Mixed Streets - Slight Angles (2) - features/guidance/collapse.feature:318
* Fixed Scenario: Segregated Intersection into Slight Turn - features/guidance/collapse.feature:581
* Updated Scenario: U-turn after a traffic light - features/guidance/turn-lanes.feature:1220
* Updated how we combine segregated steps
* Added test to Verify end of road left turn across divided roads
* Fixed divided highwat tests
* Fixed test failure
* fixed Scenario: Partitioned turn, Slight Curve - maxspeed - features/guidance/turn-lanes.feature:936
* Fixed Scenario: Partitioned turn, Slight Curve - features/guidance/turn-lanes.feature:961
* Added strategies to combine segrgated intersections
* Added setModifier alias for readability
* Added strategies to combine segrgated intersections
* Format updates
* Fixes segregated indentification to not mark `circular` edge as segregated
* Added intersection prior to turn so we still call out end of road
* updated expectation to be turn instead of continue
* Confirmed with @oxidase that the lane information if correct - updated the expectation
* Added logic to handle wider straights
Fixed tests
* Update CHANGELOG.md
Added #4925
* Removed TODO
* Process straight step prior to wider straight step
* upgrade to api version 4
* set bike pushing speed to 4km/h and add mode change penalty
* set cycleways for two-way and one-way roads according correctly
* add changelog
* adjust tests according to change of the walking speed
* adjust tests according to new behaviour of opposite, track and lane in one-ways
* refactor the cycleway identification
* comment to understand weird counterintuitive testcase
* adjust sliproad test because walking speed is now slower
* Initial internal intersection updates
paired with @oxidase and @kdiluca
TODO fix tests and add in new ones
* Added Internal Intersection Model
* removed debug info
* updates per PR 4845
* fixing build errors
* fixing all compile errors
* fixed EdgeID param
* Added is_internal_straight lambda
Added/Updated constexpr names and values
* added rejection case turn degree logic
* debug logging
* added turn angle logic to reject if there are incoming edges that have opposite turn degrees than outgoing edges or if the outgoing edges have opposing turn degrees; also merged with master v5.16
* fixed formatting
* fix to decrease tile size based on latest turn angle internal intersection updates
* Removed breaks
Breaks in code were a mistake and caused a change in the internal intersection identification.
* Update segregated_intersection_classification.cpp
* Update CHANGELOG.md
Added CHANGED #4845: Updated segregated intersection identification to Unreleased
* set and store highway and access classification for the turn function
* expose highway turn classification and access turn classification and speed to the lua profile turn function
* expose whether connection road at turn is incoming or outgoing
* add lua tests for exposed information to turn function
* update docs about attributes in process_turn
* add turn_classification info to docs
* adding warning if uturn and intersection dont match
* handle u turns that do not turn into intersection[0]
* split OSM link generation in an accessible coordinate function
- separates node-based graph creation and compression from edge-based graph creation
- moves usage of edge-based node data-container to pre-processing as well, unifying access to node-based data
- single struct instead of separate vectors for annotation data in engine (single place of modification)
