NodeJS API docs are out of sync with NodeJS C++ wrapper documentation
due to the doc build breaking back in #4043.
The commit fixes the Node dev dependencies such that `npm run docs`
works again and re-enables the check in CI.
- Fix typo in util function name for_each_indexed.
- Use the overloaded functions for_each_indexed and for_each_pair
with a container argument where possible to improve readability.
A recent change to the Ubuntu Focal CI worker - either a CMake upgrade,
or default installation of the libexpat-dev library - leads to the
32-bit expat library to not be found by CMake.
FindPackage(EXPAT) finds the library via pkg-config, so the fix is
to explicitly include the i386 pkg-config directory in the
PKG_CONFIG_PATH environment variable list.
During the CI migration to Github Actions, a bug was introduced
such that Mason CMake is not being added to the PATH environment
variable correctly, so it defaults to the CMake installed in the
OS environment.
Simple fixing the typo fails as the Mason CMake version requires
libssl 1.0 as a dependency, whilst the Ubuntu Focal runners are
on the newer libssl 1.1. Therefore, we also bump the Mason
CMake version to 3.21.2.
This is a workaround for discrepancy between MSVC 19.27 and 19.28
about static const member definition
https://developercommunity2.visualstudio.com/t/discrepancy-between-msvc-1927-vs-1928-about-static/1255338
We can not use C++17 inline variable as a workaround suggested
in the issue report linked above, because the sol2 does not
seem to compile in C++17 mode:
third_party/sol2/sol2/sol.hpp: error C2039: 'object_type': is not a member of...
* Add missing profile name to library extract test.
* Support both tzid and TZID properties on timezone geometry. Improve validation of timezone polygons.
* Missing tzid property wasn't a geojson validation issue, shouldn't have been tested there.
* Use filesystem glob to loop over all test executables so we don't miss any in the future.
Co-authored-by: Michael Bell <michael@mjjbell.com>
Moves Linux CI builds to the latest Ubuntu LTS.
- Bumps the GCC matrix to versions {7,8,9,10,11}, making 9 the default
for testing non-standard builds.
- Bump Node matrix to {12,14,16,LTS,latest} now that 10 is EOL.
- Fixes to CI builds due to library changes on newer distro.
A request to osrm-routed can be assigned to a thread which
is currently busy processing another request, even when there
are other threads/cores available. This unnecessarily delays
the response, and can make requests appear to hang when
awaiting CPU intensive requests to finish.
The issue looks like a bug in Boost.Asio multithreaded
networking stack.
osrm-routed server implementation is
heavily influenced by the HTTP server 3 example in the
Boost.Asio docs. By upgrading to Boost 1.70 and updating the
server connections to match the example provided in the 1.70
release, the problem is resolved.
The diff of the changes to the Boost.Asio stack are
vast, so it's difficult to identify the exact cause. However
the implementation change is to push the strand of execution
into the socket (and timer) objects, which suggests it could
fix the type of threading issue we are observing.
* Fix test cache to consider MLD executable changes
Currently the test cache is not invalidated if changes to
osrm-partition or osrm-customize executables. This is problematic
when these executables make breaking changes to the data format.
We fix this by including both in the list of executables that
can invalidate the test cache.
* Executables use hyphens
Each MLD cell has source and destination nodes.
MLD is keeping a |source| x |destination| sized table
for various metrics (distances, durations, etc) from each
source to all destinations in a cell.
It stores all of the values for a metric in one large array, with
an offset for each cell to find its values. The offset is currently
limited to 32 bit values, which overflows on very large graphs
(e.g. Planet OSM).
We fix this by changing the offsets to be uint64_t types.
The generation of level masks for compactly storing partition cells
supports sizes that can be stored in 64 bits.
The current implementation fails if the total bit sum is 64 bits
exactly. A bit shift mechanism is used that is undefined when the
shift size is equal to the bit size of the underlying type. This
generates an incorrect mask value.
We fix this by adding a special case for a 64 bit offset. Given this
code is called at most |level| times, there will be no effect on
performance. We also update the assertions to reflect 64 bit masks
are now supported.
osrm-routed does not immediately clean up a keep-alive connection
when the client closes it. Instead it waits for five seconds
of inactivity before removing.
Given a setup with low file limits and clients opening and
closing a lot of keep-alive connections, it's possible for
osrm-routed to run out of file descriptors whilst it waits for
the clean-up to trigger.
Furthermore, this causes the connection acceptor loop to exit.
Even after the old connections are cleaned up, new ones
will not be created. Any new requests will block until the
server is restarted.
This commit improves the situation by:
- Immediately closing connections on error. This includes EOF errors
indicating that the client has closed the connection. This releases
resources early (including the open file) and doesn't wait for the
timer.
- Log when the acceptor loop exits. Whilst this means the behaviour
can still occur for reasons other than too many open files,
we will at least have visibility of the cause and can investigate further.
In newer versions of cmake, FindBoost uses Imported Targets for library
component variables, rather than file paths to the Boost libraries.
cmake uses these targets when linking (e.g. target_link_library) and
knows how to correctly substitute the values. However, the OSRM
pkg-config file that we generate doesn't do this, and ends up writing
the actual target symbols, hence the errors trying to link Boost::<component>.
To fix this for newer cmake versions, we create an intermediate configure step
that references the linker files for the imported targets. This is followed
by a generate step that performs the correct substitution.
See this thread for more details: https://cmake.org/pipermail/cmake/2018-December/068812.html
This is backwards compatible to the existing min cmake version (3.1).
However, building using cmake 3.1 fails with a package.json parsing error,
so this commit also bumps the min version to 3.2.
Duplicate restriction nodes in the edge-based-graph are currently
not in included in a mapping (.osrm.cnbg_to_ebg) from
node-based-graph edges to edge-based-graph nodes.
This mapping is used by the MLD partitioner to assign EBG nodes
to partitions.
The omission from the mapping means all restriction nodes are
included in a special 'invalid' partition. This special partition
will break the geolocation properties of the multi-level hierarchy.
The partition and its super levels will have a large number of
border nodes and very few internal paths between them.
Given the partitioner is the only consumer of the mapping, we fix
the issue by including the duplicate restriction nodes in the mapping,
so that they are correctly assigned to a partition.
This has measurable improvement on MLD routing.
For a country-sized routing network, the fix reduces routing and table
request computation time by ~2% and ~6% respectively.
Replace Travis for continuous integration with Github Actions.
The Github Actions pipeline is functionally equivalent, with
all the same build permutations supported.
Whilst the Github Actions offering is broadly equivalent to
Travis, a few changes have been made as part of the migration.
- The 'core' and 'optional' Travis stages have been consolidated
into one build matrix. This is due to the current inability in
Github Actions to share build steps between jobs, so this avoids
having to duplicate the steps.
Optional stage jobs will now run in parallel with core jobs,
but they still remain optional in the sense that they don't fail
the build.
- A number of existing Github Action plugins are used to replace
functionality provided by Travis or other tools:
Node setup, caching, Codecov, publishing release artifacts.
- Linux builds are updated to build on Ubuntu 18.04.
MacOS builds are updated to run on 10.15. Similar to the
Travis Xenial upgrade attempt, some changes are required due
to underlying platform and compiler upgrades. This means some
Node 10 toolchains will no longer be supported.
Whilst there is opportunity to upgrade some dependencies and
make the CI steps more idiomatic, I've left this for future changes
and just focussed on functional replication.
* Handle maxspeed tags before surface and smoothness tags
Let's say we have a tertiary road with the following tags:
highway=tertiary
maxspeed=60
surface=gravel
smoothness=intermediate
While the maxspeed tag tells us the legal speed limit, the surface and
smoothness tags have much more effect on the real-world speed of a car.
We should process the maxspeed tags first, and then update the road's
forwards/backwards speeds according to any surface and smoothness tags.
For our hypothetical road the process in the car.lua profile now goes
like this:
1. Get default speed from profile (tertiary = 40 on line 150 of car.lua)
2. Change speed to 60 using maxspeed tag (WayHandlers.maxspeed function
in way_handlers.lua, lines 434-447)
3. Change speed to 40 using surface tag (WayHandlers.surface function
in way_handlers.lua, lines 360-363)
4. Check speed according to smoothness tag --- but because it's higher
than the speed according to the surface tag, leave the speed
unchanged (WayHandlers.surface function again, lines 368-371)
<ec36319232/profiles/car.lua (L150)>
<ec36319232/profiles/lib/way_handlers.lua (L354-L372)>
Note in step 3 above the speed's only changed from 60kph to 40kph
because it's a lower value. If the surface speed was higher than than
the previous value, the speed would remain unchanged. Another example:
highway=tertiary
maxspeed=60
surface=compacted
smoothness=intermediate
Here, although the profile's speed for compacted is 80, it would stay at
the lower value of 60 (see way_handlers.lua, lines 360-363).
<ec36319232/profiles/lib/way_handlers.lua (L360-L363)>
* Use Github Releases for hosting node binaries
Replaces S3 hosting of node binaries with Github Releases.
`node-pre-gyp publish` works exclusively with S3, so upload step
is now performed by the Travis deployment provider.
The behaviour for the package user should not change.
When building a new version tag, Travis will create a release for the
tag if it does not already exist.
* Switch to the osrm-release-automation machine account rather than personal credentials.
Co-authored-by: Daniel Patterson <danpat@danpat.net>
OSM node 2^33 was created in early April 2021. This and all
subsequently created IDs will be overflowing OSRM node storage
which only support 33 bit IDs.
Bump the number of bits to 34 to double node ID capacity. This
is a breaking change to the data format as it alters the layout
of .osrm.nbg_nodes.
When using process memory, MLD cell metrics are loaded twice from
.osrm.cell_metrics - once when loading static data, and again when
loading updatable data. The former appears to be the mistake,
as .osrm.cell_metrics is only listed in `GetUpdatableFiles`.
OSM data contains many mistakes that tag kerbs as highway barriers
when instead they are only describing highway crossings.
This PR updates the default car profile to handle these mistakes
and unblock routing on the affected highways.
