A request to osrm-routed can be assigned to a thread which
is currently busy processing another request, even when there
are other threads/cores available. This unnecessarily delays
the response, and can make requests appear to hang when
awaiting CPU intensive requests to finish.
The issue looks like a bug in Boost.Asio multithreaded
networking stack.
osrm-routed server implementation is
heavily influenced by the HTTP server 3 example in the
Boost.Asio docs. By upgrading to Boost 1.70 and updating the
server connections to match the example provided in the 1.70
release, the problem is resolved.
The diff of the changes to the Boost.Asio stack are
vast, so it's difficult to identify the exact cause. However
the implementation change is to push the strand of execution
into the socket (and timer) objects, which suggests it could
fix the type of threading issue we are observing.
Phew, a lot of classes were affected by this. The rationale for the
changes are as follows:
- When a type X declares any constructor, the default constructor is
not declared, so there is no need for X() = delete there. In fact,
there is brutal difference between those two: deleted members
participate in overload resolution, but not-declared members do not!
- When a type X wants to be non-copyable (e.g. to be only movable, like
threads, unique_ptrs, and so on), you can either do it by inheriting
from boost::noncopyable (the old way), or better declare both (!) the
copy constructor _and_ the copy assignment operator as deleted:
X(X const&) = delete;
X& operator=(X const&) = delete;
We had tons of types with deleted copy constructors that were lacking
a corresponding deleted copy assignment operator, making them still
copyable and you wouldn't even notice (read: scary)!
References:
- http://accu.org/content/conf2014/Howard_Hinnant_Accu_2014.pdf
- http://www.boost.org/doc/libs/master/libs/core/doc/html/core/noncopyable.html
Note: I know, I'm quoting Hinnant's extraordinary slides a lot, but
getting the sematic right here is so incredibly important.
