Commit Graph

23 Commits

Author SHA1 Message Date
Mateusz Loskot
30ed1fae99 Move setting up task_scheduler_init from executables to library
Make entry points of individual pipeline stages responsible for
configuring the task scheduler with requested number of threads
passed in corresponding configuration bundle (ie. follow extractor).
2018-02-02 14:34:20 +01:00
Mateusz Loskot
8114104a43 Rename namespace partition to partitioner
Rename module partition to partitioner.
This cultivates naming used in existing modules like extractor,
customizer, etc. - noun vs verb (word partition is both though).
2018-02-02 11:07:18 +01:00
Mateusz Loskot
7323221e3b Add --verbosity,-l command line option
Allow to fine-control logging verbosity via command line
and LogPolicy setting (useful when OSRM used as library).

Closes #4299
2017-08-31 00:35:23 +02:00
Pepijn Schoen
d9e8caf369 Use GetPath with file names over accessing member variables 2017-07-20 12:01:05 +02:00
Pepijn Schoen
67fae1d1f0 rename osrm_input_path to osrm_path, clang-format 2017-07-20 12:01:05 +02:00
Pepijn Schoen
a9b6686725 superclass partition_config with io_config 2017-07-20 12:01:05 +02:00
Daniel Patterson
3d77714c36 Make most command-line tools return useful error codes on well-known exceptions. 2017-06-08 15:03:24 +00:00
Michael Krasnyk
7a1a209168 use std::regex_token_iterator instead of boost tokenized 2017-03-29 10:23:42 +00:00
Michael Krasnyk
ac6f07a744 make adaptors::tokenized compatible with boost 1.54 2017-03-29 10:23:42 +00:00
Michael Krasnyk
dac929f8df Make max-cell-sizes parameter a comma-separated list 2017-03-29 10:23:42 +00:00
Michael Krasnyk
5ba54a62d6 Fix clang format for 3dcc713 2017-03-28 13:39:22 +00:00
Michael Krasnyk
3dcc7132b6
Add max-cell-sizes option to partitioner 2017-03-28 11:53:14 +02:00
Michael Krasnyk
fe5d1a6e74
Add MLD testing to cucumber tests 2017-03-17 12:27:47 +01:00
Patrick Niklaus
8ad9a0aa9a parameterize cell size 2017-03-08 00:56:50 +00:00
Patrick Niklaus
886421b43a Fix help texts and .nbg_to_ebg -> .cnbg_to_ebg 2017-03-01 16:09:29 +00:00
Moritz Kobitzsch
ed7438b9ba don't check for files not actually read 2017-03-01 16:09:29 +00:00
Moritz Kobitzsch
e8cca36369 fix tbb leaks 2017-03-01 16:09:29 +00:00
Daniel J. Hofmann
299d071e9d Fixes asan's detected tbb leak in osrm-partition
Here's all I could get out of a instrumented `osrm-partition`; debug build,
with all the bells and whistles I could think of to make it more verbose:

```
==17928==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 1560 byte(s) in 3 object(s) allocated from:
    #0 0x7f4244185b30 in operator new[](unsigned long) ../../../../libsanitizer/asan/asan_new_delete.cc:62
    #1 0x7f4242a788b3  (/usr/lib/libtbb.so.2+0x208b3)

SUMMARY: AddressSanitizer: 1560 byte(s) leaked in 3 allocation(s).<Paste>
``

Symbolizing the address results in

```
echo "/usr/lib/libtbb.so 0x7f4242a788b3" | llvm-symbolizer
_fini
```

Looks like a crt finalizer => static global dtor "leaking" from tbb.

Which turned out to be a missing `tbb::task_scheduler_init` on our end:

> Using task_scheduler_init is optional in Intel® Threading Building
> Blocks (Intel® TBB) 2.2. By default, Intel TBB 2.2 automatically creates
> a task scheduler the first time that a thread uses task scheduling
> services and destroys it when the last such thread exits.

https://www.threadingbuildingblocks.org/docs/help/hh_goto.htm?index.htm#reference/task_scheduler/task_scheduler_init_cls.html

Without an explicit instanz the first call to a tbb algorithm seem to initialize
a global scheduler singleton which then "leaks" until the program exits.

Phew.
2017-03-01 16:09:29 +00:00
Daniel Patterson
b62b09e5f6 Dump memory usage stats after partitioning. 2017-03-01 16:09:29 +00:00
Moritz Kobitzsch
b789da45bd add unit tests for the different components of the parttion tool 2017-03-01 16:09:29 +00:00
Daniel J. Hofmann
b9ed20bb9b Implements Compressed Node Based Graph (De-)Serialization Skeleton
Implements parallel recursion for the partitioner
Fixes osrm-extract's -dump-partition-graph: accept no further tokens

References:
- http://www.boost.org/doc/libs/1_55_0/doc/html/boost/program_options/bool_switch.html

Pulls parameters through to make them configurable from the outside

Defaults are equivalent to:

    ./osrm-partition \
      berlin-latest.osrm \
      --max-cell-size 4096 \
      --balance 1.2 \
      --boundary 0.25 \
      --optimizing-cuts 10

Fixes parallel_do call for Intel TBB 4.2 (Trusty): no range-based overload
2017-03-01 16:09:29 +00:00
Moritz Kobitzsch
786be6f570 Optimise Flow Algorithm/Datastructures in use 2017-03-01 16:09:29 +00:00
Moritz Kobitzsch
d56db500d3 Initial Skeleton for Recursive Bisection via Inertial Flow 2017-03-01 16:09:29 +00:00