Open-Harbor/osrm-backend
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 3 Wiki Activity

Squashed 'third_party/libosmium/' changes from 2282c84..80df1d6

Browse Source 
80df1d6 Release v2.9.0
110dc5c Update change log.
6ad5829 Better handling of areas with duplicate segments.
f5985ed Better exception message for invalid areas.
fa09300 Explicit cast to make intent clear.
6f9b522 Fix name of struct stat on Windows.
6b0a47b Clean up code in data tests.
aa1226c Fix progress bar.
3663a19 Extend ProgressBar class so that it works with multiple files.
40c4d5a Add version of file_size() taking a file name.
43a2fac Merge pull request #162 from osmcode/windows-build-scripts
cc2305d [skip travis] 1st iteration of new build scripts
7abe4e1 Clean up disk location cache examples.
48841d5 Update change log.
cf854e9 Change timestamp parser.
01aa8c7 Add examples osmium_pub_names and osmium_road_length.
483c9f2 Benchmark code cleanup.
3ffea2d Cleaned up some test code.
80f0ff7 Explicit conversion from int to bool.
0ba5918 Write space after progress bar to defend against glitches in output.
8584423 Change progress bar to take max_size on construction.
d2c7585 Only call gzoffset when compiling with zlib > 1.2.4.
1b417e5 Add support for a progress report in osmium::io::Reader().
3b4c8c8 Minor cleanup of appveyor config.
d787e25 Fix OPL parser: Relation member without role at end of line.
53ca080 Make lots of variables const.
d776ab2 Add to change log.
eec3b62 Properly initialize m_data field.
cc607e1 Take argument by const ref.
be1e346 Remove unused function.
2a356ee Make lots of one-argument constructors explicit.
adca74f Add comments to and cleanup up examples.
381e535 Simplify WKB code.
b49efd8 Fix opl_parse_changeset_id() return type.
bb52e57 Use uint64_t for line count and column to be on the safe side.
243f6a7 Use parentheses to make sure the right precedence is used.
5a7648e Consistently catch by const ref unless var needs to be non-const.
e3be990 Avoid some warnings.
c436d92 Do not include unistd.h on Windows.
95b228c Add dummy function to avoid warnings.
f276ca3 Fixed includes and changelog update.
8c54bd9 Change timestamp error message.
27e1d5c Add OPL parser.
1d2caab Add more includes to osm.hpp to make usual osmium use simpler.
9d88361 More tests for area CRC.
4f8964d Initialize Item::m_diff member on construction.
f2b648b Parse coordinates in scientific notations ourselves.
b01323f More include fixes.
69f39d4 Fix some includes.
156536d Make padded_length a plain function, not a template function.
65cd1dc Extend functions to set Location lon/lat.
98b7b17 Update to protozero 1.4.2.
a6420cf Add diff indicator to items and use for diff opl and debug output.
0ef02a3 Add workaround for YCM.
3a986f4 Update protozero version.
5245c5b Document osmium_count example program and add memory usage output.
796ca13 Document handler class.
2ba1c1f Add example for mercator projection and tiles.
201f744 Restrict tiles to zoom <= 30.
202291d Add member_type_string class.
494ed6e Cleaned up Tile tests.
af13a8b Add documentation and range checks to Tile struct.
9df5d91 Some small changes to avoid conversion warnings.
afac031 Explicit cast to avoid warning.
8188f66 Better contribution info.
fa89d1d Fixed a problem limiting cache file sizes on Windows to 32 bit.
23a89df Remove obsolete info about versioning from CONTRIBUTING.md file.
115ae23 Release v2.8.0
4174b3c Style fix.
1795dcb Function wait_and_pop_with_timeout() is not needed any more.
4a3a71b Fix for possible threading problem.
cc85925 Updated change log.
67bc8b1 Use unordered_map instead of map in PBF string table code.
18b7b66 Set better default for string table chunk size and document it.
e6d7410 Remove dependency on sparsehash and boost program_options for examples.
14d92d6 Fix regression: Debug output of invalid location works again.
ef91ce1 Bugfix: PBF String table corruption when there are many strings.
649af78 Remove DeltaEncodeIterator completely.
56e5ac2 Function getting queue sizes from environment uses default when getting 0.
bfaab7d Add change log.
d260339 Remove use of PROTOZERO_STRICT_API macro.
c61722d Remove use of DeltaEncodeIterator simplifying code.
f7c60b6 Updates for new protozero.
0bdfb9d Updated change log.
bb56cbb Switch to newest protozero 1.4.0.
9e19a82 Add ccache support to CMake cnfig, better travis builds.
00d8868 Make I/O max queue sizes configurable via environment.
dc7e504 Remove unused debugging code.
13f66a0 Track pop() calls and queue underruns when OSMIUM_DEBUG_QUEUE_SIZE is set.
5c2e367 Add EWKT support.
8f7c7d3 Automatically set correct SRID when creating geometries.
ff11893 Better check of optional components in CMake config.
4562429 Use fallback implementation for coordinates given in scientific notation.
3bdf46e Mark enable_debug_output() as deprecated.
ea1093e Update catch unit test framework.
8623f1e Release v2.7.2
e135dd8 Fix data corruption regression in mmap based indexes.
adbd3b0 Do not output empty discussion tags in changeset XML output.
8126fbb Formatting.
c6970fd Fix coordinate output.
3471b4b Resize output string once in output_int().
0ddf0e7 Use our own function to convert integers to strings instead of printf.
f9a1dd3 Reading and writing coordinates is now independant of locale.
8104294 Use hand-crafted function for hex output (faster than printf).
0bb452a Fix links in change log.
1862d06 Release v2.7.1
8bfe2ba Release v2.7.0
c3604f3 Use 64bit counter in area stats.
9e589b3 Update gdalcpp.hpp from upstream.
fd55d9b Cleanup of OGR-related code.
d0c53e0 Fix bug: Relation wasn't found correctly from member.
24145f9 Use make_iterator helper function.
a8a287d Refactor count_not_removed function. (No template necessary.)
389332a Also print removed flag from member_meta.
5e7c5d0 Remove unnecessary overload of begin() and end() function in iterator_range.
2ec007f Do not add rings to invalid area, even if create_empty_areas is set.
fee8b73 Optionally keep type tag in area assembler. Better doc for config.
c7e1f8a Fix timer output in assembler code.
032ab40 Update change log.
dcfa439 Node location store keeps track of whether node ids are ordered.
54d5eb8 Add tests for new file based index code.
4fe5b30 Use correct empty value when initializing index.
40b5c79 Static or not static, that is the question.
aaa9b46 Open index file with minimum size, because zero-sized mmap is not allowed.
fea2337 Fix for disk-based indexes.
428a413 More tests of corner cases for id to location index.
9d2a31b Add config option to areas assembler for only creating some areas.
d11bf8d Count and report inner rings with the same tags as relation/outer rings.
bde10c4 Speed up copy of tags.
e4c9f87 Revert "Consistently remove some tags from area."
9cd7a03 Set areas assembly config setting create_empty_areas to true by default.
660fb63 Better ordering of OSMObjects.
b4199c2 Use std::strcmp instead of just strcmp.
579c34b Better field width/precision in problem reporter.
a2ebeeb Use field names with 8 characters or less in OGR problem reporter.
ef523fe Switch remaining "typedef"s to "using".
19425f8 Switching from "typedef" to "using" in geom code.
b13c2be More cases of switching from "typedef" to "using".
7f53977 Refactoring iterators: Not derived from std::iterator any more.
1922224 Consistently remove some tags from area.
295495f Fix check for detecting wrong role.
9aa6d46 Report more IDs in problem reporters.
d7a5da7 Remove now unused spike segment reporter.
0666d66 Only report duplicate segments if they belong to the same way.
9e17f89 Improved error reporting for area assembler.
e983a48 More code cleanup and docs.
927eeda Replace awkward std::pair construct by real class.
d0543b9 Various area code refactorings.
0ae8f07 Do not build areas for ways with tag area=no.
d4cabe7 Add some convenience functions to check for tags in TagList.
99f4be9 Add missing include.
a8dda78 Travis: Only run tests if build succeeded.
9db3034 Add missing "nodes" fields.
50e9fcb Report ways that are in multiple rings as errors.
58a3669 Add some paranoia asserts.
3958c1d Use iterator_range to make equal_range results easier to use.
c12c710 Add for_each_member() function to iterate over members of an mp relation.
ca35452 Change argument order in create_area() functions.
4473ae1 Keep stats on multipolygons with no tags on the relation.
12c5335 Bugfix: Check that there is a problem reporter before using it.
ec2afce Update change log.
5af2ec9 Use new area assembler interface in multipolygon collector.
73e3440 Some code cleanup in area code and new interface for calling assembler.
7737479 Add the number of nodes in area to problem reporter.
b4f9343 Use const_iterator where possible.
02372b2 Simplify code that checks for open rings.
8d6099e Pull out location_to_ring_map into details ns and add == and < ops.
1a05042 Mixed code cleanups and added comments.
4b8d1be Ignore empty role when checking inner/outer roles on multipolygons.
e22f573 Now GCC is complaining about the clang pragma...
48000c0 Add some missing includes and forward declarations.
ba9504a Workaround for bug in old libc.
a138265 Completely new algorithm for assembling multipolygons.
74054bd Add specialization of std::hash function for Location.
5ed4c90 Use newest gdalcpp.hpp with implicit transaction support.
676949e Add "locations_on_ways" support for OPL format, too.
ce05c19 Add support for reading/writing XML/PBF files with locations on ways.
62b2ee4 Fix checksum test.
bd512a8 Added "add_crc32" file option for adding CRC32 checksum to debug output.
3a100fa Incorporate locations in NodeRefs into CRC32 checksum.
ac02f86 Update catch.hpp to newest version. Removed outdated info in README.
481f48b When assembling areas ignore ways containing no or only a single node.
a0ae33a Fix unsigned overflow in pool.hpp.
91b8adf Fix undefined behaviour in WKB writer.
697f460 Check results of dynamic casts.
f1e4571 Fix from_item_type() implementation so it also works with undefined type.
65df99b Add future_queue_type alias to simplify code.
4340e4d Removed SortedQueue implementation which was never used.
cdd8f8c Add version.hpp with macros defining version of the library.
ff5d42a Update to newest gdalcpp.hpp.
a184f66 Update change log.
0ea76f7 Add osmium::Area::outer_rings() and inner_rings() functions.
b0404b7 New ItemIteratorRange class for iteration over buffers and subitems.
eff8a7c Add default type to string_to_object_id for IDs without type prefix.
e877a6f Clean up code inner vs. outer ring in geometry factory.
9224be5 Disable use of XML entities in OSM files.
9d9fa08 Output operator of location shows full precision of coordinates.
9a8e7c0 Documentation fixes.

git-subtree-dir: third_party/libosmium
git-subtree-split: 80df1d6850bdfa661587839b77dcea0ab8fc814a
This commit is contained in:
karenzshea 2016-10-03 13:08:59 -04:00
parent 8511256779
commit 879f7eb042
208 changed files with 11590 additions and 4051 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.gitignore vendored
View File

@ -1,2 +1,4 @@
*.swp
.ycm_extra_conf.pyc
/build
/libosmium-deps

96
.travis.yml
View File

@ -4,24 +4,34 @@
#
#-----------------------------------------------------------------------------
language: cpp
language: generic
sudo: false
cache:
directories:
- $HOME/.ccache
env:
global:
- CCACHE_TEMPDIR=/tmp/.ccache-temp
- CCACHE_COMPRESS=1
- CASHER_TIME_OUT=1000
matrix:
include:
# 1/ Linux Clang Builds
- os: linux
compiler: clang
compiler: linux-clang35-release
addons:
apt:
sources: ['llvm-toolchain-precise-3.5', 'ubuntu-toolchain-r-test', 'boost-latest']
packages: ['clang-3.5', 'libboost1.55-all-dev', 'libgdal-dev', 'libgeos++-dev', 'libproj-dev', 'libsparsehash-dev', 'spatialite-bin']
packages: ['clang-3.5', 'cmake', 'libboost1.55-all-dev', 'libgdal-dev', 'libgeos++-dev', 'libproj-dev', 'libsparsehash-dev', 'spatialite-bin']
env: COMPILER='clang++-3.5' BUILD_TYPE='Release'
- os: linux
compiler: clang
compiler: linux-clang35-dev
addons:
apt:
sources: ['llvm-toolchain-precise-3.5', 'ubuntu-toolchain-r-test', 'boost-latest']
@ -30,24 +40,7 @@ matrix:
- os: linux
compiler: clang
addons:
apt:
sources: ['llvm-toolchain-precise-3.6', 'ubuntu-toolchain-r-test', 'boost-latest']
packages: ['clang-3.6', 'libboost1.55-all-dev', 'libgdal-dev', 'libgeos++-dev', 'libproj-dev', 'libsparsehash-dev', 'spatialite-bin']
env: COMPILER='clang++-3.6' BUILD_TYPE='Release'
- os: linux
compiler: clang
addons:
apt:
sources: ['llvm-toolchain-precise-3.6', 'ubuntu-toolchain-r-test', 'boost-latest']
packages: ['clang-3.6', 'libboost1.55-all-dev', 'libgdal-dev', 'libgeos++-dev', 'libproj-dev', 'libsparsehash-dev', 'spatialite-bin']
env: COMPILER='clang++-3.6' BUILD_TYPE='Dev'
- os: linux
compiler: clang
compiler: linux-clang37-release
addons:
apt:
sources: ['llvm-toolchain-precise-3.7', 'ubuntu-toolchain-r-test', 'boost-latest']
@ -55,7 +48,7 @@ matrix:
env: COMPILER='clang++-3.7' BUILD_TYPE='Release'
- os: linux
compiler: clang
compiler: linux-clang37-dev
addons:
apt:
sources: ['llvm-toolchain-precise-3.7', 'ubuntu-toolchain-r-test', 'boost-latest']
@ -63,9 +56,26 @@ matrix:
env: COMPILER='clang++-3.7' BUILD_TYPE='Dev'
- os: linux
compiler: linux-clang38-release
addons:
apt:
sources: ['llvm-toolchain-precise-3.8', 'ubuntu-toolchain-r-test', 'boost-latest']
packages: ['clang-3.8', 'libboost1.55-all-dev', 'libgdal-dev', 'libgeos++-dev', 'libproj-dev', 'libsparsehash-dev', 'spatialite-bin']
env: COMPILER='clang++-3.8' BUILD_TYPE='Release'
- os: linux
compiler: linux-clang38-dev
addons:
apt:
sources: ['llvm-toolchain-precise-3.8', 'ubuntu-toolchain-r-test', 'boost-latest']
packages: ['clang-3.8', 'libboost1.55-all-dev', 'libgdal-dev', 'libgeos++-dev', 'libproj-dev', 'libsparsehash-dev', 'spatialite-bin']
env: COMPILER='clang++-3.8' BUILD_TYPE='Dev'
# 2/ Linux GCC Builds
- os: linux
compiler: gcc
compiler: linux-gcc48-release
addons:
apt:
sources: ['ubuntu-toolchain-r-test', 'boost-latest']
@ -73,7 +83,7 @@ matrix:
env: COMPILER='g++-4.8' COMPILER_FLAGS='-Wno-return-type' BUILD_TYPE='Release'
- os: linux
compiler: gcc
compiler: linux-gcc48-dev
addons:
apt:
sources: ['ubuntu-toolchain-r-test', 'boost-latest']
@ -82,7 +92,7 @@ matrix:
- os: linux
compiler: gcc
compiler: linux-gcc49-release
addons:
apt:
sources: ['ubuntu-toolchain-r-test', 'boost-latest']
@ -90,7 +100,7 @@ matrix:
env: COMPILER='g++-4.9' BUILD_TYPE='Release'
- os: linux
compiler: gcc
compiler: linux-gcc49-dev
addons:
apt:
sources: ['ubuntu-toolchain-r-test', 'boost-latest']
@ -99,7 +109,7 @@ matrix:
- os: linux
compiler: gcc
compiler: linux-gcc50-release
addons:
apt:
sources: ['ubuntu-toolchain-r-test', 'boost-latest']
@ -107,7 +117,7 @@ matrix:
env: COMPILER='g++-5' BUILD_TYPE='Release'
- os: linux
compiler: gcc
compiler: linux-gcc50-dev
addons:
apt:
sources: ['ubuntu-toolchain-r-test', 'boost-latest']
@ -118,25 +128,25 @@ matrix:
# 3/ OSX Clang Builds
- os: osx
osx_image: xcode6.4
compiler: clang
env: COMPILER='clang++' BUILD_TYPE='Dev'
compiler: xcode64-clang-release
env: COMPILER='clang++' BUILD_TYPE='Release'
- os: osx
osx_image: xcode6.4
compiler: clang
env: COMPILER='clang++' BUILD_TYPE='Release'
- os: osx
osx_image: xcode7
compiler: clang
compiler: xcode64-clang-dev
env: COMPILER='clang++' BUILD_TYPE='Dev'
- os: osx
osx_image: xcode7
compiler: clang
compiler: xcode7-clang-release
env: COMPILER='clang++' BUILD_TYPE='Release'
- os: osx
osx_image: xcode7
compiler: xcode7-clang-dev
env: COMPILER='clang++' BUILD_TYPE='Dev'
install:
- DEPS_DIR="${TRAVIS_BUILD_DIR}/deps"
@ -145,15 +155,15 @@ install:
- |
if [[ "${TRAVIS_OS_NAME}" == "osx" ]]; then
brew remove gdal
brew install cmake boost google-sparsehash gdal
brew install cmake boost google-sparsehash gdal || true
fi
- cmake --version
before_script:
- cd ${TRAVIS_BUILD_DIR}
- mkdir build && cd build
- CXX=${COMPILER} CXXFLAGS=${COMPILER_FLAGS} cmake -LA .. -DCMAKE_BUILD_TYPE=${BUILD_TYPE} -DOSM_TESTDATA="${TRAVIS_BUILD_DIR}/deps/osm-testdata"
- CXX=${COMPILER} CXXFLAGS=${COMPILER_FLAGS} cmake -LA .. -DCMAKE_BUILD_TYPE=${BUILD_TYPE} -DBUILD_WITH_CCACHE=1 -DOSM_TESTDATA="${TRAVIS_BUILD_DIR}/deps/osm-testdata"
script:
- make VERBOSE=1
- ctest --output-on-failure
- make VERBOSE=1 && ctest --output-on-failure

5
.ycm_extra_conf.py
View File

@ -29,6 +29,11 @@ flags = [
'-x',
'c++',
# workaround for https://github.com/Valloric/YouCompleteMe/issues/303
# also see https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=800618
'-isystem',
'/usr/lib/ycmd/clang_includes/',
# libosmium include dirs
'-I%s/include' % basedir,
'-I%s/test/include' % basedir,

141
CHANGELOG.md
View File

@ -13,6 +13,140 @@ This project adheres to [Semantic Versioning](http://semver.org/).
### Fixed
## [2.9.0] - 2016-09-15
### Added
- Support for reading OPL files.
- For diff output OSM objects in buffers can be marked as only in one or the
other file. The OPL and debug output formats support diff output based on
this.
- Add documentation and range checks to `Tile` struct.
- More documentation.
- More examples and more extensive comments on examples.
- Support for a progress report in `osmium::io::Reader()` and a `ProgressBar`
utility class to use it.
- New `OSMObject::set_timestamp(const char*)` function.
### Changed
- Parse coordinates in scientific notations ourselves.
- Updated included protozero version to 1.4.2.
- Lots of one-argument constructors are now explicit.
- Timestamp parser now uses our own implementation instead of strptime.
This is faster and independant of locale settings.
- More cases of invalid areas with duplicate segments are reported as
errors.
### Fixed
- Fixed a problem limiting cache file sizes on Windows to 32 bit.
- Fixed includes.
- Exception messages for invalid areas do not report "area contains no rings"
any more, but "invalid area".
## [2.8.0] - 2016-08-04
### Added
- EWKT support.
- Track `pop` type calls and queue underruns when `OSMIUM_DEBUG_QUEUE_SIZE`
environment variable is set.
### Changed
- Switched to newest protozero v1.4.0. This should deliver some speedups
when parsing PBF files. This also removes the DeltaEncodeIterator class,
which isn't needed any more.
- Uses `std::unordered_map` instead of `std::map` in PBF string table code
speeding up writing of PBF files considerably.
- Uses less memory when writing PBF files (smaller string table by default).
- Removes dependency on sparsehash and boost program options libraries for
examples.
- Cleaned up threaded queue code.
### Fixed
- A potentially very bad bug was fixed: When there are many and/or long strings
in tag keys and values and/or user names and/or relation roles, the string
table inside a PBF block would overflow. I have never seen this happen for
normal OSM data, but that doesn't mean it can't happen. The result is that
the strings will all be mixed up, keys for values, values for user names or
whatever.
- Automatically set correct SRID when creating WKB and GEOS geometries.
Note that this changes the behaviour of libosmium when creating GEOS
geometries. Before we created them with -1 as SRID unless set otherwise.
Manual setting of the SRID on the GEOSGeometryFactory is now deprecated.
- Allow coordinates of nodes in scientific notation when reading XML files.
This shouldn't be used really, but sometimes you can find them.
## [2.7.2] - 2016-06-08
### Changed
- Much faster output of OSM files in XML, OPL, or debug formats.
### Fixed
- Parsing and output of coordinates now faster and always uses decimal dot
independant of locale setting.
- Do not output empty discussion elements in changeset XML output.
- Data corruption regression in mmap based indexes.
## [2.7.1] - 2016-06-01
### Fixes
- Update version number in version.hpp.
## [2.7.0] - 2016-06-01
### Added
- New functions for iterating over specific item types in buffers
(`osmium::memory::Buffer::select()`), over specific subitems
(`osmium::OSMObject::subitems()`), and for iterating over all rings of
an area (`osmium::Areas::outer_rings(`), `inner_rings()`).
- Debug output optionally prints CRC32 when `add_crc32` file option is set.
### Changed
- XML parser will not allow any XML entities which are usually not used in OSM
files anyway. This can help avoiding DOS attacks.
- Removed SortedQueue implementation which was never used.
- Also incorporate Locations in NodeRefs into CRC32 checksums. This means
all checksums will be different compared to earlier versions of libosmium.
- The completely new algorithm for assembling multipolygons is much faster,
has better error reporting, generates statistics and can build more complex
multipolygons correctly. The ProblemReporter classes have changed to make
this happen, if you have written your own, you have to fix it.
- Sparse node location stores are now only sorted if needed, ie. when nodes
come in unordered.
### Fixed
- Output operator for Location shows full precision.
- Undefined behaviour in WKB writer and `types_from_string()` function.
- Fix unsigned overflow in pool.hpp.
- OSM objects are now ordered by type (nodes, then ways, then relations),
then ID, then version, then timestamp. Ordering by timestamp is normally
not necessary, because there can't be two objects with same type, ID, and
version but different timestamp. But this can happen when diffs are
created from OSM extracts, so we check for this here. This change also
makes sure IDs are always ordered by absolute IDs, positives first, so
order is 0, 1, -1, 2, -2, ...
- Data corruption bug fixed in disk based indexes (used for the node
location store for instance). This only affected you, if you created
and index, closed it, and re-opened it (possibly in a different process)
and if there were missing nodes. If you looked up those nodes, you got
location (0,0) back instead of an error.
- Memory corruption bug showing up with GDAL 2.
## [2.6.1] - 2016-02-22
### Added
@ -276,7 +410,12 @@ This project adheres to [Semantic Versioning](http://semver.org/).
Doxygen (up to version 1.8.8). This version contains a workaround to fix
this.
[unreleased]: https://github.com/osmcode/libosmium/compare/v2.6.1...HEAD
[unreleased]: https://github.com/osmcode/libosmium/compare/v2.9.0...HEAD
[2.9.0]: https://github.com/osmcode/libosmium/compare/v2.8.0...v2.9.0
[2.8.0]: https://github.com/osmcode/libosmium/compare/v2.7.2...v2.8.0
[2.7.2]: https://github.com/osmcode/libosmium/compare/v2.7.1...v2.7.2
[2.7.1]: https://github.com/osmcode/libosmium/compare/v2.7.0...v2.7.1
[2.7.0]: https://github.com/osmcode/libosmium/compare/v2.6.1...v2.7.0
[2.6.1]: https://github.com/osmcode/libosmium/compare/v2.6.0...v2.6.1
[2.6.0]: https://github.com/osmcode/libosmium/compare/v2.5.4...v2.6.0
[2.5.4]: https://github.com/osmcode/libosmium/compare/v2.5.3...v2.5.4

25
CMakeLists.txt
View File

@ -24,8 +24,8 @@ set(CMAKE_CONFIGURATION_TYPES "Debug;Release;RelWithDebInfo;MinSizeRel;Dev;Cover
project(libosmium)
set(LIBOSMIUM_VERSION_MAJOR 2)
set(LIBOSMIUM_VERSION_MINOR 6)
set(LIBOSMIUM_VERSION_PATCH 1)
set(LIBOSMIUM_VERSION_MINOR 9)
set(LIBOSMIUM_VERSION_PATCH 0)
set(LIBOSMIUM_VERSION
"${LIBOSMIUM_VERSION_MAJOR}.${LIBOSMIUM_VERSION_MINOR}.${LIBOSMIUM_VERSION_PATCH}")
@ -61,6 +61,27 @@ option(INSTALL_UTFCPP "also install utfcpp headers" OFF)
option(WITH_PROFILING "add flags needed for profiling" OFF)
#-----------------------------------------------------------------------------
#
# CCache support
#
#-----------------------------------------------------------------------------
option(BUILD_WITH_CCACHE "build using ccache" OFF)
if(BUILD_WITH_CCACHE)
find_program(CCACHE_PROGRAM ccache)
if(CCACHE_PROGRAM)
set_property(GLOBAL PROPERTY RULE_LAUNCH_COMPILE "CCACHE_CPP2=1 ${CCACHE_PROGRAM}")
# workaround for some clang versions
if(CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
add_definitions(-Qunused-arguments)
endif()
endif()
endif()
#-----------------------------------------------------------------------------
#
# Coverage support

147
CONTRIBUTING.md
View File

@ -1,143 +1,12 @@
# Notes for Developers
Some rules for contributing to this project:
Read this if you want to contribute to Libosmium.
* Please open a separate issue for each problem, question, or comment you have.
Do not re-use existing issues for other topics, even if they are similar. This
keeps issues small and manageable and makes it much easier to follow through
and make sure each problem is taken care of.
## Versioning
Osmium is currently considered in beta and doesn't use versioning yet. Proper
versions will be introduced as soon as it is somewhat stable.
## Namespace
All Osmium code MUST be in the `osmium` namespace or one of its sub-namespaces.
## Include-Only
Osmium is a include-only library. You can't compile the library itself. There
is no libosmium.so.
One drawback ist that you can't have static data in classes, because there
is no place to put this data.
All free functions must be declared `inline`.
## Coding Conventions
These coding conventions have been changing over time and some code is still
different.
* All include files have `#ifdef` guards around them, macros are the path name
in all uppercase where the slashes (`/`) have been changed to underscore (`_`).
* Class names begin with uppercase chars and use CamelCase. Smaller helper
classes are usually defined as struct and have lowercase names.
* Macros (and only macros) are all uppercase. Use macros sparingly, usually
a simple (maybe constexpr) inline function is better. Undef macros after use
if possible.
* Macros should only be used for controlling which parts of the code should be
included when compiling or to avoid major code repetitions.
* Variables, attributes, and function names are lowercase with
`underscores_between_words`.
* Class attribute names start with `m_` (member).
* Use `descriptive_variable_names`, exceptions are well-established conventions
like `i` for a loop variable. Iterators are usually called `it`.
* Declare variables where they are first used (C++ style), not at the beginning
of a function (old C style).
* Names from external namespaces (even `std`) are always mentioned explicitly.
Do not use `using` (except for `std::swap`). This way we can't even by
accident pollute the namespace of the code using Osmium.
* Always use the standard swap idiom: `using std::swap; swap(foo, bar);`.
* `#include` directives appear in three "blocks" after the copyright notice.
The blocks are separated by blank lines. First block contains `#include`s for
standard C/C++ includes, second block for any external libs used, third
block for osmium internal includes. Within each block `#include`s are usually
sorted by path name. All `#include`s use `<>` syntax not `""`.
* Names not to be used from outside the library should be in a namespace
called `detail` under the namespace where they would otherwise appear. If
whole include files are never meant to be included from outside they should
be in a subdirectory called `detail`.
* All files have suffix `.hpp`.
* Closing } of all classes and namespaces should have a trailing comment
with the name of the class/namespace.
* All constructors with one (or more arguments if they have a default) should
be declared "explicit" unless there is a reason for them not to be. Document
that reason.
* If a class has any of the special methods (copy/move constructor/assigment,
destructor) it should have all of them, possibly marking them as default or
deleted.
* Typedefs have `names_like_this_type` which end in `_type`. Typedefs should
use the new `using foo_type = bar` syntax instead of the old
`typedef bar foo_type`.
* Template parameters are single uppercase letters or start with uppercase `T`
and use CamelCase.
* Always use `typename` in templates, not `class`: `template <typename T>`.
* The ellipsis in variadic template never has a space to the left of it and
always has a space to the right: `template <typename... TArgs>` etc.
Keep to the indentation and other styles used in the code. Use `make indent`
in the toplevel directory to fix indentation and styling. It calls `astyle`
with the right parameters. This program is in the `astyle` Debian package.
## C++11
Osmium uses C++11 and you can use its features such as auto, lambdas,
threading, etc. There are a few features we do not use, because even modern
compilers don't support them yet. This list might change as we get more data
about which compilers support which feature and what operating system versions
or distributions have which versions of these compilers installed.
GCC 4.6 - too old, not supported (Ubuntu 12.04 LTS)
GCC 4.7.2 - can probably not be supported (Debian wheezy)
GCC 4.7.3 - probably works
GCC 4.8 - works and is supported from here on
clang 3.0 - too old, not supported (Debian wheezy, Ubuntu 12.04 LTS)
clang 3.2 - probably works
clang 3.5 - works and is supported from here on
Use `include/osmium/util/compatibility.hpp` if there are compatibility problems
between compilers due to different C++11 support.
## Checking your code
The Osmium makefiles use pretty draconian warning options for the compiler.
This is good. Code MUST never produce any warnings, even with those settings.
If absolutely necessary pragmas can be used to disable certain warnings in
specific areas of the code.
If the static code checker `cppcheck` is installed, the CMake configuration
will add a new build target `cppcheck` that will check all `.cpp` and `.hpp`
files. Cppcheck finds some bugs that gcc/clang doesn't. But take the result
with a grain of salt, it also sometimes produces wrong warnings.
Set `BUILD_HEADERS=ON` in the CMake config to enable compiling all include
files on their own to check whether dependencies are all okay. All include
files MUST include all other include files they depend on.
Call `cmake/iwyu.sh` to check for proper includes and forward declarations.
This uses the clang-based `include-what-you-use` program. Note that it does
produce some false reports and crashes often. The `osmium.imp` file can be
used to define mappings for iwyu. See the IWYU tool at
<http://code.google.com/p/include-what-you-use/>.
## Testing
There are a unit tests using the Catch Unit Test Framework in the `test`
directory and some data tests in `test/osm-testdata`. They are built by the
default cmake config. Run `ctest` to run them. Many more tests are needed.
## Documenting the code
All namespaces, classes, functions, attributes, etc. should be documented.
Osmium uses the Doxygen (www.doxygen.org) source code documentation system.
If it is installed, the CMake configuration will add a new build target, so
you can build it with `make doc`.
* We'd love for you to send pull requests for fixes you have made or new features
you have added. Please read the [notes for developers](NOTES_FOR_DEVELOPERS.md)
beforehand which contain some coding guidelines.

143
NOTES_FOR_DEVELOPERS.md Normal file
View File

@ -0,0 +1,143 @@
# Notes for Developers
Read this if you want to contribute to Libosmium.
## Namespace
All Osmium code MUST be in the `osmium` namespace or one of its sub-namespaces.
## Include-Only
Osmium is a include-only library. You can't compile the library itself. There
is no libosmium.so.
One drawback ist that you can't have static data in classes, because there
is no place to put this data.
All free functions must be declared `inline`.
## Coding Conventions
These coding conventions have been changing over time and some code is still
different.
* All include files have `#ifdef` guards around them, macros are the path name
in all uppercase where the slashes (`/`) have been changed to underscore (`_`).
* Class names begin with uppercase chars and use CamelCase. Smaller helper
classes are usually defined as struct and have lowercase names.
* Macros (and only macros) are all uppercase. Use macros sparingly, usually
a simple (maybe constexpr) inline function is better. Undef macros after use
if possible.
* Macros should only be used for controlling which parts of the code should be
included when compiling or to avoid major code repetitions.
* Variables, attributes, and function names are lowercase with
`underscores_between_words`.
* Class attribute names start with `m_` (member).
* Use `descriptive_variable_names`, exceptions are well-established conventions
like `i` for a loop variable. Iterators are usually called `it`.
* Declare variables where they are first used (C++ style), not at the beginning
of a function (old C style).
* Names from external namespaces (even `std`) are always mentioned explicitly.
Do not use `using` (except for `std::swap`). This way we can't even by
accident pollute the namespace of the code using Osmium.
* Always use the standard swap idiom: `using std::swap; swap(foo, bar);`.
* `#include` directives appear in three "blocks" after the copyright notice.
The blocks are separated by blank lines. First block contains `#include`s for
standard C/C++ includes, second block for any external libs used, third
block for osmium internal includes. Within each block `#include`s are usually
sorted by path name. All `#include`s use `<>` syntax not `""`.
* Names not to be used from outside the library should be in a namespace
called `detail` under the namespace where they would otherwise appear. If
whole include files are never meant to be included from outside they should
be in a subdirectory called `detail`.
* All files have suffix `.hpp`.
* Closing } of all classes and namespaces should have a trailing comment
with the name of the class/namespace.
* All constructors with one (or more arguments if they have a default) should
be declared "explicit" unless there is a reason for them not to be. Document
that reason.
* If a class has any of the special methods (copy/move constructor/assigment,
destructor) it should have all of them, possibly marking them as default or
deleted.
* Typedefs have `names_like_this_type` which end in `_type`. Typedefs should
use the new `using foo_type = bar` syntax instead of the old
`typedef bar foo_type`.
* Template parameters are single uppercase letters or start with uppercase `T`
and use CamelCase.
* Always use `typename` in templates, not `class`: `template <typename T>`.
* The ellipsis in variadic template never has a space to the left of it and
always has a space to the right: `template <typename... TArgs>` etc.
Keep to the indentation and other styles used in the code. Use `make indent`
in the toplevel directory to fix indentation and styling. It calls `astyle`
with the right parameters. This program is in the `astyle` Debian package.
## C++11
Osmium uses C++11 and you can use its features such as auto, lambdas,
threading, etc. There are a few features we do not use, because even modern
compilers don't support them yet. This list might change as we get more data
about which compilers support which feature and what operating system versions
or distributions have which versions of these compilers installed.
GCC 4.6 - too old, not supported (Ubuntu 12.04 LTS)
GCC 4.7.2 - can probably not be supported (Debian wheezy)
GCC 4.7.3 - probably works
GCC 4.8 - works and is supported from here on
clang 3.0 - too old, not supported (Debian wheezy, Ubuntu 12.04 LTS)
clang 3.2 - probably works
clang 3.5 - works and is supported from here on
Use `include/osmium/util/compatibility.hpp` if there are compatibility problems
between compilers due to different C++11 support.
## Operating systems
Usually all code must work on Linux, OSX, and Windows. Execptions are allowed
for some minor functionality, but please discuss this first.
## Checking your code
The Osmium makefiles use pretty draconian warning options for the compiler.
This is good. Code MUST never produce any warnings, even with those settings.
If absolutely necessary pragmas can be used to disable certain warnings in
specific areas of the code.
If the static code checker `cppcheck` is installed, the CMake configuration
will add a new build target `cppcheck` that will check all `.cpp` and `.hpp`
files. Cppcheck finds some bugs that gcc/clang doesn't. But take the result
with a grain of salt, it also sometimes produces wrong warnings.
Set `BUILD_HEADERS=ON` in the CMake config to enable compiling all include
files on their own to check whether dependencies are all okay. All include
files MUST include all other include files they depend on.
Call `cmake/iwyu.sh` to check for proper includes and forward declarations.
This uses the clang-based `include-what-you-use` program. Note that it does
produce some false reports and crashes often. The `osmium.imp` file can be
used to define mappings for iwyu. See the IWYU tool at
<http://code.google.com/p/include-what-you-use/>.
## Testing
There are a unit tests using the Catch Unit Test Framework in the `test`
directory and some data tests in `test/osm-testdata`. They are built by the
default cmake config. Run `ctest` to run them. Many more tests are needed.
## Documenting the code
All namespaces, classes, functions, attributes, etc. should be documented.
Osmium uses the Doxygen (www.doxygen.org) source code documentation system.
If it is installed, the CMake configuration will add a new build target, so
you can build it with `make doc`.

85
appveyor.yml
View File

@ -22,88 +22,5 @@ clone_folder: c:\projects\libosmium
platform: x64
install:
# show all available env vars
- set
- echo cmake on AppVeyor
- cmake -version
- call "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\vcvarsall.bat" amd64
- set PATH=c:\projects\libosmium\cmake-3.1.0-win32-x86\bin;%PATH%
- set LODEPSDIR=c:\projects\libosmium\libosmium-deps
- set PROJ_LIB=%LODEPSDIR%\proj\share
- set GDAL_DATA=%LODEPSDIR%\gdal\data
#geos.dll
- set PATH=%LODEPSDIR%\geos\lib;%PATH%
#gdal.dll
- set PATH=%LODEPSDIR%\gdal\lib;%PATH%
#libexpat.dll
- set PATH=%LODEPSDIR%\expat\lib;%PATH%
#libtiff.dll
- set PATH=%LODEPSDIR%\libtiff\lib;%PATH%
#jpeg.dll
- set PATH=%LODEPSDIR%\jpeg\lib;%PATH%
#zlibwapi.dll
- set PATH=%LODEPSDIR%\zlib\lib;%PATH%
#convert backslashes in bzip2 path to forward slashes
#cmake cannot find it otherwise
- set LIBBZIP2=%LODEPSDIR%\bzip2\lib\libbz2.lib
- set LIBBZIP2=%LIBBZIP2:\=/%
- ps: Start-FileDownload https://mapbox.s3.amazonaws.com/windows-builds/windows-build-deps/cmake-3.1.0-win32-x86.7z -FileName cm.7z
- ps: Start-FileDownload https://mapbox.s3.amazonaws.com/windows-builds/windows-build-deps/libosmium-deps-win-14.0-x64.7z -FileName lodeps.7z
- 7z x cm.7z | %windir%\system32\find "ing archive"
- 7z x lodeps.7z | %windir%\system32\find "ing archive"
- echo %LODEPSDIR%
- dir %LODEPSDIR%
- echo our own cmake
- cmake -version
- cd c:\projects
- git clone --depth 1 https://github.com/osmcode/osm-testdata.git
build_script:
- cd c:\projects\libosmium
- mkdir build
- cd build
- echo %config%
# This will produce lots of LNK4099 warnings which can be ignored.
# Unfortunately they can't be disabled, see
# http://stackoverflow.com/questions/661606/visual-c-how-to-disable-specific-linker-warnings
- cmake -LA -G "Visual Studio 14 Win64"
-DOsmium_DEBUG=TRUE
-DCMAKE_BUILD_TYPE=%config%
-DBUILD_HEADERS=OFF
-DBOOST_ROOT=%LODEPSDIR%\boost
-DBoost_PROGRAM_OPTIONS_LIBRARY=%LODEPSDIR%\boost\lib\libboost_program_options-vc140-mt-1_58.lib
-DZLIB_LIBRARY=%LODEPSDIR%\zlib\lib\zlibwapi.lib
-DBZIP2_LIBRARY_RELEASE=%LIBBZIP2%
-DCMAKE_PREFIX_PATH=%LODEPSDIR%\zlib;%LODEPSDIR%\expat;%LODEPSDIR%\bzip2;%LODEPSDIR%\geos;%LODEPSDIR%\gdal;%LODEPSDIR%\proj;%LODEPSDIR%\sparsehash;%LODEPSDIR%\wingetopt
..
- msbuild libosmium.sln /p:Configuration=%config% /toolsversion:14.0 /p:Platform=x64 /p:PlatformToolset=v140
#- cmake .. -LA -G "NMake Makefiles"
# -DOsmium_DEBUG=TRUE
# -DCMAKE_BUILD_TYPE=%config%
# -DBOOST_ROOT=%LODEPSDIR%\boost
# -DBoost_PROGRAM_OPTIONS_LIBRARY=%LODEPSDIR%\boost\lib\libboost_program_options-vc140-mt-1_57.lib
# -DZLIB_LIBRARY=%LODEPSDIR%\zlib\lib\zlibwapi.lib
# -DZLIB_INCLUDE_DIR=%LODEPSDIR%\zlib\include
# -DEXPAT_LIBRARY=%LODEPSDIR%\expat\lib\libexpat.lib
# -DEXPAT_INCLUDE_DIR=%LODEPSDIR%\expat\include
# -DBZIP2_LIBRARIES=%LIBBZIP2%
# -DBZIP2_INCLUDE_DIR=%LODEPSDIR%\bzip2\include
# -DGDAL_LIBRARY=%LODEPSDIR%\gdal\lib\gdal_i.lib
# -DGDAL_INCLUDE_DIR=%LODEPSDIR%\gdal\include
# -DGEOS_LIBRARY=%LODEPSDIR%\geos\lib\geos.lib
# -DGEOS_INCLUDE_DIR=%LODEPSDIR%\geos\include
# -DPROJ_LIBRARY=%LODEPSDIR%\proj\lib\proj.lib
# -DPROJ_INCLUDE_DIR=%LODEPSDIR%\proj\include
# -DSPARSEHASH_INCLUDE_DIR=%LODEPSDIR%\sparsehash\include
# -DGETOPT_LIBRARY=%LODEPSDIR%\wingetopt\lib\wingetopt.lib
# -DGETOPT_INCLUDE_DIR=%LODEPSDIR%\wingetopt\include
#- nmake
test_script:
# "-E testdata-overview" exempts one test we know fails on Appveyor
# because we currently don't have spatialite support.
- ctest --output-on-failure
-C %config%
-E testdata-overview
- build-appveyor.bat

8
benchmarks/osmium_benchmark_count.cpp
View File

@ -5,7 +5,9 @@
*/
#include <cstdint>
#include <cstdlib>
#include <iostream>
#include <string>
#include <osmium/io/any_input.hpp>
#include <osmium/handler.hpp>
@ -35,12 +37,12 @@ struct CountHandler : public osmium::handler::Handler {
int main(int argc, char* argv[]) {
if (argc != 2) {
std::cerr << "Usage: " << argv[0] << " OSMFILE\n";
exit(1);
std::exit(1);
}
std::string input_filename = argv[1];
const std::string input_filename{argv[1]};
osmium::io::Reader reader(input_filename);
osmium::io::Reader reader{input_filename};
CountHandler handler;
osmium::apply(reader, handler);

8
benchmarks/osmium_benchmark_count_tag.cpp
View File

@ -5,7 +5,9 @@
*/
#include <cstdint>
#include <cstdlib>
#include <iostream>
#include <string>
#include <osmium/io/any_input.hpp>
#include <osmium/handler.hpp>
@ -38,12 +40,12 @@ struct CountHandler : public osmium::handler::Handler {
int main(int argc, char* argv[]) {
if (argc != 2) {
std::cerr << "Usage: " << argv[0] << " OSMFILE\n";
exit(1);
std::exit(1);
}
std::string input_filename = argv[1];
const std::string input_filename{argv[1]};
osmium::io::Reader reader(input_filename);
osmium::io::Reader reader{input_filename};
CountHandler handler;
osmium::apply(reader, handler);

16
benchmarks/osmium_benchmark_index_map.cpp
View File

@ -4,7 +4,9 @@
*/
#include <cstdlib>
#include <iostream>
#include <string>
#include <osmium/index/map/all.hpp>
#include <osmium/handler/node_locations_for_ways.hpp>
@ -13,24 +15,24 @@
#include <osmium/io/any_input.hpp>
#include <osmium/handler.hpp>
typedef osmium::index::map::Map<osmium::unsigned_object_id_type, osmium::Location> index_type;
using index_type = osmium::index::map::Map<osmium::unsigned_object_id_type, osmium::Location>;
typedef osmium::handler::NodeLocationsForWays<index_type> location_handler_type;
using location_handler_type = osmium::handler::NodeLocationsForWays<index_type>;
int main(int argc, char* argv[]) {
if (argc != 3) {
std::cerr << "Usage: " << argv[0] << " OSMFILE FORMAT\n";
exit(1);
std::exit(1);
}
std::string input_filename = argv[1];
std::string location_store = argv[2];
const std::string input_filename{argv[1]};
const std::string location_store{argv[2]};
osmium::io::Reader reader(input_filename);
osmium::io::Reader reader{input_filename};
const auto& map_factory = osmium::index::MapFactory<osmium::unsigned_object_id_type, osmium::Location>::instance();
std::unique_ptr<index_type> index = map_factory.create_map(location_store);
location_handler_type location_handler(*index);
location_handler_type location_handler{*index};
location_handler.ignore_errors();
osmium::apply(reader, location_handler);

58
benchmarks/osmium_benchmark_static_vs_dynamic_index.cpp
View File

@ -19,8 +19,10 @@
#include <algorithm>
#include <chrono>
#include <cmath>
#include <cstdlib>
#include <iostream>
#include <limits>
#include <string>
#include <osmium/index/map/all.hpp>
#include <osmium/handler/node_locations_for_ways.hpp>
@ -29,23 +31,23 @@
#include <osmium/io/any_input.hpp>
#include <osmium/handler.hpp>
typedef osmium::index::map::SparseMemArray<osmium::unsigned_object_id_type, osmium::Location> static_index_type;
const std::string location_store="sparse_mem_array";
using static_index_type = osmium::index::map::SparseMemArray<osmium::unsigned_object_id_type, osmium::Location>;
const std::string location_store{"sparse_mem_array"};
typedef osmium::index::map::Map<osmium::unsigned_object_id_type, osmium::Location> dynamic_index_type;
using dynamic_index_type = osmium::index::map::Map<osmium::unsigned_object_id_type, osmium::Location>;
typedef osmium::handler::NodeLocationsForWays<static_index_type> static_location_handler_type;
typedef osmium::handler::NodeLocationsForWays<dynamic_index_type> dynamic_location_handler_type;
using static_location_handler_type = osmium::handler::NodeLocationsForWays<static_index_type>;
using dynamic_location_handler_type = osmium::handler::NodeLocationsForWays<dynamic_index_type>;
int main(int argc, char* argv[]) {
if (argc != 2) {
std::cerr << "Usage: " << argv[0] << " OSMFILE\n";
exit(1);
std::exit(1);
}
std::string input_filename = argv[1];
const std::string input_filename{argv[1]};
osmium::memory::Buffer buffer = osmium::io::read_file(input_filename);
osmium::memory::Buffer buffer{osmium::io::read_file(input_filename)};
const auto& map_factory = osmium::index::MapFactory<osmium::unsigned_object_id_type, osmium::Location>::instance();
@ -67,20 +69,20 @@ int main(int argc, char* argv[]) {
{
// static index
osmium::memory::Buffer tmp_buffer(buffer.committed());
osmium::memory::Buffer tmp_buffer{buffer.committed()};
for (const auto& item : buffer) {
tmp_buffer.add_item(item);
tmp_buffer.commit();
}
static_index_type static_index;
static_location_handler_type static_location_handler(static_index);
static_location_handler_type static_location_handler{static_index};
auto start = std::chrono::steady_clock::now();
const auto start = std::chrono::steady_clock::now();
osmium::apply(tmp_buffer, static_location_handler);
auto end = std::chrono::steady_clock::now();
const auto end = std::chrono::steady_clock::now();
double duration = std::chrono::duration<double, std::milli>(end-start).count();
const double duration = std::chrono::duration<double, std::milli>(end-start).count();
if (duration < static_min) static_min = duration;
if (duration > static_max) static_max = duration;
@ -89,21 +91,21 @@ int main(int argc, char* argv[]) {
{
// dynamic index
osmium::memory::Buffer tmp_buffer(buffer.committed());
osmium::memory::Buffer tmp_buffer{buffer.committed()};
for (const auto& item : buffer) {
tmp_buffer.add_item(item);
tmp_buffer.commit();
}
std::unique_ptr<dynamic_index_type> index = map_factory.create_map(location_store);
dynamic_location_handler_type dynamic_location_handler(*index);
dynamic_location_handler_type dynamic_location_handler{*index};
dynamic_location_handler.ignore_errors();
auto start = std::chrono::steady_clock::now();
const auto start = std::chrono::steady_clock::now();
osmium::apply(tmp_buffer, dynamic_location_handler);
auto end = std::chrono::steady_clock::now();
const auto end = std::chrono::steady_clock::now();
double duration = std::chrono::duration<double, std::milli>(end-start).count();
const double duration = std::chrono::duration<double, std::milli>(end-start).count();
if (duration < dynamic_min) dynamic_min = duration;
if (duration > dynamic_max) dynamic_max = duration;
@ -111,21 +113,21 @@ int main(int argc, char* argv[]) {
}
}
double static_avg = static_sum/runs;
double dynamic_avg = dynamic_sum/runs;
const double static_avg = static_sum/runs;
const double dynamic_avg = dynamic_sum/runs;
std::cout << "static min=" << static_min << "ms avg=" << static_avg << "ms max=" << static_max << "ms\n";
std::cout << "dynamic min=" << dynamic_min << "ms avg=" << dynamic_avg << "ms max=" << dynamic_max << "ms\n";
double rfactor = 100.0;
double diff_min = std::round((dynamic_min - static_min) * rfactor) / rfactor;
double diff_avg = std::round((dynamic_avg - static_avg) * rfactor) / rfactor;
double diff_max = std::round((dynamic_max - static_max) * rfactor) / rfactor;
const double rfactor = 100.0;
const double diff_min = std::round((dynamic_min - static_min) * rfactor) / rfactor;
const double diff_avg = std::round((dynamic_avg - static_avg) * rfactor) / rfactor;
const double diff_max = std::round((dynamic_max - static_max) * rfactor) / rfactor;
double prfactor = 10.0;
double percent_min = std::round((100.0 * diff_min / static_min) * prfactor) / prfactor;
double percent_avg = std::round((100.0 * diff_avg / static_avg) * prfactor) / prfactor;
double percent_max = std::round((100.0 * diff_max / static_max) * prfactor) / prfactor;
const double prfactor = 10.0;
const double percent_min = std::round((100.0 * diff_min / static_min) * prfactor) / prfactor;
const double percent_avg = std::round((100.0 * diff_avg / static_avg) * prfactor) / prfactor;
const double percent_max = std::round((100.0 * diff_max / static_max) * prfactor) / prfactor;
std::cout << "difference:";
std::cout << " min=" << diff_min << "ms (" << percent_min << "%)";

17
benchmarks/osmium_benchmark_write_pbf.cpp
View File

@ -4,8 +4,9 @@
*/
#include <cstdint>
#include <vector>
#include <cstdlib>
#include <iostream>
#include <string>
#include <osmium/io/any_input.hpp>
#include <osmium/io/any_output.hpp>
@ -13,16 +14,16 @@
int main(int argc, char* argv[]) {
if (argc != 3) {
std::cerr << "Usage: " << argv[0] << " INPUT-FILE OUTPUT-FILE\n";
exit(1);
std::exit(1);
}
std::string input_filename = argv[1];
std::string output_filename = argv[2];
std::string input_filename{argv[1]};
std::string output_filename{argv[2]};
osmium::io::Reader reader(input_filename);
osmium::io::File output_file(output_filename, "pbf");
osmium::io::Reader reader{input_filename};
osmium::io::File output_file{output_filename, "pbf"};
osmium::io::Header header;
osmium::io::Writer writer(output_file, header, osmium::io::overwrite::allow);
osmium::io::Writer writer{output_file, header, osmium::io::overwrite::allow};
while (osmium::memory::Buffer buffer = reader.read()) {
writer(std::move(buffer));

123
build-appveyor.bat Normal file
View File

@ -0,0 +1,123 @@
@ECHO OFF
SETLOCAL
SET EL=0
ECHO ~~~~~~ %~f0 ~~~~~~
SET CUSTOM_CMAKE=cmake-3.6.2-win64-x64
::show all available env vars
SET
ECHO cmake on AppVeyor
cmake -version
ECHO activating VS cmd prompt && CALL "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\vcvarsall.bat" amd64
IF %ERRORLEVEL% NEQ 0 GOTO ERROR
SET lodir=%CD%
SET PATH=%lodir%\%CUSTOM_CMAKE%\bin;%PATH%
SET LODEPSDIR=%lodir%\libosmium-deps
SET PROJ_LIB=%LODEPSDIR%\proj\share
SET GDAL_DATA=%LODEPSDIR%\gdal\data
::gdal.dll
SET PATH=%LODEPSDIR%\gdal\lib;%PATH%
::geos.dll
SET PATH=%LODEPSDIR%\geos\lib;%PATH%
::libtiff.dll
SET PATH=%LODEPSDIR%\libtiff\lib;%PATH%
::jpeg.dll
SET PATH=%LODEPSDIR%\jpeg\lib;%PATH%
::libexpat.dll
SET PATH=%LODEPSDIR%\expat\lib;%PATH%
::zlibwapi.dll
SET PATH=%LODEPSDIR%\zlib\lib;%PATH%
::convert backslashes in bzip2 path to forward slashes
::cmake cannot find it otherwise
SET LIBBZIP2=%LODEPSDIR%\bzip2\lib\libbz2.lib
SET LIBBZIP2=%LIBBZIP2:\=/%
IF NOT EXIST cm.7z ECHO downloading cmake %CUSTOM_CMAKE% ... && powershell Invoke-WebRequest https://mapbox.s3.amazonaws.com/windows-builds/windows-build-deps/%CUSTOM_CMAKE%.7z -OutFile cm.7z
IF %ERRORLEVEL% NEQ 0 GOTO ERROR
IF NOT EXIST lodeps.7z ECHO downloading binary dependencies... && powershell Invoke-WebRequest https://mapbox.s3.amazonaws.com/windows-builds/windows-build-deps/libosmium-deps-win-14.0-x64.7z -OutFile lodeps.7z
IF %ERRORLEVEL% NEQ 0 GOTO ERROR
IF NOT EXIST %CUSTOM_CMAKE% ECHO extracting cmake... && 7z x cm.7z | %windir%\system32\find "ing archive"
IF %ERRORLEVEL% NEQ 0 GOTO ERROR
IF NOT EXIST %LODEPSDIR% ECHO extracting binary dependencies... && 7z x lodeps.7z | %windir%\system32\find "ing archive"
IF %ERRORLEVEL% NEQ 0 GOTO ERROR
ECHO %LODEPSDIR%
DIR %LODEPSDIR%
::TREE %LODEPSDIR%
::powershell (Get-ChildItem $env:LODEPSDIR\boost\lib -Filter *boost*.dll)[0].BaseName.split('_')[-1]
FOR /F "tokens=1 usebackq" %%i in (`powershell ^(Get-ChildItem %LODEPSDIR%\boost\lib -Filter *boost*.dll^)[0].BaseName.split^('_'^)[-1]`) DO SET BOOST_VERSION=%%i
IF %ERRORLEVEL% NEQ 0 GOTO ERROR
ECHO BOOST_VERSION^: %BOOST_VERSION%
ECHO our own cmake
cmake -version
CD %lodir%\..
IF NOT EXIST osm-testdata ECHO cloning osm-testdata && git clone --depth 1 https://github.com/osmcode/osm-testdata.git
IF %ERRORLEVEL% NEQ 0 GOTO ERROR
CD osm-testdata
git fetch
IF %ERRORLEVEL% NEQ 0 GOTO ERROR
git pull
IF %ERRORLEVEL% NEQ 0 GOTO ERROR
CD %lodir%
IF EXIST build ECHO deleting build dir... && RD /Q /S build
IF %ERRORLEVEL% NEQ 0 GOTO ERROR
MKDIR build
IF %ERRORLEVEL% NEQ 0 GOTO ERROR
CD build
ECHO config^: %config%
::This will produce lots of LNK4099 warnings which can be ignored.
::Unfortunately they can't be disabled, see
::http://stackoverflow.com/questions/661606/visual-c-how-to-disable-specific-linker-warnings
SET CMAKE_CMD=cmake .. ^
-LA -G "Visual Studio 14 Win64" ^
-DOsmium_DEBUG=TRUE ^
-DCMAKE_BUILD_TYPE=%config% ^
-DBUILD_HEADERS=OFF ^
-DBOOST_ROOT=%LODEPSDIR%\boost ^
-DZLIB_LIBRARY=%LODEPSDIR%\zlib\lib\zlibwapi.lib ^
-DBZIP2_LIBRARY_RELEASE=%LIBBZIP2% ^
-DCMAKE_PREFIX_PATH=%LODEPSDIR%\zlib;%LODEPSDIR%\expat;%LODEPSDIR%\bzip2;%LODEPSDIR%\geos;%LODEPSDIR%\gdal;%LODEPSDIR%\proj;%LODEPSDIR%\sparsehash;%LODEPSDIR%\wingetopt
ECHO calling^: %CMAKE_CMD%
%CMAKE_CMD%
IF %ERRORLEVEL% NEQ 0 GOTO ERROR
msbuild libosmium.sln ^
/p:Configuration=%config% ^
/toolsversion:14.0 ^
/p:Platform=x64 ^
/p:PlatformToolset=v140
IF %ERRORLEVEL% NEQ 0 GOTO ERROR
ctest --output-on-failure ^
-C %config% ^
-E testdata-overview
IF %ERRORLEVEL% NEQ 0 GOTO ERROR
GOTO DONE
:ERROR
ECHO ~~~~~~ ERROR %~f0 ~~~~~~
SET EL=%ERRORLEVEL%
:DONE
IF %EL% NEQ 0 ECHO. && ECHO !!! ERRORLEVEL^: %EL% !!! && ECHO.
ECHO ~~~~~~ DONE %~f0 ~~~~~~
EXIT /b %EL%

43
build-local.bat Normal file
View File

@ -0,0 +1,43 @@
@ECHO OFF
SETLOCAL
SET EL=0
ECHO ~~~~~~ %~f0 ~~~~~~
ECHO.
ECHO build-local ["config=Dev"]
ECHO default config^: RelWithDebInfo
ECHO.
SET platform=x64
SET config=RelWithDebInfo
:: OVERRIDE PARAMETERS >>>>>>>>
:NEXT-ARG
IF '%1'=='' GOTO ARGS-DONE
ECHO setting %1
SET %1
SHIFT
GOTO NEXT-ARG
:ARGS-DONE
::<<<<< OVERRIDE PARAMETERS
WHERE 7z
IF %ERRORLEVEL% NEQ 0 ECHO 7zip not on PATH && GOTO ERROR
CALL build-appveyor.bat
IF %ERRORLEVEL% NEQ 0 GOTO ERROR
GOTO DONE
:ERROR
ECHO ~~~~~~ ERROR %~f0 ~~~~~~
SET EL=%ERRORLEVEL%
:DONE
IF %EL% NEQ 0 ECHO. && ECHO !!! ERRORLEVEL^: %EL% !!! && ECHO.
ECHO ~~~~~~ DONE %~f0 ~~~~~~
EXIT /b %EL%

5
cmake/iwyu.sh
View File

@ -6,6 +6,11 @@
# TODO: This script should be integrated with cmake in some way...
#
# If these are set, the wrong compiler is used by iwyu and there will be
# errors about missing includes.
unset CC
unset CXX
cmdline="iwyu -Xiwyu --mapping_file=osmium.imp -std=c++11 -I include"
log=build/iwyu.log

40
examples/CMakeLists.txt
View File

@ -12,13 +12,17 @@ set(EXAMPLES
area_test
convert
count
create_node_cache
debug
filter_discussions
index
location_cache_create
location_cache_use
pub_names
read
read_with_progress
road_length
serdump
use_node_cache
tiles
CACHE STRING "Example programs"
)
@ -28,7 +32,7 @@ set(EXAMPLES
# Examples depending on wingetopt
#
#-----------------------------------------------------------------------------
set(GETOPT_EXAMPLES area_test convert serdump)
set(GETOPT_EXAMPLES area_test convert index serdump)
if(NOT GETOPT_MISSING)
foreach(example ${GETOPT_EXAMPLES})
list(APPEND EXAMPLE_LIBS_${example} ${GETOPT_LIBRARY})
@ -42,36 +46,6 @@ else()
endif()
#-----------------------------------------------------------------------------
#
# Examples depending on SparseHash
#
#-----------------------------------------------------------------------------
if(NOT SPARSEHASH_FOUND)
list(REMOVE_ITEM EXAMPLES area_test)
message(STATUS "Configuring examples - Skipping examples because Google SparseHash not found:")
message(STATUS " - osmium_area_test")
endif()
#-----------------------------------------------------------------------------
#
# Examples depending on Boost Program Options
#
#-----------------------------------------------------------------------------
unset(Boost_LIBRARIES)
unset(Boost_FOUND)
find_package(Boost 1.38 COMPONENTS program_options)
if(Boost_PROGRAM_OPTIONS_FOUND)
list(APPEND EXAMPLE_LIBS_index ${Boost_PROGRAM_OPTIONS_LIBRARY})
else()
list(REMOVE_ITEM EXAMPLES index)
message(STATUS "Configuring examples - Skipping examples because Boost program_options not found:")
message(STATUS " - osmium_index")
endif()
#-----------------------------------------------------------------------------
#
# Configure examples

32
examples/README.md Normal file
View File

@ -0,0 +1,32 @@
# Osmium example programs
The programs in this directory are intended as examples for developers. They
contain extensive comments explaining what's going on. Note that the examples
only cover a small part of what Osmium can do, you should also read the
documentation and API documentation.
All programs can be run without arguments and they will tell you how to run
them.
## Very simple examples
* `osmium_read`
* `osmium_count`
* `osmium_debug`
* `osmium_tiles`
## Still reasonably simple examples
* `osmium_filter_discussions`
* `osmium_convert`
## More advanced examples
* `osmium_area_test`
## License
The code in these example files is released into the Public Domain. Feel free
to copy the code and build on it.

127
examples/osmium_area_test.cpp
View File

@ -1,48 +1,81 @@
/*
This is an example tool that creates multipolygons from OSM data
and dumps them to stdout.
EXAMPLE osmium_area_test
Create multipolygons from OSM data and dump them to stdout in one of two
formats: WKT or using the built-in Dump format.
DEMONSTRATES USE OF:
* file input
* location indexes and the NodeLocationsForWays handler
* the MultipolygonCollector and Assembler to assemble areas (multipolygons)
* your own handler that works with areas (multipolygons)
* the WKTFactory to write geometries in WKT format
* the Dump handler
* the DynamicHandler
SIMPLER EXAMPLES you might want to understand first:
* osmium_read
* osmium_count
* osmium_debug
LICENSE
The code in this example file is released into the Public Domain.
*/
#include <iostream>
#include <getopt.h>
#include <cstdlib> // for std::exit
#include <getopt.h> // for getopt_long
#include <iostream> // for std::cout, std::cerr
// For assembling multipolygons
#include <osmium/area/assembler.hpp>
#include <osmium/area/multipolygon_collector.hpp>
// For the DynamicHandler class
#include <osmium/dynamic_handler.hpp>
// For the WKT factory
#include <osmium/geom/wkt.hpp>
// For the Dump handler
#include <osmium/handler/dump.hpp>
// For the NodeLocationForWays handler
#include <osmium/handler/node_locations_for_ways.hpp>
#include <osmium/index/map/dummy.hpp>
#include <osmium/index/map/sparse_mem_array.hpp>
// Allow any format of input files (XML, PBF, ...)
#include <osmium/io/any_input.hpp>
// For osmium::apply()
#include <osmium/visitor.hpp>
typedef osmium::index::map::Dummy<osmium::unsigned_object_id_type, osmium::Location> index_neg_type;
typedef osmium::index::map::SparseMemArray<osmium::unsigned_object_id_type, osmium::Location> index_pos_type;
typedef osmium::handler::NodeLocationsForWays<index_pos_type, index_neg_type> location_handler_type;
// For the location index. There are different types of indexes available.
// This will work for small and medium sized input files.
#include <osmium/index/map/sparse_mem_array.hpp>
// The type of index used. This must match the include file above
using index_type = osmium::index::map::SparseMemArray<osmium::unsigned_object_id_type, osmium::Location>;
// The location handler always depends on the index type
using location_handler_type = osmium::handler::NodeLocationsForWays<index_type>;
// This handler writes all area geometries out in WKT (Well Known Text) format.
class WKTDump : public osmium::handler::Handler {
// This factory is used to create a geometry in WKT format from OSM
// objects. The template parameter is empty here, because we output WGS84
// coordinates, but could be used for a projection.
osmium::geom::WKTFactory<> m_factory;
std::ostream& m_out;
public:
WKTDump(std::ostream& out) :
m_out(out) {
}
// This callback is called by osmium::apply for each area in the data.
void area(const osmium::Area& area) {
try {
m_out << m_factory.create_multipolygon(area) << "\n";
} catch (osmium::geometry_error& e) {
m_out << "GEOMETRY ERROR: " << e.what() << "\n";
std::cout << m_factory.create_multipolygon(area) << "\n";
} catch (const osmium::geometry_error& e) {
std::cout << "GEOMETRY ERROR: " << e.what() << "\n";
}
}
@ -65,8 +98,13 @@ int main(int argc, char* argv[]) {
{0, 0, 0, 0}
};
// Initialize an empty DynamicHandler. Later it will be associated
// with one of the handlers. You can think of the DynamicHandler as
// a kind of "variant handler" or a "pointer handler" pointing to the
// real handler.
osmium::handler::DynamicHandler handler;
// Read options from command line.
while (true) {
int c = getopt_long(argc, argv, "hwo", long_options, 0);
if (c == -1) {
@ -76,54 +114,81 @@ int main(int argc, char* argv[]) {
switch (c) {
case 'h':
print_help();
exit(0);
std::exit(0);
case 'w':
handler.set<WKTDump>(std::cout);
handler.set<WKTDump>();
break;
case 'o':
handler.set<osmium::handler::Dump>(std::cout);
break;
default:
exit(1);
std::exit(1);
}
}
int remaining_args = argc - optind;
if (remaining_args != 1) {
std::cerr << "Usage: " << argv[0] << " [OPTIONS] OSMFILE\n";
exit(1);
std::exit(1);
}
osmium::io::File infile(argv[optind]);
osmium::io::File input_file{argv[optind]};
// Configuration for the multipolygon assembler. Here the default settings
// are used, but you could change multiple settings.
osmium::area::Assembler::config_type assembler_config;
osmium::area::MultipolygonCollector<osmium::area::Assembler> collector(assembler_config);
// Initialize the MultipolygonCollector. Its job is to collect all
// relations and member ways needed for each area. It then calls an
// instance of the osmium::area::Assembler class (with the given config)
// to actually assemble one area.
osmium::area::MultipolygonCollector<osmium::area::Assembler> collector{assembler_config};
// We read the input file twice. In the first pass, only relations are
// read and fed into the multipolygon collector.
std::cerr << "Pass 1...\n";
osmium::io::Reader reader1(infile, osmium::osm_entity_bits::relation);
osmium::io::Reader reader1{input_file, osmium::osm_entity_bits::relation};
collector.read_relations(reader1);
reader1.close();
std::cerr << "Pass 1 done\n";
// Output the amount of main memory used so far. All multipolygon relations
// are in memory now.
std::cerr << "Memory:\n";
collector.used_memory();
index_pos_type index_pos;
index_neg_type index_neg;
location_handler_type location_handler(index_pos, index_neg);
location_handler.ignore_errors(); // XXX
// The index storing all node locations.
index_type index;
// The handler that stores all node locations in the index and adds them
// to the ways.
location_handler_type location_handler{index};
// If a location is not available in the index, we ignore it. It might
// not be needed (if it is not part of a multipolygon relation), so why
// create an error?
location_handler.ignore_errors();
// On the second pass we read all objects and run them first through the
// node location handler and then the multipolygon collector. The collector
// will put the areas it has created into the "buffer" which are then
// fed through our "handler".
std::cerr << "Pass 2...\n";
osmium::io::Reader reader2(infile);
osmium::io::Reader reader2{input_file};
osmium::apply(reader2, location_handler, collector.handler([&handler](osmium::memory::Buffer&& buffer) {
osmium::apply(buffer, handler);
}));
reader2.close();
std::cerr << "Pass 2 done\n";
// Output the amount of main memory used so far. All complete multipolygon
// relations have been cleaned up.
std::cerr << "Memory:\n";
collector.used_memory();
// If there were multipolgyon relations in the input, but some of their
// members are not in the input file (which often happens for extracts)
// this will write the IDs of the incomplete relations to stderr.
std::vector<const osmium::Relation*> incomplete_relations = collector.get_incomplete_relations();
if (!incomplete_relations.empty()) {
std::cerr << "Warning! Some member ways missing for these multipolygon relations:";

94
examples/osmium_convert.cpp
View File

@ -1,16 +1,32 @@
/*
EXAMPLE osmium_convert
Convert OSM files from one format into another.
DEMONSTRATES USE OF:
* file input and output
* file types
* Osmium buffers
SIMPLER EXAMPLES you might want to understand first:
* osmium_read
LICENSE
The code in this example file is released into the Public Domain.
*/
#include <iostream>
#include <getopt.h>
#include <cstdlib> // for std::exit
#include <exception> // for std::exception
#include <getopt.h> // for getopt_long
#include <iostream> // for std::cout, std::cerr
#include <string> // for std::string
// Allow any format of input files (XML, PBF, ...)
#include <osmium/io/any_input.hpp>
// Allow any format of output files (XML, PBF, ...)
#include <osmium/io/any_output.hpp>
void print_help() {
@ -43,9 +59,13 @@ int main(int argc, char* argv[]) {
{0, 0, 0, 0}
};
// Input and output format are empty by default. Later this will mean that
// the format should be taken from the input and output file suffix,
// respectively.
std::string input_format;
std::string output_format;
// Read options from command line.
while (true) {
int c = getopt_long(argc, argv, "dhf:t:", long_options, 0);
if (c == -1) {
@ -55,7 +75,7 @@ int main(int argc, char* argv[]) {
switch (c) {
case 'h':
print_help();
exit(0);
std::exit(0);
case 'f':
input_format = optarg;
break;
@ -63,49 +83,73 @@ int main(int argc, char* argv[]) {
output_format = optarg;
break;
default:
exit(1);
std::exit(1);
}
}
std::string input;
std::string output;
int remaining_args = argc - optind;
if (remaining_args > 2) {
std::cerr << "Usage: " << argv[0] << " [OPTIONS] [INFILE [OUTFILE]]" << std::endl;
exit(1);
} else if (remaining_args == 2) {
input = argv[optind];
output = argv[optind+1];
} else if (remaining_args == 1) {
input = argv[optind];
std::cerr << "Usage: " << argv[0] << " [OPTIONS] [INFILE [OUTFILE]]\n";
std::exit(1);
}
osmium::io::File infile(input, input_format);
// Get input file name from command line.
std::string input_file_name;
if (remaining_args >= 1) {
input_file_name = argv[optind];
}
osmium::io::File outfile(output, output_format);
// Get output file name from command line.
std::string output_file_name;
if (remaining_args == 2) {
output_file_name = argv[optind+1];
}
if (infile.has_multiple_object_versions() && !outfile.has_multiple_object_versions()) {
// This declares the input and output files using either the suffix of
// the file names or the format in the 2nd argument. It does not yet open
// the files.
osmium::io::File input_file{input_file_name, input_format};
osmium::io::File output_file{output_file_name, output_format};
// Input and output files can be OSM data files (without history) or
// OSM history files. History files are detected if they use the '.osh'
// file suffix.
if ( input_file.has_multiple_object_versions() &&
!output_file.has_multiple_object_versions()) {
std::cerr << "Warning! You are converting from an OSM file with (potentially) several versions of the same object to one that is not marked as such.\n";
}
int exit_code = 0;
try {
osmium::io::Reader reader(infile);
// Initialize Reader
osmium::io::Reader reader{input_file};
// Get header from input file and change the "generator" setting to
// outselves.
osmium::io::Header header = reader.header();
header.set("generator", "osmium_convert");
osmium::io::Writer writer(outfile, header, osmium::io::overwrite::allow);
// Initialize Writer using the header from above and tell it that it
// is allowed to overwrite a possibly existing file.
osmium::io::Writer writer(output_file, header, osmium::io::overwrite::allow);
// Copy the contents from the input to the output file one buffer at
// a time. This is much easier and faster than copying each object
// in the file. Buffers are moved around, so there is no cost for
// copying in memory.
while (osmium::memory::Buffer buffer = reader.read()) {
writer(std::move(buffer));
}
// Explicitly close the writer and reader. Will throw an exception if
// there is a problem. If you wait for the destructor to close the writer
// and reader, you will not notice the problem, because destructors must
// not throw.
writer.close();
reader.close();
} catch (std::exception& e) {
} catch (const std::exception& e) {
// All exceptions used by the Osmium library derive from std::exception.
std::cerr << e.what() << "\n";
exit_code = 1;
}
return exit_code;
std::exit(1);
}
}

69
examples/osmium_count.cpp
View File

@ -1,56 +1,95 @@
/*
This is a small tool that counts the number of nodes, ways, and relations in
the input file.
EXAMPLE osmium_count
Counts the number of nodes, ways, and relations in the input file.
DEMONSTRATES USE OF:
* OSM file input
* your own handler
* the memory usage utility class
SIMPLER EXAMPLES you might want to understand first:
* osmium_read
LICENSE
The code in this example file is released into the Public Domain.
*/
#include <cstdint>
#include <iostream>
#include <cstdint> // for std::uint64_t
#include <cstdlib> // for std::exit
#include <iostream> // for std::cout, std::cerr
// Allow any format of input files (XML, PBF, ...)
#include <osmium/io/any_input.hpp>
// We want to use the handler interface
#include <osmium/handler.hpp>
// Utility class gives us access to memory usage information
#include <osmium/util/memory.hpp>
// For osmium::apply()
#include <osmium/visitor.hpp>
// Handler derive from the osmium::handler::Handler base class. Usually you
// overwrite functions node(), way(), and relation(). Other functions are
// available, too. Read the API documentation for details.
struct CountHandler : public osmium::handler::Handler {
uint64_t nodes = 0;
uint64_t ways = 0;
uint64_t relations = 0;
std::uint64_t nodes = 0;
std::uint64_t ways = 0;
std::uint64_t relations = 0;
void node(osmium::Node&) {
// This callback is called by osmium::apply for each node in the data.
void node(const osmium::Node&) noexcept {
++nodes;
}
void way(osmium::Way&) {
// This callback is called by osmium::apply for each way in the data.
void way(const osmium::Way&) noexcept {
++ways;
}
void relation(osmium::Relation&) {
// This callback is called by osmium::apply for each relation in the data.
void relation(const osmium::Relation&) noexcept {
++relations;
}
};
}; // struct CountHandler
int main(int argc, char* argv[]) {
if (argc != 2) {
std::cerr << "Usage: " << argv[0] << " OSMFILE\n";
exit(1);
std::exit(1);
}
osmium::io::File infile(argv[1]);
osmium::io::Reader reader(infile);
// The Reader is initialized here with an osmium::io::File, but could
// also be directly initialized with a file name.
osmium::io::File input_file{argv[1]};
osmium::io::Reader reader{input_file};
// Create an instance of our own CountHandler and push the data from the
// input file through it.
CountHandler handler;
osmium::apply(reader, handler);
// You do not have to close the Reader explicitly, but because the
// destructor can't throw, you will not see any errors otherwise.
reader.close();
std::cout << "Nodes: " << handler.nodes << "\n";
std::cout << "Ways: " << handler.ways << "\n";
std::cout << "Relations: " << handler.relations << "\n";
// Because of the huge amount of OSM data, some Osmium-based programs
// (though not this one) can use huge amounts of data. So checking actual
// memore usage is often useful and can be done easily with this class.
// (Currently only works on Linux, not OSX and Windows.)
osmium::MemoryUsage memory;
std::cout << "\nMemory used: " << memory.peak() << " MBytes\n";
}

55
examples/osmium_create_node_cache.cpp
View File

@ -1,55 +0,0 @@
/*
This reads an OSM file and writes out the node locations to a cache
file.
The code in this example file is released into the Public Domain.
*/
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <iostream>
#include <osmium/io/any_input.hpp>
#include <osmium/index/map/dummy.hpp>
#include <osmium/index/map/dense_mmap_array.hpp>
#include <osmium/index/map/dense_file_array.hpp>
#include <osmium/handler/node_locations_for_ways.hpp>
#include <osmium/visitor.hpp>
typedef osmium::index::map::Dummy<osmium::unsigned_object_id_type, osmium::Location> index_neg_type;
//typedef osmium::index::map::DenseMmapArray<osmium::unsigned_object_id_type, osmium::Location> index_pos_type;
typedef osmium::index::map::DenseFileArray<osmium::unsigned_object_id_type, osmium::Location> index_pos_type;
typedef osmium::handler::NodeLocationsForWays<index_pos_type, index_neg_type> location_handler_type;
int main(int argc, char* argv[]) {
if (argc != 3) {
std::cerr << "Usage: " << argv[0] << " OSM_FILE CACHE_FILE\n";
return 1;
}
std::string input_filename(argv[1]);
osmium::io::Reader reader(input_filename, osmium::osm_entity_bits::node);
int fd = open(argv[2], O_RDWR | O_CREAT, 0666);
if (fd == -1) {
std::cerr << "Can not open node cache file '" << argv[2] << "': " << strerror(errno) << "\n";
return 1;
}
index_pos_type index_pos {fd};
index_neg_type index_neg;
location_handler_type location_handler(index_pos, index_neg);
location_handler.ignore_errors();
osmium::apply(reader, location_handler);
reader.close();
return 0;
}

46
examples/osmium_debug.cpp
View File

@ -1,27 +1,46 @@
/*
This is a small tool to dump the contents of the input file.
EXAMPLE osmium_debug
Dump the contents of the input file in a debug format.
DEMONSTRATES USE OF:
* file input reading only some types
* the dump handler
SIMPLER EXAMPLES you might want to understand first:
* osmium_read
* osmium_count
LICENSE
The code in this example file is released into the Public Domain.
*/
#include <iostream>
#include <cstdlib> // for std::exit
#include <iostream> // for std::cout, std::cerr
#include <string> // for std::string
// The Dump handler
#include <osmium/handler/dump.hpp>
// Allow any format of input files (XML, PBF, ...)
#include <osmium/io/any_input.hpp>
int main(int argc, char* argv[]) {
// Speed up output (not Osmium-specific)
std::ios_base::sync_with_stdio(false);
if (argc < 2 || argc > 3) {
std::cerr << "Usage: " << argv[0] << " OSMFILE [TYPES]\n";
std::cerr << "TYPES can be any combination of 'n', 'w', 'r', and 'c' to indicate what types of OSM entities you want (default: all).\n";
exit(1);
std::exit(1);
}
// Default is all entity types: nodes, ways, relations, and changesets
osmium::osm_entity_bits::type read_types = osmium::osm_entity_bits::all;
// Get entity types from command line if there is a 2nd argument.
if (argc == 3) {
read_types = osmium::osm_entity_bits::nothing;
std::string types = argv[2];
@ -31,20 +50,27 @@ int main(int argc, char* argv[]) {
if (types.find('c') != std::string::npos) read_types |= osmium::osm_entity_bits::changeset;
}
osmium::io::Reader reader(argv[1], read_types);
osmium::io::Header header = reader.header();
// Initialize Reader with file name and the types of entities we want to
// read.
osmium::io::Reader reader{argv[1], read_types};
// The file header can contain metadata such as the program that generated
// the file and the bounding box of the data.
osmium::io::Header header = reader.header();
std::cout << "HEADER:\n generator=" << header.get("generator") << "\n";
for (auto& bbox : header.boxes()) {
for (const auto& bbox : header.boxes()) {
std::cout << " bbox=" << bbox << "\n";
}
osmium::handler::Dump dump(std::cout);
while (osmium::memory::Buffer buffer = reader.read()) {
osmium::apply(buffer, dump);
}
// Initialize Dump handler.
osmium::handler::Dump dump{std::cout};
// Read from input and send everything to Dump handler.
osmium::apply(reader, dump);
// You do not have to close the Reader explicitly, but because the
// destructor can't throw, you will not see any errors otherwise.
reader.close();
}

46
examples/osmium_filter_discussions.cpp
View File

@ -1,53 +1,73 @@
/*
EXAMPLE osmium_filter_discussions
Read OSM changesets with discussions from a changeset dump like the one
you get from http://planet.osm.org/planet/discussions-latest.osm.bz2
and write out only those changesets which have discussions (ie comments).
DEMONSTRATES USE OF:
* file input and output
* setting file formats using the osmium::io::File class
* OSM file headers
* input and output iterators
SIMPLER EXAMPLES you might want to understand first:
* osmium_read
* osmium_count
LICENSE
The code in this example file is released into the Public Domain.
*/
#include <algorithm> // for std::copy_if
#include <cstdlib> // for std::exit
#include <iostream> // for std::cout, std::cerr
// we want to read OSM files in XML format
// (other formats don't support full changesets, so only XML is needed here)
// We want to read OSM files in XML format
// (other formats don't support full changesets, so only XML is needed here).
#include <osmium/io/xml_input.hpp>
#include <osmium/io/input_iterator.hpp>
// we want to write OSM files in XML format
// We want to write OSM files in XML format.
#include <osmium/io/xml_output.hpp>
// We want to use input and output iterators for easy integration with the
// algorithms of the standard library.
#include <osmium/io/input_iterator.hpp>
#include <osmium/io/output_iterator.hpp>
// we want to support any compressioon (.gz2 and .bz2)
// We want to support any compression (none, gzip, and bzip2).
#include <osmium/io/any_compression.hpp>
int main(int argc, char* argv[]) {
if (argc != 3) {
std::cout << "Usage: " << argv[0] << " INFILE OUTFILE\n";
exit(1);
std::exit(1);
}
// The input file, deduce file format from file suffix
osmium::io::File infile(argv[1]);
// The input file, deduce file format from file suffix.
osmium::io::File input_file{argv[1]};
// The output file, force class XML OSM file format
osmium::io::File outfile(argv[2], "osm");
// The output file, force XML OSM file format.
osmium::io::File output_file{argv[2], "osm"};
// Initialize Reader for the input file.
// Read only changesets (will ignore nodes, ways, and
// relations if there are any).
osmium::io::Reader reader(infile, osmium::osm_entity_bits::changeset);
osmium::io::Reader reader{input_file, osmium::osm_entity_bits::changeset};
// Get the header from the input file
// Get the header from the input file.
osmium::io::Header header = reader.header();
// Set the "generator" on the header to ourselves.
header.set("generator", "osmium_filter_discussions");
// Initialize writer for the output file. Use the header from the input
// file for the output file. This will copy over some header information.
// The last parameter will tell the writer that it is allowed to overwrite
// an existing file. Without it, it will refuse to do so.
osmium::io::Writer writer(outfile, header, osmium::io::overwrite::allow);
osmium::io::Writer writer(output_file, header, osmium::io::overwrite::allow);
// Create range of input iterators that will iterator over all changesets
// delivered from input file through the "reader".

147
examples/osmium_index.cpp
View File

@ -12,7 +12,7 @@
#include <sys/stat.h>
#include <sys/types.h>
#include <boost/program_options.hpp>
#include <getopt.h>
#include <osmium/index/map/dense_file_array.hpp>
#include <osmium/index/map/sparse_file_array.hpp>
@ -31,7 +31,7 @@ class IndexSearch {
void dump_dense() {
dense_index_type index(m_fd);
for (size_t i = 0; i < index.size(); ++i) {
for (std::size_t i = 0; i < index.size(); ++i) {
if (index.get(i) != TValue()) {
std::cout << i << " " << index.get(i) << "\n";
}
@ -51,9 +51,9 @@ class IndexSearch {
try {
TValue value = index.get(key);
std::cout << key << " " << value << std::endl;
std::cout << key << " " << value << "\n";
} catch (...) {
std::cout << key << " not found" << std::endl;
std::cout << key << " not found\n";
return false;
}
@ -69,7 +69,7 @@ class IndexSearch {
return lhs.first < rhs.first;
});
if (positions.first == positions.second) {
std::cout << key << " not found" << std::endl;
std::cout << key << " not found\n";
return false;
}
@ -103,7 +103,7 @@ public:
}
}
bool search(std::vector<TKey> keys) {
bool search(const std::vector<TKey>& keys) {
bool found_all = true;
for (const auto key : keys) {
@ -124,82 +124,105 @@ enum return_code : int {
fatal = 3
};
namespace po = boost::program_options;
class Options {
po::variables_map vm;
std::vector<osmium::unsigned_object_id_type> m_ids;
std::string m_type;
std::string m_filename;
bool m_dump = false;
bool m_array_format = false;
bool m_list_format = false;
void print_help() {
std::cout << "Usage: osmium_index [OPTIONS]\n\n"
<< "-h, --help Print this help message\n"
<< "-a, --array=FILE Read given index file in array format\n"
<< "-l, --list=FILE Read given index file in list format\n"
<< "-d, --dump Dump contents of index file to STDOUT\n"
<< "-s, --search=ID Search for given id (Option can appear multiple times)\n"
<< "-t, --type=TYPE Type of value ('location' or 'offset')\n"
;
}
public:
Options(int argc, char* argv[]) {
try {
po::options_description desc("Allowed options");
desc.add_options()
("help,h", "Print this help message")
("array,a", po::value<std::string>(), "Read given index file in array format")
("list,l", po::value<std::string>(), "Read given index file in list format")
("dump,d", "Dump contents of index file to STDOUT")
("search,s", po::value<std::vector<osmium::unsigned_object_id_type>>(), "Search for given id (Option can appear multiple times)")
("type,t", po::value<std::string>(), "Type of value ('location' or 'offset')")
;
static struct option long_options[] = {
{"array", required_argument, 0, 'a'},
{"dump", no_argument, 0, 'd'},
{"help", no_argument, 0, 'h'},
{"list", required_argument, 0, 'l'},
{"search", required_argument, 0, 's'},
{"type", required_argument, 0, 't'},
{0, 0, 0, 0}
};
po::store(po::parse_command_line(argc, argv, desc), vm);
po::notify(vm);
if (vm.count("help")) {
std::cout << desc << "\n";
exit(return_code::okay);
while (true) {
int c = getopt_long(argc, argv, "a:dhl:s:t:", long_options, 0);
if (c == -1) {
break;
}
if (vm.count("array") && vm.count("list")) {
std::cerr << "Only option --array or --list allowed." << std::endl;
exit(return_code::fatal);
switch (c) {
case 'a':
m_array_format = true;
m_filename = optarg;
break;
case 'd':
m_dump = true;
break;
case 'h':
print_help();
std::exit(return_code::okay);
case 'l':
m_list_format = true;
m_filename = optarg;
break;
case 's':
m_ids.push_back(std::atoll(optarg));
break;
case 't':
m_type = optarg;
if (m_type != "location" && m_type != "offset") {
std::cerr << "Unknown type '" << m_type << "'. Must be 'location' or 'offset'.\n";
std::exit(return_code::fatal);
}
if (!vm.count("array") && !vm.count("list")) {
std::cerr << "Need one of option --array or --list." << std::endl;
exit(return_code::fatal);
}
if (!vm.count("type")) {
std::cerr << "Need --type argument." << std::endl;
exit(return_code::fatal);
}
const std::string& type = vm["type"].as<std::string>();
if (type != "location" && type != "offset") {
std::cerr << "Unknown type '" << type << "'. Must be 'location' or 'offset'." << std::endl;
exit(return_code::fatal);
}
} catch (boost::program_options::error& e) {
std::cerr << "Error parsing command line: " << e.what() << std::endl;
exit(return_code::fatal);
break;
default:
std::exit(return_code::fatal);
}
}
const std::string& filename() const {
if (vm.count("array")) {
return vm["array"].as<std::string>();
} else {
return vm["list"].as<std::string>();
}
if (m_array_format == m_list_format) {
std::cerr << "Need option --array or --list, but not both\n";
std::exit(return_code::fatal);
}
bool dense_format() const {
return vm.count("array") != 0;
if (m_type.empty()) {
std::cerr << "Need --type argument.\n";
std::exit(return_code::fatal);
}
bool do_dump() const {
return vm.count("dump") != 0;
}
std::vector<osmium::unsigned_object_id_type> search_keys() const {
return vm["search"].as<std::vector<osmium::unsigned_object_id_type>>();
const std::string& filename() const noexcept {
return m_filename;
}
bool type_is(const char* type) const {
return vm["type"].as<std::string>() == type;
bool dense_format() const noexcept {
return m_array_format;
}
bool do_dump() const noexcept {
return m_dump;
}
const std::vector<osmium::unsigned_object_id_type>& search_keys() const noexcept {
return m_ids;
}
bool type_is(const char* type) const noexcept {
return m_type == type;
}
}; // class Options
@ -232,6 +255,6 @@ int main(int argc, char* argv[]) {
}
}
exit(result_okay ? return_code::okay : return_code::not_found);
std::exit(result_okay ? return_code::okay : return_code::not_found);
}

87
examples/osmium_location_cache_create.cpp Normal file
View File

@ -0,0 +1,87 @@
/*
EXAMPLE osmium_location_cache_create
Reads nodes from an OSM file and writes out their locations to a cache
file. The cache file can then be read with osmium_location_cache_use.
Warning: The locations cache file will get huge (>32GB) if you are using
the DenseFileArray index even if the input file is small, because
it depends on the *largest* node ID, not the number of nodes.
DEMONSTRATES USE OF:
* file input
* location indexes and the NodeLocationsForWays handler
* location indexes on disk
SIMPLER EXAMPLES you might want to understand first:
* osmium_read
* osmium_count
* osmium_road_length
LICENSE
The code in this example file is released into the Public Domain.
*/
#include <cerrno> // for errno
#include <cstdlib> // for std::exit
#include <cstring> // for strerror
#include <fcntl.h> // for open
#include <iostream> // for std::cout, std::cerr
#include <string> // for std::string
#include <sys/stat.h> // for open
#include <sys/types.h> // for open
// Allow any format of input files (XML, PBF, ...)
#include <osmium/io/any_input.hpp>
// For the location index. There are different types of index implementation
// available. These implementations put the index on disk. See below.
#include <osmium/index/map/sparse_file_array.hpp>
#include <osmium/index/map/dense_file_array.hpp>
// For the NodeLocationForWays handler
#include <osmium/handler/node_locations_for_ways.hpp>
// For osmium::apply()
#include <osmium/visitor.hpp>
// Chose one of these two. "sparse" is best used for small and medium extracts,
// the "dense" index for large extracts or the whole planet.
using index_type = osmium::index::map::SparseFileArray<osmium::unsigned_object_id_type, osmium::Location>;
//using index_type = osmium::index::map::DenseFileArray<osmium::unsigned_object_id_type, osmium::Location>;
// The location handler always depends on the index type
using location_handler_type = osmium::handler::NodeLocationsForWays<index_type>;
int main(int argc, char* argv[]) {
if (argc != 3) {
std::cerr << "Usage: " << argv[0] << " OSM_FILE CACHE_FILE\n";
std::exit(1);
}
const std::string input_filename{argv[1]};
const std::string cache_filename{argv[2]};
// Construct Reader reading only nodes
osmium::io::Reader reader{input_filename, osmium::osm_entity_bits::node};
// Initialize location index on disk creating a new file.
const int fd = open(cache_filename.c_str(), O_RDWR | O_CREAT, 0666);
if (fd == -1) {
std::cerr << "Can not open location cache file '" << cache_filename << "': " << std::strerror(errno) << "\n";
std::exit(1);
}
index_type index{fd};
// The handler that stores all node locations in the index.
location_handler_type location_handler{index};
// Feed all nodes through the location handler.
osmium::apply(reader, location_handler);
// Explicitly close input so we get notified of any errors.
reader.close();
}

101
examples/osmium_location_cache_use.cpp Normal file
View File

@ -0,0 +1,101 @@
/*
EXAMPLE osmium_location_cache_use
This reads ways from an OSM file and writes out the way node locations
it got from a location cache generated with osmium_location_cache_create.
Warning: The locations cache file will get huge (>32GB) if you are using
the DenseFileArray index even if the input file is small, because
it depends on the *largest* node ID, not the number of nodes.
DEMONSTRATES USE OF:
* file input
* location indexes and the NodeLocationsForWays handler
* location indexes on disk
SIMPLER EXAMPLES you might want to understand first:
* osmium_read
* osmium_count
* osmium_road_length
LICENSE
The code in this example file is released into the Public Domain.
*/
#include <cerrno> // for errno
#include <cstdlib> // for std::exit
#include <cstring> // for strerror
#include <fcntl.h> // for open
#include <iostream> // for std::cout, std::cerr
#include <string> // for std::string
#include <sys/stat.h> // for open
#include <sys/types.h> // for open
// Allow any format of input files (XML, PBF, ...)
#include <osmium/io/any_input.hpp>
// For the location index. There are different types of index implementation
// available. These implementations put the index on disk. See below.
#include <osmium/index/map/dense_file_array.hpp>
#include <osmium/index/map/sparse_file_array.hpp>
// For the NodeLocationForWays handler
#include <osmium/handler/node_locations_for_ways.hpp>
// For osmium::apply()
#include <osmium/visitor.hpp>
// Chose one of these two. "sparse" is best used for small and medium extracts,
// the "dense" index for large extracts or the whole planet.
using index_type = osmium::index::map::SparseFileArray<osmium::unsigned_object_id_type, osmium::Location>;
//using index_type = osmium::index::map::DenseFileArray<osmium::unsigned_object_id_type, osmium::Location>;
// The location handler always depends on the index type
using location_handler_type = osmium::handler::NodeLocationsForWays<index_type>;
// This handler only implements the way() function which prints out the way
// ID and all nodes IDs and locations in those ways.
struct MyHandler : public osmium::handler::Handler {
void way(const osmium::Way& way) {
std::cout << "way " << way.id() << "\n";
for (const auto& nr : way.nodes()) {
std::cout << " node " << nr.ref() << " " << nr.location() << "\n";
}
}
}; // struct MyHandler
int main(int argc, char* argv[]) {
if (argc != 3) {
std::cerr << "Usage: " << argv[0] << " OSM_FILE CACHE_FILE\n";
std::exit(1);
}
const std::string input_filename{argv[1]};
const std::string cache_filename{argv[2]};
// Construct Reader reading only ways
osmium::io::Reader reader{input_filename, osmium::osm_entity_bits::way};
// Initialize location index on disk using an existing file
const int fd = open(cache_filename.c_str(), O_RDWR);
if (fd == -1) {
std::cerr << "Can not open location cache file '" << cache_filename << "': " << std::strerror(errno) << "\n";
return 1;
}
index_type index{fd};
// The handler that adds node locations from the index to the ways.
location_handler_type location_handler{index};
// Feed all ways through the location handler and then our own handler.
MyHandler handler;
osmium::apply(reader, location_handler, handler);
// Explicitly close input so we get notified of any errors.
reader.close();
}

89
examples/osmium_pub_names.cpp Executable file
View File

@ -0,0 +1,89 @@
/*
EXAMPLE osmium_pub_names
Show the names and addresses of all pubs found in an OSM file.
DEMONSTRATES USE OF:
* file input
* your own handler
* access to tags
SIMPLER EXAMPLES you might want to understand first:
* osmium_read
* osmium_count
LICENSE
The code in this example file is released into the Public Domain.
*/
#include <cstdlib> // for std::exit
#include <cstring> // for std::strncmp
#include <iostream> // for std::cout, std::cerr
// Allow any format of input files (XML, PBF, ...)
#include <osmium/io/any_input.hpp>
// We want to use the handler interface
#include <osmium/handler.hpp>
// For osmium::apply()
#include <osmium/visitor.hpp>
class NamesHandler : public osmium::handler::Handler {
void output_pubs(const osmium::OSMObject& object) {
const osmium::TagList& tags = object.tags();
if (tags.has_tag("amenity", "pub")) {
// Print name of the pub if it is set.
const char* name = tags["name"];
if (name) {
std::cout << name << "\n";
} else {
std::cout << "pub with unknown name\n";
}
// Iterate over all tags finding those which start with "addr:"
// and print them.
for (const osmium::Tag& tag : tags) {
if (!std::strncmp(tag.key(), "addr:", 5)) {
std::cout << " " << tag.key() << ": " << tag.value() << "\n";
}
}
}
}
public:
// Nodes can be tagged amenity=pub.
void node(const osmium::Node& node) {
output_pubs(node);
}
// Ways can be tagged amenity=pub, too (typically buildings).
void way(const osmium::Way& way) {
output_pubs(way);
}
}; // class NamesHandler
int main(int argc, char* argv[]) {
if (argc != 2) {
std::cerr << "Usage: " << argv[0] << " OSMFILE\n";
std::exit(1);
}
// Construct the handler defined above
NamesHandler names_handler;
// Initialize the reader with the filename from the command line and
// tell it to only read nodes and ways. We are ignoring multipolygon
// relations in this simple example.
osmium::io::Reader reader{argv[1], osmium::osm_entity_bits::node | osmium::osm_entity_bits::way};
// Apply input data to our own handler
osmium::apply(reader, names_handler);
}

26
examples/osmium_read.cpp
View File

@ -1,30 +1,42 @@
/*
This is a small tool that reads and discards the contents of the input file.
(Used for timing.)
EXAMPLE osmium_read
Reads and discards the contents of the input file.
(It can be used for timing.)
DEMONSTRATES USE OF:
* file input
LICENSE
The code in this example file is released into the Public Domain.
*/
#include <iostream>
#include <cstdlib> // for std::exit
#include <iostream> // for std::cerr
// Allow any format of input files (XML, PBF, ...)
#include <osmium/io/any_input.hpp>
int main(int argc, char* argv[]) {
if (argc != 2) {
std::cerr << "Usage: " << argv[0] << " OSMFILE\n";
exit(1);
std::exit(1);
}
osmium::io::File infile(argv[1]);
osmium::io::Reader reader(infile);
// The Reader is initialized here with an osmium::io::File, but could
// also be directly initialized with a file name.
osmium::io::File input_file{argv[1]};
osmium::io::Reader reader{input_file};
// OSM data comes in buffers, read until there are no more.
while (osmium::memory::Buffer buffer = reader.read()) {
// do nothing
}
// You do not have to close the Reader explicitly, but because the
// destructor can't throw, you will not see any errors otherwise.
reader.close();
}

56
examples/osmium_read_with_progress.cpp Normal file
View File

@ -0,0 +1,56 @@
/*
EXAMPLE osmium_read_with_progress
Reads the contents of the input file showing a progress bar.
DEMONSTRATES USE OF:
* file input
* ProgressBar utility function
SIMPLER EXAMPLES you might want to understand first:
* osmium_read
LICENSE
The code in this example file is released into the Public Domain.
*/
#include <cstdlib> // for std::exit
#include <iostream> // for std::cerr
// Allow any format of input files (XML, PBF, ...)
#include <osmium/io/any_input.hpp>
// Get access to isatty utility function and progress bar utility class.
#include <osmium/util/file.hpp>
#include <osmium/util/progress_bar.hpp>
int main(int argc, char* argv[]) {
if (argc != 2) {
std::cerr << "Usage: " << argv[0] << " OSMFILE\n";
std::exit(1);
}
// The Reader is initialized here with an osmium::io::File, but could
// also be directly initialized with a file name.
osmium::io::File input_file{argv[1]};
osmium::io::Reader reader{input_file};
// Initialize progress bar, enable it only if STDERR is a TTY.
osmium::ProgressBar progress{reader.file_size(), osmium::util::isatty(2)};
// OSM data comes in buffers, read until there are no more.
while (osmium::memory::Buffer buffer = reader.read()) {
// Update progress bar for each buffer.
progress.update(reader.offset());
}
// Progress bar is done.
progress.done();
// You do not have to close the Reader explicitly, but because the
// destructor can't throw, you will not see any errors otherwise.
reader.close();
}

92
examples/osmium_road_length.cpp Executable file
View File

@ -0,0 +1,92 @@
/*
EXAMPLE osmium_road_length
Calculate the length of the road network (everything tagged `highway=*`)
from the given OSM file.
DEMONSTRATES USE OF:
* file input
* location indexes and the NodeLocationsForWays handler
* length calculation on the earth using the haversine function
SIMPLER EXAMPLES you might want to understand first:
* osmium_read
* osmium_count
* osmium_pub_names
LICENSE
The code in this example file is released into the Public Domain.
*/
#include <cstdlib> // for std::exit
#include <iostream> // for std::cout, std::cerr
// Allow any format of input files (XML, PBF, ...)
#include <osmium/io/any_input.hpp>
// For the osmium::geom::haversine::distance() function
#include <osmium/geom/haversine.hpp>
// For osmium::apply()
#include <osmium/visitor.hpp>
// For the location index. There are different types of indexes available.
// This will work for small and medium sized input files.
#include <osmium/index/map/sparse_mem_array.hpp>
// For the NodeLocationForWays handler
#include <osmium/handler/node_locations_for_ways.hpp>
// The type of index used. This must match the include file above
using index_type = osmium::index::map::SparseMemArray<osmium::unsigned_object_id_type, osmium::Location>;
// The location handler always depends on the index type
using location_handler_type = osmium::handler::NodeLocationsForWays<index_type>;
// This handler only implements the way() function, we are not interested in
// any other objects.
struct RoadLengthHandler : public osmium::handler::Handler {
double length = 0;
// If the way has a "highway" tag, find its length and add it to the
// overall length.
void way(const osmium::Way& way) {
const char* highway = way.tags()["highway"];
if (highway) {
length += osmium::geom::haversine::distance(way.nodes());
}
}
}; // struct RoadLengthHandler
int main(int argc, char* argv[]) {
if (argc != 2) {
std::cerr << "Usage: " << argv[0] << " OSMFILE\n";
std::exit(1);
}
// Initialize the reader with the filename from the command line and
// tell it to only read nodes and ways.
osmium::io::Reader reader{argv[1], osmium::osm_entity_bits::node | osmium::osm_entity_bits::way};
// The index to hold node locations.
index_type index;
// The location handler will add the node locations to the index and then
// to the ways
location_handler_type location_handler{index};
// Our handler defined above
RoadLengthHandler road_length_handler;
// Apply input data to first the location handler and then our own handler
osmium::apply(reader, location_handler, road_length_handler);
// Output the length. The haversine function calculates it in meters,
// so we first devide by 1000 to get kilometers.
std::cout << "Length: " << road_length_handler.length / 1000 << " km\n";
}

24
examples/osmium_serdump.cpp
View File

@ -69,9 +69,9 @@ int main(int argc, char* argv[]) {
switch (c) {
case 'h':
print_help();
exit(0);
std::exit(0);
default:
exit(2);
std::exit(2);
}
}
@ -79,7 +79,7 @@ int main(int argc, char* argv[]) {
if (remaining_args != 2) {
std::cerr << "Usage: " << argv[0] << " OSMFILE DIR\n";
exit(2);
std::exit(2);
}
std::string dir(argv[optind+1]);
@ -90,14 +90,14 @@ int main(int argc, char* argv[]) {
#endif
if (result == -1 && errno != EEXIST) {
std::cerr << "Problem creating directory '" << dir << "': " << strerror(errno) << "\n";
exit(2);
std::exit(2);
}
std::string data_file(dir + "/data.osm.ser");
int data_fd = ::open(data_file.c_str(), O_WRONLY | O_CREAT | O_TRUNC, 0666);
if (data_fd < 0) {
std::cerr << "Can't open data file '" << data_file << "': " << strerror(errno) << "\n";
exit(2);
std::exit(2);
}
offset_index_type node_index;
@ -127,7 +127,7 @@ int main(int argc, char* argv[]) {
int fd = ::open(index_file.c_str(), O_WRONLY | O_CREAT | O_TRUNC, 0666);
if (fd < 0) {
std::cerr << "Can't open nodes index file '" << index_file << "': " << strerror(errno) << "\n";
exit(2);
std::exit(2);
}
node_index.dump_as_list(fd);
close(fd);
@ -138,7 +138,7 @@ int main(int argc, char* argv[]) {
int fd = ::open(index_file.c_str(), O_WRONLY | O_CREAT | O_TRUNC, 0666);
if (fd < 0) {
std::cerr << "Can't open ways index file '" << index_file << "': " << strerror(errno) << "\n";
exit(2);
std::exit(2);
}
way_index.dump_as_list(fd);
close(fd);
@ -149,7 +149,7 @@ int main(int argc, char* argv[]) {
int fd = ::open(index_file.c_str(), O_WRONLY | O_CREAT | O_TRUNC, 0666);
if (fd < 0) {
std::cerr << "Can't open relations index file '" << index_file << "': " << strerror(errno) << "\n";
exit(2);
std::exit(2);
}
relation_index.dump_as_list(fd);
close(fd);
@ -161,7 +161,7 @@ int main(int argc, char* argv[]) {
int fd = ::open(index_file.c_str(), O_WRONLY | O_CREAT | O_TRUNC, 0666);
if (fd < 0) {
std::cerr << "Can't open node->way map file '" << index_file << "': " << strerror(errno) << "\n";
exit(2);
std::exit(2);
}
map_node2way.dump_as_list(fd);
close(fd);
@ -173,7 +173,7 @@ int main(int argc, char* argv[]) {
int fd = ::open(index_file.c_str(), O_WRONLY | O_CREAT | O_TRUNC, 0666);
if (fd < 0) {
std::cerr << "Can't open node->rel map file '" << index_file << "': " << strerror(errno) << "\n";
exit(2);
std::exit(2);
}
map_node2relation.dump_as_list(fd);
close(fd);
@ -185,7 +185,7 @@ int main(int argc, char* argv[]) {
int fd = ::open(index_file.c_str(), O_WRONLY | O_CREAT | O_TRUNC, 0666);
if (fd < 0) {
std::cerr << "Can't open way->rel map file '" << index_file << "': " << strerror(errno) << "\n";
exit(2);
std::exit(2);
}
map_way2relation.dump_as_list(fd);
close(fd);
@ -197,7 +197,7 @@ int main(int argc, char* argv[]) {
int fd = ::open(index_file.c_str(), O_WRONLY | O_CREAT | O_TRUNC, 0666);
if (fd < 0) {
std::cerr << "Can't open rel->rel map file '" << index_file << "': " << strerror(errno) << "\n";
exit(2);
std::exit(2);
}
map_relation2relation.dump_as_list(fd);
close(fd);

72
examples/osmium_tiles.cpp Normal file
View File

@ -0,0 +1,72 @@
/*
EXAMPLE osmium_tiles
Convert WGS84 longitude and latitude to Mercator coordinates and tile
coordinates.
DEMONSTRATES USE OF:
* the Location and Coordinates classes
* the Mercator projection function
* the Tile class
LICENSE
The code in this example file is released into the Public Domain.
*/
#include <cstdlib> // for std::exit, std::atoi, std::atof
#include <iostream> // for std::cout, std::cerr
// The Location contains a longitude and latitude and is usually used inside
// a node to store its location in the world.
#include <osmium/osm/location.hpp>
// Needed for the Mercator projection function. Osmium supports the Mercator
// projection out of the box, or pretty much any projection using the Proj.4
// library (with the osmium::geom::Projection class).
#include <osmium/geom/mercator_projection.hpp>
// The Tile class handles tile coordinates and zoom levels.
#include <osmium/geom/tile.hpp>
int main(int argc, char* argv[]) {
if (argc != 4) {
std::cerr << "Usage: " << argv[0] << " ZOOM LON LAT\n";
std::exit(1);
}
const int zoom = std::atoi(argv[1]);
if (zoom < 0 || zoom > 30) {
std::cerr << "ERROR: Zoom must be between 0 and 30\n";
std::exit(1);
}
const double lon = std::atof(argv[2]);
const double lat = std::atof(argv[3]);
// Create location from WGS84 coordinates. In Osmium the order of
// coordinate values is always x/longitude first, then y/latitude.
const osmium::Location location{lon, lat};
std::cout << "WGS84: lon=" << lon << " lat=" << lat << "\n";
// A location can store some invalid locations, ie locations outside the
// -180 to 180 and -90 to 90 degree range. This function checks for that.
if (!location.valid()) {
std::cerr << "ERROR: Location is invalid\n";
std::exit(1);
}
// Project the coordinates using a helper function. You can also use the
// osmium::geom::MercatorProjection class.
const osmium::geom::Coordinates c = osmium::geom::lonlat_to_mercator(location);
std::cout << "Mercator: x=" << c.x << " y=" << c.y << "\n";
// Create a tile at this location. This will also internally use the
// Mercator projection and then calculate the tile coordinates.
const osmium::geom::Tile tile{uint32_t(zoom), location};
std::cout << "Tile: zoom=" << tile.z << " x=" << tile.x << " y=" << tile.y << "\n";
}

68
examples/osmium_use_node_cache.cpp
View File

@ -1,68 +0,0 @@
/*
This reads ways from an OSM file and writes out the node locations
it got from a node cache generated with osmium_create_node_cache.
The code in this example file is released into the Public Domain.
*/
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <iostream>
#include <osmium/io/any_input.hpp>
#include <osmium/index/map/dummy.hpp>
#include <osmium/index/map/dense_file_array.hpp>
#include <osmium/index/map/dense_mmap_array.hpp>
#include <osmium/handler/node_locations_for_ways.hpp>
#include <osmium/visitor.hpp>
typedef osmium::index::map::Dummy<osmium::unsigned_object_id_type, osmium::Location> index_neg_type;
//typedef osmium::index::map::DenseMmapArray<osmium::unsigned_object_id_type, osmium::Location> index_pos_type;
typedef osmium::index::map::DenseFileArray<osmium::unsigned_object_id_type, osmium::Location> index_pos_type;
typedef osmium::handler::NodeLocationsForWays<index_pos_type, index_neg_type> location_handler_type;
class MyHandler : public osmium::handler::Handler {
public:
void way(osmium::Way& way) {
for (auto& nr : way.nodes()) {
std::cout << nr << "\n";
}
}
}; // class MyHandler
int main(int argc, char* argv[]) {
if (argc != 3) {
std::cerr << "Usage: " << argv[0] << " OSM_FILE CACHE_FILE\n";
return 1;
}
std::string input_filename(argv[1]);
osmium::io::Reader reader(input_filename, osmium::osm_entity_bits::way);
int fd = open(argv[2], O_RDWR);
if (fd == -1) {
std::cerr << "Can not open node cache file '" << argv[2] << "': " << strerror(errno) << "\n";
return 1;
}
index_pos_type index_pos {fd};
index_neg_type index_neg;
location_handler_type location_handler(index_pos, index_neg);
location_handler.ignore_errors();
MyHandler handler;
osmium::apply(reader, location_handler, handler);
reader.close();
return 0;
}

91
include/gdalcpp.hpp
View File

@ -110,8 +110,12 @@ namespace gdalcpp {
namespace detail {
struct init_wrapper {
#if GDAL_VERSION_MAJOR >= 2
init_wrapper() { GDALAllRegister(); }
#else
init_wrapper() { OGRRegisterAll(); }
~init_wrapper() { OGRCleanupAll(); }
#endif
};
struct init_library {
@ -237,6 +241,8 @@ namespace gdalcpp {
detail::Options m_options;
SRS m_srs;
std::unique_ptr<gdal_dataset_type, gdal_dataset_deleter> m_dataset;
uint64_t m_edit_count = 0;
uint64_t m_max_edit_count = 0;
public:
@ -255,6 +261,15 @@ namespace gdalcpp {
}
}
~Dataset() {
try {
if (m_edit_count > 0) {
commit_transaction();
}
} catch (...) {
}
}
const std::string& driver_name() const {
return m_driver_name;
}
@ -282,10 +297,14 @@ namespace gdalcpp {
exec(sql.c_str());
}
Dataset& start_transaction() {
#if GDAL_VERSION_MAJOR >= 2
m_dataset->StartTransaction();
#else
OGRLayer* layer = m_dataset->GetLayer(0);
if (layer) {
layer->StartTransaction();
}
#endif
return *this;
}
@ -293,7 +312,38 @@ namespace gdalcpp {
Dataset& commit_transaction() {
#if GDAL_VERSION_MAJOR >= 2
m_dataset->CommitTransaction();
#else
OGRLayer* layer = m_dataset->GetLayer(0);
if (layer) {
layer->CommitTransaction();
}
#endif
m_edit_count = 0;
return *this;
}
void prepare_edit() {
if (m_max_edit_count != 0 && m_edit_count == 0) {
start_transaction();
}
}
void finalize_edit() {
if (m_max_edit_count != 0 && ++m_edit_count > m_max_edit_count) {
commit_transaction();
}
}
Dataset& enable_auto_transactions(uint64_t edits = 100000) {
m_max_edit_count = edits;
return *this;
}
Dataset& disable_auto_transactions() {
if (m_max_edit_count != 0 && m_edit_count > 0) {
commit_transaction();
}
m_max_edit_count = 0;
return *this;
}
@ -346,19 +396,32 @@ namespace gdalcpp {
return *this;
}
void create_feature(OGRFeature* feature) {
dataset().prepare_edit();
OGRErr result = m_layer->CreateFeature(feature);
if (result != OGRERR_NONE) {
throw gdal_error(std::string("creating feature in layer '") + name() + "' failed", result, dataset().driver_name(), dataset().dataset_name());
}
dataset().finalize_edit();
}
Layer& start_transaction() {
#if GDAL_VERSION_MAJOR < 2
OGRErr result = m_layer->StartTransaction();
if (result != OGRERR_NONE) {
throw gdal_error(std::string("starting transaction on layer '") + name() + "' failed", result, m_dataset.driver_name(), m_dataset.dataset_name(), name());
}
#endif
return *this;
}
Layer& commit_transaction() {
#if GDAL_VERSION_MAJOR < 2
OGRErr result = m_layer->CommitTransaction();
if (result != OGRERR_NONE) {
throw gdal_error(std::string("committing transaction on layer '") + name() + "' failed", result, m_dataset.driver_name(), m_dataset.dataset_name(), name());
}
#endif
return *this;
}
@ -366,36 +429,44 @@ namespace gdalcpp {
class Feature {
struct ogr_feature_deleter {
void operator()(OGRFeature* feature) {
OGRFeature::DestroyFeature(feature);
}
}; // struct ogr_feature_deleter
Layer& m_layer;
OGRFeature m_feature;
std::unique_ptr<OGRFeature, ogr_feature_deleter> m_feature;
public:
Feature(Layer& layer, std::unique_ptr<OGRGeometry>&& geometry) :
m_layer(layer),
m_feature(m_layer.get().GetLayerDefn()) {
OGRErr result = m_feature.SetGeometryDirectly(geometry.release());
m_feature(OGRFeature::CreateFeature(m_layer.get().GetLayerDefn())) {
if (!m_feature) {
throw std::bad_alloc();
}
OGRErr result = m_feature->SetGeometryDirectly(geometry.release());
if (result != OGRERR_NONE) {
throw gdal_error(std::string("setting feature geometry in layer '") + m_layer.name() + "' failed", result, m_layer.dataset().driver_name(), m_layer.dataset().dataset_name());
}
}
void add_to_layer() {
OGRErr result = m_layer.get().CreateFeature(&m_feature);
if (result != OGRERR_NONE) {
throw gdal_error(std::string("creating feature in layer '") + m_layer.name() + "' failed", result, m_layer.dataset().driver_name(), m_layer.dataset().dataset_name());
}
m_layer.create_feature(m_feature.get());
}
template <class T>
Feature& set_field(int n, T&& arg) {
m_feature.SetField(n, std::forward<T>(arg));
m_feature->SetField(n, std::forward<T>(arg));
return *this;
}
template <class T>
Feature& set_field(const char* name, T&& arg) {
m_feature.SetField(name, std::forward<T>(arg));
m_feature->SetField(name, std::forward<T>(arg));
return *this;
}

1760
include/osmium/area/assembler.hpp
View File

File diff suppressed because it is too large Load Diff

314
include/osmium/area/detail/node_ref_segment.hpp
View File

@ -34,11 +34,12 @@ DEALINGS IN THE SOFTWARE.
*/
#include <algorithm>
#include <cassert>
#include <cstdint>
#include <cstring>
#include <iosfwd>
#include <utility>
#include <osmium/area/detail/vector.hpp>
#include <osmium/osm/location.hpp>
#include <osmium/osm/node_ref.hpp>
@ -53,108 +54,178 @@ namespace osmium {
*/
namespace detail {
class ProtoRing;
enum class role_type : uint8_t {
unknown = 0,
outer = 1,
inner = 2,
empty = 3
};
/**
* This helper class for the Assembler class models a segment.
* Segments are the connection between
* two nodes and they all have their smaller coordinate at the
* beginning of the segment. Smaller, in this case, means smaller x
* coordinate, and if they are the same smaller y coordinate.
* This helper class for the Assembler class models a segment,
* the connection between two nodes.
*
* Internally segments have their smaller coordinate at the
* beginning of the segment. Smaller, in this case, means smaller
* x coordinate, and, if they are the same, smaller y coordinate.
*/
class NodeRefSegment {
// First node in order described above.
osmium::NodeRef m_first;
// Second node in order described above.
osmium::NodeRef m_second;
/// Role of the member this segment was from.
const char* m_role;
/// Way this segment was from.
// Way this segment was from.
const osmium::Way* m_way;
// The ring this segment is part of. Initially nullptr, this
// will be filled in once we know which ring the segment is in.
ProtoRing* m_ring;
// The role of this segment from the member role.
role_type m_role;
// Nodes have to be reversed to get the intended order.
bool m_reverse = false;
// We found the right direction for this segment in the ring.
// (This depends on whether it is an inner or outer ring.)
bool m_direction_done = false;
public:
void swap_locations() {
NodeRefSegment() noexcept :
m_first(),
m_second(),
m_way(nullptr),
m_ring(nullptr),
m_role(role_type::unknown) {
}
NodeRefSegment(const osmium::NodeRef& nr1, const osmium::NodeRef& nr2, role_type role = role_type::unknown, const osmium::Way* way = nullptr) noexcept :
m_first(nr1),
m_second(nr2),
m_way(way),
m_ring(nullptr),
m_role(role) {
if (nr2.location() < nr1.location()) {
using std::swap;
swap(m_first, m_second);
}
explicit NodeRefSegment() noexcept :
m_first(),
m_second(),
m_role(nullptr),
m_way(nullptr) {
}
explicit NodeRefSegment(const osmium::NodeRef& nr1, const osmium::NodeRef& nr2, const char* role, const osmium::Way* way) :
m_first(nr1),
m_second(nr2),
m_role(role),
m_way(way) {
if (nr2.location() < nr1.location()) {
swap_locations();
}
/**
* The ring this segment is a part of. nullptr if we don't
* have the ring yet.
*/
ProtoRing* ring() const noexcept {
return m_ring;
}
NodeRefSegment(const NodeRefSegment&) = default;
NodeRefSegment(NodeRefSegment&&) = default;
/**
* Returns true if the segment has already been placed in a
* ring.
*/
bool is_done() const noexcept {
return m_ring != nullptr;
}
NodeRefSegment& operator=(const NodeRefSegment&) = default;
NodeRefSegment& operator=(NodeRefSegment&&) = default;
void set_ring(ProtoRing* ring) noexcept {
assert(ring);
m_ring = ring;
}
~NodeRefSegment() = default;
bool is_reverse() const noexcept {
return m_reverse;
}
/// Return first NodeRef of Segment according to sorting order (bottom left to top right).
void reverse() noexcept {
m_reverse = !m_reverse;
}
bool is_direction_done() const noexcept {
return m_direction_done;
}
void mark_direction_done() noexcept {
m_direction_done = true;
}
void mark_direction_not_done() noexcept {
m_direction_done = false;
}
/**
* Return first NodeRef of Segment according to sorting
* order (bottom left to top right).
*/
const osmium::NodeRef& first() const noexcept {
return m_first;
}
/// Return second NodeRef of Segment according to sorting order (bottom left to top right).
/**
* Return second NodeRef of Segment according to sorting
* order (bottom left to top right).
*/
const osmium::NodeRef& second() const noexcept {
return m_second;
}
bool to_left_of(const osmium::Location& location) const {
// std::cerr << "segment " << first() << "--" << second() << " to_left_of(" << location << "\n";
if (first().location() == location || second().location() == location) {
return false;
/**
* Return real first NodeRef of Segment.
*/
const osmium::NodeRef& start() const noexcept {
return m_reverse ? m_second : m_first;
}
const std::pair<osmium::Location, osmium::Location> mm = std::minmax(first().location(), second().location(), [](const osmium::Location a, const osmium::Location b) {
return a.y() < b.y();
});
if (mm.first.y() >= location.y() || mm.second.y() < location.y() || first().location().x() > location.x()) {
// std::cerr << " false\n";
return false;
}
int64_t ax = mm.first.x();
int64_t bx = mm.second.x();
int64_t lx = location.x();
int64_t ay = mm.first.y();
int64_t by = mm.second.y();
int64_t ly = location.y();
return ((bx - ax)*(ly - ay) - (by - ay)*(lx - ax)) <= 0;
/**
* Return real second NodeRef of Segment.
*/
const osmium::NodeRef& stop() const noexcept {
return m_reverse ? m_first : m_second;
}
bool role_outer() const noexcept {
return !strcmp(m_role, "outer");
return m_role == role_type::outer;
}
bool role_inner() const noexcept {
return !strcmp(m_role, "inner");
return m_role == role_type::inner;
}
bool role_empty() const noexcept {
return m_role == role_type::empty;
}
const char* role_name() const noexcept {
static const char* names[] = { "unknown", "outer", "inner", "empty" };
return names[int(m_role)];
}
const osmium::Way* way() const noexcept {
return m_way;
}
/**
* The "determinant" of this segment. Used for calculating
* the winding order or a ring.
*/
int64_t det() const noexcept {
const vec a{start()};
const vec b{stop()};
return a * b;
}
}; // class NodeRefSegment
/// NodeRefSegments are equal if both their locations are equal
inline bool operator==(const NodeRefSegment& lhs, const NodeRefSegment& rhs) noexcept {
return lhs.first().location() == rhs.first().location() && lhs.second().location() == rhs.second().location();
return lhs.first().location() == rhs.first().location() &&
lhs.second().location() == rhs.second().location();
}
inline bool operator!=(const NodeRefSegment& lhs, const NodeRefSegment& rhs) noexcept {
@ -162,12 +233,33 @@ namespace osmium {
}
/**
* NodeRefSegments are "smaller" if they are to the left and down of another
* segment. The first() location is checked first() and only if they have the
* same first() location the second() location is taken into account.
* A NodeRefSegment is "smaller" if the first point is to the
* left and down of the first point of the second segment.
* If both first points are the same, the segment with the higher
* slope comes first. If the slope is the same, the shorter
* segment comes first.
*/
inline bool operator<(const NodeRefSegment& lhs, const NodeRefSegment& rhs) noexcept {
return (lhs.first().location() == rhs.first().location() && lhs.second().location() < rhs.second().location()) || lhs.first().location() < rhs.first().location();
if (lhs.first().location() == rhs.first().location()) {
const vec p0{lhs.first().location()};
const vec p1{lhs.second().location()};
const vec q0{rhs.first().location()};
const vec q1{rhs.second().location()};
const vec p = p1 - p0;
const vec q = q1 - q0;
if (p.x == 0 && q.x == 0) {
return p.y < q.y;
}
const auto a = p.y * q.x;
const auto b = q.y * p.x;
if (a == b) {
return p.x < q.x;
}
return a > b;
}
return lhs.first().location() < rhs.first().location();
}
inline bool operator>(const NodeRefSegment& lhs, const NodeRefSegment& rhs) noexcept {
@ -184,7 +276,10 @@ namespace osmium {
template <typename TChar, typename TTraits>
inline std::basic_ostream<TChar, TTraits>& operator<<(std::basic_ostream<TChar, TTraits>& out, const NodeRefSegment& segment) {
return out << segment.first() << "--" << segment.second();
return out << segment.start() << "--" << segment.stop()
<< "[" << (segment.is_reverse() ? 'R' : '_')
<< (segment.is_done() ? 'd' : '_')
<< (segment.is_direction_done() ? 'D' : '_') << "]";
}
inline bool outside_x_range(const NodeRefSegment& s1, const NodeRefSegment& s2) noexcept {
@ -194,7 +289,7 @@ namespace osmium {
return false;
}
inline bool y_range_overlap(const NodeRefSegment& s1, const NodeRefSegment& s2) {
inline bool y_range_overlap(const NodeRefSegment& s1, const NodeRefSegment& s2) noexcept {
const std::pair<int32_t, int32_t> m1 = std::minmax(s1.first().location().y(), s1.second().location().y());
const std::pair<int32_t, int32_t> m2 = std::minmax(s2.first().location().y(), s2.second().location().y());
if (m1.first > m2.second || m2.first > m1.second) {
@ -210,55 +305,94 @@ namespace osmium {
* might be slightly different than the numerically correct
* location.
*
* This function uses integer arithmentic as much as possible and
* This function uses integer arithmetic as much as possible and
* will not work if the segments are longer than about half the
* planet. This shouldn't happen with real data, so it isn't a big
* problem.
*
* If the segments touch in one of their endpoints, it doesn't
* count as an intersection.
* If the segments touch in one or both of their endpoints, it
* doesn't count as an intersection.
*
* If the segments intersect not in a single point but in multiple
* points, ie if they overlap, this is NOT detected.
* points, ie if they are collinear and overlap, the smallest
* of the endpoints that is in the overlapping section is returned.
*
* @returns Undefined osmium::Location if there is no intersection
* or a defined Location if the segments intersect.
*/
inline osmium::Location calculate_intersection(const NodeRefSegment& s1, const NodeRefSegment& s2) {
if (s1.first().location() == s2.first().location() ||
s1.first().location() == s2.second().location() ||
s1.second().location() == s2.first().location() ||
s1.second().location() == s2.second().location()) {
inline osmium::Location calculate_intersection(const NodeRefSegment& s1, const NodeRefSegment& s2) noexcept {
// See http://stackoverflow.com/questions/563198/how-do-you-detect-where-two-line-segments-intersect
// for some hints about how the algorithm works.
const vec p0{s1.first()};
const vec p1{s1.second()};
const vec q0{s2.first()};
const vec q1{s2.second()};
if ((p0 == q0 && p1 == q1) ||
(p0 == q1 && p1 == q0)) {
// segments are the same
return osmium::Location();
}
int64_t s1ax = s1.first().x();
int64_t s1ay = s1.first().y();
int64_t s1bx = s1.second().x();
int64_t s1by = s1.second().y();
int64_t s2ax = s2.first().x();
int64_t s2ay = s2.first().y();
int64_t s2bx = s2.second().x();
int64_t s2by = s2.second().y();
int64_t d = (s2by - s2ay) * (s1bx - s1ax) -
(s2bx - s2ax) * (s1by - s1ay);
const vec pd = p1 - p0;
const int64_t d = pd * (q1 - q0);
if (d != 0) {
int64_t na = (s2bx - s2ax) * (s1ay - s2ay) -
(s2by - s2ay) * (s1ax - s2ax);
// segments are not collinear
int64_t nb = (s1bx - s1ax) * (s1ay - s2ay) -
(s1by - s1ay) * (s1ax - s2ax);
if (p0 == q0 || p0 == q1 || p1 == q0 || p1 == q1) {
// touching at an end point
return osmium::Location();
}
// intersection in a point
const int64_t na = (q1.x - q0.x) * (p0.y - q0.y) -
(q1.y - q0.y) * (p0.x - q0.x);
const int64_t nb = (p1.x - p0.x) * (p0.y - q0.y) -
(p1.y - p0.y) * (p0.x - q0.x);
if ((d > 0 && na >= 0 && na <= d && nb >= 0 && nb <= d) ||
(d < 0 && na <= 0 && na >= d && nb <= 0 && nb >= d)) {
const double ua = double(na) / d;
const vec i = p0 + ua * (p1 - p0);
return osmium::Location(int32_t(i.x), int32_t(i.y));
}
double ua = double(na) / d;
int32_t ix = int32_t(s1ax + ua*(s1bx - s1ax));
int32_t iy = int32_t(s1ay + ua*(s1by - s1ay));
return osmium::Location();
}
return osmium::Location(ix, iy);
// segments are collinear
if (pd * (q0 - p0) == 0) {
// segments are on the same line
struct seg_loc {
int segment;
osmium::Location location;
};
seg_loc sl[4];
sl[0] = {0, s1.first().location() };
sl[1] = {0, s1.second().location()};
sl[2] = {1, s2.first().location() };
sl[3] = {1, s2.second().location()};
std::sort(sl, sl+4, [](const seg_loc& a, const seg_loc& b) {
return a.location < b.location;
});
if (sl[1].location == sl[2].location) {
return osmium::Location();
}
if (sl[0].segment != sl[1].segment) {
if (sl[0].location == sl[1].location) {
return sl[2].location;
} else {
return sl[1].location;
}
}
}

254
include/osmium/area/detail/proto_ring.hpp
View File

@ -34,10 +34,9 @@ DEALINGS IN THE SOFTWARE.
*/
#include <algorithm>
#include <cassert>
#include <cstdint>
#include <cstdlib>
#include <iostream>
#include <iterator>
#include <set>
#include <vector>
@ -47,6 +46,8 @@ DEALINGS IN THE SOFTWARE.
namespace osmium {
class Way;
namespace area {
namespace detail {
@ -58,214 +59,155 @@ namespace osmium {
public:
typedef std::vector<NodeRefSegment> segments_type;
using segments_type = std::vector<NodeRefSegment*>;
private:
// segments in this ring
// Segments in this ring.
segments_type m_segments;
bool m_outer {true};
// if this is an outer ring, these point to it's inner rings (if any)
// If this is an outer ring, these point to it's inner rings
// (if any).
std::vector<ProtoRing*> m_inner;
// The smallest segment. Will be kept current whenever a new
// segment is added to the ring.
NodeRefSegment* m_min_segment;
// If this is an inner ring, points to the outer ring.
ProtoRing* m_outer_ring;
int64_t m_sum;
public:
explicit ProtoRing(const NodeRefSegment& segment) noexcept :
m_segments() {
explicit ProtoRing(NodeRefSegment* segment) noexcept :
m_segments(),
m_inner(),
m_min_segment(segment),
m_outer_ring(nullptr),
m_sum(0) {
add_segment_back(segment);
}
explicit ProtoRing(segments_type::const_iterator sbegin, segments_type::const_iterator send) :
m_segments(static_cast<size_t>(std::distance(sbegin, send))) {
std::copy(sbegin, send, m_segments.begin());
void add_segment_back(NodeRefSegment* segment) {
assert(segment);
if (*segment < *m_min_segment) {
m_min_segment = segment;
}
m_segments.push_back(segment);
segment->set_ring(this);
m_sum += segment->det();
}
bool outer() const noexcept {
return m_outer;
NodeRefSegment* min_segment() const noexcept {
return m_min_segment;
}
void set_inner() noexcept {
m_outer = false;
ProtoRing* outer_ring() const noexcept {
return m_outer_ring;
}
segments_type& segments() noexcept {
return m_segments;
void set_outer_ring(ProtoRing* outer_ring) noexcept {
assert(outer_ring);
assert(m_inner.empty());
m_outer_ring = outer_ring;
}
const std::vector<ProtoRing*>& inner_rings() const noexcept {
return m_inner;
}
void add_inner_ring(ProtoRing* ring) {
assert(ring);
assert(!m_outer_ring);
m_inner.push_back(ring);
}
bool is_outer() const noexcept {
return !m_outer_ring;
}
const segments_type& segments() const noexcept {
return m_segments;
}
void remove_segments(segments_type::iterator sbegin, segments_type::iterator send) {
m_segments.erase(sbegin, send);
const NodeRef& get_node_ref_start() const noexcept {
return m_segments.front()->start();
}
void add_segment_front(const NodeRefSegment& segment) {
m_segments.insert(m_segments.begin(), segment);
const NodeRef& get_node_ref_stop() const noexcept {
return m_segments.back()->stop();
}
void add_segment_back(const NodeRefSegment& segment) {
m_segments.push_back(segment);
bool closed() const noexcept {
return get_node_ref_start().location() == get_node_ref_stop().location();
}
const NodeRefSegment& get_segment_front() const {
return m_segments.front();
void reverse() {
std::for_each(m_segments.begin(), m_segments.end(), [](NodeRefSegment* segment) {
segment->reverse();
});
std::reverse(m_segments.begin(), m_segments.end());
m_sum = -m_sum;
}
NodeRefSegment& get_segment_front() {
return m_segments.front();
void mark_direction_done() {
std::for_each(m_segments.begin(), m_segments.end(), [](NodeRefSegment* segment) {
segment->mark_direction_done();
});
}
const NodeRef& get_node_ref_front() const {
return get_segment_front().first();
bool is_cw() const noexcept {
return m_sum <= 0;
}
const NodeRefSegment& get_segment_back() const {
return m_segments.back();
int64_t sum() const noexcept {
return m_sum;
}
NodeRefSegment& get_segment_back() {
return m_segments.back();
void fix_direction() noexcept {
if (is_cw() == is_outer()) {
reverse();
}
}
const NodeRef& get_node_ref_back() const {
return get_segment_back().second();
void reset() {
m_inner.clear();
m_outer_ring = nullptr;
std::for_each(m_segments.begin(), m_segments.end(), [](NodeRefSegment* segment) {
segment->mark_direction_not_done();
});
}
bool closed() const {
return m_segments.front().first().location() == m_segments.back().second().location();
}
int64_t sum() const {
int64_t sum = 0;
void get_ways(std::set<const osmium::Way*>& ways) const {
for (const auto& segment : m_segments) {
sum += static_cast<int64_t>(segment.first().location().x()) * static_cast<int64_t>(segment.second().location().y()) -
static_cast<int64_t>(segment.second().location().x()) * static_cast<int64_t>(segment.first().location().y());
ways.insert(segment->way());
}
}
return sum;
void join_forward(ProtoRing& other) {
for (NodeRefSegment* segment : other.m_segments) {
add_segment_back(segment);
}
}
bool is_cw() const {
return sum() <= 0;
void join_backward(ProtoRing& other) {
for (auto it = other.m_segments.rbegin(); it != other.m_segments.rend(); ++it) {
(*it)->reverse();
add_segment_back(*it);
}
int64_t area() const {
return std::abs(sum()) / 2;
}
void swap_segments(ProtoRing& other) {
using std::swap;
swap(m_segments, other.m_segments);
}
void add_inner_ring(ProtoRing* ring) {
m_inner.push_back(ring);
}
const std::vector<ProtoRing*>& inner_rings() const {
return m_inner;
}
void print(std::ostream& out) const {
out << "[";
bool first = true;
if (!m_segments.empty()) {
out << m_segments.front()->start().ref();
}
for (const auto& segment : m_segments) {
if (first) {
out << segment.first().ref();
out << ',' << segment->stop().ref();
}
out << ',' << segment.second().ref();
first = false;
}
out << "]";
}
void reverse() {
std::for_each(m_segments.begin(), m_segments.end(), [](NodeRefSegment& segment) {
segment.swap_locations();
});
std::reverse(m_segments.begin(), m_segments.end());
}
/**
* Merge other ring to end of this ring.
*/
void merge_ring(const ProtoRing& other, bool debug) {
if (debug) {
std::cerr << " MERGE rings ";
print(std::cerr);
std::cerr << " to ";
other.print(std::cerr);
std::cerr << "\n";
}
m_segments.insert(m_segments.end(), other.m_segments.begin(), other.m_segments.end());
if (debug) {
std::cerr << " result ring: ";
print(std::cerr);
std::cerr << "\n";
}
}
void merge_ring_reverse(const ProtoRing& other, bool debug) {
if (debug) {
std::cerr << " MERGE rings (reverse) ";
print(std::cerr);
std::cerr << " to ";
other.print(std::cerr);
std::cerr << "\n";
}
size_t n = m_segments.size();
m_segments.resize(n + other.m_segments.size());
std::transform(other.m_segments.rbegin(), other.m_segments.rend(), m_segments.begin() + static_cast<segments_type::difference_type>(n), [](NodeRefSegment segment) {
segment.swap_locations();
return segment;
});
if (debug) {
std::cerr << " result ring: ";
print(std::cerr);
std::cerr << "\n";
}
}
const NodeRef& min_node() const {
auto it = std::min_element(m_segments.begin(), m_segments.end());
if (location_less()(it->first(), it->second())) {
return it->first();
} else {
return it->second();
}
}
bool is_in(ProtoRing* outer) {
osmium::Location testpoint = segments().front().first().location();
bool is_in = false;
for (size_t i = 0, j = outer->segments().size()-1; i < outer->segments().size(); j = i++) {
if (((outer->segments()[i].first().location().y() > testpoint.y()) != (outer->segments()[j].first().location().y() > testpoint.y())) &&
(testpoint.x() < (outer->segments()[j].first().location().x() - outer->segments()[i].first().location().x()) * (testpoint.y() - outer->segments()[i].first().location().y()) / (outer->segments()[j].first().location().y() - outer->segments()[i].first().location().y()) + outer->segments()[i].first().location().x()) ) {
is_in = !is_in;
}
}
return is_in;
}
void get_ways(std::set<const osmium::Way*>& ways) {
for (const auto& segment : m_segments) {
ways.insert(segment.way());
}
}
bool contains(const NodeRefSegment& segment) const {
for (const auto& s : m_segments) {
if (s == segment || (s.first() == segment.second() && s.second() == segment.first())) {
return true;
}
}
return false;
out << "]-" << (is_outer() ? "OUTER" : "INNER");
}
}; // class ProtoRing

173
include/osmium/area/detail/segment_list.hpp
View File

@ -35,12 +35,16 @@ DEALINGS IN THE SOFTWARE.
#include <algorithm>
#include <cassert>
#include <cstdint>
#include <cstring>
#include <iostream>
#include <iterator>
#include <numeric>
#include <vector>
#include <osmium/area/problem_reporter.hpp>
#include <osmium/area/detail/node_ref_segment.hpp>
#include <osmium/memory/buffer.hpp>
#include <osmium/area/problem_reporter.hpp>
#include <osmium/osm/item_type.hpp>
#include <osmium/osm/location.hpp>
#include <osmium/osm/node_ref.hpp>
#include <osmium/osm/relation.hpp>
@ -52,6 +56,24 @@ namespace osmium {
namespace detail {
/**
* Iterate over all relation members and the vector of ways at the
* same time and call given function with the relation member and
* way as parameter. This takes into account that there might be
* non-way members in the relation.
*/
template <typename F>
inline void for_each_member(const osmium::Relation& relation, const std::vector<const osmium::Way*>& ways, F&& func) {
auto way_it = ways.cbegin();
for (const osmium::RelationMember& member : relation.members()) {
if (member.type() == osmium::item_type::way) {
assert(way_it != ways.cend());
func(member, **way_it);
++way_it;
}
}
}
/**
* This is a helper class for the area assembler. It models
* a list of segments.
@ -64,6 +86,51 @@ namespace osmium {
bool m_debug;
static role_type parse_role(const char* role) noexcept {
if (role[0] == '\0') {
return role_type::empty;
} else if (!std::strcmp(role, "outer")) {
return role_type::outer;
} else if (!std::strcmp(role, "inner")) {
return role_type::inner;
}
return role_type::unknown;
}
/**
* Calculate the number of segments in all the ways together.
*/
static size_t get_num_segments(const std::vector<const osmium::Way*>& members) noexcept {
return std::accumulate(members.cbegin(), members.cend(), 0, [](size_t sum, const osmium::Way* way) {
if (way->nodes().empty()) {
return sum;
} else {
return sum + way->nodes().size() - 1;
}
});
}
uint32_t extract_segments_from_way_impl(osmium::area::ProblemReporter* problem_reporter, const osmium::Way& way, role_type role) {
uint32_t duplicate_nodes = 0;
osmium::NodeRef previous_nr;
for (const osmium::NodeRef& nr : way.nodes()) {
if (previous_nr.location()) {
if (previous_nr.location() != nr.location()) {
m_segments.emplace_back(previous_nr, nr, role, &way);
} else {
++duplicate_nodes;
if (problem_reporter) {
problem_reporter->report_duplicate_node(previous_nr.ref(), nr.ref(), nr.location());
}
}
}
previous_nr = nr;
}
return duplicate_nodes;
}
public:
explicit SegmentList(bool debug) noexcept :
@ -84,12 +151,31 @@ namespace osmium {
return m_segments.size();
}
/// Is the segment list empty?
bool empty() const noexcept {
return m_segments.empty();
}
typedef slist_type::const_iterator const_iterator;
typedef slist_type::iterator iterator;
using const_iterator = slist_type::const_iterator;
using iterator = slist_type::iterator;
NodeRefSegment& front() {
return m_segments.front();
}
NodeRefSegment& back() {
return m_segments.back();
}
const NodeRefSegment& operator[](size_t n) const noexcept {
assert(n < m_segments.size());
return m_segments[n];
}
NodeRefSegment& operator[](size_t n) noexcept {
assert(n < m_segments.size());
return m_segments[n];
}
iterator begin() noexcept {
return m_segments.begin();
@ -115,11 +201,6 @@ namespace osmium {
m_debug = debug;
}
/// Clear the list of segments. All segments are removed.
void clear() {
m_segments.clear();
}
/// Sort the list of segments.
void sort() {
std::sort(m_segments.begin(), m_segments.end());
@ -128,32 +209,37 @@ namespace osmium {
/**
* Extract segments from given way and add them to the list.
*
* Segments connecting two nodes with the same location (ie same
* node or different node with same location) are removed.
*
* XXX should two nodes with same location be reported?
* Segments connecting two nodes with the same location (ie
* same node or different nodes with same location) are
* removed after reporting the duplicate node.
*/
void extract_segments_from_way(const osmium::Way& way, const char* role) {
osmium::NodeRef last_nr;
for (const osmium::NodeRef& nr : way.nodes()) {
if (last_nr.location() && last_nr.location() != nr.location()) {
m_segments.emplace_back(last_nr, nr, role, &way);
}
last_nr = nr;
uint32_t extract_segments_from_way(osmium::area::ProblemReporter* problem_reporter, const osmium::Way& way) {
if (way.nodes().empty()) {
return 0;
}
m_segments.reserve(way.nodes().size() - 1);
return extract_segments_from_way_impl(problem_reporter, way, role_type::outer);
}
/**
* Extract all segments from all ways that make up this
* multipolygon relation and add them to the list.
*/
void extract_segments_from_ways(const osmium::Relation& relation, const std::vector<size_t>& members, const osmium::memory::Buffer& in_buffer) {
auto member_it = relation.members().begin();
for (size_t offset : members) {
const osmium::Way& way = in_buffer.get<const osmium::Way>(offset);
extract_segments_from_way(way, member_it->role());
++member_it;
uint32_t extract_segments_from_ways(osmium::area::ProblemReporter* problem_reporter, const osmium::Relation& relation, const std::vector<const osmium::Way*>& members) {
assert(relation.members().size() >= members.size());
const size_t num_segments = get_num_segments(members);
if (problem_reporter) {
problem_reporter->set_nodes(num_segments);
}
m_segments.reserve(num_segments);
uint32_t duplicate_nodes = 0;
for_each_member(relation, members, [this, &problem_reporter, &duplicate_nodes](const osmium::RelationMember& member, const osmium::Way& way) {
duplicate_nodes += extract_segments_from_way_impl(problem_reporter, way, parse_role(member.role()));
});
return duplicate_nodes;
}
/**
@ -162,17 +248,35 @@ namespace osmium {
* same segment. So if there are three, for instance, two will
* be removed and one will be left.
*/
void erase_duplicate_segments() {
uint32_t erase_duplicate_segments(osmium::area::ProblemReporter* problem_reporter) {
uint32_t duplicate_segments = 0;
while (true) {
auto it = std::adjacent_find(m_segments.begin(), m_segments.end());
if (it == m_segments.end()) {
return;
break;
}
if (m_debug) {
std::cerr << " erase duplicate segment: " << *it << "\n";
}
// Only count and report duplicate segments if they
// belong to the same way or if they don't both have
// the role "inner". Those cases are definitely wrong.
// If the duplicate segments belong to different
// "inner" ways, they could be touching inner rings
// which are perfectly okay. Note that for this check
// the role has to be correct in the member data.
if (it->way() == std::next(it)->way() || !it->role_inner() || !std::next(it)->role_inner()) {
++duplicate_segments;
if (problem_reporter) {
problem_reporter->report_duplicate_segment(it->first(), it->second());
}
}
m_segments.erase(it, it+2);
}
return duplicate_segments;
}
/**
@ -182,14 +286,14 @@ namespace osmium {
* reported to this object.
* @returns true if there are intersections.
*/
bool find_intersections(osmium::area::ProblemReporter* problem_reporter) const {
uint32_t find_intersections(osmium::area::ProblemReporter* problem_reporter) const {
if (m_segments.empty()) {
return false;
return 0;
}
bool found_intersections = false;
uint32_t found_intersections = 0;
for (auto it1 = m_segments.begin(); it1 != m_segments.end()-1; ++it1) {
for (auto it1 = m_segments.cbegin(); it1 != m_segments.cend()-1; ++it1) {
const NodeRefSegment& s1 = *it1;
for (auto it2 = it1+1; it2 != m_segments.end(); ++it2) {
const NodeRefSegment& s2 = *it2;
@ -203,12 +307,13 @@ namespace osmium {
if (y_range_overlap(s1, s2)) {
osmium::Location intersection = calculate_intersection(s1, s2);
if (intersection) {
found_intersections = true;
++found_intersections;
if (m_debug) {
std::cerr << " segments " << s1 << " and " << s2 << " intersecting at " << intersection << "\n";
}
if (problem_reporter) {
problem_reporter->report_intersection(s1.way()->id(), s1.first().location(), s1.second().location(), s2.way()->id(), s2.first().location(), s2.second().location(), intersection);
problem_reporter->report_intersection(s1.way()->id(), s1.first().location(), s1.second().location(),
s2.way()->id(), s2.first().location(), s2.second().location(), intersection);
}
}
}

121
include/osmium/area/detail/vector.hpp Normal file
View File

@ -0,0 +1,121 @@
#ifndef OSMIUM_AREA_DETAIL_VECTOR_HPP
#define OSMIUM_AREA_DETAIL_VECTOR_HPP
/*
This file is part of Osmium (http://osmcode.org/libosmium).
Copyright 2013-2016 Jochen Topf <jochen@topf.org> and others (see README).
Boost Software License - Version 1.0 - August 17th, 2003
Permission is hereby granted, free of charge, to any person or organization
obtaining a copy of the software and accompanying documentation covered by
this license (the "Software") to use, reproduce, display, distribute,
execute, and transmit the Software, and to prepare derivative works of the
Software, and to permit third-parties to whom the Software is furnished to
do so, all subject to the following:
The copyright notices in the Software and this entire statement, including
the above license grant, this restriction and the following disclaimer,
must be included in all copies of the Software, in whole or in part, and
all derivative works of the Software, unless such copies or derivative
works are solely in the form of machine-executable object code generated by
a source language processor.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT
SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE
FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE,
ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
*/
#include <cstdint>
#include <iosfwd>
#include <osmium/osm/location.hpp>
#include <osmium/osm/node_ref.hpp>
namespace osmium {
namespace area {
namespace detail {
/**
* This helper class models a 2D vector in the mathematical sense.
* It uses 64 bit integers internally which has enough precision
* for most operations with inputs based on 32 bit locations.
*/
struct vec {
int64_t x;
int64_t y;
constexpr vec(int64_t a, int64_t b) noexcept :
x(a),
y(b) {
}
constexpr explicit vec(const osmium::Location& l) noexcept :
x(l.x()),
y(l.y()) {
}
constexpr explicit vec(const osmium::NodeRef& nr) noexcept :
x(nr.x()),
y(nr.y()) {
}
}; // struct vec
// addition
constexpr inline vec operator+(const vec& a, const vec& b) noexcept {
return vec{a.x + b.x, a.y + b.y};
}
// subtraction
constexpr inline vec operator-(const vec& a, const vec& b) noexcept {
return vec{a.x - b.x, a.y - b.y};
}
// cross product
constexpr inline int64_t operator*(const vec& a, const vec& b) noexcept {
return a.x * b.y - a.y * b.x;
}
// scale vector
constexpr inline vec operator*(double s, const vec& v) noexcept {
return vec{int64_t(s * v.x), int64_t(s * v.y)};
}
// scale vector
constexpr inline vec operator*(const vec& v, double s) noexcept {
return vec{int64_t(s * v.x), int64_t(s * v.y)};
}
// equality
constexpr inline bool operator==(const vec& a, const vec& b) noexcept {
return a.x == b.x && a.y == b.y;
}
// inequality
constexpr inline bool operator!=(const vec& a, const vec& b) noexcept {
return !(a == b);
}
template <typename TChar, typename TTraits>
inline std::basic_ostream<TChar, TTraits>& operator<<(std::basic_ostream<TChar, TTraits>& out, const vec& v) {
return out << '(' << v.x << ',' << v.y << ')';
}
} // namespace detail
} // namespace area
} // namespace osmium
#endif // OSMIUM_AREA_DETAIL_VECTOR_HPP

31
include/osmium/area/multipolygon_collector.hpp
View File

@ -34,11 +34,11 @@ DEALINGS IN THE SOFTWARE.
*/
#include <algorithm>
#include <cassert>
#include <cstddef>
#include <cstring>
#include <vector>
#include <osmium/area/stats.hpp>
#include <osmium/memory/buffer.hpp>
#include <osmium/osm/item_type.hpp>
#include <osmium/osm/location.hpp>
@ -47,7 +47,6 @@ DEALINGS IN THE SOFTWARE.
#include <osmium/osm/tag.hpp>
#include <osmium/osm/way.hpp>
#include <osmium/relations/collector.hpp>
#include <osmium/relations/detail/member_meta.hpp>
namespace osmium {
@ -74,13 +73,15 @@ namespace osmium {
template <typename TAssembler>
class MultipolygonCollector : public osmium::relations::Collector<MultipolygonCollector<TAssembler>, false, true, false> {
typedef typename osmium::relations::Collector<MultipolygonCollector<TAssembler>, false, true, false> collector_type;
using collector_type = osmium::relations::Collector<MultipolygonCollector<TAssembler>, false, true, false>;
typedef typename TAssembler::config_type assembler_config_type;
using assembler_config_type = typename TAssembler::config_type;
const assembler_config_type m_assembler_config;
osmium::memory::Buffer m_output_buffer;
osmium::area::area_stats m_stats;
static constexpr size_t initial_output_buffer_size = 1024 * 1024;
static constexpr size_t max_buffer_size_for_flush = 100 * 1024;
@ -107,6 +108,10 @@ namespace osmium {
m_output_buffer(initial_output_buffer_size, osmium::memory::Buffer::auto_grow::yes) {
}
const osmium::area::area_stats& stats() const noexcept {
return m_stats;
}
/**
* We are interested in all relations tagged with type=multipolygon
* or type=boundary.
@ -121,7 +126,7 @@ namespace osmium {
return false;
}
if ((!strcmp(type, "multipolygon")) || (!strcmp(type, "boundary"))) {
if ((!std::strcmp(type, "multipolygon")) || (!std::strcmp(type, "boundary"))) {
return true;
}
@ -155,26 +160,32 @@ namespace osmium {
// way is closed and has enough nodes, build simple multipolygon
TAssembler assembler(m_assembler_config);
assembler(way, m_output_buffer);
m_stats += assembler.stats();
possibly_flush_output_buffer();
}
} catch (osmium::invalid_location&) {
} catch (const osmium::invalid_location&) {
// XXX ignore
}
}
void complete_relation(osmium::relations::RelationMeta& relation_meta) {
const osmium::Relation& relation = this->get_relation(relation_meta);
std::vector<size_t> offsets;
const osmium::memory::Buffer& buffer = this->members_buffer();
std::vector<const osmium::Way*> ways;
for (const auto& member : relation.members()) {
if (member.ref() != 0) {
offsets.push_back(this->get_offset(member.type(), member.ref()));
const size_t offset = this->get_offset(member.type(), member.ref());
ways.push_back(&buffer.get<const osmium::Way>(offset));
}
}
try {
TAssembler assembler(m_assembler_config);
assembler(relation, offsets, this->members_buffer(), m_output_buffer);
assembler(relation, ways, m_output_buffer);
m_stats += assembler.stats();
possibly_flush_output_buffer();
} catch (osmium::invalid_location&) {
} catch (const osmium::invalid_location&) {
// XXX ignore
}
}

66
include/osmium/area/problem_reporter.hpp
View File

@ -33,12 +33,17 @@ DEALINGS IN THE SOFTWARE.
*/
#include <cstddef>
#include <osmium/osm/item_type.hpp>
#include <osmium/osm/location.hpp>
#include <osmium/osm/types.hpp>
namespace osmium {
class NodeRef;
class Way;
namespace area {
/**
@ -62,6 +67,9 @@ namespace osmium {
// ID of the relation/way we are currently working on
osmium::object_id_type m_object_id;
// Number of nodes in the area
size_t m_nodes;
public:
ProblemReporter() = default;
@ -79,6 +87,10 @@ namespace osmium {
m_object_id = object_id;
}
void set_nodes(size_t nodes) noexcept {
m_nodes = nodes;
}
// Disable "unused-parameter" warning, so that the compiler will not complain.
// We can't remove the parameter names, because then doxygen will complain.
#pragma GCC diagnostic push
@ -94,6 +106,16 @@ namespace osmium {
virtual void report_duplicate_node(osmium::object_id_type node_id1, osmium::object_id_type node_id2, osmium::Location location) {
}
/**
* Report a node/location where rings touch. This is often wrong,
* but not necessarily so.
*
* @param node_id ID of the node.
* @param location Location of the node.
*/
virtual void report_touching_ring(osmium::object_id_type node_id, osmium::Location location) {
}
/**
* Report an intersection between two segments.
*
@ -109,13 +131,25 @@ namespace osmium {
osmium::object_id_type way2_id, osmium::Location way2_seg_start, osmium::Location way2_seg_end, osmium::Location intersection) {
}
/**
* Report a duplicate segments. Two or more segments are directly
* on top of each other. This can be a problem, if there is a
* spike for instance, or it could be okay, if there are touching
* inner rings.
*
* @param nr1 NodeRef of one end of the segment.
* @param nr2 NodeRef of the other end of the segment.
*/
virtual void report_duplicate_segment(const osmium::NodeRef& nr1, const osmium::NodeRef& nr2) {
}
/**
* Report an open ring.
*
* @param end1 Location of the first open end.
* @param end2 Location of the second open end.
* @param nr NodeRef of one end of the ring.
* @param way Optional pointer to way the end node is in.
*/
virtual void report_ring_not_closed(osmium::Location end1, osmium::Location end2) {
virtual void report_ring_not_closed(const osmium::NodeRef& nr, const osmium::Way* way = nullptr) {
}
/**
@ -138,6 +172,32 @@ namespace osmium {
virtual void report_role_should_be_inner(osmium::object_id_type way_id, osmium::Location seg_start, osmium::Location seg_end) {
}
/**
* Report a way that is in multiple rings.
*
* @param way The way.
*/
virtual void report_way_in_multiple_rings(const osmium::Way& way) {
}
/**
* Report a way with role inner that has the same tags as the
* relation or outer ways.
*
* @param way The way.
*/
virtual void report_inner_with_same_tags(const osmium::Way& way) {
}
/**
* In addition to reporting specific problems, this is used to
* report all ways belonging to areas having problems.
*
* @param way The way
*/
virtual void report_way(const osmium::Way& way) {
}
#pragma GCC diagnostic pop
}; // class ProblemReporter

31
include/osmium/area/problem_reporter_exception.hpp
View File

@ -42,6 +42,9 @@ DEALINGS IN THE SOFTWARE.
namespace osmium {
class NodeRef;
class Way;
namespace area {
class ProblemReporterException : public ProblemReporterStream {
@ -62,6 +65,12 @@ namespace osmium {
throw std::runtime_error(m_sstream.str());
}
void report_touching_ring(osmium::object_id_type node_id, osmium::Location location) override {
m_sstream.str();
ProblemReporterStream::report_touching_ring(node_id, location);
throw std::runtime_error(m_sstream.str());
}
void report_intersection(osmium::object_id_type way1_id, osmium::Location way1_seg_start, osmium::Location way1_seg_end,
osmium::object_id_type way2_id, osmium::Location way2_seg_start, osmium::Location way2_seg_end, osmium::Location intersection) override {
m_sstream.str();
@ -69,9 +78,15 @@ namespace osmium {
throw std::runtime_error(m_sstream.str());
}
void report_ring_not_closed(osmium::Location end1, osmium::Location end2) override {
void report_duplicate_segment(const osmium::NodeRef& nr1, const osmium::NodeRef& nr2) override {
m_sstream.str();
ProblemReporterStream::report_ring_not_closed(end1, end2);
ProblemReporterStream::report_duplicate_segment(nr1, nr2);
throw std::runtime_error(m_sstream.str());
}
void report_ring_not_closed(const osmium::NodeRef& nr, const osmium::Way* way = nullptr) override {
m_sstream.str();
ProblemReporterStream::report_ring_not_closed(nr, way);
throw std::runtime_error(m_sstream.str());
}
@ -87,6 +102,18 @@ namespace osmium {
throw std::runtime_error(m_sstream.str());
}
void report_way_in_multiple_rings(const osmium::Way& way) override {
m_sstream.str();
ProblemReporterStream::report_way_in_multiple_rings(way);
throw std::runtime_error(m_sstream.str());
}
void report_inner_with_same_tags(const osmium::Way& way) override {
m_sstream.str();
ProblemReporterStream::report_inner_with_same_tags(way);
throw std::runtime_error(m_sstream.str());
}
}; // class ProblemReporterException
} // namespace area

143
include/osmium/area/problem_reporter_ogr.hpp
View File

@ -49,8 +49,12 @@ DEALINGS IN THE SOFTWARE.
#include <osmium/area/problem_reporter.hpp>
#include <osmium/geom/factory.hpp>
#include <osmium/geom/ogr.hpp>
#include <osmium/osm/item_type.hpp>
#include <osmium/osm/location.hpp>
#include <osmium/osm/node_ref.hpp>
#include <osmium/osm/node_ref_list.hpp>
#include <osmium/osm/types.hpp>
#include <osmium/osm/way.hpp>
namespace osmium {
@ -66,26 +70,34 @@ namespace osmium {
gdalcpp::Layer m_layer_perror;
gdalcpp::Layer m_layer_lerror;
gdalcpp::Layer m_layer_ways;
void set_object(gdalcpp::Feature& feature) {
const char t[2] = { osmium::item_type_to_char(m_object_type), '\0' };
feature.set_field("obj_type", t);
feature.set_field("obj_id", int32_t(m_object_id));
feature.set_field("nodes", int32_t(m_nodes));
}
void write_point(const char* problem_type, osmium::object_id_type id1, osmium::object_id_type id2, osmium::Location location) {
gdalcpp::Feature feature(m_layer_perror, m_ogr_factory.create_point(location));
feature.set_field("id1", static_cast<double>(id1));
feature.set_field("id2", static_cast<double>(id2));
feature.set_field("problem_type", problem_type);
set_object(feature);
feature.set_field("id1", double(id1));
feature.set_field("id2", double(id2));
feature.set_field("problem", problem_type);
feature.add_to_layer();
}
void write_line(const char* problem_type, osmium::object_id_type id1, osmium::object_id_type id2, osmium::Location loc1, osmium::Location loc2) {
std::unique_ptr<OGRPoint> ogr_point1 = m_ogr_factory.create_point(loc1);
std::unique_ptr<OGRPoint> ogr_point2 = m_ogr_factory.create_point(loc2);
std::unique_ptr<OGRLineString> ogr_linestring = std::unique_ptr<OGRLineString>(new OGRLineString());
ogr_linestring->addPoint(ogr_point1.get());
ogr_linestring->addPoint(ogr_point2.get());
auto ogr_linestring = std::unique_ptr<OGRLineString>{new OGRLineString{}};
ogr_linestring->addPoint(loc1.lon(), loc1.lat());
ogr_linestring->addPoint(loc2.lon(), loc2.lat());
gdalcpp::Feature feature(m_layer_lerror, std::move(ogr_linestring));
set_object(feature);
feature.set_field("id1", static_cast<double>(id1));
feature.set_field("id2", static_cast<double>(id2));
feature.set_field("problem_type", problem_type);
feature.set_field("problem", problem_type);
feature.add_to_layer();
}
@ -93,15 +105,36 @@ namespace osmium {
explicit ProblemReporterOGR(gdalcpp::Dataset& dataset) :
m_layer_perror(dataset, "perrors", wkbPoint),
m_layer_lerror(dataset, "lerrors", wkbLineString) {
m_layer_lerror(dataset, "lerrors", wkbLineString),
m_layer_ways(dataset, "ways", wkbLineString) {
m_layer_perror.add_field("id1", OFTReal, 10);
m_layer_perror.add_field("id2", OFTReal, 10);
m_layer_perror.add_field("problem_type", OFTString, 30);
// 64bit integers are not supported in GDAL < 2, so we
// are using a workaround here in fields where we expect
// node IDs, we use real numbers.
m_layer_perror
.add_field("obj_type", OFTString, 1)
.add_field("obj_id", OFTInteger, 10)
.add_field("nodes", OFTInteger, 8)
.add_field("id1", OFTReal, 12, 1)
.add_field("id2", OFTReal, 12, 1)
.add_field("problem", OFTString, 30)
;
m_layer_lerror.add_field("id1", OFTReal, 10);
m_layer_lerror.add_field("id2", OFTReal, 10);
m_layer_lerror.add_field("problem_type", OFTString, 30);
m_layer_lerror
.add_field("obj_type", OFTString, 1)
.add_field("obj_id", OFTInteger, 10)
.add_field("nodes", OFTInteger, 8)
.add_field("id1", OFTReal, 12, 1)
.add_field("id2", OFTReal, 12, 1)
.add_field("problem", OFTString, 30)
;
m_layer_ways
.add_field("obj_type", OFTString, 1)
.add_field("obj_id", OFTInteger, 10)
.add_field("way_id", OFTInteger, 10)
.add_field("nodes", OFTInteger, 8)
;
}
~ProblemReporterOGR() override = default;
@ -110,24 +143,82 @@ namespace osmium {
write_point("duplicate_node", node_id1, node_id2, location);
}
void report_intersection(osmium::object_id_type way1_id, osmium::Location way1_seg_start, osmium::Location way1_seg_end,
osmium::object_id_type way2_id, osmium::Location way2_seg_start, osmium::Location way2_seg_end, osmium::Location intersection) override {
write_point("intersection", m_object_id, 0, intersection);
write_line("intersection", m_object_id, way1_id, way1_seg_start, way1_seg_end);
write_line("intersection", m_object_id, way2_id, way2_seg_start, way2_seg_end);
void report_touching_ring(osmium::object_id_type node_id, osmium::Location location) override {
write_point("touching_ring", node_id, 0, location);
}
void report_ring_not_closed(osmium::Location end1, osmium::Location end2) override {
write_point("ring_not_closed", m_object_id, 0, end1);
write_point("ring_not_closed", m_object_id, 0, end2);
void report_intersection(osmium::object_id_type way1_id, osmium::Location way1_seg_start, osmium::Location way1_seg_end,
osmium::object_id_type way2_id, osmium::Location way2_seg_start, osmium::Location way2_seg_end, osmium::Location intersection) override {
write_point("intersection", way1_id, way2_id, intersection);
write_line("intersection", way1_id, way2_id, way1_seg_start, way1_seg_end);
write_line("intersection", way2_id, way1_id, way2_seg_start, way2_seg_end);
}
void report_duplicate_segment(const osmium::NodeRef& nr1, const osmium::NodeRef& nr2) override {
write_line("duplicate_segment", nr1.ref(), nr2.ref(), nr1.location(), nr2.location());
}
void report_ring_not_closed(const osmium::NodeRef& nr, const osmium::Way* way = nullptr) override {
write_point("ring_not_closed", nr.ref(), way ? way->id() : 0, nr.location());
}
void report_role_should_be_outer(osmium::object_id_type way_id, osmium::Location seg_start, osmium::Location seg_end) override {
write_line("role_should_be_outer", m_object_id, way_id, seg_start, seg_end);
write_line("role_should_be_outer", way_id, 0, seg_start, seg_end);
}
void report_role_should_be_inner(osmium::object_id_type way_id, osmium::Location seg_start, osmium::Location seg_end) override {
write_line("role_should_be_inner", m_object_id, way_id, seg_start, seg_end);
write_line("role_should_be_inner", way_id, 0, seg_start, seg_end);
}
void report_way_in_multiple_rings(const osmium::Way& way) override {
if (way.nodes().size() < 2) {
return;
}
try {
gdalcpp::Feature feature(m_layer_lerror, m_ogr_factory.create_linestring(way));
set_object(feature);
feature.set_field("id1", int32_t(way.id()));
feature.set_field("id2", 0);
feature.set_field("problem", "way_in_multiple_rings");
feature.add_to_layer();
} catch (const osmium::geometry_error&) {
// XXX
}
}
void report_inner_with_same_tags(const osmium::Way& way) override {
if (way.nodes().size() < 2) {
return;
}
try {
gdalcpp::Feature feature(m_layer_lerror, m_ogr_factory.create_linestring(way));
set_object(feature);
feature.set_field("id1", int32_t(way.id()));
feature.set_field("id2", 0);
feature.set_field("problem", "inner_with_same_tags");
feature.add_to_layer();
} catch (const osmium::geometry_error&) {
// XXX
}
}
void report_way(const osmium::Way& way) override {
if (way.nodes().empty()) {
return;
}
if (way.nodes().size() == 1) {
const auto& first_nr = way.nodes()[0];
write_point("single_node_in_way", way.id(), first_nr.ref(), first_nr.location());
return;
}
try {
gdalcpp::Feature feature(m_layer_ways, m_ogr_factory.create_linestring(way));
set_object(feature);
feature.set_field("way_id", int32_t(way.id()));
feature.add_to_layer();
} catch (const osmium::geometry_error&) {
// XXX
}
}
}; // class ProblemReporterOGR

33
include/osmium/area/problem_reporter_stream.hpp
View File

@ -38,7 +38,9 @@ DEALINGS IN THE SOFTWARE.
#include <osmium/area/problem_reporter.hpp>
#include <osmium/osm/item_type.hpp>
#include <osmium/osm/location.hpp>
#include <osmium/osm/node_ref.hpp>
#include <osmium/osm/types.hpp>
#include <osmium/osm/way.hpp>
namespace osmium {
@ -57,7 +59,7 @@ namespace osmium {
~ProblemReporterStream() override = default;
void header(const char* msg) {
*m_out << "DATA PROBLEM: " << msg << " on " << item_type_to_char(m_object_type) << m_object_id << ": ";
*m_out << "DATA PROBLEM: " << msg << " on " << item_type_to_char(m_object_type) << m_object_id << " (with " << m_nodes << " nodes): ";
}
void report_duplicate_node(osmium::object_id_type node_id1, osmium::object_id_type node_id2, osmium::Location location) override {
@ -65,6 +67,11 @@ namespace osmium {
*m_out << "node_id1=" << node_id1 << " node_id2=" << node_id2 << " location=" << location << "\n";
}
void report_touching_ring(osmium::object_id_type node_id, osmium::Location location) override {
header("touching ring");
*m_out << "node_id=" << node_id << " location=" << location << "\n";
}
void report_intersection(osmium::object_id_type way1_id, osmium::Location way1_seg_start, osmium::Location way1_seg_end,
osmium::object_id_type way2_id, osmium::Location way2_seg_start, osmium::Location way2_seg_end, osmium::Location intersection) override {
header("intersection");
@ -72,9 +79,19 @@ namespace osmium {
<< " way2_id=" << way2_id << " way2_seg_start=" << way2_seg_start << " way2_seg_end=" << way2_seg_end << " intersection=" << intersection << "\n";
}
void report_ring_not_closed(osmium::Location end1, osmium::Location end2) override {
void report_duplicate_segment(const osmium::NodeRef& nr1, const osmium::NodeRef& nr2) override {
header("duplicate segment");
*m_out << "node_id1=" << nr1.ref() << " location1=" << nr1.location()
<< " node_id2=" << nr2.ref() << " location2=" << nr2.location() << "\n";
}
void report_ring_not_closed(const osmium::NodeRef& nr, const osmium::Way* way = nullptr) override {
header("ring not closed");
*m_out << "end1=" << end1 << " end2=" << end2 << "\n";
*m_out << "node_id=" << nr.ref() << " location=" << nr.location();
if (way) {
*m_out << " on way " << way->id();
}
*m_out << "\n";
}
void report_role_should_be_outer(osmium::object_id_type way_id, osmium::Location seg_start, osmium::Location seg_end) override {
@ -87,6 +104,16 @@ namespace osmium {
*m_out << "way_id=" << way_id << " seg_start=" << seg_start << " seg_end=" << seg_end << "\n";
}
void report_way_in_multiple_rings(const osmium::Way& way) override {
header("way in multiple rings");
*m_out << "way_id=" << way.id() << '\n';
}
void report_inner_with_same_tags(const osmium::Way& way) override {
header("inner way with same tags as relation or outer");
*m_out << "way_id=" << way.id() << '\n';
}
}; // class ProblemReporterStream
} // namespace area

128
include/osmium/area/stats.hpp Normal file
View File

@ -0,0 +1,128 @@
#ifndef OSMIUM_AREA_STATS_HPP
#define OSMIUM_AREA_STATS_HPP
/*
This file is part of Osmium (http://osmcode.org/libosmium).
Copyright 2013-2016 Jochen Topf <jochen@topf.org> and others (see README).
Boost Software License - Version 1.0 - August 17th, 2003
Permission is hereby granted, free of charge, to any person or organization
obtaining a copy of the software and accompanying documentation covered by
this license (the "Software") to use, reproduce, display, distribute,
execute, and transmit the Software, and to prepare derivative works of the
Software, and to permit third-parties to whom the Software is furnished to
do so, all subject to the following:
The copyright notices in the Software and this entire statement, including
the above license grant, this restriction and the following disclaimer,
must be included in all copies of the Software, in whole or in part, and
all derivative works of the Software, unless such copies or derivative
works are solely in the form of machine-executable object code generated by
a source language processor.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT
SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE
FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE,
ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
*/
#include <cstdint>
#include <ostream>
namespace osmium {
namespace area {
/**
* These statistics are generated by the area assembler code. They
* tell the user of the assembler a lot about the objects this area
* is made out of, what happened during the assembly, and what errors
* there were.
*/
struct area_stats {
uint64_t area_really_complex_case = 0; ///< Most difficult case with rings touching in multiple points
uint64_t area_simple_case = 0; ///< Simple case, no touching rings
uint64_t area_touching_rings_case = 0; ///< More difficult case with touching rings
uint64_t duplicate_nodes = 0; ///< Consecutive identical nodes or consecutive nodes with same location
uint64_t duplicate_segments = 0; ///< Segments duplicated (going back and forth)
uint64_t from_relations = 0; ///< Area created from multipolygon relation
uint64_t from_ways = 0; ///< Area created from way
uint64_t inner_rings = 0; ///< Number of inner rings
uint64_t inner_with_same_tags = 0; ///< Number of inner ways with same tags as area
uint64_t intersections = 0; ///< Number of intersections between segments
uint64_t member_ways = 0; ///< Number of ways in the area
uint64_t no_tags_on_relation = 0; ///< No tags on relation (old-style multipolygon with tags on outer ways)
uint64_t no_way_in_mp_relation = 0; ///< Multipolygon relation with no way members
uint64_t nodes = 0; ///< Number of nodes in the area
uint64_t open_rings = 0; ///< Number of open rings in the area
uint64_t outer_rings = 0; ///< Number of outer rings in the area
uint64_t short_ways = 0; ///< Number of ways with less than two nodes
uint64_t single_way_in_mp_relation = 0; ///< Multipolygon relation containing a single way
uint64_t touching_rings = 0; ///< Rings touching in a node
uint64_t ways_in_multiple_rings = 0; ///< Different segments of a way ended up in different rings
uint64_t wrong_role = 0; ///< Member has wrong role (not "outer", "inner", or empty)
area_stats& operator+=(const area_stats& other) noexcept {
area_really_complex_case += other.area_really_complex_case;
area_simple_case += other.area_simple_case;
area_touching_rings_case += other.area_touching_rings_case;
duplicate_nodes += other.duplicate_nodes;
duplicate_segments += other.duplicate_segments;
from_relations += other.from_relations;
from_ways += other.from_ways;
inner_rings += other.inner_rings;
inner_with_same_tags += other.inner_with_same_tags;
intersections += other.intersections;
member_ways += other.member_ways;
no_tags_on_relation += other.no_tags_on_relation;
no_way_in_mp_relation += other.no_way_in_mp_relation;
nodes += other.nodes;
open_rings += other.open_rings;
outer_rings += other.outer_rings;
short_ways += other.short_ways;
single_way_in_mp_relation += other.single_way_in_mp_relation;
touching_rings += other.touching_rings;
ways_in_multiple_rings += other.ways_in_multiple_rings;
wrong_role += other.wrong_role;
return *this;
}
}; // struct area_stats
template <typename TChar, typename TTraits>
inline std::basic_ostream<TChar, TTraits>& operator<<(std::basic_ostream<TChar, TTraits>& out, const area_stats& s) {
return out << " area_really_complex_case=" << s.area_really_complex_case
<< " area_simple_case=" << s.area_simple_case
<< " area_touching_rings_case=" << s.area_touching_rings_case
<< " duplicate_nodes=" << s.duplicate_nodes
<< " duplicate_segments=" << s.duplicate_segments
<< " from_relations=" << s.from_relations
<< " from_ways=" << s.from_ways
<< " inner_rings=" << s.inner_rings
<< " inner_with_same_tags=" << s.inner_with_same_tags
<< " intersections=" << s.intersections
<< " member_ways=" << s.member_ways
<< " no_tags_on_relation=" << s.no_tags_on_relation
<< " no_way_in_mp_relation=" << s.no_way_in_mp_relation
<< " nodes=" << s.nodes
<< " open_rings=" << s.open_rings
<< " outer_rings=" << s.outer_rings
<< " short_ways=" << s.short_ways
<< " single_way_in_mp_relation=" << s.single_way_in_mp_relation
<< " touching_rings=" << s.touching_rings
<< " ways_in_multiple_rings=" << s.ways_in_multiple_rings
<< " wrong_role=" << s.wrong_role;
}
} // namespace area
} // namespace osmium
#endif // OSMIUM_AREA_STATS_HPP

37
include/osmium/builder/attr.hpp
View File

@ -46,8 +46,15 @@ DEALINGS IN THE SOFTWARE.
#include <osmium/builder/builder.hpp>
#include <osmium/builder/osm_object_builder.hpp>
#include <osmium/memory/buffer.hpp>
#include <osmium/osm/changeset.hpp>
#include <osmium/osm/item_type.hpp>
#include <osmium/osm/location.hpp>
#include <osmium/osm/node.hpp>
#include <osmium/osm/node_ref.hpp>
#include <osmium/osm/object.hpp>
#include <osmium/osm/relation.hpp>
#include <osmium/osm/timestamp.hpp>
#include <osmium/osm/types.hpp>
#include <osmium/osm.hpp>
namespace osmium {
@ -261,6 +268,34 @@ namespace osmium {
}; // class member_type
class member_type_string {
osmium::item_type m_type;
osmium::object_id_type m_ref;
std::string m_role;
public:
member_type_string(osmium::item_type type, osmium::object_id_type ref, std::string&& role) :
m_type(type),
m_ref(ref),
m_role(std::move(role)) {
}
osmium::item_type type() const noexcept {
return m_type;
}
osmium::object_id_type ref() const noexcept {
return m_ref;
}
const char* role() const noexcept {
return m_role.c_str();
}
}; // class member_type_string
class comment_type {
osmium::Timestamp m_date;

3
include/osmium/builder/builder.hpp
View File

@ -35,7 +35,6 @@ DEALINGS IN THE SOFTWARE.
#include <algorithm>
#include <cassert>
#include <cstddef>
#include <cstdint>
#include <cstring>
#include <new>
@ -101,7 +100,7 @@ namespace osmium {
*
*/
void add_padding(bool self = false) {
auto padding = osmium::memory::align_bytes - (size() % osmium::memory::align_bytes);
const auto padding = osmium::memory::align_bytes - (size() % osmium::memory::align_bytes);
if (padding != osmium::memory::align_bytes) {
std::fill_n(m_buffer.reserve_space(padding), padding, 0);
if (self) {

8
include/osmium/builder/builder_helper.hpp
View File

@ -56,7 +56,7 @@ namespace osmium {
* Use osmium::builder::add_way_node_list() instead.
*/
OSMIUM_DEPRECATED inline const osmium::WayNodeList& build_way_node_list(osmium::memory::Buffer& buffer, const std::initializer_list<osmium::NodeRef>& nodes) {
size_t pos = buffer.committed();
const size_t pos = buffer.committed();
{
osmium::builder::WayNodeListBuilder wnl_builder(buffer);
for (const auto& node_ref : nodes) {
@ -72,7 +72,7 @@ namespace osmium {
* Use osmium::builder::add_tag_list() instead.
*/
inline const osmium::TagList& build_tag_list(osmium::memory::Buffer& buffer, const std::initializer_list<std::pair<const char*, const char*>>& tags) {
size_t pos = buffer.committed();
const size_t pos = buffer.committed();
{
osmium::builder::TagListBuilder tl_builder(buffer);
for (const auto& p : tags) {
@ -88,7 +88,7 @@ namespace osmium {
* Use osmium::builder::add_tag_list() instead.
*/
inline const osmium::TagList& build_tag_list_from_map(osmium::memory::Buffer& buffer, const std::map<const char*, const char*>& tags) {
size_t pos = buffer.committed();
const size_t pos = buffer.committed();
{
osmium::builder::TagListBuilder tl_builder(buffer);
for (const auto& p : tags) {
@ -104,7 +104,7 @@ namespace osmium {
* Use osmium::builder::add_tag_list() instead.
*/
inline const osmium::TagList& build_tag_list_from_func(osmium::memory::Buffer& buffer, std::function<void(osmium::builder::TagListBuilder&)> func) {
size_t pos = buffer.committed();
const size_t pos = buffer.committed();
{
osmium::builder::TagListBuilder tl_builder(buffer);
func(tl_builder);

22
include/osmium/builder/osm_object_builder.hpp
View File

@ -34,7 +34,6 @@ DEALINGS IN THE SOFTWARE.
*/
#include <cassert>
#include <cstddef>
#include <cstring>
#include <initializer_list>
#include <limits>
@ -44,16 +43,23 @@ DEALINGS IN THE SOFTWARE.
#include <utility>
#include <osmium/builder/builder.hpp>
#include <osmium/osm.hpp>
#include <osmium/osm/item_type.hpp>
#include <osmium/osm/location.hpp>
#include <osmium/osm/node_ref.hpp>
#include <osmium/osm/object.hpp>
#include <osmium/osm/tag.hpp>
#include <osmium/osm/types.hpp>
#include <osmium/memory/item.hpp>
#include <osmium/osm/area.hpp>
#include <osmium/osm/changeset.hpp>
#include <osmium/osm/relation.hpp>
#include <osmium/osm/timestamp.hpp>
#include <osmium/osm/way.hpp>
namespace osmium {
class Node;
namespace memory {
class Buffer;
} // namespace memory
@ -186,9 +192,9 @@ namespace osmium {
}; // class NodeRefListBuilder
typedef NodeRefListBuilder<WayNodeList> WayNodeListBuilder;
typedef NodeRefListBuilder<OuterRing> OuterRingBuilder;
typedef NodeRefListBuilder<InnerRing> InnerRingBuilder;
using WayNodeListBuilder = NodeRefListBuilder<WayNodeList>;
using OuterRingBuilder = NodeRefListBuilder<OuterRing>;
using InnerRingBuilder = NodeRefListBuilder<InnerRing>;
class RelationMemberListBuilder : public ObjectBuilder<RelationMemberList> {
@ -353,8 +359,8 @@ namespace osmium {
}; // class OSMObjectBuilder
typedef OSMObjectBuilder<osmium::Node> NodeBuilder;
typedef OSMObjectBuilder<osmium::Relation> RelationBuilder;
using NodeBuilder = OSMObjectBuilder<osmium::Node>;
using RelationBuilder = OSMObjectBuilder<osmium::Relation>;
class WayBuilder : public OSMObjectBuilder<osmium::Way> {
@ -398,7 +404,7 @@ namespace osmium {
}; // class AreaBuilder
typedef ObjectBuilder<osmium::Changeset> ChangesetBuilder;
using ChangesetBuilder = ObjectBuilder<osmium::Changeset>;
} // namespace builder

10
include/osmium/diff_iterator.hpp
View File

@ -34,8 +34,10 @@ DEALINGS IN THE SOFTWARE.
*/
#include <cassert>
#include <cstddef>
#include <iterator>
#include <type_traits>
#include <utility>
#include <osmium/osm/diff_object.hpp>
@ -49,7 +51,7 @@ namespace osmium {
* underlying OSMObjects.
*/
template <typename TBasicIterator>
class DiffIterator : public std::iterator<std::input_iterator_tag, const osmium::DiffObject> {
class DiffIterator {
static_assert(std::is_base_of<osmium::OSMObject, typename TBasicIterator::value_type>::value, "TBasicIterator::value_type must derive from osmium::OSMObject");
@ -76,6 +78,12 @@ namespace osmium {
public:
using iterator_category = std::input_iterator_tag;
using value_type = const osmium::DiffObject;
using difference_type = std::ptrdiff_t;
using pointer = value_type*;
using reference = value_type&;
DiffIterator(TBasicIterator begin, TBasicIterator end) :
m_prev(begin),
m_curr(begin),

9
include/osmium/dynamic_handler.hpp
View File

@ -36,11 +36,16 @@ DEALINGS IN THE SOFTWARE.
#include <memory>
#include <utility>
#include <osmium/fwd.hpp>
#include <osmium/handler.hpp>
namespace osmium {
class Node;
class Way;
class Relation;
class Area;
class Changeset;
namespace handler {
namespace detail {
@ -143,7 +148,7 @@ auto _name_##_dispatch(THandler& handler, const osmium::_type_& object, long) ->
class DynamicHandler : public osmium::handler::Handler {
typedef std::unique_ptr<osmium::handler::detail::HandlerWrapperBase> impl_ptr;
using impl_ptr = std::unique_ptr<osmium::handler::detail::HandlerWrapperBase>;
impl_ptr m_impl;
public:

3
include/osmium/experimental/flex_reader.hpp
View File

@ -34,11 +34,12 @@ DEALINGS IN THE SOFTWARE.
*/
#include <string>
#include <utility>
#include <vector>
#include <osmium/area/assembler.hpp>
#include <osmium/area/multipolygon_collector.hpp>
#include <osmium/handler/node_locations_for_ways.hpp>
#include <osmium/handler/node_locations_for_ways.hpp> // IWYU pragma: keep
#include <osmium/io/file.hpp>
#include <osmium/io/header.hpp>
#include <osmium/io/reader.hpp>

30
include/osmium/geom/factory.hpp
View File

@ -46,6 +46,8 @@ DEALINGS IN THE SOFTWARE.
#include <osmium/osm/location.hpp>
#include <osmium/osm/node.hpp>
#include <osmium/osm/node_ref.hpp>
#include <osmium/osm/node_ref_list.hpp>
#include <osmium/osm/types.hpp>
#include <osmium/osm/way.hpp>
namespace osmium {
@ -148,7 +150,7 @@ namespace osmium {
/**
* Add all points of an outer or inner ring to a multipolygon.
*/
void add_points(const osmium::OuterRing& nodes) {
void add_points(const osmium::NodeRefList& nodes) {
osmium::Location last_location;
for (const osmium::NodeRef& node_ref : nodes) {
if (last_location != node_ref.location()) {
@ -169,7 +171,7 @@ namespace osmium {
template <typename... TArgs>
explicit GeometryFactory<TGeomImpl, TProjection>(TArgs&&... args) :
m_projection(),
m_impl(std::forward<TArgs>(args)...) {
m_impl(m_projection.epsg(), std::forward<TArgs>(args)...) {
}
/**
@ -179,15 +181,16 @@ namespace osmium {
template <typename... TArgs>
explicit GeometryFactory<TGeomImpl, TProjection>(TProjection&& projection, TArgs&&... args) :
m_projection(std::move(projection)),
m_impl(std::forward<TArgs>(args)...) {
m_impl(m_projection.epsg(), std::forward<TArgs>(args)...) {
}
typedef TProjection projection_type;
typedef typename TGeomImpl::point_type point_type;
typedef typename TGeomImpl::linestring_type linestring_type;
typedef typename TGeomImpl::polygon_type polygon_type;
typedef typename TGeomImpl::multipolygon_type multipolygon_type;
typedef typename TGeomImpl::ring_type ring_type;
using projection_type = TProjection;
using point_type = typename TGeomImpl::point_type;
using linestring_type = typename TGeomImpl::linestring_type;
using polygon_type = typename TGeomImpl::polygon_type;
using multipolygon_type = typename TGeomImpl::multipolygon_type;
using ring_type = typename TGeomImpl::ring_type;
int epsg() const {
return m_projection.epsg();
@ -280,7 +283,7 @@ namespace osmium {
}
if (num_points < 2) {
throw osmium::geometry_error("need at least two points for linestring");
throw osmium::geometry_error{"need at least two points for linestring"};
}
return linestring_finish(num_points);
@ -354,7 +357,7 @@ namespace osmium {
}
if (num_points < 4) {
throw osmium::geometry_error("need at least four points for polygon");
throw osmium::geometry_error{"need at least four points for polygon"};
}
return polygon_finish(num_points);
@ -378,8 +381,8 @@ namespace osmium {
m_impl.multipolygon_start();
for (auto it = area.cbegin(); it != area.cend(); ++it) {
const osmium::OuterRing& ring = static_cast<const osmium::OuterRing&>(*it);
if (it->type() == osmium::item_type::outer_ring) {
auto& ring = static_cast<const osmium::OuterRing&>(*it);
if (num_polygons > 0) {
m_impl.multipolygon_polygon_finish();
}
@ -390,6 +393,7 @@ namespace osmium {
++num_rings;
++num_polygons;
} else if (it->type() == osmium::item_type::inner_ring) {
auto& ring = static_cast<const osmium::InnerRing&>(*it);
m_impl.multipolygon_inner_ring_start();
add_points(ring);
m_impl.multipolygon_inner_ring_finish();
@ -399,7 +403,7 @@ namespace osmium {
// if there are no rings, this area is invalid
if (num_rings == 0) {
throw osmium::geometry_error("area contains no rings");
throw osmium::geometry_error{"invalid area"};
}
m_impl.multipolygon_polygon_finish();

13
include/osmium/geom/geojson.hpp
View File

@ -34,6 +34,7 @@ DEALINGS IN THE SOFTWARE.
*/
#include <cassert>
#include <cstddef>
#include <string>
#include <utility>
@ -53,13 +54,13 @@ namespace osmium {
public:
typedef std::string point_type;
typedef std::string linestring_type;
typedef std::string polygon_type;
typedef std::string multipolygon_type;
typedef std::string ring_type;
using point_type = std::string;
using linestring_type = std::string;
using polygon_type = std::string;
using multipolygon_type = std::string;
using ring_type = std::string;
GeoJSONFactoryImpl(int precision = 7) :
GeoJSONFactoryImpl(int /* srid */, int precision = 7) :
m_precision(precision) {
}

57
include/osmium/geom/geos.hpp
View File

@ -42,9 +42,14 @@ DEALINGS IN THE SOFTWARE.
* @attention If you include this file, you'll need to link with `libgeos`.
*/
#include <algorithm>
#include <cassert>
#include <cstddef>
#include <iterator>
#include <memory>
#include <string>
#include <utility>
#include <vector>
#include <geos/geom/Coordinate.h>
#include <geos/geom/CoordinateSequence.h>
@ -59,11 +64,13 @@ DEALINGS IN THE SOFTWARE.
#include <osmium/geom/factory.hpp>
#include <osmium/geom/coordinates.hpp>
#include <osmium/util/compatibility.hpp>
// MSVC doesn't support throw_with_nested yet
#ifdef _MSC_VER
# define THROW throw
#else
# include <exception>
# define THROW std::throw_with_nested
#endif
@ -71,8 +78,8 @@ namespace osmium {
struct geos_geometry_error : public geometry_error {
geos_geometry_error(const char* message) :
geometry_error(std::string("geometry creation failed in GEOS library: ") + message) {
explicit geos_geometry_error(const char* message) :
geometry_error(std::string{"geometry creation failed in GEOS library: "} + message) {
}
}; // struct geos_geometry_error
@ -93,19 +100,29 @@ namespace osmium {
public:
typedef std::unique_ptr<geos::geom::Point> point_type;
typedef std::unique_ptr<geos::geom::LineString> linestring_type;
typedef std::unique_ptr<geos::geom::Polygon> polygon_type;
typedef std::unique_ptr<geos::geom::MultiPolygon> multipolygon_type;
typedef std::unique_ptr<geos::geom::LinearRing> ring_type;
using point_type = std::unique_ptr<geos::geom::Point>;
using linestring_type = std::unique_ptr<geos::geom::LineString>;
using polygon_type = std::unique_ptr<geos::geom::Polygon>;
using multipolygon_type = std::unique_ptr<geos::geom::MultiPolygon>;
using ring_type = std::unique_ptr<geos::geom::LinearRing>;
explicit GEOSFactoryImpl(geos::geom::GeometryFactory& geos_factory) :
explicit GEOSFactoryImpl(int /* srid */, geos::geom::GeometryFactory& geos_factory) :
m_precision_model(nullptr),
m_our_geos_factory(nullptr),
m_geos_factory(&geos_factory) {
}
explicit GEOSFactoryImpl(int srid = -1) :
/**
* @deprecated Do not set SRID explicitly. It will be set to the
* correct value automatically.
*/
OSMIUM_DEPRECATED explicit GEOSFactoryImpl(int /* srid */, int srid) :
m_precision_model(new geos::geom::PrecisionModel),
m_our_geos_factory(new geos::geom::GeometryFactory(m_precision_model.get(), srid)),
m_geos_factory(m_our_geos_factory.get()) {
}
explicit GEOSFactoryImpl(int srid) :
m_precision_model(new geos::geom::PrecisionModel),
m_our_geos_factory(new geos::geom::GeometryFactory(m_precision_model.get(), srid)),
m_geos_factory(m_our_geos_factory.get()) {
@ -116,7 +133,7 @@ namespace osmium {
point_type make_point(const osmium::geom::Coordinates& xy) const {
try {
return point_type(m_geos_factory->createPoint(geos::geom::Coordinate(xy.x, xy.y)));
} catch (geos::util::GEOSException& e) {
} catch (const geos::util::GEOSException& e) {
THROW(osmium::geos_geometry_error(e.what()));
}
}
@ -126,7 +143,7 @@ namespace osmium {
void linestring_start() {
try {
m_coordinate_sequence.reset(m_geos_factory->getCoordinateSequenceFactory()->create(static_cast<size_t>(0), 2));
} catch (geos::util::GEOSException& e) {
} catch (const geos::util::GEOSException& e) {
THROW(osmium::geos_geometry_error(e.what()));
}
}
@ -134,7 +151,7 @@ namespace osmium {
void linestring_add_location(const osmium::geom::Coordinates& xy) {
try {
m_coordinate_sequence->add(geos::geom::Coordinate(xy.x, xy.y));
} catch (geos::util::GEOSException& e) {
} catch (const geos::util::GEOSException& e) {
THROW(osmium::geos_geometry_error(e.what()));
}
}
@ -142,7 +159,7 @@ namespace osmium {
linestring_type linestring_finish(size_t /* num_points */) {
try {
return linestring_type(m_geos_factory->createLineString(m_coordinate_sequence.release()));
} catch (geos::util::GEOSException& e) {
} catch (const geos::util::GEOSException& e) {
THROW(osmium::geos_geometry_error(e.what()));
}
}
@ -166,7 +183,7 @@ namespace osmium {
});
m_polygons.emplace_back(m_geos_factory->createPolygon(m_rings[0].release(), inner_rings));
m_rings.clear();
} catch (geos::util::GEOSException& e) {
} catch (const geos::util::GEOSException& e) {
THROW(osmium::geos_geometry_error(e.what()));
}
}
@ -174,7 +191,7 @@ namespace osmium {
void multipolygon_outer_ring_start() {
try {
m_coordinate_sequence.reset(m_geos_factory->getCoordinateSequenceFactory()->create(static_cast<size_t>(0), 2));
} catch (geos::util::GEOSException& e) {
} catch (const geos::util::GEOSException& e) {
THROW(osmium::geos_geometry_error(e.what()));
}
}
@ -182,7 +199,7 @@ namespace osmium {
void multipolygon_outer_ring_finish() {
try {
m_rings.emplace_back(m_geos_factory->createLinearRing(m_coordinate_sequence.release()));
} catch (geos::util::GEOSException& e) {
} catch (const geos::util::GEOSException& e) {
THROW(osmium::geos_geometry_error(e.what()));
}
}
@ -190,7 +207,7 @@ namespace osmium {
void multipolygon_inner_ring_start() {
try {
m_coordinate_sequence.reset(m_geos_factory->getCoordinateSequenceFactory()->create(static_cast<size_t>(0), 2));
} catch (geos::util::GEOSException& e) {
} catch (const geos::util::GEOSException& e) {
THROW(osmium::geos_geometry_error(e.what()));
}
}
@ -198,7 +215,7 @@ namespace osmium {
void multipolygon_inner_ring_finish() {
try {
m_rings.emplace_back(m_geos_factory->createLinearRing(m_coordinate_sequence.release()));
} catch (geos::util::GEOSException& e) {
} catch (const geos::util::GEOSException& e) {
THROW(osmium::geos_geometry_error(e.what()));
}
}
@ -206,7 +223,7 @@ namespace osmium {
void multipolygon_add_location(const osmium::geom::Coordinates& xy) {
try {
m_coordinate_sequence->add(geos::geom::Coordinate(xy.x, xy.y));
} catch (geos::util::GEOSException& e) {
} catch (const geos::util::GEOSException& e) {
THROW(osmium::geos_geometry_error(e.what()));
}
}
@ -219,7 +236,7 @@ namespace osmium {
});
m_polygons.clear();
return multipolygon_type(m_geos_factory->createMultiPolygon(polygons));
} catch (geos::util::GEOSException& e) {
} catch (const geos::util::GEOSException& e) {
THROW(osmium::geos_geometry_error(e.what()));
}
}

2
include/osmium/geom/haversine.hpp
View File

@ -56,7 +56,7 @@ namespace osmium {
namespace haversine {
/// @brief Earth's quadratic mean radius for WGS84
constexpr double EARTH_RADIUS_IN_METERS = 6372797.560856;
constexpr const double EARTH_RADIUS_IN_METERS = 6372797.560856;
/**
* Calculate distance in meters between two sets of coordinates.

38
include/osmium/geom/ogr.hpp
View File

@ -62,33 +62,34 @@ namespace osmium {
public:
typedef std::unique_ptr<OGRPoint> point_type;
typedef std::unique_ptr<OGRLineString> linestring_type;
typedef std::unique_ptr<OGRPolygon> polygon_type;
typedef std::unique_ptr<OGRMultiPolygon> multipolygon_type;
typedef std::unique_ptr<OGRLinearRing> ring_type;
using point_type = std::unique_ptr<OGRPoint>;
using linestring_type = std::unique_ptr<OGRLineString>;
using polygon_type = std::unique_ptr<OGRPolygon>;
using multipolygon_type = std::unique_ptr<OGRMultiPolygon>;
using ring_type = std::unique_ptr<OGRLinearRing>;
private:
linestring_type m_linestring;
multipolygon_type m_multipolygon;
polygon_type m_polygon;
ring_type m_ring;
linestring_type m_linestring{nullptr};
multipolygon_type m_multipolygon{nullptr};
polygon_type m_polygon{nullptr};
ring_type m_ring{nullptr};
public:
OGRFactoryImpl() = default;
explicit OGRFactoryImpl(int /* srid */) {
}
/* Point */
point_type make_point(const osmium::geom::Coordinates& xy) const {
return point_type(new OGRPoint(xy.x, xy.y));
return point_type{new OGRPoint{xy.x, xy.y}};
}
/* LineString */
void linestring_start() {
m_linestring = std::unique_ptr<OGRLineString>(new OGRLineString());
m_linestring.reset(new OGRLineString{});
}
void linestring_add_location(const osmium::geom::Coordinates& xy) {
@ -97,13 +98,14 @@ namespace osmium {
}
linestring_type linestring_finish(size_t /* num_points */) {
assert(!!m_linestring);
return std::move(m_linestring);
}
/* Polygon */
void polygon_start() {
m_ring = std::unique_ptr<OGRLinearRing>(new OGRLinearRing());
m_ring.reset(new OGRLinearRing{});
}
void polygon_add_location(const osmium::geom::Coordinates& xy) {
@ -112,7 +114,7 @@ namespace osmium {
}
polygon_type polygon_finish(size_t /* num_points */) {
std::unique_ptr<OGRPolygon> polygon = std::unique_ptr<OGRPolygon>(new OGRPolygon());
auto polygon = std::unique_ptr<OGRPolygon>{new OGRPolygon{}};
polygon->addRingDirectly(m_ring.release());
return polygon;
}
@ -120,11 +122,11 @@ namespace osmium {
/* MultiPolygon */
void multipolygon_start() {
m_multipolygon.reset(new OGRMultiPolygon());
m_multipolygon.reset(new OGRMultiPolygon{});
}
void multipolygon_polygon_start() {
m_polygon.reset(new OGRPolygon());
m_polygon.reset(new OGRPolygon{});
}
void multipolygon_polygon_finish() {
@ -134,7 +136,7 @@ namespace osmium {
}
void multipolygon_outer_ring_start() {
m_ring.reset(new OGRLinearRing());
m_ring.reset(new OGRLinearRing{});
}
void multipolygon_outer_ring_finish() {
@ -144,7 +146,7 @@ namespace osmium {
}
void multipolygon_inner_ring_start() {
m_ring.reset(new OGRLinearRing());
m_ring.reset(new OGRLinearRing{});
}
void multipolygon_inner_ring_finish() {

22
include/osmium/geom/projection.hpp
View File

@ -70,15 +70,19 @@ namespace osmium {
public:
CRS(const std::string& crs) :
explicit CRS(const std::string& crs) :
m_crs(pj_init_plus(crs.c_str()), ProjCRSDeleter()) {
if (!m_crs) {
throw osmium::projection_error(std::string("creation of CRS failed: ") + pj_strerrno(*pj_get_errno_ref()));
throw osmium::projection_error(std::string{"creation of CRS failed: "} + pj_strerrno(*pj_get_errno_ref()));
}
}
CRS(int epsg) :
CRS(std::string("+init=epsg:") + std::to_string(epsg)) {
explicit CRS(const char* crs) :
CRS(std::string{crs}) {
}
explicit CRS(int epsg) :
CRS(std::string{"+init=epsg:"} + std::to_string(epsg)) {
}
/**
@ -127,13 +131,19 @@ namespace osmium {
public:
Projection(const std::string& proj_string) :
explicit Projection(const std::string& proj_string) :
m_epsg(-1),
m_proj_string(proj_string),
m_crs_user(proj_string) {
}
Projection(int epsg) :
explicit Projection(const char* proj_string) :
m_epsg(-1),
m_proj_string(proj_string),
m_crs_user(proj_string) {
}
explicit Projection(int epsg) :
m_epsg(epsg),
m_proj_string(std::string("+init=epsg:") + std::to_string(epsg)),
m_crs_user(epsg) {

14
include/osmium/geom/rapid_geojson.hpp
View File

@ -33,6 +33,8 @@ DEALINGS IN THE SOFTWARE.
*/
#include <cstddef>
#include <osmium/geom/coordinates.hpp>
#include <osmium/geom/factory.hpp>
@ -53,13 +55,13 @@ namespace osmium {
public:
typedef void point_type;
typedef void linestring_type;
typedef void polygon_type;
typedef void multipolygon_type;
typedef void ring_type;
using point_type = void;
using linestring_type = void;
using polygon_type = void;
using multipolygon_type = void;
using ring_type = void;
RapidGeoJSONFactoryImpl(TWriter& writer) :
RapidGeoJSONFactoryImpl(int /* srid */, TWriter& writer) :
m_writer(&writer) {
}

45
include/osmium/geom/tile.hpp
View File

@ -33,9 +33,12 @@ DEALINGS IN THE SOFTWARE.
*/
#include <cassert>
#include <cstdint>
#include <osmium/geom/coordinates.hpp>
#include <osmium/geom/mercator_projection.hpp>
#include <osmium/osm/location.hpp>
namespace osmium {
@ -57,24 +60,64 @@ namespace osmium {
*/
struct Tile {
/// x coordinate
uint32_t x;
/// y coordinate
uint32_t y;
/// Zoom level
uint32_t z;
/**
* Create a tile with the given zoom level and x any y tile
* coordinates.
*
* The values are not checked for validity.
*
* @pre @code zoom <= 30 && x < 2^zoom && y < 2^zoom @endcode
*/
explicit Tile(uint32_t zoom, uint32_t tx, uint32_t ty) noexcept : x(tx), y(ty), z(zoom) {
assert(zoom <= 30u);
assert(x < (1u << zoom));
assert(y < (1u << zoom));
}
/**
* Create a tile with the given zoom level that contains the given
* location.
*
* The values are not checked for validity.
*
* @pre @code location.valid() && zoom <= 30 @endcode
*/
explicit Tile(uint32_t zoom, const osmium::Location& location) :
z(zoom) {
osmium::geom::Coordinates c = lonlat_to_mercator(location);
assert(zoom <= 30u);
assert(location.valid());
const osmium::geom::Coordinates c = lonlat_to_mercator(location);
const int32_t n = 1 << zoom;
const double scale = detail::max_coordinate_epsg3857 * 2 / n;
x = uint32_t(detail::restrict_to_range<int32_t>(int32_t((c.x + detail::max_coordinate_epsg3857) / scale), 0, n-1));
y = uint32_t(detail::restrict_to_range<int32_t>(int32_t((detail::max_coordinate_epsg3857 - c.y) / scale), 0, n-1));
}
/**
* Check whether this tile is valid. For a tile to be valid the
* zoom level must be between 0 and 30 and the coordinates must
* each be between 0 and 2^zoom-1.
*/
bool valid() const noexcept {
if (z > 30) {
return false;
}
const uint32_t max = 1 << z;
return x < max && y < max;
}
}; // struct Tile
/// Tiles are equal if all their attributes are equal.
inline bool operator==(const Tile& a, const Tile& b) {
return a.z == b.z && a.x == b.x && a.y == b.y;
}

4
include/osmium/geom/util.hpp
View File

@ -44,11 +44,11 @@ namespace osmium {
*/
struct projection_error : public std::runtime_error {
projection_error(const std::string& what) :
explicit projection_error(const std::string& what) :
std::runtime_error(what) {
}
projection_error(const char* what) :
explicit projection_error(const char* what) :
std::runtime_error(what) {
}

30
include/osmium/geom/wkb.hpp
View File

@ -33,6 +33,7 @@ DEALINGS IN THE SOFTWARE.
*/
#include <algorithm>
#include <cstddef>
#include <cstdint>
#include <string>
@ -60,14 +61,13 @@ namespace osmium {
template <typename T>
inline void str_push(std::string& str, T data) {
size_t size = str.size();
str.resize(size + sizeof(T));
std::copy_n(reinterpret_cast<char*>(&data), sizeof(T), &str[size]);
str.append(reinterpret_cast<const char*>(&data), sizeof(T));
}
inline std::string convert_to_hex(const std::string& str) {
static const char* lookup_hex = "0123456789ABCDEF";
std::string out;
out.reserve(str.size() * 2);
for (char c : str) {
out += lookup_hex[(c >> 4) & 0xf];
@ -79,9 +79,6 @@ namespace osmium {
class WKBFactoryImpl {
/// OSM data always uses SRID 4326 (WGS84).
static constexpr uint32_t srid = 4326;
/**
* Type of WKB geometry.
* These definitions are from
@ -112,6 +109,7 @@ namespace osmium {
std::string m_data;
uint32_t m_points {0};
int m_srid;
wkb_type m_wkb_type;
out_type m_out_type;
@ -130,11 +128,11 @@ namespace osmium {
#endif
if (m_wkb_type == wkb_type::ewkb) {
str_push(str, type | wkbSRID);
str_push(str, srid);
str_push(str, m_srid);
} else {
str_push(str, type);
}
size_t offset = str.size();
const size_t offset = str.size();
if (add_length) {
str_push(str, static_cast<uint32_t>(0));
}
@ -142,18 +140,20 @@ namespace osmium {
}
void set_size(const size_t offset, const size_t size) {
*reinterpret_cast<uint32_t*>(&m_data[offset]) = static_cast_with_assert<uint32_t>(size);
uint32_t s = static_cast_with_assert<uint32_t>(size);
std::copy_n(reinterpret_cast<char*>(&s), sizeof(uint32_t), &m_data[offset]);
}
public:
typedef std::string point_type;
typedef std::string linestring_type;
typedef std::string polygon_type;
typedef std::string multipolygon_type;
typedef std::string ring_type;
using point_type = std::string;
using linestring_type = std::string;
using polygon_type = std::string;
using multipolygon_type = std::string;
using ring_type = std::string;
explicit WKBFactoryImpl(wkb_type wtype = wkb_type::wkb, out_type otype = out_type::binary) :
explicit WKBFactoryImpl(int srid, wkb_type wtype = wkb_type::wkb, out_type otype = out_type::binary) :
m_srid(srid),
m_wkb_type(wtype),
m_out_type(otype) {
}

37
include/osmium/geom/wkt.hpp
View File

@ -45,29 +45,44 @@ namespace osmium {
namespace geom {
enum class wkt_type : bool {
wkt = false,
ewkt = true
}; // enum class wkt_type
namespace detail {
class WKTFactoryImpl {
std::string m_srid_prefix;
std::string m_str;
int m_precision;
wkt_type m_wkt_type;
public:
typedef std::string point_type;
typedef std::string linestring_type;
typedef std::string polygon_type;
typedef std::string multipolygon_type;
typedef std::string ring_type;
using point_type = std::string;
using linestring_type = std::string;
using polygon_type = std::string;
using multipolygon_type = std::string;
using ring_type = std::string;
WKTFactoryImpl(int precision = 7) :
m_precision(precision) {
WKTFactoryImpl(int srid, int precision = 7, wkt_type wtype = wkt_type::wkt) :
m_srid_prefix(),
m_precision(precision),
m_wkt_type(wtype) {
if (m_wkt_type == wkt_type::ewkt) {
m_srid_prefix = "SRID=";
m_srid_prefix += std::to_string(srid);
m_srid_prefix += ';';
}
}
/* Point */
point_type make_point(const osmium::geom::Coordinates& xy) const {
std::string str {"POINT"};
std::string str {m_srid_prefix};
str += "POINT";
xy.append_to_string(str, '(', ' ', ')', m_precision);
return str;
}
@ -75,7 +90,8 @@ namespace osmium {
/* LineString */
void linestring_start() {
m_str = "LINESTRING(";
m_str = m_srid_prefix;
m_str += "LINESTRING(";
}
void linestring_add_location(const osmium::geom::Coordinates& xy) {
@ -97,7 +113,8 @@ namespace osmium {
/* MultiPolygon */
void multipolygon_start() {
m_str = "MULTIPOLYGON(";
m_str = m_srid_prefix;
m_str += "MULTIPOLYGON(";
}
void multipolygon_polygon_start() {

56
include/osmium/handler.hpp
View File

@ -33,56 +33,82 @@ DEALINGS IN THE SOFTWARE.
*/
#include <osmium/fwd.hpp>
namespace osmium {
class Area;
class Changeset;
class ChangesetDiscussion;
class InnerRing;
class Node;
class OSMObject;
class OuterRing;
class Relation;
class RelationMemberList;
class TagList;
class Way;
class WayNodeList;
/**
* @brief Osmium handlers provide callbacks for OSM objects
*/
namespace handler {
/**
* Handler base class. Never used directly. Derive your own class from
* this class and "overwrite" the functions. Your functions must be
* named the same, but don't have to be const or noexcept or take
* their argument as const.
*
* Usually you will overwrite the node(), way(), and relation()
* functions. If your program supports multipolygons, also the area()
* function. You can also use the osm_object() function which is
* called for all OSM objects (nodes, ways, relations, and areas)
* right before each of their specific callbacks is called.
*
* If you are working with changesets, implement the changeset()
* function.
*/
class Handler {
public:
void osm_object(const osmium::OSMObject&) const {
void osm_object(const osmium::OSMObject&) const noexcept {
}
void node(const osmium::Node&) const {
void node(const osmium::Node&) const noexcept {
}
void way(const osmium::Way&) const {
void way(const osmium::Way&) const noexcept {
}
void relation(const osmium::Relation&) const {
void relation(const osmium::Relation&) const noexcept {
}
void area(const osmium::Area&) const {
void area(const osmium::Area&) const noexcept {
}
void changeset(const osmium::Changeset&) const {
void changeset(const osmium::Changeset&) const noexcept {
}
void tag_list(const osmium::TagList&) const {
void tag_list(const osmium::TagList&) const noexcept {
}
void way_node_list(const osmium::WayNodeList&) const {
void way_node_list(const osmium::WayNodeList&) const noexcept {
}
void relation_member_list(const osmium::RelationMemberList&) const {
void relation_member_list(const osmium::RelationMemberList&) const noexcept {
}
void outer_ring(const osmium::OuterRing&) const {
void outer_ring(const osmium::OuterRing&) const noexcept {
}
void inner_ring(const osmium::InnerRing&) const {
void inner_ring(const osmium::InnerRing&) const noexcept {
}
void changeset_discussion(const osmium::ChangesetDiscussion&) const {
void changeset_discussion(const osmium::ChangesetDiscussion&) const noexcept {
}
void flush() const {
void flush() const noexcept {
}
}; // class Handler

2
include/osmium/handler/chain.hpp
View File

@ -67,7 +67,7 @@ namespace osmium {
template <typename... THandler>
class ChainHandler : public osmium::handler::Handler {
typedef std::tuple<THandler&...> handlers_type;
using handlers_type = std::tuple<THandler&...>;
handlers_type m_handlers;
template <int N, int SIZE, typename THandlers>

4
include/osmium/handler/check_order.hpp
View File

@ -51,11 +51,11 @@ namespace osmium {
*/
struct out_of_order_error : public std::runtime_error {
out_of_order_error(const std::string& what) :
explicit out_of_order_error(const std::string& what) :
std::runtime_error(what) {
}
out_of_order_error(const char* what) :
explicit out_of_order_error(const char* what) :
std::runtime_error(what) {
}

2
include/osmium/handler/disk_store.hpp
View File

@ -57,7 +57,7 @@ namespace osmium {
*/
class DiskStore : public osmium::handler::Handler {
typedef osmium::index::map::Map<unsigned_object_id_type, size_t> offset_index_type;
using offset_index_type = osmium::index::map::Map<unsigned_object_id_type, size_t>;
size_t m_offset = 0;
int m_data_fd;

16
include/osmium/handler/node_locations_for_ways.hpp
View File

@ -33,6 +33,7 @@ DEALINGS IN THE SOFTWARE.
*/
#include <limits>
#include <type_traits>
#include <osmium/handler.hpp>
@ -50,7 +51,7 @@ namespace osmium {
namespace handler {
typedef osmium::index::map::Dummy<osmium::unsigned_object_id_type, osmium::Location> dummy_type;
using dummy_type = osmium::index::map::Dummy<osmium::unsigned_object_id_type, osmium::Location>;
/**
* Handler to retrieve locations from nodes and add them to ways.
@ -69,8 +70,8 @@ namespace osmium {
public:
typedef TStoragePosIDs index_pos_type;
typedef TStorageNegIDs index_neg_type;
using index_pos_type = TStoragePosIDs;
using index_neg_type = TStorageNegIDs;
private:
@ -80,6 +81,8 @@ namespace osmium {
/// Object that handles the actual storage of the node locations (with negative IDs).
TStorageNegIDs& m_storage_neg;
osmium::unsigned_object_id_type m_last_id{0};
bool m_ignore_errors {false};
bool m_must_sort {false};
@ -115,7 +118,11 @@ namespace osmium {
* Store the location of the node in the storage.
*/
void node(const osmium::Node& node) {
if (node.positive_id() < m_last_id) {
m_must_sort = true;
}
m_last_id = node.positive_id();
const osmium::object_id_type id = node.id();
if (id >= 0) {
m_storage_pos.set(static_cast<osmium::unsigned_object_id_type>( id), node.location());
@ -144,6 +151,7 @@ namespace osmium {
m_storage_pos.sort();
m_storage_neg.sort();
m_must_sort = false;
m_last_id = std::numeric_limits<osmium::unsigned_object_id_type>::max();
}
bool error = false;
for (auto& node_ref : way.nodes()) {
@ -152,7 +160,7 @@ namespace osmium {
if (!node_ref.location()) {
error = true;
}
} catch (osmium::not_found&) {
} catch (const osmium::not_found&) {
error = true;
}
}

2
include/osmium/handler/object_relations.hpp
View File

@ -52,7 +52,7 @@ namespace osmium {
*/
class ObjectRelations : public osmium::handler::Handler {
typedef osmium::index::multimap::Multimap<unsigned_object_id_type, unsigned_object_id_type> index_type;
using index_type = osmium::index::multimap::Multimap<unsigned_object_id_type, unsigned_object_id_type>;
index_type& m_index_n2w;
index_type& m_index_n2r;

4
include/osmium/index/detail/create_map_with_fd.hpp
View File

@ -54,9 +54,9 @@ namespace osmium {
}
assert(config.size() > 1);
const std::string& filename = config[1];
int fd = ::open(filename.c_str(), O_CREAT | O_RDWR, 0644);
const int fd = ::open(filename.c_str(), O_CREAT | O_RDWR, 0644);
if (fd == -1) {
throw std::runtime_error(std::string("can't open file '") + filename + "': " + strerror(errno));
throw std::runtime_error(std::string("can't open file '") + filename + "': " + std::strerror(errno));
}
return new T(fd);
}

36
include/osmium/index/detail/mmap_vector_base.hpp
View File

@ -33,10 +33,13 @@ DEALINGS IN THE SOFTWARE.
*/
#include <algorithm>
#include <cassert>
#include <cstddef>
#include <new> // IWYU pragma: keep
#include <stdexcept>
#include <osmium/index/index.hpp>
#include <osmium/util/memory_mapping.hpp>
namespace osmium {
@ -63,22 +66,26 @@ namespace osmium {
mmap_vector_base(int fd, size_t capacity, size_t size = 0) :
m_size(size),
m_mapping(capacity, osmium::util::MemoryMapping::mapping_mode::write_shared, fd) {
assert(size <= capacity);
std::fill(data() + size, data() + capacity, osmium::index::empty_value<T>());
shrink_to_fit();
}
explicit mmap_vector_base(size_t capacity = mmap_vector_size_increment) :
m_size(0),
m_mapping(capacity) {
std::fill_n(data(), capacity, osmium::index::empty_value<T>());
}
~mmap_vector_base() noexcept = default;
typedef T value_type;
typedef T& reference;
typedef const T& const_reference;
typedef T* pointer;
typedef const T* const_pointer;
typedef T* iterator;
typedef const T* const_iterator;
using value_type = T;
using pointer = value_type*;
using const_pointer = const value_type*;
using reference = value_type&;
using const_reference = const value_type&;
using iterator = value_type*;
using const_iterator = const value_type*;
void close() {
m_mapping.unmap();
@ -105,6 +112,7 @@ namespace osmium {
}
T& operator[](size_t n) {
assert(n < m_size);
return data()[n];
}
@ -120,20 +128,21 @@ namespace osmium {
}
void shrink_to_fit() {
// XXX do something here
while (m_size > 0 && data()[m_size - 1] == osmium::index::empty_value<T>()) {
--m_size;
}
}
void push_back(const T& value) {
if (m_size >= capacity()) {
resize(m_size+1);
}
data()[m_size] = value;
++m_size;
data()[m_size-1] = value;
}
void reserve(size_t new_capacity) {
if (new_capacity > capacity()) {
const size_t old_capacity = capacity();
m_mapping.resize(new_capacity);
std::fill(data() + old_capacity, data() + new_capacity, osmium::index::empty_value<T>());
}
}
@ -141,9 +150,6 @@ namespace osmium {
if (new_size > capacity()) {
reserve(new_size + osmium::detail::mmap_vector_size_increment);
}
if (new_size > size()) {
new (data() + size()) T[new_size - size()];
}
m_size = new_size;
}

19
include/osmium/index/detail/mmap_vector_file.hpp
View File

@ -33,6 +33,11 @@ DEALINGS IN THE SOFTWARE.
*/
#include <algorithm>
#include <cstddef>
#include <stdexcept>
#include <string>
#include <osmium/index/detail/mmap_vector_base.hpp>
#include <osmium/index/detail/tmpfile.hpp>
#include <osmium/util/file.hpp>
@ -48,6 +53,16 @@ namespace osmium {
template <typename T>
class mmap_vector_file : public mmap_vector_base<T> {
size_t filesize(int fd) const {
const size_t size = osmium::util::file_size(fd);
if (size % sizeof(T) != 0) {
throw std::runtime_error("Index file has wrong size (must be multiple of " + std::to_string(sizeof(T)) + ").");
}
return size / sizeof(T);
}
public:
mmap_vector_file() :
@ -59,8 +74,8 @@ namespace osmium {
explicit mmap_vector_file(int fd) :
mmap_vector_base<T>(
fd,
osmium::util::file_size(fd) / sizeof(T),
osmium::util::file_size(fd) / sizeof(T)) {
std::max(osmium::detail::mmap_vector_size_increment, filesize(fd)),
filesize(fd)) {
}
~mmap_vector_file() noexcept = default;

18
include/osmium/index/detail/vector_map.hpp
View File

@ -55,10 +55,10 @@ namespace osmium {
public:
typedef TValue element_type;
typedef TVector vector_type;
typedef typename vector_type::iterator iterator;
typedef typename vector_type::const_iterator const_iterator;
using element_type = TValue;
using vector_type = TVector;
using iterator = typename vector_type::iterator;
using const_iterator = typename vector_type::const_iterator;
VectorBasedDenseMap() :
m_vector() {
@ -88,7 +88,7 @@ namespace osmium {
not_found_error(id);
}
return value;
} catch (std::out_of_range&) {
} catch (const std::out_of_range&) {
not_found_error(id);
}
}
@ -146,10 +146,10 @@ namespace osmium {
public:
typedef typename std::pair<TId, TValue> element_type;
typedef TVector<element_type> vector_type;
typedef typename vector_type::iterator iterator;
typedef typename vector_type::const_iterator const_iterator;
using element_type = typename std::pair<TId, TValue>;
using vector_type = TVector<element_type>;
using iterator = typename vector_type::iterator;
using const_iterator = typename vector_type::const_iterator;
private:

10
include/osmium/index/detail/vector_multimap.hpp
View File

@ -52,10 +52,10 @@ namespace osmium {
public:
typedef typename std::pair<TId, TValue> element_type;
typedef TVector<element_type> vector_type;
typedef typename vector_type::iterator iterator;
typedef typename vector_type::const_iterator const_iterator;
using element_type = typename std::pair<TId, TValue>;
using vector_type = TVector<element_type>;
using iterator = typename vector_type::iterator;
using const_iterator = typename vector_type::const_iterator;
private:
@ -127,7 +127,7 @@ namespace osmium {
}
void remove(const TId id, const TValue value) {
auto r = get_all(id);
const auto r = get_all(id);
for (auto it = r.first; it != r.second; ++it) {
if (it->second == value) {
it->second = 0;

4
include/osmium/index/index.hpp
View File

@ -49,11 +49,11 @@ namespace osmium {
*/
struct not_found : public std::runtime_error {
not_found(const std::string& what) :
explicit not_found(const std::string& what) :
std::runtime_error(what) {
}
not_found(const char* what) :
explicit not_found(const char* what) :
std::runtime_error(what) {
}

14
include/osmium/index/map.hpp
View File

@ -98,10 +98,10 @@ namespace osmium {
public:
/// The "key" type, usually osmium::unsigned_object_id_type.
typedef TId key_type;
using key_type = TId;
/// The "value" type, usually a Location or size_t.
typedef TValue value_type;
using value_type = TValue;
Map() = default;
@ -171,10 +171,10 @@ namespace osmium {
public:
typedef TId id_type;
typedef TValue value_type;
typedef osmium::index::map::Map<id_type, value_type> map_type;
typedef std::function<map_type*(const std::vector<std::string>&)> create_map_func;
using id_type = TId;
using value_type = TValue;
using map_type = osmium::index::map::Map<id_type, value_type>;
using create_map_func = std::function<map_type*(const std::vector<std::string>&)>;
private:
@ -207,7 +207,7 @@ namespace osmium {
}
bool has_map_type(const std::string& map_type_name) const {
return m_callbacks.count(map_type_name);
return m_callbacks.count(map_type_name) != 0;
}
std::vector<std::string> map_types() const {

3
include/osmium/index/map/sparse_mem_map.hpp
View File

@ -37,7 +37,6 @@ DEALINGS IN THE SOFTWARE.
#include <cstddef>
#include <iterator>
#include <map>
#include <stdexcept>
#include <vector>
#include <osmium/index/map.hpp>
@ -98,7 +97,7 @@ namespace osmium {
}
void dump_as_list(const int fd) final {
typedef typename std::map<TId, TValue>::value_type t;
using t = typename std::map<TId, TValue>::value_type;
std::vector<t> v;
v.reserve(m_elements.size());
std::copy(m_elements.cbegin(), m_elements.cend(), std::back_inserter(v));

8
include/osmium/index/multimap.hpp
View File

@ -52,7 +52,7 @@ namespace osmium {
static_assert(std::is_integral<TId>::value && std::is_unsigned<TId>::value, "TId template parameter for class Multimap must be unsigned integral type");
typedef typename std::pair<TId, TValue> element_type;
using element_type = typename std::pair<TId, TValue>;
Multimap(const Multimap&) = delete;
Multimap& operator=(const Multimap&) = delete;
@ -65,10 +65,10 @@ namespace osmium {
public:
/// The "key" type, usually osmium::unsigned_object_id_type.
typedef TId key_type;
using key_type = TId;
/// The "value" type, usually a Location or size_t.
typedef TValue value_type;
using value_type = TValue;
Multimap() = default;
@ -77,7 +77,7 @@ namespace osmium {
/// Set the field with id to value.
virtual void set(const TId id, const TValue value) = 0;
typedef element_type* iterator;
using iterator = element_type*;
// virtual std::pair<iterator, iterator> get_all(const TId id) const = 0;

14
include/osmium/index/multimap/hybrid.hpp
View File

@ -50,10 +50,10 @@ namespace osmium {
template <typename TId, typename TValue>
class HybridIterator {
typedef SparseMemArray<TId, TValue> main_map_type;
typedef SparseMemMultimap<TId, TValue> extra_map_type;
using main_map_type = SparseMemArray<TId, TValue>;
using extra_map_type = SparseMemMultimap<TId, TValue>;
typedef typename std::pair<TId, TValue> element_type;
using element_type = typename std::pair<TId, TValue>;
typename main_map_type::iterator m_begin_main;
typename main_map_type::iterator m_end_main;
@ -120,16 +120,16 @@ namespace osmium {
template <typename TId, typename TValue>
class Hybrid : public Multimap<TId, TValue> {
typedef SparseMemArray<TId, TValue> main_map_type;
typedef SparseMemMultimap<TId, TValue> extra_map_type;
using main_map_type = SparseMemArray<TId, TValue>;
using extra_map_type = SparseMemMultimap<TId, TValue>;
main_map_type m_main;
extra_map_type m_extra;
public:
typedef HybridIterator<TId, TValue> iterator;
typedef const HybridIterator<TId, TValue> const_iterator;
using iterator = HybridIterator<TId, TValue>;
using const_iterator = const HybridIterator<TId, TValue>;
Hybrid() :
m_main(),

11
include/osmium/index/multimap/sparse_mem_multimap.hpp
View File

@ -63,12 +63,11 @@ namespace osmium {
public:
typedef typename std::multimap<const TId, TValue> collection_type;
typedef typename collection_type::iterator iterator;
typedef typename collection_type::const_iterator const_iterator;
typedef typename collection_type::value_type value_type;
typedef typename std::pair<TId, TValue> element_type;
using collection_type = typename std::multimap<const TId, TValue>;
using iterator = typename collection_type::iterator;
using const_iterator = typename collection_type::const_iterator;
using value_type = typename collection_type::value_type;
using element_type = typename std::pair<TId, TValue>;
private:

3
include/osmium/io/any_input.hpp
View File

@ -45,8 +45,9 @@ DEALINGS IN THE SOFTWARE.
#include <osmium/io/any_compression.hpp> // IWYU pragma: export
#include <osmium/io/o5m_input.hpp> // IWYU pragma: export
#include <osmium/io/opl_input.hpp> // IWYU pragma: export
#include <osmium/io/pbf_input.hpp> // IWYU pragma: export
#include <osmium/io/xml_input.hpp> // IWYU pragma: export
#include <osmium/io/o5m_input.hpp> // IWYU pragma: export
#endif // OSMIUM_IO_ANY_INPUT_HPP

10
include/osmium/io/bzip2_compression.hpp
View File

@ -43,10 +43,9 @@ DEALINGS IN THE SOFTWARE.
*/
#include <cerrno>
#include <cstddef>
#include <cstdio>
#include <stdexcept>
#include <string>
#include <system_error>
#include <bzlib.h>
@ -55,6 +54,7 @@ DEALINGS IN THE SOFTWARE.
#endif
#include <osmium/io/compression.hpp>
#include <osmium/io/detail/read_write.hpp>
#include <osmium/io/error.hpp>
#include <osmium/io/file_compression.hpp>
#include <osmium/io/writer_options.hpp>
@ -109,7 +109,7 @@ namespace osmium {
explicit Bzip2Compressor(int fd, fsync sync) :
Compressor(sync),
m_file(fdopen(dup(fd), "wb")),
m_file(fdopen(::dup(fd), "wb")),
m_bzerror(BZ_OK),
m_bzfile(::BZ2_bzWriteOpen(&m_bzerror, m_file, 6, 0, 0)) {
if (!m_bzfile) {
@ -165,7 +165,7 @@ namespace osmium {
explicit Bzip2Decompressor(int fd) :
Decompressor(),
m_file(fdopen(dup(fd), "rb")),
m_file(fdopen(::dup(fd), "rb")),
m_bzerror(BZ_OK),
m_bzfile(::BZ2_bzReadOpen(&m_bzerror, m_file, 0, 0, nullptr, 0)) {
if (!m_bzfile) {
@ -215,6 +215,8 @@ namespace osmium {
buffer.resize(static_cast<std::string::size_type>(nread));
}
set_offset(size_t(ftell(m_file)));
return buffer;
}

41
include/osmium/io/compression.hpp
View File

@ -33,11 +33,12 @@ DEALINGS IN THE SOFTWARE.
*/
#include <atomic>
#include <cerrno>
#include <cstddef>
#include <functional>
#include <map>
#include <memory>
#include <stdexcept>
#include <string>
#include <system_error>
#include <tuple>
@ -54,6 +55,7 @@ DEALINGS IN THE SOFTWARE.
#include <osmium/io/file_compression.hpp>
#include <osmium/io/writer_options.hpp>
#include <osmium/util/compatibility.hpp>
#include <osmium/util/file.hpp>
namespace osmium {
@ -86,6 +88,9 @@ namespace osmium {
class Decompressor {
std::atomic<size_t> m_file_size {0};
std::atomic<size_t> m_offset {0};
public:
static constexpr unsigned int input_buffer_size = 1024 * 1024;
@ -105,6 +110,22 @@ namespace osmium {
virtual void close() = 0;
size_t file_size() const noexcept {
return m_file_size;
}
void set_file_size(size_t size) noexcept {
m_file_size = size;
}
size_t offset() const noexcept {
return m_offset;
}
void set_offset(size_t offset) noexcept {
m_offset = offset;
}
}; // class Decompressor
/**
@ -118,16 +139,16 @@ namespace osmium {
public:
typedef std::function<osmium::io::Compressor*(int, fsync)> create_compressor_type;
typedef std::function<osmium::io::Decompressor*(int)> create_decompressor_type_fd;
typedef std::function<osmium::io::Decompressor*(const char*, size_t)> create_decompressor_type_buffer;
using create_compressor_type = std::function<osmium::io::Compressor*(int, fsync)>;
using create_decompressor_type_fd = std::function<osmium::io::Decompressor*(int)>;
using create_decompressor_type_buffer = std::function<osmium::io::Decompressor*(const char*, size_t)>;
private:
typedef std::map<const osmium::io::file_compression,
using compression_map_type = std::map<const osmium::io::file_compression,
std::tuple<create_compressor_type,
create_decompressor_type_fd,
create_decompressor_type_buffer>> compression_map_type;
create_decompressor_type_buffer>>;
compression_map_type m_callbacks;
@ -182,7 +203,9 @@ namespace osmium {
auto it = m_callbacks.find(compression);
if (it != m_callbacks.end()) {
return std::unique_ptr<osmium::io::Decompressor>(std::get<1>(it->second)(fd));
auto p = std::unique_ptr<osmium::io::Decompressor>(std::get<1>(it->second)(fd));
p->set_file_size(osmium::util::file_size(fd));
return p;
}
error(compression);
@ -241,6 +264,7 @@ namespace osmium {
int m_fd;
const char *m_buffer;
size_t m_buffer_size;
size_t m_offset = 0;
public:
@ -284,6 +308,9 @@ namespace osmium {
buffer.resize(std::string::size_type(nread));
}
m_offset += buffer.size();
set_offset(m_offset);
return buffer;
}

181
include/osmium/io/detail/debug_output_format.hpp
View File

@ -34,29 +34,35 @@ DEALINGS IN THE SOFTWARE.
*/
#include <cinttypes>
#include <cstddef>
#include <cstdint>
#include <cstdio>
#include <future>
#include <cmath>
#include <cstring>
#include <iterator>
#include <memory>
#include <string>
#include <thread>
#include <utility>
#include <boost/crc.hpp>
#include <osmium/io/detail/output_format.hpp>
#include <osmium/io/detail/queue_util.hpp>
#include <osmium/io/detail/string_util.hpp>
#include <osmium/io/file.hpp>
#include <osmium/io/file_format.hpp>
#include <osmium/io/header.hpp>
#include <osmium/memory/buffer.hpp>
#include <osmium/memory/collection.hpp>
#include <osmium/memory/item_iterator.hpp>
#include <osmium/osm/box.hpp>
#include <osmium/osm/changeset.hpp>
#include <osmium/osm/crc.hpp>
#include <osmium/osm/item_type.hpp>
#include <osmium/osm/location.hpp>
#include <osmium/osm/node.hpp>
#include <osmium/osm/node_ref.hpp>
#include <osmium/osm/object.hpp>
#include <osmium/osm/relation.hpp>
#include <osmium/osm/tag.hpp>
#include <osmium/osm/timestamp.hpp>
#include <osmium/osm/types.hpp>
#include <osmium/osm/way.hpp>
#include <osmium/thread/pool.hpp>
#include <osmium/util/minmax.hpp>
@ -66,8 +72,6 @@ namespace osmium {
namespace io {
class File;
namespace detail {
constexpr const char* color_bold = "\x1b[1m";
@ -80,6 +84,10 @@ namespace osmium {
constexpr const char* color_magenta = "\x1b[35m";
constexpr const char* color_cyan = "\x1b[36m";
constexpr const char* color_white = "\x1b[37m";
constexpr const char* color_backg_red = "\x1b[41m";
constexpr const char* color_backg_green = "\x1b[42m";
constexpr const char* color_reset = "\x1b[0m";
struct debug_output_options {
@ -90,6 +98,11 @@ namespace osmium {
/// Output with ANSI colors?
bool use_color;
/// Add CRC32 checksum to each object?
bool add_crc32;
/// Write in form of a diff file?
bool format_as_diff;
};
/**
@ -102,16 +115,47 @@ namespace osmium {
const char* m_utf8_prefix = "";
const char* m_utf8_suffix = "";
char m_diff_char = '\0';
void append_encoded_string(const char* data) {
append_debug_encoded_string(*m_out, data, m_utf8_prefix, m_utf8_suffix);
}
template <typename... TArgs>
void output_formatted(const char* format, TArgs&&... args) {
append_printf_formatted_string(*m_out, format, std::forward<TArgs>(args)...);
}
void write_color(const char* color) {
if (m_options.use_color) {
*m_out += color;
}
}
void write_diff() {
if (!m_diff_char) {
return;
}
if (m_options.use_color) {
if (m_diff_char == '-') {
*m_out += color_backg_red;
*m_out += color_white;
*m_out += color_bold;
*m_out += '-';
*m_out += color_reset;
return;
} else if (m_diff_char == '+') {
*m_out += color_backg_green;
*m_out += color_white;
*m_out += color_bold;
*m_out += '+';
*m_out += color_reset;
return;
}
}
*m_out += m_diff_char;
}
void write_string(const char* string) {
*m_out += '"';
write_color(color_blue);
@ -121,6 +165,7 @@ namespace osmium {
}
void write_object_type(const char* object_type, bool visible = true) {
write_diff();
if (visible) {
write_color(color_bold);
} else {
@ -132,6 +177,7 @@ namespace osmium {
}
void write_fieldname(const char* name) {
write_diff();
*m_out += " ";
write_color(color_cyan);
*m_out += name;
@ -161,7 +207,9 @@ namespace osmium {
void write_timestamp(const osmium::Timestamp& timestamp) {
if (timestamp.valid()) {
*m_out += timestamp.to_iso();
output_formatted(" (%d)", timestamp.seconds_since_epoch());
*m_out += " (";
output_int(timestamp.seconds_since_epoch());
*m_out += ')';
} else {
write_error("NOT SET");
}
@ -169,37 +217,47 @@ namespace osmium {
}
void write_meta(const osmium::OSMObject& object) {
output_formatted("%" PRId64 "\n", object.id());
output_int(object.id());
*m_out += '\n';
if (m_options.add_metadata) {
write_fieldname("version");
output_formatted(" %d", object.version());
*m_out += " ";
output_int(object.version());
if (object.visible()) {
*m_out += " visible\n";
} else {
write_error(" deleted\n");
}
write_fieldname("changeset");
output_formatted("%d\n", object.changeset());
output_int(object.changeset());
*m_out += '\n';
write_fieldname("timestamp");
write_timestamp(object.timestamp());
write_fieldname("user");
output_formatted(" %d ", object.uid());
*m_out += " ";
output_int(object.uid());
*m_out += ' ';
write_string(object.user());
*m_out += '\n';
}
}
void write_tags(const osmium::TagList& tags, const char* padding="") {
if (!tags.empty()) {
if (tags.empty()) {
return;
}
write_fieldname("tags");
*m_out += padding;
output_formatted(" %d\n", tags.size());
*m_out += " ";
output_int(tags.size());
*m_out += '\n';
osmium::max_op<size_t> max;
for (const auto& tag : tags) {
max.update(std::strlen(tag.key()));
}
for (const auto& tag : tags) {
write_diff();
*m_out += " ";
write_string(tag.key());
auto spacing = max() - std::strlen(tag.key());
@ -211,11 +269,11 @@ namespace osmium {
*m_out += '\n';
}
}
}
void write_location(const osmium::Location& location) {
write_fieldname("lon/lat");
output_formatted(" %.7f,%.7f", location.lon_without_check(), location.lat_without_check());
*m_out += " ";
location.as_string_without_check(std::back_inserter(*m_out));
if (!location.valid()) {
write_error(" INVALID LOCATION!");
}
@ -230,13 +288,30 @@ namespace osmium {
}
const auto& bl = box.bottom_left();
const auto& tr = box.top_right();
output_formatted("%.7f,%.7f %.7f,%.7f", bl.lon_without_check(), bl.lat_without_check(), tr.lon_without_check(), tr.lat_without_check());
bl.as_string(std::back_inserter(*m_out));
*m_out += ' ';
tr.as_string(std::back_inserter(*m_out));
if (!box.valid()) {
write_error(" INVALID BOX!");
}
*m_out += '\n';
}
template <typename T>
void write_crc32(const T& object) {
write_fieldname("crc32");
osmium::CRC<boost::crc_32_type> crc32;
crc32.update(object);
output_formatted(" %x\n", crc32().checksum());
}
void write_crc32(const osmium::Changeset& object) {
write_fieldname("crc32");
osmium::CRC<boost::crc_32_type> crc32;
crc32.update(object);
output_formatted(" %x\n", crc32().checksum());
}
public:
DebugOutputBlock(osmium::memory::Buffer&& buffer, const debug_output_options& options) :
@ -265,6 +340,8 @@ namespace osmium {
}
void node(const osmium::Node& node) {
m_diff_char = m_options.format_as_diff ? node.diff_as_char() : '\0';
write_object_type("node", node.visible());
write_meta(node);
@ -274,17 +351,24 @@ namespace osmium {
write_tags(node.tags());
if (m_options.add_crc32) {
write_crc32(node);
}
*m_out += '\n';
}
void way(const osmium::Way& way) {
m_diff_char = m_options.format_as_diff ? way.diff_as_char() : '\0';
write_object_type("way", way.visible());
write_meta(way);
write_tags(way.tags());
write_fieldname("nodes");
output_formatted(" %d", way.nodes().size());
*m_out += " ";
output_int(way.nodes().size());
if (way.nodes().size() < 2) {
write_error(" LESS THAN 2 NODES!\n");
} else if (way.nodes().size() > 2000) {
@ -295,32 +379,45 @@ namespace osmium {
*m_out += " (open)\n";
}
int width = int(log10(way.nodes().size())) + 1;
const int width = int(std::log10(way.nodes().size())) + 1;
int n = 0;
for (const auto& node_ref : way.nodes()) {
write_diff();
write_counter(width, n++);
output_formatted("%10" PRId64, node_ref.ref());
if (node_ref.location().valid()) {
output_formatted(" (%.7f,%.7f)", node_ref.location().lon_without_check(), node_ref.location().lat_without_check());
*m_out += " (";
node_ref.location().as_string(std::back_inserter(*m_out));
*m_out += ')';
}
*m_out += '\n';
}
if (m_options.add_crc32) {
write_crc32(way);
}
*m_out += '\n';
}
void relation(const osmium::Relation& relation) {
static const char* short_typename[] = { "node", "way ", "rel " };
m_diff_char = m_options.format_as_diff ? relation.diff_as_char() : '\0';
write_object_type("relation", relation.visible());
write_meta(relation);
write_tags(relation.tags());
write_fieldname("members");
output_formatted(" %d\n", relation.members().size());
*m_out += " ";
output_int(relation.members().size());
*m_out += '\n';
int width = int(log10(relation.members().size())) + 1;
const int width = int(std::log10(relation.members().size())) + 1;
int n = 0;
for (const auto& member : relation.members()) {
write_diff();
write_counter(width, n++);
*m_out += short_typename[item_type_to_nwr_index(member.type())];
output_formatted(" %10" PRId64 " ", member.ref());
@ -328,15 +425,20 @@ namespace osmium {
*m_out += '\n';
}
if (m_options.add_crc32) {
write_crc32(relation);
}
*m_out += '\n';
}
void changeset(const osmium::Changeset& changeset) {
write_object_type("changeset");
output_formatted("%d\n", changeset.id());
output_int(changeset.id());
*m_out += '\n';
write_fieldname("num changes");
output_formatted("%d", changeset.num_changes());
output_int(changeset.num_changes());
if (changeset.num_changes() == 0) {
write_error(" NO CHANGES!");
}
@ -355,7 +457,9 @@ namespace osmium {
}
write_fieldname("user");
output_formatted(" %d ", changeset.uid());
*m_out += " ";
output_int(changeset.uid());
*m_out += ' ';
write_string(changeset.user());
*m_out += '\n';
@ -364,9 +468,11 @@ namespace osmium {
if (changeset.num_comments() > 0) {
write_fieldname("comments");
output_formatted(" %d\n", changeset.num_comments());
*m_out += " ";
output_int(changeset.num_comments());
*m_out += '\n';
int width = int(log10(changeset.num_comments())) + 1;
const int width = int(std::log10(changeset.num_comments())) + 1;
int n = 0;
for (const auto& comment : changeset.discussion()) {
write_counter(width, n++);
@ -376,7 +482,8 @@ namespace osmium {
output_formatted(" %*s", width, "");
write_comment_field("user");
output_formatted("%d ", comment.uid());
output_int(comment.uid());
*m_out += ' ';
write_string(comment.user());
output_formatted("\n %*s", width, "");
@ -386,6 +493,10 @@ namespace osmium {
}
}
if (m_options.add_crc32) {
write_crc32(changeset);
}
*m_out += '\n';
}
@ -414,6 +525,8 @@ namespace osmium {
m_options() {
m_options.add_metadata = file.is_not_false("add_metadata");
m_options.use_color = file.is_true("color");
m_options.add_crc32 = file.is_true("add_crc32");
m_options.format_as_diff = file.is_true("diff");
}
DebugOutputFormat(const DebugOutputFormat&) = delete;
@ -422,6 +535,10 @@ namespace osmium {
~DebugOutputFormat() noexcept final = default;
void write_header(const osmium::io::Header& header) final {
if (m_options.format_as_diff) {
return;
}
std::string out;
if (m_options.use_color) {
@ -439,9 +556,9 @@ namespace osmium {
out += '\n';
for (const auto& box : header.boxes()) {
out += " ";
box.bottom_left().as_string(std::back_inserter(out), ',');
out += " ";
box.top_right().as_string(std::back_inserter(out), ',');
box.bottom_left().as_string(std::back_inserter(out));
out += ' ';
box.top_right().as_string(std::back_inserter(out));
out += '\n';
}
write_fieldname(out, "options");

12
include/osmium/io/detail/input_format.hpp
View File

@ -38,11 +38,11 @@ DEALINGS IN THE SOFTWARE.
#include <future>
#include <map>
#include <memory>
#include <stdexcept>
#include <string>
#include <utility>
#include <osmium/io/detail/queue_util.hpp>
#include <osmium/io/error.hpp>
#include <osmium/io/file.hpp>
#include <osmium/io/file_format.hpp>
#include <osmium/io/header.hpp>
@ -154,18 +154,14 @@ namespace osmium {
public:
typedef std::function<
std::unique_ptr<Parser>(
future_string_queue_type&,
using create_parser_type = std::function<std::unique_ptr<Parser>(future_string_queue_type&,
future_buffer_queue_type&,
std::promise<osmium::io::Header>& header_promise,
osmium::osm_entity_bits::type read_which_entities
)
> create_parser_type;
osmium::osm_entity_bits::type read_which_entities)>;
private:
typedef std::map<osmium::io::file_format, create_parser_type> map_type;
using map_type = std::map<osmium::io::file_format, create_parser_type>;
map_type m_callbacks;

13
include/osmium/io/detail/o5m_input_format.hpp
View File

@ -42,9 +42,9 @@ DEALINGS IN THE SOFTWARE.
#include <string>
#include <utility>
#include <protozero/exception.hpp>
#include <protozero/varint.hpp>
#include <osmium/builder/builder.hpp>
#include <osmium/builder/osm_object_builder.hpp>
#include <osmium/io/detail/input_format.hpp>
#include <osmium/io/detail/queue_util.hpp>
@ -52,19 +52,26 @@ DEALINGS IN THE SOFTWARE.
#include <osmium/io/file_format.hpp>
#include <osmium/io/header.hpp>
#include <osmium/memory/buffer.hpp>
#include <osmium/osm.hpp>
#include <osmium/osm/box.hpp>
#include <osmium/osm/entity_bits.hpp>
#include <osmium/osm/item_type.hpp>
#include <osmium/osm/location.hpp>
#include <osmium/osm/node.hpp>
#include <osmium/osm/object.hpp>
#include <osmium/osm/relation.hpp>
#include <osmium/osm/timestamp.hpp>
#include <osmium/osm/types.hpp>
#include <osmium/osm/way.hpp>
#include <osmium/thread/util.hpp>
#include <osmium/util/cast.hpp>
#include <osmium/util/delta.hpp>
namespace osmium {
namespace builder {
class Builder;
} // namespace builder
/**
* Exception thrown when the o5m deocder failed. The exception contains
* (if available) information about the place where the error happened
@ -529,7 +536,7 @@ namespace osmium {
uint64_t length = 0;
try {
length = protozero::decode_varint(&m_data, m_end);
} catch (protozero::end_of_buffer_exception&) {
} catch (const protozero::end_of_buffer_exception&) {
throw o5m_error("premature end of file");
}

156
include/osmium/io/detail/opl_input_format.hpp Normal file
View File

@ -0,0 +1,156 @@
#ifndef OSMIUM_IO_DETAIL_OPL_INPUT_FORMAT_HPP
#define OSMIUM_IO_DETAIL_OPL_INPUT_FORMAT_HPP
/*
This file is part of Osmium (http://osmcode.org/libosmium).
Copyright 2013-2016 Jochen Topf <jochen@topf.org> and others (see README).
Boost Software License - Version 1.0 - August 17th, 2003
Permission is hereby granted, free of charge, to any person or organization
obtaining a copy of the software and accompanying documentation covered by
this license (the "Software") to use, reproduce, display, distribute,
execute, and transmit the Software, and to prepare derivative works of the
Software, and to permit third-parties to whom the Software is furnished to
do so, all subject to the following:
The copyright notices in the Software and this entire statement, including
the above license grant, this restriction and the following disclaimer,
must be included in all copies of the Software, in whole or in part, and
all derivative works of the Software, unless such copies or derivative
works are solely in the form of machine-executable object code generated by
a source language processor.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT
SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE
FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE,
ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
*/
#include <cstdlib>
#include <future>
#include <memory>
#include <string>
#include <utility>
#include <osmium/io/detail/input_format.hpp>
#include <osmium/io/detail/opl_parser_functions.hpp>
#include <osmium/io/detail/queue_util.hpp>
#include <osmium/io/file_format.hpp>
#include <osmium/io/header.hpp>
#include <osmium/memory/buffer.hpp>
#include <osmium/osm/entity_bits.hpp>
#include <osmium/thread/util.hpp>
namespace osmium {
namespace io {
namespace detail {
class OPLParser : public Parser {
osmium::memory::Buffer m_buffer{1024*1024};
const char* m_data = nullptr;
uint64_t m_line_count = 0;
void maybe_flush() {
if (m_buffer.committed() > 800*1024) {
osmium::memory::Buffer buffer{1024*1024};
using std::swap;
swap(m_buffer, buffer);
send_to_output_queue(std::move(buffer));
}
}
void parse_line() {
if (opl_parse_line(m_line_count, m_data, m_buffer, read_types())) {
maybe_flush();
}
++m_line_count;
}
public:
OPLParser(future_string_queue_type& input_queue,
future_buffer_queue_type& output_queue,
std::promise<osmium::io::Header>& header_promise,
osmium::osm_entity_bits::type read_types) :
Parser(input_queue, output_queue, header_promise, read_types) {
set_header_value(osmium::io::Header{});
}
~OPLParser() noexcept final = default;
void run() final {
osmium::thread::set_thread_name("_osmium_opl_in");
std::string rest;
while (!input_done()) {
std::string input = get_input();
std::string::size_type ppos = 0;
if (!rest.empty()) {
ppos = input.find('\n');
if (ppos == std::string::npos) {
rest.append(input);
continue;
}
rest.append(input.substr(0, ppos));
m_data = rest.data();
parse_line();
rest.clear();
}
std::string::size_type pos = input.find('\n', ppos);
while (pos != std::string::npos) {
m_data = &input[ppos];
input[pos] = '\0';
parse_line();
ppos = pos + 1;
if (ppos >= input.size()) {
break;
}
pos = input.find('\n', ppos);
}
rest = input.substr(ppos);
}
if (m_buffer.committed() > 0) {
send_to_output_queue(std::move(m_buffer));
}
}
}; // class OPLParser
// we want the register_parser() function to run, setting
// the variable is only a side-effect, it will never be used
const bool registered_opl_parser = ParserFactory::instance().register_parser(
file_format::opl,
[](future_string_queue_type& input_queue,
future_buffer_queue_type& output_queue,
std::promise<osmium::io::Header>& header_promise,
osmium::osm_entity_bits::type read_which_entities) {
return std::unique_ptr<Parser>(new OPLParser(input_queue, output_queue, header_promise, read_which_entities));
});
// dummy function to silence the unused variable warning from above
inline bool get_registered_opl_parser() noexcept {
return registered_opl_parser;
}
} // namespace detail
} // namespace io
} // namespace osmium
#endif // OSMIUM_IO_DETAIL_OPL_INPUT_FORMAT_HPP

180
include/osmium/io/detail/opl_output_format.hpp
View File

@ -33,26 +33,26 @@ DEALINGS IN THE SOFTWARE.
*/
#include <cinttypes>
#include <cstddef>
#include <cstdint>
#include <cstdio>
#include <future>
#include <iterator>
#include <memory>
#include <string>
#include <thread>
#include <utility>
#include <osmium/io/detail/output_format.hpp>
#include <osmium/io/detail/queue_util.hpp>
#include <osmium/io/detail/string_util.hpp>
#include <osmium/io/file.hpp>
#include <osmium/io/file_format.hpp>
#include <osmium/memory/buffer.hpp>
#include <osmium/memory/collection.hpp>
#include <osmium/memory/item_iterator.hpp>
#include <osmium/osm/box.hpp>
#include <osmium/osm/changeset.hpp>
#include <osmium/osm/item_type.hpp>
#include <osmium/osm/location.hpp>
#include <osmium/osm/node.hpp>
#include <osmium/osm/node_ref.hpp>
#include <osmium/osm/object.hpp>
#include <osmium/osm/relation.hpp>
#include <osmium/osm/tag.hpp>
@ -65,8 +65,6 @@ namespace osmium {
namespace io {
class File;
namespace detail {
struct opl_output_options {
@ -74,6 +72,12 @@ namespace osmium {
/// Should metadata of objects be added?
bool add_metadata;
/// Should node locations be added to ways?
bool locations_on_ways;
/// Write in form of a diff file?
bool format_as_diff;
};
/**
@ -87,38 +91,71 @@ namespace osmium {
osmium::io::detail::append_utf8_encoded_string(*m_out, data);
}
void write_meta(const osmium::OSMObject& object) {
output_formatted("%" PRId64, object.id());
if (m_options.add_metadata) {
output_formatted(" v%d d", object.version());
*m_out += (object.visible() ? 'V' : 'D');
output_formatted(" c%d t", object.changeset());
*m_out += object.timestamp().to_iso();
output_formatted(" i%d u", object.uid());
append_encoded_string(object.user());
void write_field_int(char c, int64_t value) {
*m_out += c;
output_int(value);
}
void write_field_timestamp(char c, const osmium::Timestamp& timestamp) {
*m_out += c;
*m_out += timestamp.to_iso();
}
void write_tags(const osmium::TagList& tags) {
*m_out += " T";
bool first = true;
for (const auto& tag : object.tags()) {
if (first) {
first = false;
} else {
*m_out += ',';
if (tags.empty()) {
return;
}
append_encoded_string(tag.key());
auto it = tags.begin();
append_encoded_string(it->key());
*m_out += '=';
append_encoded_string(tag.value());
append_encoded_string(it->value());
for (++it; it != tags.end(); ++it) {
*m_out += ',';
append_encoded_string(it->key());
*m_out += '=';
append_encoded_string(it->value());
}
}
void write_meta(const osmium::OSMObject& object) {
output_int(object.id());
if (m_options.add_metadata) {
*m_out += ' ';
write_field_int('v', object.version());
*m_out += " d";
*m_out += (object.visible() ? 'V' : 'D');
*m_out += ' ';
write_field_int('c', object.changeset());
*m_out += ' ';
write_field_timestamp('t', object.timestamp());
*m_out += ' ';
write_field_int('i', object.uid());
*m_out += " u";
append_encoded_string(object.user());
}
write_tags(object.tags());
}
void write_location(const osmium::Location& location, const char x, const char y) {
if (location) {
output_formatted(" %c%.7f %c%.7f", x, location.lon_without_check(), y, location.lat_without_check());
} else {
*m_out += ' ';
*m_out += x;
if (location) {
osmium::detail::append_location_coordinate_to_string(std::back_inserter(*m_out), location.x());
}
*m_out += ' ';
*m_out += y;
if (location) {
osmium::detail::append_location_coordinate_to_string(std::back_inserter(*m_out), location.y());
}
}
void write_diff(const osmium::OSMObject& object) {
if (m_options.format_as_diff) {
*m_out += object.diff_as_char();
}
}
@ -148,70 +185,93 @@ namespace osmium {
}
void node(const osmium::Node& node) {
write_diff(node);
*m_out += 'n';
write_meta(node);
write_location(node.location(), 'x', 'y');
*m_out += '\n';
}
void write_field_ref(const osmium::NodeRef& node_ref) {
write_field_int('n', node_ref.ref());
*m_out += 'x';
if (node_ref.location()) {
node_ref.location().as_string(std::back_inserter(*m_out), 'y');
} else {
*m_out += 'y';
}
}
void way(const osmium::Way& way) {
write_diff(way);
*m_out += 'w';
write_meta(way);
*m_out += " N";
bool first = true;
for (const auto& node_ref : way.nodes()) {
if (first) {
first = false;
} else {
if (!way.nodes().empty()) {
auto it = way.nodes().begin();
if (m_options.locations_on_ways) {
write_field_ref(*it);
for (++it; it != way.nodes().end(); ++it) {
*m_out += ',';
write_field_ref(*it);
}
output_formatted("n%" PRId64, node_ref.ref());
} else {
write_field_int('n', it->ref());
for (++it; it != way.nodes().end(); ++it) {
*m_out += ',';
write_field_int('n', it->ref());
}
}
}
*m_out += '\n';
}
void relation_member(const osmium::RelationMember& member) {
*m_out += item_type_to_char(member.type());
output_int(member.ref());
*m_out += '@';
append_encoded_string(member.role());
}
void relation(const osmium::Relation& relation) {
write_diff(relation);
*m_out += 'r';
write_meta(relation);
*m_out += " M";
bool first = true;
for (const auto& member : relation.members()) {
if (first) {
first = false;
} else {
if (!relation.members().empty()) {
auto it = relation.members().begin();
relation_member(*it);
for (++it; it != relation.members().end(); ++it) {
*m_out += ',';
relation_member(*it);
}
*m_out += item_type_to_char(member.type());
output_formatted("%" PRId64 "@", member.ref());
append_encoded_string(member.role());
}
*m_out += '\n';
}
void changeset(const osmium::Changeset& changeset) {
output_formatted("c%d k%d s", changeset.id(), changeset.num_changes());
*m_out += changeset.created_at().to_iso();
*m_out += " e";
*m_out += changeset.closed_at().to_iso();
output_formatted(" d%d i%d u", changeset.num_comments(), changeset.uid());
write_field_int('c', changeset.id());
*m_out += ' ';
write_field_int('k', changeset.num_changes());
*m_out += ' ';
write_field_timestamp('s', changeset.created_at());
*m_out += ' ';
write_field_timestamp('e', changeset.closed_at());
*m_out += ' ';
write_field_int('d', changeset.num_comments());
*m_out += ' ';
write_field_int('i', changeset.uid());
*m_out += " u";
append_encoded_string(changeset.user());
write_location(changeset.bounds().bottom_left(), 'x', 'y');
write_location(changeset.bounds().top_right(), 'X', 'Y');
*m_out += " T";
bool first = true;
for (const auto& tag : changeset.tags()) {
if (first) {
first = false;
} else {
*m_out += ',';
}
append_encoded_string(tag.key());
*m_out += '=';
append_encoded_string(tag.value());
}
write_tags(changeset.tags());
*m_out += '\n';
}
@ -227,6 +287,8 @@ namespace osmium {
OutputFormat(output_queue),
m_options() {
m_options.add_metadata = file.is_not_false("add_metadata");
m_options.locations_on_ways = file.is_true("locations_on_ways");
m_options.format_as_diff = file.is_true("diff");
}
OPLOutputFormat(const OPLOutputFormat&) = delete;

747
include/osmium/io/detail/opl_parser_functions.hpp Normal file
View File

@ -0,0 +1,747 @@
#ifndef OSMIUM_IO_DETAIL_OPL_PARSER_FUNCTIONS_HPP
#define OSMIUM_IO_DETAIL_OPL_PARSER_FUNCTIONS_HPP
/*
This file is part of Osmium (http://osmcode.org/libosmium).
Copyright 2013-2016 Jochen Topf <jochen@topf.org> and others (see README).
Boost Software License - Version 1.0 - August 17th, 2003
Permission is hereby granted, free of charge, to any person or organization
obtaining a copy of the software and accompanying documentation covered by
this license (the "Software") to use, reproduce, display, distribute,
execute, and transmit the Software, and to prepare derivative works of the
Software, and to permit third-parties to whom the Software is furnished to
do so, all subject to the following:
The copyright notices in the Software and this entire statement, including
the above license grant, this restriction and the following disclaimer,
must be included in all copies of the Software, in whole or in part, and
all derivative works of the Software, unless such copies or derivative
works are solely in the form of machine-executable object code generated by
a source language processor.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT
SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE
FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE,
ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
*/
#include <cstdint>
#include <cstdlib>
#include <iterator>
#include <limits>
#include <stdexcept>
#include <string>
#include <utf8.h>
#include <osmium/builder/osm_object_builder.hpp>
#include <osmium/io/error.hpp>
#include <osmium/memory/buffer.hpp>
#include <osmium/osm/box.hpp>
#include <osmium/osm/changeset.hpp>
#include <osmium/osm/entity_bits.hpp>
#include <osmium/osm/item_type.hpp>
#include <osmium/osm/location.hpp>
#include <osmium/osm/node.hpp>
#include <osmium/osm/relation.hpp>
#include <osmium/osm/timestamp.hpp>
#include <osmium/osm/types.hpp>
#include <osmium/osm/way.hpp>
namespace osmium {
namespace builder {
class Builder;
} // namespace builder
/**
* Exception thrown when there was a problem with parsing the OPL format
* of a file.
*/
struct opl_error : public io_error {
uint64_t line = 0;
uint64_t column = 0;
const char* data;
std::string msg;
explicit opl_error(const std::string& what, const char* d = nullptr) :
io_error(std::string("OPL error: ") + what),
data(d),
msg("OPL error: ") {
msg.append(what);
}
explicit opl_error(const char* what, const char* d = nullptr) :
io_error(std::string("OPL error: ") + what),
data(d),
msg("OPL error: ") {
msg.append(what);
}
void set_pos(uint64_t l, uint64_t col) {
line = l;
column = col;
msg.append(" on line ");
msg.append(std::to_string(line));
msg.append(" column ");
msg.append(std::to_string(column));
}
const char* what() const noexcept override {
return msg.c_str();
}
}; // struct opl_error
namespace io {
namespace detail {
/**
* Consume consecutive space and tab characters. There must be
* at least one.
*/
inline void opl_parse_space(const char** s) {
if (**s != ' ' && **s != '\t') {
throw opl_error{"expected space or tab character", *s};
}
do {
++*s;
} while (**s == ' ' || **s == '\t');
}
/**
* Check whether s points to something else than the end of the
* string or a space or tab.
*/
inline bool opl_non_empty(const char *s) {
return *s != '\0' && *s != ' ' && *s != '\t';
}
/**
* Skip to the next space or tab character or the end of the
* string.
*/
inline const char* opl_skip_section(const char** s) noexcept {
while (opl_non_empty(*s)) {
++*s;
}
return *s;
}
/**
* Parse OPL-escaped strings with hex code with a '%' at the end.
* Appends resulting unicode character to the result string.
*
* Returns a pointer to next character that needs to be consumed.
*/
inline void opl_parse_escaped(const char** data, std::string& result) {
const char* s = *data;
uint32_t value = 0;
const int max_length = sizeof(value) * 2 /* hex chars per byte */;
int length = 0;
while (++length <= max_length) {
if (*s == '\0') {
throw opl_error{"eol", s};
}
if (*s == '%') {
++s;
utf8::utf32to8(&value, &value + 1, std::back_inserter(result));
*data = s;
return;
}
value <<= 4;
if (*s >= '0' && *s <= '9') {
value += *s - '0';
} else if (*s >= 'a' && *s <= 'f') {
value += *s - 'a' + 10;
} else if (*s >= 'A' && *s <= 'F') {
value += *s - 'A' + 10;
} else {
throw opl_error{"not a hex char", s};
}
++s;
}
throw opl_error{"hex escape too long", s};
}
/**
* Parse a string up to end of string or next space, tab, comma, or
* equal sign.
*
* Appends characters to the result string.
*
* Returns a pointer to next character that needs to be consumed.
*/
inline void opl_parse_string(const char** data, std::string& result) {
const char* s = *data;
while (true) {
if (*s == '\0' || *s == ' ' || *s == '\t' || *s == ',' || *s == '=') {
break;
} else if (*s == '%') {
++s;
opl_parse_escaped(&s, result);
} else {
result += *s;
++s;
}
}
*data = s;
}
// Arbitrary limit how long integers can get
constexpr const int max_int_len = 16;
template <typename T>
inline T opl_parse_int(const char** s) {
if (**s == '\0') {
throw opl_error{"expected integer", *s};
}
const bool negative = (**s == '-');
if (negative) {
++*s;
}
int64_t value = 0;
int n = max_int_len;
while (**s >= '0' && **s <= '9') {
if (--n == 0) {
throw opl_error{"integer too long", *s};
}
value *= 10;
value += **s - '0';
++*s;
}
if (n == max_int_len) {
throw opl_error{"expected integer", *s};
}
if (negative) {
value = -value;
if (value < std::numeric_limits<T>::min()) {
throw opl_error{"integer too long", *s};
}
} else {
if (value > std::numeric_limits<T>::max()) {
throw opl_error{"integer too long", *s};
}
}
return T(value);
}
inline osmium::object_id_type opl_parse_id(const char** s) {
return opl_parse_int<osmium::object_id_type>(s);
}
inline osmium::changeset_id_type opl_parse_changeset_id(const char** s) {
return opl_parse_int<osmium::changeset_id_type>(s);
}
inline osmium::object_version_type opl_parse_version(const char** s) {
return opl_parse_int<osmium::object_version_type>(s);
}
inline bool opl_parse_visible(const char** data) {
if (**data == 'V') {
++*data;
return true;
}
if (**data == 'D') {
++*data;
return false;
}
throw opl_error{"invalid visible flag", *data};
}
inline osmium::user_id_type opl_parse_uid(const char** s) {
return opl_parse_int<osmium::user_id_type>(s);
}
inline osmium::Timestamp opl_parse_timestamp(const char** s) {
try {
if (**s == '\0' || **s == ' ' || **s == '\t') {
return osmium::Timestamp{};
}
osmium::Timestamp timestamp{*s};
*s += 20;
return timestamp;
} catch (const std::invalid_argument&) {
throw opl_error{"can not parse timestamp", *s};
}
}
/**
* Check if data points to given character and consume it.
* Throw error otherwise.
*/
inline void opl_parse_char(const char** data, char c) {
if (**data == c) {
++*data;
return;
}
std::string msg = "expected '";
msg += c;
msg += "'";
throw opl_error{msg, *data};
}
/**
* Parse a list of tags in the format 'key=value,key=value,...'
*
* Tags will be added to the buffer using a TagListBuilder.
*/
inline void opl_parse_tags(const char* s, osmium::memory::Buffer& buffer, osmium::builder::Builder* parent_builder = nullptr) {
osmium::builder::TagListBuilder builder{buffer, parent_builder};
std::string key;
std::string value;
while (true) {
opl_parse_string(&s, key);
opl_parse_char(&s, '=');
opl_parse_string(&s, value);
builder.add_tag(key, value);
if (*s == ' ' || *s == '\t' || *s == '\0') {
break;
}
opl_parse_char(&s, ',');
key.clear();
value.clear();
}
}
/**
* Parse a number of nodes in the format "nID,nID,nID..."
*
* Nodes will be added to the buffer using a WayNodeListBuilder.
*/
inline void opl_parse_way_nodes(const char* s, const char* e, osmium::memory::Buffer& buffer, osmium::builder::WayBuilder* parent_builder = nullptr) {
if (s == e) {
return;
}
osmium::builder::WayNodeListBuilder builder{buffer, parent_builder};
while (s < e) {
opl_parse_char(&s, 'n');
if (s == e) {
throw opl_error{"expected integer", s};
}
const osmium::object_id_type ref = opl_parse_id(&s);
if (s == e) {
builder.add_node_ref(ref);
return;
}
osmium::Location location;
if (*s == 'x') {
++s;
location.set_lon_partial(&s);
if (*s == 'y') {
++s;
location.set_lat_partial(&s);
}
}
builder.add_node_ref(ref, location);
if (s == e) {
return;
}
opl_parse_char(&s, ',');
}
}
inline void opl_parse_node(const char** data, osmium::memory::Buffer& buffer) {
osmium::builder::NodeBuilder builder{buffer};
osmium::Node& node = builder.object();
node.set_id(opl_parse_id(data));
const char* tags_begin = nullptr;
std::string user;
osmium::Location location;
while (**data) {
opl_parse_space(data);
const char c = **data;
if (c == '\0') {
break;
}
++(*data);
switch (c) {
case 'v':
node.set_version(opl_parse_version(data));
break;
case 'd':
node.set_visible(opl_parse_visible(data));
break;
case 'c':
node.set_changeset(opl_parse_changeset_id(data));
break;
case 't':
node.set_timestamp(opl_parse_timestamp(data));
break;
case 'i':
node.set_uid(opl_parse_uid(data));
break;
case 'u':
opl_parse_string(data, user);
break;
case 'T':
if (opl_non_empty(*data)) {
tags_begin = *data;
opl_skip_section(data);
}
break;
case 'x':
if (opl_non_empty(*data)) {
location.set_lon_partial(data);
}
break;
case 'y':
if (opl_non_empty(*data)) {
location.set_lat_partial(data);
}
break;
default:
--(*data);
throw opl_error{"unknown attribute", *data};
}
}
if (location.valid()) {
node.set_location(location);
}
builder.add_user(user);
if (tags_begin) {
opl_parse_tags(tags_begin, buffer, &builder);
}
buffer.commit();
}
inline void opl_parse_way(const char** data, osmium::memory::Buffer& buffer) {
osmium::builder::WayBuilder builder{buffer};
osmium::Way& way = builder.object();
way.set_id(opl_parse_id(data));
const char* tags_begin = nullptr;
const char* nodes_begin = nullptr;
const char* nodes_end = nullptr;
std::string user;
while (**data) {
opl_parse_space(data);
const char c = **data;
if (c == '\0') {
break;
}
++(*data);
switch (c) {
case 'v':
way.set_version(opl_parse_version(data));
break;
case 'd':
way.set_visible(opl_parse_visible(data));
break;
case 'c':
way.set_changeset(opl_parse_changeset_id(data));
break;
case 't':
way.set_timestamp(opl_parse_timestamp(data));
break;
case 'i':
way.set_uid(opl_parse_uid(data));
break;
case 'u':
opl_parse_string(data, user);
break;
case 'T':
if (opl_non_empty(*data)) {
tags_begin = *data;
opl_skip_section(data);
}
break;
case 'N':
nodes_begin = *data;
nodes_end = opl_skip_section(data);
break;
default:
--(*data);
throw opl_error{"unknown attribute", *data};
}
}
builder.add_user(user);
if (tags_begin) {
opl_parse_tags(tags_begin, buffer, &builder);
}
opl_parse_way_nodes(nodes_begin, nodes_end, buffer, &builder);
buffer.commit();
}
inline void opl_parse_relation_members(const char* s, const char* e, osmium::memory::Buffer& buffer, osmium::builder::RelationBuilder* parent_builder = nullptr) {
if (s == e) {
return;
}
osmium::builder::RelationMemberListBuilder builder{buffer, parent_builder};
while (s < e) {
osmium::item_type type = osmium::char_to_item_type(*s);
if (type != osmium::item_type::node &&
type != osmium::item_type::way &&
type != osmium::item_type::relation) {
throw opl_error{"unknown object type", s};
}
++s;
if (s == e) {
throw opl_error{"expected integer", s};
}
osmium::object_id_type ref = opl_parse_id(&s);
opl_parse_char(&s, '@');
if (s == e) {
builder.add_member(type, ref, "");
return;
}
std::string role;
opl_parse_string(&s, role);
builder.add_member(type, ref, role);
if (s == e) {
return;
}
opl_parse_char(&s, ',');
}
}
inline void opl_parse_relation(const char** data, osmium::memory::Buffer& buffer) {
osmium::builder::RelationBuilder builder{buffer};
osmium::Relation& relation = builder.object();
relation.set_id(opl_parse_id(data));
const char* tags_begin = nullptr;
const char* members_begin = nullptr;
const char* members_end = nullptr;
std::string user;
while (**data) {
opl_parse_space(data);
const char c = **data;
if (c == '\0') {
break;
}
++(*data);
switch (c) {
case 'v':
relation.set_version(opl_parse_version(data));
break;
case 'd':
relation.set_visible(opl_parse_visible(data));
break;
case 'c':
relation.set_changeset(opl_parse_changeset_id(data));
break;
case 't':
relation.set_timestamp(opl_parse_timestamp(data));
break;
case 'i':
relation.set_uid(opl_parse_uid(data));
break;
case 'u':
opl_parse_string(data, user);
break;
case 'T':
if (opl_non_empty(*data)) {
tags_begin = *data;
opl_skip_section(data);
}
break;
case 'M':
members_begin = *data;
members_end = opl_skip_section(data);
break;
default:
--(*data);
throw opl_error{"unknown attribute", *data};
}
}
builder.add_user(user);
if (tags_begin) {
opl_parse_tags(tags_begin, buffer, &builder);
}
if (members_begin != members_end) {
opl_parse_relation_members(members_begin, members_end, buffer, &builder);
}
buffer.commit();
}
inline void opl_parse_changeset(const char** data, osmium::memory::Buffer& buffer) {
osmium::builder::ChangesetBuilder builder{buffer};
osmium::Changeset& changeset = builder.object();
changeset.set_id(opl_parse_changeset_id(data));
const char* tags_begin = nullptr;
osmium::Location location1;
osmium::Location location2;
std::string user;
while (**data) {
opl_parse_space(data);
const char c = **data;
if (c == '\0') {
break;
}
++(*data);
switch (c) {
case 'k':
changeset.set_num_changes(opl_parse_int<osmium::num_changes_type>(data));
break;
case 's':
changeset.set_created_at(opl_parse_timestamp(data));
break;
case 'e':
changeset.set_closed_at(opl_parse_timestamp(data));
break;
case 'd':
changeset.set_num_comments(opl_parse_int<osmium::num_comments_type>(data));
break;
case 'i':
changeset.set_uid(opl_parse_uid(data));
break;
case 'u':
opl_parse_string(data, user);
break;
case 'x':
if (opl_non_empty(*data)) {
location1.set_lon_partial(data);
}
break;
case 'y':
if (opl_non_empty(*data)) {
location1.set_lat_partial(data);
}
break;
case 'X':
if (opl_non_empty(*data)) {
location2.set_lon_partial(data);
}
break;
case 'Y':
if (opl_non_empty(*data)) {
location2.set_lat_partial(data);
}
break;
case 'T':
if (opl_non_empty(*data)) {
tags_begin = *data;
opl_skip_section(data);
}
break;
default:
--(*data);
throw opl_error{"unknown attribute", *data};
}
}
if (location1.valid() && location2.valid()) {
changeset.bounds().extend(location1);
changeset.bounds().extend(location2);
}
builder.add_user(user);
if (tags_begin) {
opl_parse_tags(tags_begin, buffer, &builder);
}
buffer.commit();
}
inline bool opl_parse_line(uint64_t line_count,
const char* data,
osmium::memory::Buffer& buffer,
osmium::osm_entity_bits::type read_types = osmium::osm_entity_bits::all) {
const char* start_of_line = data;
try {
switch (*data) {
case '\0':
// ignore empty lines
break;
case '#':
// ignore lines starting with #
break;
case 'n':
if (read_types & osmium::osm_entity_bits::node) {
++data;
opl_parse_node(&data, buffer);
return true;
}
break;
case 'w':
if (read_types & osmium::osm_entity_bits::way) {
++data;
opl_parse_way(&data, buffer);
return true;
}
break;
case 'r':
if (read_types & osmium::osm_entity_bits::relation) {
++data;
opl_parse_relation(&data, buffer);
return true;
}
break;
case 'c':
if (read_types & osmium::osm_entity_bits::changeset) {
++data;
opl_parse_changeset(&data, buffer);
return true;
}
break;
default:
throw opl_error{"unknown type", data};
}
} catch (opl_error& e) {
e.set_pos(line_count, e.data ? e.data - start_of_line : 0);
throw;
}
return false;
}
} // namespace detail
} // namespace io
} // namespace osmium
#endif // OSMIUM_IO_DETAIL_OPL_PARSER_FUNCTIONS_HPP

33
include/osmium/io/detail/output_format.hpp
View File

@ -33,16 +33,16 @@ DEALINGS IN THE SOFTWARE.
*/
#include <cstdint>
#include <functional>
#include <map>
#include <memory>
#include <stdexcept>
#include <string>
#include <utility>
#include <osmium/handler.hpp>
#include <osmium/io/detail/queue_util.hpp>
#include <osmium/io/detail/string_util.hpp>
#include <osmium/io/error.hpp>
#include <osmium/io/file.hpp>
#include <osmium/io/file_format.hpp>
#include <osmium/memory/buffer.hpp>
@ -70,9 +70,28 @@ namespace osmium {
m_out(std::make_shared<std::string>()) {
}
template <typename... TArgs>
void output_formatted(const char* format, TArgs&&... args) {
append_printf_formatted_string(*m_out, format, std::forward<TArgs>(args)...);
// Simple function to convert integer to string. This is much
// faster than using sprintf, but could be further optimized.
// See https://github.com/miloyip/itoa-benchmark .
void output_int(int64_t value) {
if (value < 0) {
*m_out += '-';
value = -value;
}
char temp[20];
char *t = temp;
do {
*t++ = char(value % 10) + '0';
value /= 10;
} while (value > 0);
const auto old_size = m_out->size();
m_out->resize(old_size + (t - temp));
char* data = &(*m_out)[old_size];
do {
*data++ += *--t;
} while (t != temp);
}
}; // class OutputBlock;
@ -133,11 +152,11 @@ namespace osmium {
public:
typedef std::function<osmium::io::detail::OutputFormat*(const osmium::io::File&, future_string_queue_type&)> create_output_type;
using create_output_type = std::function<osmium::io::detail::OutputFormat*(const osmium::io::File&, future_string_queue_type&)>;
private:
typedef std::map<osmium::io::file_format, create_output_type> map_type;
using map_type = std::map<osmium::io::file_format, create_output_type>;
map_type m_callbacks;

2
include/osmium/io/detail/pbf.hpp
View File

@ -78,7 +78,7 @@ namespace osmium {
// between representation as double and as int
const int64_t lonlat_resolution = 1000 * 1000 * 1000;
const int64_t resolution_convert = lonlat_resolution / osmium::Location::coordinate_precision;
const int64_t resolution_convert = lonlat_resolution / osmium::detail::coordinate_precision;
} // namespace detail

237
include/osmium/io/detail/pbf_decoder.hpp
View File

@ -33,10 +33,8 @@ DEALINGS IN THE SOFTWARE.
*/
#include <cstddef>
#include <cstdint>
#include <cstring>
#include <algorithm>
#include <limits>
#include <memory>
#include <stdexcept>
@ -44,35 +42,47 @@ DEALINGS IN THE SOFTWARE.
#include <utility>
#include <vector>
#include <protozero/iterators.hpp>
#include <protozero/pbf_message.hpp>
#include <protozero/types.hpp>
#include <osmium/builder/osm_object_builder.hpp>
#include <osmium/io/detail/pbf.hpp> // IWYU pragma: export
#include <osmium/io/detail/protobuf_tags.hpp>
#include <osmium/io/detail/zlib.hpp>
#include <osmium/io/header.hpp>
#include <osmium/memory/buffer.hpp>
#include <osmium/osm/box.hpp>
#include <osmium/osm/entity_bits.hpp>
#include <osmium/osm/item_type.hpp>
#include <osmium/osm/location.hpp>
#include <osmium/osm/node.hpp>
#include <osmium/osm/object.hpp>
#include <osmium/osm/relation.hpp>
#include <osmium/osm/timestamp.hpp>
#include <osmium/osm/types.hpp>
#include <osmium/memory/buffer.hpp>
#include <osmium/osm/entity_bits.hpp>
#include <osmium/osm/way.hpp>
#include <osmium/util/cast.hpp>
#include <osmium/util/delta.hpp>
namespace osmium {
namespace builder {
class Builder;
} // namespace builder
namespace io {
namespace detail {
using ptr_len_type = std::pair<const char*, size_t>;
using protozero::data_view;
using osm_string_len_type = std::pair<const char*, osmium::string_size_type>;
class PBFPrimitiveBlockDecoder {
static constexpr size_t initial_buffer_size = 2 * 1024 * 1024;
static constexpr const size_t initial_buffer_size = 2 * 1024 * 1024;
ptr_len_type m_data;
data_view m_data;
std::vector<osm_string_len_type> m_stringtable;
int64_t m_lon_offset = 0;
@ -84,18 +94,18 @@ namespace osmium {
osmium::memory::Buffer m_buffer { initial_buffer_size };
void decode_stringtable(const ptr_len_type& data) {
void decode_stringtable(const data_view& data) {
if (!m_stringtable.empty()) {
throw osmium::pbf_error("more than one stringtable in pbf file");
}
protozero::pbf_message<OSMFormat::StringTable> pbf_string_table(data);
while (pbf_string_table.next(OSMFormat::StringTable::repeated_bytes_s)) {
auto str_len = pbf_string_table.get_data();
if (str_len.second > osmium::max_osm_string_length) {
const auto str_view = pbf_string_table.get_view();
if (str_view.size() > osmium::max_osm_string_length) {
throw osmium::pbf_error("overlong string in string table");
}
m_stringtable.emplace_back(str_len.first, osmium::string_size_type(str_len.second));
m_stringtable.emplace_back(str_view.data(), osmium::string_size_type(str_view.size()));
}
}
@ -104,7 +114,7 @@ namespace osmium {
while (pbf_primitive_block.next()) {
switch (pbf_primitive_block.tag()) {
case OSMFormat::PrimitiveBlock::required_StringTable_stringtable:
decode_stringtable(pbf_primitive_block.get_data());
decode_stringtable(pbf_primitive_block.get_view());
break;
case OSMFormat::PrimitiveBlock::optional_int32_granularity:
m_granularity = pbf_primitive_block.get_int32();
@ -132,28 +142,28 @@ namespace osmium {
switch (pbf_primitive_group.tag()) {
case OSMFormat::PrimitiveGroup::repeated_Node_nodes:
if (m_read_types & osmium::osm_entity_bits::node) {
decode_node(pbf_primitive_group.get_data());
decode_node(pbf_primitive_group.get_view());
} else {
pbf_primitive_group.skip();
}
break;
case OSMFormat::PrimitiveGroup::optional_DenseNodes_dense:
if (m_read_types & osmium::osm_entity_bits::node) {
decode_dense_nodes(pbf_primitive_group.get_data());
decode_dense_nodes(pbf_primitive_group.get_view());
} else {
pbf_primitive_group.skip();
}
break;
case OSMFormat::PrimitiveGroup::repeated_Way_ways:
if (m_read_types & osmium::osm_entity_bits::way) {
decode_way(pbf_primitive_group.get_data());
decode_way(pbf_primitive_group.get_view());
} else {
pbf_primitive_group.skip();
}
break;
case OSMFormat::PrimitiveGroup::repeated_Relation_relations:
if (m_read_types & osmium::osm_entity_bits::relation) {
decode_relation(pbf_primitive_group.get_data());
decode_relation(pbf_primitive_group.get_view());
} else {
pbf_primitive_group.skip();
}
@ -165,7 +175,7 @@ namespace osmium {
}
}
osm_string_len_type decode_info(const ptr_len_type& data, osmium::OSMObject& object) {
osm_string_len_type decode_info(const data_view& data, osmium::OSMObject& object) {
osm_string_len_type user = std::make_pair("", 0);
protozero::pbf_message<OSMFormat::Info> pbf_info(data);
@ -173,7 +183,7 @@ namespace osmium {
switch (pbf_info.tag()) {
case OSMFormat::Info::optional_int32_version:
{
auto version = pbf_info.get_int32();
const auto version = pbf_info.get_int32();
if (version < 0) {
throw osmium::pbf_error("object version must not be negative");
}
@ -185,7 +195,7 @@ namespace osmium {
break;
case OSMFormat::Info::optional_int64_changeset:
{
auto changeset_id = pbf_info.get_int64();
const auto changeset_id = pbf_info.get_int64();
if (changeset_id < 0) {
throw osmium::pbf_error("object changeset_id must not be negative");
}
@ -209,15 +219,15 @@ namespace osmium {
return user;
}
using kv_type = std::pair<protozero::pbf_reader::const_uint32_iterator, protozero::pbf_reader::const_uint32_iterator>;
using kv_type = protozero::iterator_range<protozero::pbf_reader::const_uint32_iterator>;
void build_tag_list(osmium::builder::Builder& builder, const kv_type& keys, const kv_type& vals) {
if (keys.first != keys.second) {
if (!keys.empty()) {
osmium::builder::TagListBuilder tl_builder(m_buffer, &builder);
auto kit = keys.first;
auto vit = vals.first;
while (kit != keys.second) {
if (vit == vals.second) {
auto kit = keys.begin();
auto vit = vals.begin();
while (kit != keys.end()) {
if (vit == vals.end()) {
// this is against the spec, must have same number of elements
throw osmium::pbf_error("PBF format error");
}
@ -232,7 +242,7 @@ namespace osmium {
return int32_t((c * m_granularity + m_lon_offset) / resolution_convert);
}
void decode_node(const ptr_len_type& data) {
void decode_node(const data_view& data) {
osmium::builder::NodeBuilder builder(m_buffer);
osmium::Node& node = builder.object();
@ -256,7 +266,7 @@ namespace osmium {
vals = pbf_node.get_packed_uint32();
break;
case OSMFormat::Node::optional_Info_info:
user = decode_info(pbf_node.get_data(), builder.object());
user = decode_info(pbf_node.get_view(), builder.object());
break;
case OSMFormat::Node::required_sint64_lat:
lat = pbf_node.get_sint64();
@ -287,12 +297,14 @@ namespace osmium {
m_buffer.commit();
}
void decode_way(const ptr_len_type& data) {
void decode_way(const data_view& data) {
osmium::builder::WayBuilder builder(m_buffer);
kv_type keys;
kv_type vals;
std::pair<protozero::pbf_reader::const_sint64_iterator, protozero::pbf_reader::const_sint64_iterator> refs;
protozero::iterator_range<protozero::pbf_reader::const_sint64_iterator> refs;
protozero::iterator_range<protozero::pbf_reader::const_sint64_iterator> lats;
protozero::iterator_range<protozero::pbf_reader::const_sint64_iterator> lons;
osm_string_len_type user = { "", 0 };
@ -309,11 +321,17 @@ namespace osmium {
vals = pbf_way.get_packed_uint32();
break;
case OSMFormat::Way::optional_Info_info:
user = decode_info(pbf_way.get_data(), builder.object());
user = decode_info(pbf_way.get_view(), builder.object());
break;
case OSMFormat::Way::packed_sint64_refs:
refs = pbf_way.get_packed_sint64();
break;
case OSMFormat::Way::packed_sint64_lat:
lats = pbf_way.get_packed_sint64();
break;
case OSMFormat::Way::packed_sint64_lon:
lons = pbf_way.get_packed_sint64();
break;
default:
pbf_way.skip();
}
@ -321,11 +339,26 @@ namespace osmium {
builder.add_user(user.first, user.second);
if (refs.first != refs.second) {
if (!refs.empty()) {
osmium::builder::WayNodeListBuilder wnl_builder(m_buffer, &builder);
osmium::util::DeltaDecode<int64_t> ref;
while (refs.first != refs.second) {
wnl_builder.add_node_ref(ref.update(*refs.first++));
if (lats.empty()) {
for (const auto& ref_value : refs) {
wnl_builder.add_node_ref(ref.update(ref_value));
}
} else {
osmium::util::DeltaDecode<int64_t> lon;
osmium::util::DeltaDecode<int64_t> lat;
while (!refs.empty() && !lons.empty() && !lats.empty()) {
wnl_builder.add_node_ref(
ref.update(refs.front()),
osmium::Location{convert_pbf_coordinate(lon.update(lons.front())),
convert_pbf_coordinate(lat.update(lats.front()))}
);
refs.drop_front();
lons.drop_front();
lats.drop_front();
}
}
}
@ -334,14 +367,14 @@ namespace osmium {
m_buffer.commit();
}
void decode_relation(const ptr_len_type& data) {
void decode_relation(const data_view& data) {
osmium::builder::RelationBuilder builder(m_buffer);
kv_type keys;
kv_type vals;
std::pair<protozero::pbf_reader::const_int32_iterator, protozero::pbf_reader::const_int32_iterator> roles;
std::pair<protozero::pbf_reader::const_sint64_iterator, protozero::pbf_reader::const_sint64_iterator> refs;
std::pair<protozero::pbf_reader::const_int32_iterator, protozero::pbf_reader::const_int32_iterator> types;
protozero::iterator_range<protozero::pbf_reader::const_int32_iterator> roles;
protozero::iterator_range<protozero::pbf_reader::const_sint64_iterator> refs;
protozero::iterator_range<protozero::pbf_reader::const_int32_iterator> types;
osm_string_len_type user = { "", 0 };
@ -358,7 +391,7 @@ namespace osmium {
vals = pbf_relation.get_packed_uint32();
break;
case OSMFormat::Relation::optional_Info_info:
user = decode_info(pbf_relation.get_data(), builder.object());
user = decode_info(pbf_relation.get_view(), builder.object());
break;
case OSMFormat::Relation::packed_int32_roles_sid:
roles = pbf_relation.get_packed_int32();
@ -376,21 +409,24 @@ namespace osmium {
builder.add_user(user.first, user.second);
if (refs.first != refs.second) {
if (!refs.empty()) {
osmium::builder::RelationMemberListBuilder rml_builder(m_buffer, &builder);
osmium::util::DeltaDecode<int64_t> ref;
while (roles.first != roles.second && refs.first != refs.second && types.first != types.second) {
const auto& r = m_stringtable.at(*roles.first++);
int type = *types.first++;
while (!roles.empty() && !refs.empty() && !types.empty()) {
const auto& r = m_stringtable.at(roles.front());
const int type = types.front();
if (type < 0 || type > 2) {
throw osmium::pbf_error("unknown relation member type");
}
rml_builder.add_member(
osmium::item_type(type + 1),
ref.update(*refs.first++),
ref.update(refs.front()),
r.first,
r.second
);
roles.drop_front();
refs.drop_front();
types.drop_front();
}
}
@ -399,22 +435,22 @@ namespace osmium {
m_buffer.commit();
}
void decode_dense_nodes(const ptr_len_type& data) {
void decode_dense_nodes(const data_view& data) {
bool has_info = false;
bool has_visibles = false;
std::pair<protozero::pbf_reader::const_sint64_iterator, protozero::pbf_reader::const_sint64_iterator> ids;
std::pair<protozero::pbf_reader::const_sint64_iterator, protozero::pbf_reader::const_sint64_iterator> lats;
std::pair<protozero::pbf_reader::const_sint64_iterator, protozero::pbf_reader::const_sint64_iterator> lons;
protozero::iterator_range<protozero::pbf_reader::const_sint64_iterator> ids;
protozero::iterator_range<protozero::pbf_reader::const_sint64_iterator> lats;
protozero::iterator_range<protozero::pbf_reader::const_sint64_iterator> lons;
std::pair<protozero::pbf_reader::const_int32_iterator, protozero::pbf_reader::const_int32_iterator> tags;
protozero::iterator_range<protozero::pbf_reader::const_int32_iterator> tags;
std::pair<protozero::pbf_reader::const_int32_iterator, protozero::pbf_reader::const_int32_iterator> versions;
std::pair<protozero::pbf_reader::const_sint64_iterator, protozero::pbf_reader::const_sint64_iterator> timestamps;
std::pair<protozero::pbf_reader::const_sint64_iterator, protozero::pbf_reader::const_sint64_iterator> changesets;
std::pair<protozero::pbf_reader::const_sint32_iterator, protozero::pbf_reader::const_sint32_iterator> uids;
std::pair<protozero::pbf_reader::const_sint32_iterator, protozero::pbf_reader::const_sint32_iterator> user_sids;
std::pair<protozero::pbf_reader::const_int32_iterator, protozero::pbf_reader::const_int32_iterator> visibles;
protozero::iterator_range<protozero::pbf_reader::const_int32_iterator> versions;
protozero::iterator_range<protozero::pbf_reader::const_sint64_iterator> timestamps;
protozero::iterator_range<protozero::pbf_reader::const_sint64_iterator> changesets;
protozero::iterator_range<protozero::pbf_reader::const_sint32_iterator> uids;
protozero::iterator_range<protozero::pbf_reader::const_sint32_iterator> user_sids;
protozero::iterator_range<protozero::pbf_reader::const_int32_iterator> visibles;
protozero::pbf_message<OSMFormat::DenseNodes> pbf_dense_nodes(data);
while (pbf_dense_nodes.next()) {
@ -475,11 +511,11 @@ namespace osmium {
osmium::util::DeltaDecode<int64_t> dense_changeset;
osmium::util::DeltaDecode<int64_t> dense_timestamp;
auto tag_it = tags.first;
auto tag_it = tags.begin();
while (ids.first != ids.second) {
if (lons.first == lons.second ||
lats.first == lats.second) {
while (!ids.empty()) {
if (lons.empty() ||
lats.empty()) {
// this is against the spec, must have same number of elements
throw osmium::pbf_error("PBF format error");
}
@ -489,43 +525,50 @@ namespace osmium {
osmium::builder::NodeBuilder builder(m_buffer);
osmium::Node& node = builder.object();
node.set_id(dense_id.update(*ids.first++));
node.set_id(dense_id.update(ids.front()));
ids.drop_front();
if (has_info) {
if (versions.first == versions.second ||
changesets.first == changesets.second ||
timestamps.first == timestamps.second ||
uids.first == uids.second ||
user_sids.first == user_sids.second) {
if (versions.empty() ||
changesets.empty() ||
timestamps.empty() ||
uids.empty() ||
user_sids.empty()) {
// this is against the spec, must have same number of elements
throw osmium::pbf_error("PBF format error");
}
auto version = *versions.first++;
const auto version = versions.front();
versions.drop_front();
if (version < 0) {
throw osmium::pbf_error("object version must not be negative");
}
node.set_version(static_cast<osmium::object_version_type>(version));
auto changeset_id = dense_changeset.update(*changesets.first++);
const auto changeset_id = dense_changeset.update(changesets.front());
changesets.drop_front();
if (changeset_id < 0) {
throw osmium::pbf_error("object changeset_id must not be negative");
}
node.set_changeset(static_cast<osmium::changeset_id_type>(changeset_id));
node.set_timestamp(dense_timestamp.update(*timestamps.first++) * m_date_factor / 1000);
node.set_uid_from_signed(static_cast<osmium::signed_user_id_type>(dense_uid.update(*uids.first++)));
node.set_timestamp(dense_timestamp.update(timestamps.front()) * m_date_factor / 1000);
timestamps.drop_front();
node.set_uid_from_signed(static_cast<osmium::signed_user_id_type>(dense_uid.update(uids.front())));
uids.drop_front();
if (has_visibles) {
if (visibles.first == visibles.second) {
if (visibles.empty()) {
// this is against the spec, must have same number of elements
throw osmium::pbf_error("PBF format error");
}
visible = (*visibles.first++) != 0;
visible = (visibles.front() != 0);
visibles.drop_front();
}
node.set_visible(visible);
const auto& u = m_stringtable.at(dense_user_sid.update(*user_sids.first++));
const auto& u = m_stringtable.at(dense_user_sid.update(user_sids.front()));
user_sids.drop_front();
builder.add_user(u.first, u.second);
} else {
builder.add_user("");
@ -533,8 +576,10 @@ namespace osmium {
// even if the node isn't visible, there's still a record
// of its lat/lon in the dense arrays.
const auto lon = dense_longitude.update(*lons.first++);
const auto lat = dense_latitude.update(*lats.first++);
const auto lon = dense_longitude.update(lons.front());
lons.drop_front();
const auto lat = dense_latitude.update(lats.front());
lats.drop_front();
if (visible) {
builder.object().set_location(osmium::Location(
convert_pbf_coordinate(lon),
@ -542,18 +587,18 @@ namespace osmium {
));
}
if (tag_it != tags.second) {
if (tag_it != tags.end()) {
osmium::builder::TagListBuilder tl_builder(m_buffer, &builder);
while (tag_it != tags.second && *tag_it != 0) {
while (tag_it != tags.end() && *tag_it != 0) {
const auto& k = m_stringtable.at(*tag_it++);
if (tag_it == tags.second) {
if (tag_it == tags.end()) {
throw osmium::pbf_error("PBF format error"); // this is against the spec, keys/vals must come in pairs
}
const auto& v = m_stringtable.at(*tag_it++);
tl_builder.add_tag(k.first, k.second, v.first, v.second);
}
if (tag_it != tags.second) {
if (tag_it != tags.end()) {
++tag_it;
}
}
@ -565,7 +610,7 @@ namespace osmium {
public:
PBFPrimitiveBlockDecoder(const ptr_len_type& data, osmium::osm_entity_bits::type read_types) :
PBFPrimitiveBlockDecoder(const data_view& data, osmium::osm_entity_bits::type read_types) :
m_data(data),
m_read_types(read_types) {
}
@ -582,7 +627,7 @@ namespace osmium {
try {
decode_primitive_block_metadata();
decode_primitive_block_data();
} catch (std::out_of_range&) {
} catch (const std::out_of_range&) {
throw osmium::pbf_error("string id out of range");
}
@ -591,17 +636,17 @@ namespace osmium {
}; // class PBFPrimitiveBlockDecoder
inline ptr_len_type decode_blob(const std::string& blob_data, std::string& output) {
inline data_view decode_blob(const std::string& blob_data, std::string& output) {
int32_t raw_size = 0;
std::pair<const char*, protozero::pbf_length_type> zlib_data = {nullptr, 0};
protozero::data_view zlib_data;
protozero::pbf_message<FileFormat::Blob> pbf_blob(blob_data);
while (pbf_blob.next()) {
switch (pbf_blob.tag()) {
case FileFormat::Blob::optional_bytes_raw:
{
auto data_len = pbf_blob.get_data();
if (data_len.second > max_uncompressed_blob_size) {
auto data_len = pbf_blob.get_view();
if (data_len.size() > max_uncompressed_blob_size) {
throw osmium::pbf_error("illegal blob size");
}
return data_len;
@ -613,7 +658,7 @@ namespace osmium {
}
break;
case FileFormat::Blob::optional_bytes_zlib_data:
zlib_data = pbf_blob.get_data();
zlib_data = pbf_blob.get_view();
break;
case FileFormat::Blob::optional_bytes_lzma_data:
throw osmium::pbf_error("lzma blobs not implemented");
@ -622,10 +667,10 @@ namespace osmium {
}
}
if (zlib_data.second != 0 && raw_size != 0) {
if (zlib_data.size() != 0 && raw_size != 0) {
return osmium::io::detail::zlib_uncompress_string(
zlib_data.first,
static_cast<unsigned long>(zlib_data.second),
zlib_data.data(),
static_cast<unsigned long>(zlib_data.size()),
static_cast<unsigned long>(raw_size),
output
);
@ -634,7 +679,7 @@ namespace osmium {
throw osmium::pbf_error("blob contains no data");
}
inline osmium::Box decode_header_bbox(const ptr_len_type& data) {
inline osmium::Box decode_header_bbox(const data_view& data) {
int64_t left = std::numeric_limits<int64_t>::max();
int64_t right = std::numeric_limits<int64_t>::max();
int64_t top = std::numeric_limits<int64_t>::max();
@ -674,7 +719,7 @@ namespace osmium {
return box;
}
inline osmium::io::Header decode_header_block(const ptr_len_type& data) {
inline osmium::io::Header decode_header_block(const data_view& data) {
osmium::io::Header header;
int i = 0;
@ -682,20 +727,20 @@ namespace osmium {
while (pbf_header_block.next()) {
switch (pbf_header_block.tag()) {
case OSMFormat::HeaderBlock::optional_HeaderBBox_bbox:
header.add_box(decode_header_bbox(pbf_header_block.get_data()));
header.add_box(decode_header_bbox(pbf_header_block.get_view()));
break;
case OSMFormat::HeaderBlock::repeated_string_required_features:
{
auto feature = pbf_header_block.get_data();
if (!strncmp("OsmSchema-V0.6", feature.first, feature.second)) {
auto feature = pbf_header_block.get_view();
if (!std::strncmp("OsmSchema-V0.6", feature.data(), feature.size())) {
// intentionally left blank
} else if (!strncmp("DenseNodes", feature.first, feature.second)) {
} else if (!std::strncmp("DenseNodes", feature.data(), feature.size())) {
header.set("pbf_dense_nodes", true);
} else if (!strncmp("HistoricalInformation", feature.first, feature.second)) {
} else if (!std::strncmp("HistoricalInformation", feature.data(), feature.size())) {
header.set_has_multiple_object_versions(true);
} else {
std::string msg("required feature not supported: ");
msg.append(feature.first, feature.second);
msg.append(feature.data(), feature.size());
throw osmium::pbf_error(msg);
}
}
@ -708,7 +753,7 @@ namespace osmium {
break;
case OSMFormat::HeaderBlock::optional_int64_osmosis_replication_timestamp:
{
auto timestamp = osmium::Timestamp(pbf_header_block.get_int64()).to_iso();
const auto timestamp = osmium::Timestamp(pbf_header_block.get_int64()).to_iso();
header.set("osmosis_replication_timestamp", timestamp);
header.set("timestamp", timestamp);
}

21
include/osmium/io/detail/pbf_input_format.hpp
View File

@ -38,25 +38,22 @@ DEALINGS IN THE SOFTWARE.
#include <cstddef>
#include <cstdint>
#include <cstring>
#include <future>
#include <memory>
#include <sstream>
#include <string>
#include <thread>
#include <type_traits>
#include <protozero/pbf_message.hpp>
#include <protozero/types.hpp>
#include <osmium/io/detail/input_format.hpp>
#include <osmium/io/detail/pbf.hpp> // IWYU pragma: export
#include <osmium/io/detail/pbf_decoder.hpp>
#include <osmium/io/detail/protobuf_tags.hpp>
#include <osmium/io/error.hpp>
#include <osmium/io/file.hpp>
#include <osmium/io/detail/queue_util.hpp>
#include <osmium/io/file_format.hpp>
#include <osmium/osm.hpp>
#include <osmium/io/header.hpp>
#include <osmium/osm/entity_bits.hpp>
#include <osmium/osm/object.hpp>
#include <osmium/osm/timestamp.hpp>
#include <osmium/thread/pool.hpp>
#include <osmium/thread/util.hpp>
#include <osmium/util/config.hpp>
@ -80,7 +77,7 @@ namespace osmium {
*/
std::string read_from_input_queue(size_t size) {
while (m_input_buffer.size() < size) {
std::string new_data = get_input();
const std::string new_data = get_input();
if (input_done()) {
throw osmium::pbf_error("truncated data (EOF encountered)");
}
@ -106,7 +103,7 @@ namespace osmium {
try {
const std::string input_data = read_from_input_queue(sizeof(size_in_network_byte_order));
size_in_network_byte_order = *reinterpret_cast<const uint32_t*>(input_data.data());
} catch (osmium::pbf_error&) {
} catch (const osmium::pbf_error&) {
return 0; // EOF
}
@ -123,13 +120,13 @@ namespace osmium {
* type. Return the size of the following Blob.
*/
size_t decode_blob_header(protozero::pbf_message<FileFormat::BlobHeader>&& pbf_blob_header, const char* expected_type) {
std::pair<const char*, size_t> blob_header_type;
protozero::data_view blob_header_type;
size_t blob_header_datasize = 0;
while (pbf_blob_header.next()) {
switch (pbf_blob_header.tag()) {
case FileFormat::BlobHeader::required_string_type:
blob_header_type = pbf_blob_header.get_data();
blob_header_type = pbf_blob_header.get_view();
break;
case FileFormat::BlobHeader::required_int32_datasize:
blob_header_datasize = pbf_blob_header.get_int32();
@ -143,7 +140,7 @@ namespace osmium {
throw osmium::pbf_error("PBF format error: BlobHeader.datasize missing or zero.");
}
if (strncmp(expected_type, blob_header_type.first, blob_header_type.second)) {
if (std::strncmp(expected_type, blob_header_type.data(), blob_header_type.size())) {
throw osmium::pbf_error("blob does not have expected type (OSMHeader in first blob, OSMData in following blobs)");
}

68
include/osmium/io/detail/pbf_output_format.hpp
View File

@ -41,30 +41,34 @@ DEALINGS IN THE SOFTWARE.
#include <iterator>
#include <memory>
#include <string>
#include <time.h>
#include <utility>
#include <protozero/pbf_builder.hpp>
#include <protozero/pbf_writer.hpp>
#include <protozero/types.hpp>
#include <osmium/handler.hpp>
#include <osmium/io/detail/output_format.hpp>
#include <osmium/io/detail/pbf.hpp> // IWYU pragma: export
#include <osmium/io/detail/protobuf_tags.hpp>
#include <osmium/io/detail/queue_util.hpp>
#include <osmium/io/detail/string_table.hpp>
#include <osmium/io/detail/zlib.hpp>
#include <osmium/io/file.hpp>
#include <osmium/io/file_format.hpp>
#include <osmium/io/header.hpp>
#include <osmium/memory/buffer.hpp>
#include <osmium/memory/collection.hpp>
#include <osmium/memory/item_iterator.hpp>
#include <osmium/osm/box.hpp>
#include <osmium/osm/item_type.hpp>
#include <osmium/osm/location.hpp>
#include <osmium/osm/node.hpp>
#include <osmium/osm/node_ref.hpp>
#include <osmium/osm/object.hpp>
#include <osmium/osm/relation.hpp>
#include <osmium/osm/tag.hpp>
#include <osmium/osm/timestamp.hpp>
#include <osmium/osm/types.hpp>
#include <osmium/osm/way.hpp>
#include <osmium/thread/pool.hpp>
#include <osmium/util/cast.hpp>
@ -101,6 +105,9 @@ namespace osmium {
/// Should the visible flag be added to all OSM objects?
bool add_visible_flag;
/// Should node locations be added to ways?
bool locations_on_ways;
};
/**
@ -483,6 +490,7 @@ namespace osmium {
m_options.add_metadata = file.is_not_false("pbf_add_metadata") && file.is_not_false("add_metadata");
m_options.add_historical_information_flag = file.has_multiple_object_versions();
m_options.add_visible_flag = file.has_multiple_object_versions();
m_options.locations_on_ways = file.is_true("locations_on_ways");
}
PBFOutputFormat(const PBFOutputFormat&) = delete;
@ -514,20 +522,24 @@ namespace osmium {
pbf_header_block.add_string(OSMFormat::HeaderBlock::repeated_string_required_features, "HistoricalInformation");
}
if (m_options.locations_on_ways) {
pbf_header_block.add_string(OSMFormat::HeaderBlock::repeated_string_optional_features, "LocationsOnWays");
}
pbf_header_block.add_string(OSMFormat::HeaderBlock::optional_string_writingprogram, header.get("generator"));
std::string osmosis_replication_timestamp = header.get("osmosis_replication_timestamp");
const std::string osmosis_replication_timestamp = header.get("osmosis_replication_timestamp");
if (!osmosis_replication_timestamp.empty()) {
osmium::Timestamp ts(osmosis_replication_timestamp.c_str());
pbf_header_block.add_int64(OSMFormat::HeaderBlock::optional_int64_osmosis_replication_timestamp, uint32_t(ts));
}
std::string osmosis_replication_sequence_number = header.get("osmosis_replication_sequence_number");
const std::string osmosis_replication_sequence_number = header.get("osmosis_replication_sequence_number");
if (!osmosis_replication_sequence_number.empty()) {
pbf_header_block.add_int64(OSMFormat::HeaderBlock::optional_int64_osmosis_replication_sequence_number, std::atoll(osmosis_replication_sequence_number.c_str()));
}
std::string osmosis_replication_base_url = header.get("osmosis_replication_base_url");
const std::string osmosis_replication_base_url = header.get("osmosis_replication_base_url");
if (!osmosis_replication_base_url.empty()) {
pbf_header_block.add_string(OSMFormat::HeaderBlock::optional_string_osmosis_replication_base_url, osmosis_replication_base_url);
}
@ -571,15 +583,30 @@ namespace osmium {
pbf_way.add_int64(OSMFormat::Way::required_int64_id, way.id());
add_meta(way, pbf_way);
static auto map_node_ref = [](osmium::NodeRefList::const_iterator node_ref) noexcept -> osmium::object_id_type {
return node_ref->ref();
};
typedef osmium::util::DeltaEncodeIterator<osmium::NodeRefList::const_iterator, decltype(map_node_ref), osmium::object_id_type> it_type;
{
osmium::util::DeltaEncode<object_id_type, int64_t> delta_id;
protozero::packed_field_sint64 field{pbf_way, protozero::pbf_tag_type(OSMFormat::Way::packed_sint64_refs)};
for (const auto& node_ref : way.nodes()) {
field.add_element(delta_id.update(node_ref.ref()));
}
}
const auto& nodes = way.nodes();
it_type first { nodes.cbegin(), nodes.cend(), map_node_ref };
it_type last { nodes.cend(), nodes.cend(), map_node_ref };
pbf_way.add_packed_sint64(OSMFormat::Way::packed_sint64_refs, first, last);
if (m_options.locations_on_ways) {
{
osmium::util::DeltaEncode<int64_t, int64_t> delta_id;
protozero::packed_field_sint64 field{pbf_way, protozero::pbf_tag_type(OSMFormat::Way::packed_sint64_lon)};
for (const auto& node_ref : way.nodes()) {
field.add_element(delta_id.update(lonlat2int(node_ref.location().lon_without_check())));
}
}
{
osmium::util::DeltaEncode<int64_t, int64_t> delta_id;
protozero::packed_field_sint64 field{pbf_way, protozero::pbf_tag_type(OSMFormat::Way::packed_sint64_lat)};
for (const auto& node_ref : way.nodes()) {
field.add_element(delta_id.update(lonlat2int(node_ref.location().lat_without_check())));
}
}
}
}
void relation(const osmium::Relation& relation) {
@ -596,14 +623,13 @@ namespace osmium {
}
}
static auto map_member_ref = [](osmium::RelationMemberList::const_iterator member) noexcept -> osmium::object_id_type {
return member->ref();
};
typedef osmium::util::DeltaEncodeIterator<osmium::RelationMemberList::const_iterator, decltype(map_member_ref), osmium::object_id_type> it_type;
const auto& members = relation.members();
it_type first { members.cbegin(), members.cend(), map_member_ref };
it_type last { members.cend(), members.cend(), map_member_ref };
pbf_relation.add_packed_sint64(OSMFormat::Relation::packed_sint64_memids, first, last);
{
osmium::util::DeltaEncode<object_id_type, int64_t> delta_id;
protozero::packed_field_sint64 field{pbf_relation, protozero::pbf_tag_type(OSMFormat::Relation::packed_sint64_memids)};
for (const auto& member : relation.members()) {
field.add_element(delta_id.update(member.ref()));
}
}
{
protozero::packed_field_int32 field{pbf_relation, protozero::pbf_tag_type(OSMFormat::Relation::packed_MemberType_types)};

4
include/osmium/io/detail/protobuf_tags.hpp
View File

@ -146,7 +146,9 @@ namespace osmium {
packed_uint32_keys = 2,
packed_uint32_vals = 3,
optional_Info_info = 4,
packed_sint64_refs = 8
packed_sint64_refs = 8,
packed_sint64_lat = 9,
packed_sint64_lon = 10
};
enum class Relation : protozero::pbf_tag_type {

19
include/osmium/io/detail/queue_util.hpp
View File

@ -47,6 +47,9 @@ namespace osmium {
namespace detail {
template <typename T>
using future_queue_type = osmium::thread::Queue<std::future<T>>;
/**
* This type of queue contains buffers with OSM data in them.
* The "end of file" is marked by an invalid Buffer.
@ -54,7 +57,7 @@ namespace osmium {
* transport exceptions. The future also helps with keeping the
* data in order.
*/
using future_buffer_queue_type = osmium::thread::Queue<std::future<osmium::memory::Buffer>>;
using future_buffer_queue_type = future_queue_type<osmium::memory::Buffer>;
/**
* This type of queue contains OSM file data in the form it is
@ -71,24 +74,24 @@ namespace osmium {
* transport exceptions. The future also helps with keeping the
* data in order.
*/
using future_string_queue_type = osmium::thread::Queue<std::future<std::string>>;
using future_string_queue_type = future_queue_type<std::string>;
template <typename T>
inline void add_to_queue(osmium::thread::Queue<std::future<T>>& queue, T&& data) {
inline void add_to_queue(future_queue_type<T>& queue, T&& data) {
std::promise<T> promise;
queue.push(promise.get_future());
promise.set_value(std::forward<T>(data));
}
template <typename T>
inline void add_to_queue(osmium::thread::Queue<std::future<T>>& queue, std::exception_ptr&& exception) {
inline void add_to_queue(future_queue_type<T>& queue, std::exception_ptr&& exception) {
std::promise<T> promise;
queue.push(promise.get_future());
promise.set_exception(std::move(exception));
}
template <typename T>
inline void add_end_of_data_to_queue(osmium::thread::Queue<std::future<T>>& queue) {
inline void add_end_of_data_to_queue(future_queue_type<T>& queue) {
add_to_queue<T>(queue, T{});
}
@ -103,14 +106,12 @@ namespace osmium {
template <typename T>
class queue_wrapper {
using queue_type = osmium::thread::Queue<std::future<T>>;
queue_type& m_queue;
future_queue_type<T>& m_queue;
bool m_has_reached_end_of_data;
public:
explicit queue_wrapper(queue_type& queue) :
explicit queue_wrapper(future_queue_type<T>& queue) :
m_queue(queue),
m_has_reached_end_of_data(false) {
}

7
include/osmium/io/detail/read_write.hpp
View File

@ -35,7 +35,6 @@ DEALINGS IN THE SOFTWARE.
#include <cerrno>
#include <cstddef>
#include <errno.h>
#include <fcntl.h>
#include <string>
#include <system_error>
@ -84,7 +83,7 @@ namespace osmium {
#ifdef _WIN32
flags |= O_BINARY;
#endif
int fd = ::open(filename.c_str(), flags, 0666);
const int fd = ::open(filename.c_str(), flags, 0666);
if (fd < 0) {
throw std::system_error(errno, std::system_category(), std::string("Open failed for '") + filename + "'");
}
@ -108,7 +107,7 @@ namespace osmium {
#ifdef _WIN32
flags |= O_BINARY;
#endif
int fd = ::open(filename.c_str(), flags);
const int fd = ::open(filename.c_str(), flags);
if (fd < 0) {
throw std::system_error(errno, std::system_category(), std::string("Open failed for '") + filename + "'");
}
@ -133,7 +132,7 @@ namespace osmium {
if (write_count > max_write) {
write_count = max_write;
}
auto length = ::write(fd, output_buffer + offset, static_cast<unsigned int>(write_count));
const auto length = ::write(fd, output_buffer + offset, static_cast<unsigned int>(write_count));
if (length < 0) {
throw std::system_error(errno, std::system_category(), "Write failed");
}

77
include/osmium/io/detail/string_table.hpp
View File

@ -34,13 +34,14 @@ DEALINGS IN THE SOFTWARE.
*/
#include <cassert>
#include <cstddef>
#include <cstdint>
#include <cstdlib>
#include <cstring>
#include <iterator>
#include <list>
#include <map>
#include <string>
#include <unordered_map>
#include <utility>
#include <osmium/io/detail/pbf.hpp>
@ -69,8 +70,8 @@ namespace osmium {
std::list<std::string> m_chunks;
void add_chunk() {
m_chunks.push_front(std::string());
m_chunks.front().reserve(m_chunk_size);
m_chunks.emplace_back();
m_chunks.back().reserve(m_chunk_size);
}
public:
@ -82,6 +83,7 @@ namespace osmium {
}
void clear() noexcept {
assert(!m_chunks.empty());
m_chunks.erase(std::next(m_chunks.begin()), m_chunks.end());
m_chunks.front().clear();
}
@ -93,31 +95,38 @@ namespace osmium {
* allocated.
*/
const char* add(const char* string) {
size_t len = std::strlen(string) + 1;
const size_t len = std::strlen(string) + 1;
assert(len <= m_chunk_size);
size_t chunk_len = m_chunks.front().size();
if (chunk_len + len > m_chunks.front().capacity()) {
size_t chunk_len = m_chunks.back().size();
if (chunk_len + len > m_chunks.back().capacity()) {
add_chunk();
chunk_len = 0;
}
m_chunks.front().append(string);
m_chunks.front().append(1, '\0');
m_chunks.back().append(string);
m_chunks.back().append(1, '\0');
return m_chunks.front().c_str() + chunk_len;
return m_chunks.back().c_str() + chunk_len;
}
class const_iterator : public std::iterator<std::forward_iterator_tag, const char*> {
class const_iterator {
using it_type = std::list<std::string>::const_iterator;
typedef std::list<std::string>::const_iterator it_type;
it_type m_it;
const it_type m_last;
const char* m_pos;
public:
using iterator_category = std::forward_iterator_tag;
using value_type = const char*;
using difference_type = std::ptrdiff_t;
using pointer = value_type*;
using reference = value_type&;
const_iterator(it_type it, it_type last) :
m_it(it),
m_last(last),
@ -126,7 +135,7 @@ namespace osmium {
const_iterator& operator++() {
assert(m_it != m_last);
auto last_pos = m_it->c_str() + m_it->size();
const auto last_pos = m_it->c_str() + m_it->size();
while (m_pos != last_pos && *m_pos) ++m_pos;
if (m_pos != last_pos) ++m_pos;
if (m_pos == last_pos) {
@ -184,18 +193,33 @@ namespace osmium {
}
size_t get_used_bytes_in_last_chunk() const noexcept {
return m_chunks.front().size();
return m_chunks.back().size();
}
}; // class StringStore
struct StrComp {
struct str_equal {
bool operator()(const char* lhs, const char* rhs) const {
return strcmp(lhs, rhs) < 0;
bool operator()(const char* lhs, const char* rhs) const noexcept {
return lhs == rhs || std::strcmp(lhs, rhs) == 0;
}
}; // struct StrComp
}; // struct str_equal
struct djb2_hash {
size_t operator()(const char* str) const noexcept {
size_t hash = 5381;
int c;
while ((c = *str++)) {
hash = ((hash << 5) + hash) + c; /* hash * 33 + c */
}
return hash;
}
}; // struct djb2_hash
class StringTable {
@ -206,14 +230,23 @@ namespace osmium {
// Blob.
static constexpr const uint32_t max_entries = max_uncompressed_blob_size;
// There is one string table per PBF primitive block. Most of
// them are really small, because most blocks are full of nodes
// with no tags. But string tables can get really large for
// ways with many tags or for large relations.
// The chosen size is enough so that 99% of all string tables
// in typical OSM files will only need a single memory
// allocation.
static constexpr const size_t default_stringtable_chunk_size = 100 * 1024;
StringStore m_strings;
std::map<const char*, size_t, StrComp> m_index;
std::unordered_map<const char*, size_t, djb2_hash, str_equal> m_index;
uint32_t m_size;
public:
StringTable() :
m_strings(1024 * 1024),
explicit StringTable(size_t size = default_stringtable_chunk_size) :
m_strings(size),
m_index(),
m_size(0) {
m_strings.add("");
@ -231,7 +264,7 @@ namespace osmium {
}
uint32_t add(const char* s) {
auto f = m_index.find(s);
const auto f = m_index.find(s);
if (f != m_index.end()) {
return uint32_t(f->second);
}

39
include/osmium/io/detail/string_util.hpp
View File

@ -35,6 +35,7 @@ DEALINGS IN THE SOFTWARE.
#include <cassert>
#include <cstdint>
#include <cstdio>
#include <cstring>
#include <string>
#include <utility>
@ -100,9 +101,9 @@ namespace osmium {
static const size_t max_size = 0;
#endif
size_t old_size = out.size();
const size_t old_size = out.size();
int len = string_snprintf(out,
const int len = string_snprintf(out,
old_size,
max_size,
format,
@ -111,7 +112,7 @@ namespace osmium {
if (size_t(len) >= max_size) {
#ifndef NDEBUG
int len2 =
const int len2 =
#endif
string_snprintf(out,
old_size,
@ -124,12 +125,33 @@ namespace osmium {
out.resize(old_size + size_t(len));
}
// Write out the value with exactly two hex digits.
inline void append_2_hex_digits(std::string& out, uint32_t value, const char* const hex_digits) {
out += hex_digits[(value >> 4) & 0xf];
out += hex_digits[ value & 0xf];
}
// Write out the value with four or more hex digits.
inline void append_min_4_hex_digits(std::string& out, uint32_t value, const char* const hex_digits) {
auto
v = value & 0xf0000000; if (v) out += hex_digits[v >> 28];
v = value & 0x0f000000; if (v) out += hex_digits[v >> 24];
v = value & 0x00f00000; if (v) out += hex_digits[v >> 20];
v = value & 0x000f0000; if (v) out += hex_digits[v >> 16];
out += hex_digits[(value >> 12) & 0xf];
out += hex_digits[(value >> 8) & 0xf];
out += hex_digits[(value >> 4) & 0xf];
out += hex_digits[ value & 0xf];
}
inline void append_utf8_encoded_string(std::string& out, const char* data) {
static const char* lookup_hex = "0123456789abcdef";
const char* end = data + std::strlen(data);
while (data != end) {
const char* last = data;
uint32_t c = utf8::next(data, end);
const uint32_t c = utf8::next(data, end);
// This is a list of Unicode code points that we let
// through instead of escaping them. It is incomplete
@ -148,9 +170,9 @@ namespace osmium {
} else {
out += '%';
if (c <= 0xff) {
append_printf_formatted_string(out, "%02x", c);
append_2_hex_digits(out, c, lookup_hex);
} else {
append_printf_formatted_string(out, "%04x", c);
append_min_4_hex_digits(out, c, lookup_hex);
}
out += '%';
}
@ -174,6 +196,7 @@ namespace osmium {
}
inline void append_debug_encoded_string(std::string& out, const char* data, const char* prefix, const char* suffix) {
static const char* lookup_hex = "0123456789ABCDEF";
const char* end = data + std::strlen(data);
while (data != end) {
@ -194,7 +217,9 @@ namespace osmium {
out.append(last, data);
} else {
out.append(prefix);
append_printf_formatted_string(out, "<U+%04X>", c);
out.append("<U+");
append_min_4_hex_digits(out, c, lookup_hex);
out.append(">");
out.append(suffix);
}
}

Some files were not shown because too many files have changed in this diff Show More