We need to have a proper sed, even on MacOSX. So let's use the variable
introduced in 191aa0fd (🔧 overworked maintaner targets,
2019-03-28) in more places.
The current code uses std::stoi to convert the input string to an int
array_index. This limits the maximum addressable array size to ~2GB on
most platforms.
But all callers immediately convert the result of array_index to
BasicJsonType::size_type.
So let's parse it as unsigned long long, which allows us to have as
big arrays as available memory. And also makes the call sites nicer to
read.
One complication arises on platforms where size_type is smaller than
unsigned long long. We need to bail out on these if the parsed array
index does not fit into size_type.
In a future commit we want to add support for MSVC 2019. For that
version cmake requires us to pass in the architecture not part of the
generator but explicitly.
So let's pass that in always like that. This also removes the doubling
of the platform/architecture.
Using GLOB is slow and considered bad practice.
From https://cmake.org/cmake/help/latest/command/file.html:
> We do not recommend using GLOB to collect a list of source files from
> your source tree. If no CMakeLists.txt file changes when a source is
> added or removed then the generated build system cannot know when to ask
> CMake to regenerate. The CONFIGURE_DEPENDS flag may not work reliably on
> all generators, or if a new generator is added in the future that cannot
> support it, projects using it will be stuck. Even if CONFIGURE_DEPENDS
> works reliably, there is still a cost to perform the check on every
> rebuild.
For some gcc version (Ubuntu 5.5.0-12ubuntu1~16.04) the existing code
crashes when the minimum value of int64_t is outputted.
Resurrect the code from before 546e2cbf (🚨 fixed some warnings,
2019-03-13) but delegate the sign removal so that the compilers don't
complain about taking the negative value of an unsigned value.
In addition we also rewrite the expression so that we first increment
and then negate.
The definition of remove_sign(number_unsigned_t) is never called as
unsigned values are never negative.
The clang sanitizer tests, and there especially the unicode tests, can
hit the default timeout of 25 minutes (1500 seconds) quite easily, so
let's raise the timeout to 45 minutes (2700 seconds).
Clang UBSAN currently complains that the char * to input_buffer_adapter
is a nullptr.
Turns out it is actually required to accept nullptr, see for example
line 415 in input_adapters.hpp
...
// the address of first cannot be used: use nullptr
ia = std::make_shared<input_buffer_adapter>(nullptr, len);
....
Therefore we have to handle it gracefully here. We now also ignore the
length parameter l if b is a nullptr.
- Switch to clang-7
- Adapt PATH so that llvm-symbolizer can be found for useful stacktraces
- Adapt compile flags
"-O0" ensures much faster compile times
"-fno-sanitize-recover=all
-fsanitize-recover=unsigned-integer-overflow" this fails the build on
all issues except unsigned integer overflows. Not failing in this case
is required in combination with the sanitizer suppression file as only
recoverable errors can be suppressed.
The UBSAN suppression file ignores errors from stl_bvector.h (which
holds std::vector<bool>).
Clang reports that error as
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /usr/lib/gcc/x86_64-linux-gnu/6.3.0/../../../../include/c++/6.3.0/bits/stl_bvector.h:158:20 in
Start 34: test-deserialization_all
28/88 Test #71: test-testsuites_default .............***Failed 0.32 sec
/usr/lib/gcc/x86_64-linux-gnu/6.3.0/../../../../include/c++/6.3.0/bits/stl_bvector.h:158:20: runtime error: unsigned integer overflow: 0 - 1 cannot be represented in type 'unsigned int'
#0 0x628f72 in std::_Bit_iterator_base::_M_bump_down() /usr/lib/gcc/x86_64-linux-gnu/6.3.0/../../../../include/c++/6.3.0/bits/stl_bvector.h:158:20
#1 0x628d16 in std::_Bit_iterator::operator--() /usr/lib/gcc/x86_64-linux-gnu/6.3.0/../../../../include/c++/6.3.0/bits/stl_bvector.h:251:7
#2 0x634aac in std::vector<bool, std::allocator<bool> >::pop_back() /usr/lib/gcc/x86_64-linux-gnu/6.3.0/../../../../include/c++/6.3.0/bits/stl_bvector.h:1010:7
#3 0x61eff0 in bool nlohmann::detail::parser<nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer> >::sax_parse_internal<nlohmann::detail::json_sax_dom_parser<nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer> > >(nlohmann::detail::json_sax_dom_parser<nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer> >*) /home/firma/devel/json/include/nlohmann/detail/input/parser.hpp:439:28
#4 0x604864 in nlohmann::detail::parser<nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer> >::parse(bool, nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer>&) /home/firma/devel/json/include/nlohmann/detail/input/parser.hpp:116:13
#5 0x5f8079 in nlohmann::operator>>(std::istream&, nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer>&) /home/firma/devel/json/include/nlohmann/json.hpp:6356:42
#6 0x5e1d92 in _DOCTEST_ANON_FUNC_21() /home/firma/devel/json/test/src/unit-testsuites.cpp:343:9
#7 0x7207fe in doctest::Context::run() /home/firma/devel/json/test/thirdparty/doctest/doctest.h:5938:21
#8 0x72681a in main /home/firma/devel/json/test/thirdparty/doctest/doctest.h:6016:71
#9 0x7f75d22362e0 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x202e0)
#10 0x4c28b9 in _start (/home/firma/devel/json/build/test/test-testsuites+0x4c28b9)
The pop_back() in parser.hpp
assert(not states.empty());
-> states.pop_back();
triggers the UBSAN report. But the assertion above ensure that we only
call pop_back() on an non-empty vector, therefore this is a STL library
bug and thus must be ignored for us.
Thomas Braun