mirror of
https://github.com/nlohmann/json.git
synced 2026-09-29 19:20:30 +00:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
 Niels Lohmann
|
f6c115a9a6 | ||
|
Niels Lohmann
|
33ef25099d |
@@ -115,4 +115,3 @@ The default value is `0` (disabled — existing behavior is preserved).
|
||||
## Version history
|
||||
|
||||
- Added in version 3.13.0.
|
||||
- Planned to become the default (with the macro removed) in version 4.0.0.
|
||||
|
||||
@@ -14,7 +14,7 @@ work items are tracked in the [GitHub milestones](https://github.com/nlohmann/js
|
||||
opt-in.
|
||||
- **Keep the 3.x public API stable.** Releases follow [semantic versioning](https://semver.org). Changes that would
|
||||
break existing code are only added behind a feature macro, so users can opt in and test their code before a next
|
||||
major release, see [Version 4.0](#version-40).
|
||||
major release.
|
||||
- **Support a broad range of compilers and platforms.** The [CI](quality_assurance.md) keeps testing old and new
|
||||
versions of GCC, Clang, MSVC, and other compilers on Linux, macOS, and Windows.
|
||||
- **Keep the quality assurance up.** Every change keeps the test coverage at 100%, passes the static and dynamic
|
||||
@@ -37,65 +37,7 @@ work items are tracked in the [GitHub milestones](https://github.com/nlohmann/js
|
||||
|
||||
## Version 4.0
|
||||
|
||||
There is no release date for version 4.0 yet. Proposals that need a major version, for instance stricter type
|
||||
conversions, are collected in issue [#3453](https://github.com/nlohmann/json/issues/3453).
|
||||
|
||||
!!! note "Not final"
|
||||
|
||||
The plan for version 4.0 described below is not final and may still change: macros may be added to or removed from
|
||||
the list, and planned defaults may be revised. Any such change will be documented on this page.
|
||||
|
||||
### Trying out 4.0 today
|
||||
|
||||
Version 4.0 will not be developed on a separate branch. Instead, every breaking change is first added to a 3.x release
|
||||
behind a macro whose default keeps the 3.x behavior. Version 4.0 then switches the defaults and removes the macros.
|
||||
Version 4.0 is therefore the sum of these macros: you can try it on the 3.x release train today by defining each macro
|
||||
to its 4.0 value and fixing what no longer compiles or behaves differently. Once your code works with all of them, it
|
||||
is ready for version 4.0.
|
||||
|
||||
The following macros guard changes that are planned to become the default in version 4.0:
|
||||
|
||||
| Macro | 3.x default | 4.0 behavior | CMake option | Added |
|
||||
|------------------------------------------------------------------------------------------------------------------|-------------|-------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------|--------|
|
||||
| [`JSON_USE_IMPLICIT_CONVERSIONS`](../api/macros/json_use_implicit_conversions.md) | `1` | `0`: no implicit conversions from `basic_json` to other types; use [`get`](../api/basic_json/get.md) instead | [`JSON_ImplicitConversions`](../integration/cmake.md#json_implicitconversions) | 3.9.0 |
|
||||
| [`JSON_USE_GLOBAL_UDLS`](../api/macros/json_use_global_udls.md) | `1` | `0`: the string literals `_json` and `_json_pointer` are only available in namespace `nlohmann::literals` | [`JSON_GlobalUDLs`](../integration/cmake.md#json_globaludls) | 3.11.0 |
|
||||
| [`JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON`](../api/macros/json_use_legacy_discarded_value_comparison.md) | `0` | removed: the deprecated legacy comparison of discarded values can no longer be enabled | [`JSON_LegacyDiscardedValueComparison`](../integration/cmake.md#json_legacydiscardedvaluecomparison) | 3.11.0 |
|
||||
| [`JSON_BRACE_INIT_COPY_SEMANTICS`](../api/macros/json_brace_init_copy_semantics.md) | `0` | `1`: single-element brace initialization such as `#!cpp json j{obj};` copies the element instead of creating an array | – | 3.13.0 |
|
||||
| [`JSON_PRECISE_STREAM_POSITION`](../api/macros/json_precise_stream_position.md) | `0` | `1`: reading from a stream does not consume the character after a number | – | 3.13.0 |
|
||||
| [`JSON_STRICT_NUL_HANDLING`](../api/macros/json_strict_nul_handling.md) | `0` | `1`: a NUL byte in the input is a parse error instead of the end of input | [`JSON_StrictNulHandling`](../integration/cmake.md#json_strictnulhandling) | 3.13.0 |
|
||||
|
||||
For example, the following makes a 3.x release behave like version 4.0 with respect to these changes:
|
||||
|
||||
```cpp
|
||||
#define JSON_USE_IMPLICIT_CONVERSIONS 0
|
||||
#define JSON_USE_GLOBAL_UDLS 0
|
||||
#define JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON 0
|
||||
#define JSON_BRACE_INIT_COPY_SEMANTICS 1
|
||||
#define JSON_PRECISE_STREAM_POSITION 1
|
||||
#define JSON_STRICT_NUL_HANDLING 1
|
||||
#include <nlohmann/json.hpp>
|
||||
```
|
||||
|
||||
The macros must be defined before the library header is included; setting them once in the build system is the easiest
|
||||
way to achieve this.
|
||||
|
||||
### Removal of deprecated functions
|
||||
|
||||
Version 4.0 will remove all deprecated functions. Compiling with deprecation warnings enabled shows which of them your
|
||||
code still uses. The [migration guide](../integration/migration_guide.md#replace-deprecated-functions) shows how to
|
||||
replace each of them.
|
||||
|
||||
| Deprecated | Since | Migration |
|
||||
|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------|----------------------------------------------------------------------------------|
|
||||
| `#!cpp operator<<(basic_json&, std::istream&)` | 3.0.0 | [Parsing](../integration/migration_guide.md#parsing) |
|
||||
| `#!cpp operator>>(const basic_json&, std::ostream&)` | 3.0.0 | [Miscellaneous functions](../integration/migration_guide.md#miscellaneous-functions) |
|
||||
| `iterator_wrapper` | 3.1.0 | [Miscellaneous functions](../integration/migration_guide.md#miscellaneous-functions) |
|
||||
| [`parse`](../api/basic_json/parse.md), [`accept`](../api/basic_json/accept.md), and [`sax_parse`](../api/basic_json/sax_parse.md) with an initializer list `{ptr, len}` or `{first, last}` | 3.8.0 | [Parsing](../integration/migration_guide.md#parsing) |
|
||||
| [`from_bson`](../api/basic_json/from_bson.md), [`from_cbor`](../api/basic_json/from_cbor.md), [`from_msgpack`](../api/basic_json/from_msgpack.md), and [`from_ubjson`](../api/basic_json/from_ubjson.md) with `(ptr, len)` or an initializer list | 3.8.0 | [Parsing](../integration/migration_guide.md#parsing) |
|
||||
| [`json_pointer::operator string_t`](../api/json_pointer/operator_string_t.md) | 3.11.0 | [JSON Pointers](../integration/migration_guide.md#json-pointers) |
|
||||
| [`json_pointer`](../api/json_pointer/index.md) with a `basic_json` type as template argument, and the overloads of `value`, `contains`, `operator[]`, and `at` accepting such a pointer | 3.11.0 | [JSON Pointers](../integration/migration_guide.md#json-pointers) |
|
||||
| Comparing a [`json_pointer`](../api/json_pointer/index.md) with a string via [`operator==`](../api/json_pointer/operator_eq.md) or [`operator!=`](../api/json_pointer/operator_ne.md) | 3.11.2 | [JSON Pointers](../integration/migration_guide.md#json-pointers) |
|
||||
|
||||
The deprecated legacy comparison of discarded values is controlled by a macro and therefore listed in the table above.
|
||||
|
||||
New breaking changes will follow the same path: they are added to these tables when they land in a 3.x release.
|
||||
There is no decision yet on whether or when a version 4.0 with breaking changes will be released. Proposals that need
|
||||
a major version, for instance stricter type conversions, are collected in issue
|
||||
[#3453](https://github.com/nlohmann/json/issues/3453). Until then, such changes are only added as opt-in behavior
|
||||
behind feature macros.
|
||||
|
||||
@@ -1,13 +1,10 @@
|
||||
# Migration Guide
|
||||
|
||||
This page collects some guidelines on how to future-proof your code for future versions of this library.
|
||||
The [roadmap](../community/roadmap.md#version-40) lists what will change in version 4.0, including the macros that let
|
||||
you try its behavior with a 3.x release; this page describes how to adjust your code.
|
||||
|
||||
## Replace deprecated functions
|
||||
|
||||
The following functions have been deprecated and will be removed in the next major version (i.e., 4.0.0), see the
|
||||
[roadmap](../community/roadmap.md#removal-of-deprecated-functions) for an overview. All
|
||||
The following functions have been deprecated and will be removed in the next major version (i.e., 4.0.0). All
|
||||
deprecations are annotated with
|
||||
[`HEDLEY_DEPRECATED_FOR`](https://nemequ.github.io/hedley/api-reference.html#HEDLEY_DEPRECATED_FOR) to report which
|
||||
function to use instead.
|
||||
@@ -37,9 +34,8 @@ function to use instead.
|
||||
[`accept`](../api/basic_json/accept.md), [`sax_parse`](../api/basic_json/sax_parse.md),
|
||||
[`from_cbor`](../api/basic_json/from_cbor.md), [`from_msgpack`](../api/basic_json/from_msgpack.md),
|
||||
[`from_ubjson`](../api/basic_json/from_ubjson.md), and [`from_bson`](../api/basic_json/from_bson.md) via initializer
|
||||
lists is deprecated since 3.8.0. The same holds for passing a pointer and a length as two arguments to the `from_*`
|
||||
functions. Instead, pass two iterators; for instance, call `from_cbor(ptr, ptr+len)` instead of
|
||||
`from_cbor({ptr, len})` or `from_cbor(ptr, len)`.
|
||||
lists is deprecated since 3.8.0. Instead, pass two iterators; for instance, call `from_cbor(ptr, ptr+len)` instead of
|
||||
`from_cbor({ptr, len})`.
|
||||
|
||||
=== "Deprecated"
|
||||
|
||||
|
||||
@@ -9,10 +9,8 @@
|
||||
#pragma once
|
||||
|
||||
#include <array> // array
|
||||
#include <clocale> // localeconv
|
||||
#include <cstddef> // size_t
|
||||
#include <cstdio> // snprintf
|
||||
#include <cstdlib> // strtof, strtod, strtold, strtoll, strtoull
|
||||
#include <initializer_list> // initializer_list
|
||||
#include <string> // char_traits, string
|
||||
#include <utility> // move
|
||||
@@ -217,18 +215,6 @@ class lexer : public lexer_base<BasicJsonType>
|
||||
~lexer() = default;
|
||||
|
||||
private:
|
||||
/////////////////////
|
||||
// locales
|
||||
/////////////////////
|
||||
|
||||
/// return the decimal point of the current locale
|
||||
static char get_decimal_point() noexcept
|
||||
{
|
||||
const auto* loc = localeconv();
|
||||
JSON_ASSERT(loc != nullptr);
|
||||
return (loc->decimal_point == nullptr) ? '.' : *(loc->decimal_point);
|
||||
}
|
||||
|
||||
/////////////////////
|
||||
// scan functions
|
||||
/////////////////////
|
||||
@@ -1036,24 +1022,6 @@ class lexer : public lexer_base<BasicJsonType>
|
||||
}
|
||||
}
|
||||
|
||||
JSON_HEDLEY_NON_NULL(2)
|
||||
static void strtof(float& f, const char* str, char** endptr) noexcept
|
||||
{
|
||||
f = std::strtof(str, endptr);
|
||||
}
|
||||
|
||||
JSON_HEDLEY_NON_NULL(2)
|
||||
static void strtof(double& f, const char* str, char** endptr) noexcept
|
||||
{
|
||||
f = std::strtod(str, endptr);
|
||||
}
|
||||
|
||||
JSON_HEDLEY_NON_NULL(2)
|
||||
static void strtof(long double& f, const char* str, char** endptr) noexcept
|
||||
{
|
||||
f = std::strtold(str, endptr);
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief scan a number literal
|
||||
|
||||
@@ -1093,7 +1061,7 @@ class lexer : public lexer_base<BasicJsonType>
|
||||
@note The scanner is independent of the current locale: token_buffer
|
||||
always holds `.`. Only the std::strtod fallback of convert_number()
|
||||
depends on the locale, and it looks up the decimal point right
|
||||
before converting (see convert_float_locale_aware()).
|
||||
before converting (see detail::convert_float_locale_aware()).
|
||||
*/
|
||||
token_type scan_number() // lgtm [cpp/use-of-goto] `goto` is used in this function to implement the number-parsing state machine described above. By design, any finite input will eventually reach the "done" state or return token_type::parse_error. In each intermediate state, 1 byte of the input is appended to the token_buffer vector, and only the already initialized variables token_buffer, number_type, and error_message are manipulated.
|
||||
{
|
||||
@@ -1424,59 +1392,6 @@ scan_number_done:
|
||||
return token_type::uninitialized;
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief check whether Clinger's fast path can still succeed for this token
|
||||
|
||||
parse_float_fast() needs a significand below 2^53. A mantissa with 17 or
|
||||
more significant digits is at least 10^16 and therefore always exceeds it,
|
||||
so calling the fast path would walk the token one extra time only to
|
||||
decline before strtod has to run anyway.
|
||||
|
||||
Significant digits are the mantissa's digits from the first nonzero one on;
|
||||
the sign, the decimal point, leading zeros, and the exponent do not count.
|
||||
The answer is derived from indices - the digits are not scanned again - so
|
||||
this stays off the hot path of the number scanners.
|
||||
|
||||
@param[in] mantissa_end offset just past the last mantissa byte in
|
||||
token_buffer
|
||||
@return false if parse_float_fast() is guaranteed to decline
|
||||
*/
|
||||
bool mantissa_fits_clinger(std::size_t mantissa_end) const
|
||||
{
|
||||
// 10^16 already exceeds 2^53, so 17 digits can never fit
|
||||
constexpr std::size_t limit = 17;
|
||||
|
||||
const std::size_t neg = (!token_buffer.empty() && token_buffer[0] == '-') ? 1u : 0u;
|
||||
const std::size_t has_dot = (decimal_point_position != std::string::npos) ? 1u : 0u;
|
||||
// the JSON grammar restricts the integer part to "0" or [1-9][0-9]*, so
|
||||
// a leading zero can only be a lone "0", which is not significant
|
||||
const std::size_t lead_zero = (token_buffer[neg] == '0') ? 1u : 0u;
|
||||
JSON_ASSERT(mantissa_end >= neg + has_dot + lead_zero);
|
||||
std::size_t digits = mantissa_end - neg - has_dot - lead_zero;
|
||||
|
||||
if (JSON_HEDLEY_LIKELY(digits < limit))
|
||||
{
|
||||
return true;
|
||||
}
|
||||
|
||||
// Only a number below 1 can carry further insignificant zeros, and only
|
||||
// while the count stays at the limit does removing them change the
|
||||
// answer - so this loop is skipped for all but a few tokens. The
|
||||
// fraction is located through decimal_point_position rather than by
|
||||
// searching '.'.
|
||||
if (lead_zero != 0)
|
||||
{
|
||||
JSON_ASSERT(has_dot != 0); // an integer "0" cannot reach the limit
|
||||
for (std::size_t i = decimal_point_position + 1;
|
||||
digits >= limit && i < mantissa_end && token_buffer[i] == '0'; ++i)
|
||||
{
|
||||
--digits;
|
||||
}
|
||||
}
|
||||
|
||||
return digits < limit;
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief convert the number text in token_buffer to its value and token type
|
||||
|
||||
@@ -1490,7 +1405,7 @@ scan_number_done:
|
||||
token_buffer (the index of 'e'/'E', or
|
||||
token_buffer.size() when there is no exponent);
|
||||
used to skip Clinger's fast path when it cannot
|
||||
possibly succeed - see mantissa_fits_clinger()
|
||||
possibly succeed - see detail::mantissa_fits_clinger()
|
||||
*/
|
||||
token_type convert_number(token_type number_type, std::size_t mantissa_end)
|
||||
{
|
||||
@@ -1563,77 +1478,15 @@ scan_number_done:
|
||||
// (Eisel-Lemire, locale-independent, correctly rounded) when available;
|
||||
// otherwise the exact Clinger fast path (double only); otherwise the
|
||||
// locale-aware strtof/strtod/strtold.
|
||||
if (parse_float_from_chars(num_begin, num_end, value_float))
|
||||
{
|
||||
return token_type::value_float;
|
||||
}
|
||||
// Skipping a fast path that cannot succeed is lossless and saves a full
|
||||
// extra pass over the token's bytes, which otherwise shows up on
|
||||
// high-precision inputs such as canada.json
|
||||
if (mantissa_fits_clinger(mantissa_end)
|
||||
&& parse_float_fast(num_begin, num_end, value_float))
|
||||
if (convert_float_fast(num_begin, num_end, decimal_point_position, mantissa_end, value_float))
|
||||
{
|
||||
return token_type::value_float;
|
||||
}
|
||||
|
||||
convert_float_locale_aware();
|
||||
convert_float_locale_aware(token_buffer, decimal_point_position, value_float);
|
||||
return token_type::value_float;
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief convert the float in token_buffer with strtof/strtod/strtold
|
||||
|
||||
These functions expect the decimal point of the *current* locale, so it is
|
||||
looked up right before the conversion instead of once when the lexer is
|
||||
constructed: a locale change in between (by a parser callback, a SAX
|
||||
handler, or another thread) must not truncate the value (#5198). The
|
||||
token has been validated before, so if the conversion stops early and the
|
||||
decimal point changed in the meantime, the locale changed between the
|
||||
lookup and the call, and the conversion is repeated with the new decimal
|
||||
point. If the decimal point did not change, a retry cannot succeed: the
|
||||
locale's decimal point is not a single character (e.g., the two-byte
|
||||
U+066B of ar_EG.UTF-8 or fa_IR.UTF-8) and cannot be substituted in place.
|
||||
The value strtod parsed up to that point is kept, as before this change.
|
||||
|
||||
Note that changing the locale in another thread *while* strtod runs is
|
||||
undefined behavior of the C library, which this function cannot prevent.
|
||||
*/
|
||||
void convert_float_locale_aware()
|
||||
{
|
||||
const bool has_dot = decimal_point_position != std::string::npos;
|
||||
char decimal_point = get_decimal_point();
|
||||
for (;;)
|
||||
{
|
||||
const bool substitute = has_dot && decimal_point != '.';
|
||||
if (substitute)
|
||||
{
|
||||
token_buffer[decimal_point_position] = static_cast<typename string_t::value_type>(decimal_point);
|
||||
}
|
||||
|
||||
char* endptr = nullptr; // NOLINT(misc-const-correctness,cppcoreguidelines-pro-type-vararg,hicpp-vararg)
|
||||
strtof(value_float, token_buffer.data(), &endptr);
|
||||
|
||||
if (substitute)
|
||||
{
|
||||
// get_string() hands the token to the SAX interface with '.'
|
||||
token_buffer[decimal_point_position] = '.';
|
||||
}
|
||||
|
||||
if (JSON_HEDLEY_LIKELY(endptr == token_buffer.data() + token_buffer.size()))
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
// retry only if the locale changed; otherwise, this would loop forever
|
||||
const char current_decimal_point = get_decimal_point();
|
||||
if (current_decimal_point == decimal_point)
|
||||
{
|
||||
return;
|
||||
}
|
||||
decimal_point = current_decimal_point;
|
||||
}
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief contiguous fast path for scanning a number
|
||||
|
||||
|
||||
@@ -10,9 +10,12 @@
|
||||
|
||||
#include <array> // array
|
||||
#include <cfloat> // FLT_EVAL_METHOD
|
||||
#include <clocale> // localeconv
|
||||
#include <cstddef> // size_t
|
||||
#include <cstdint> // int64_t, uint64_t
|
||||
#include <cstdlib> // strtof, strtod, strtold
|
||||
#include <limits> // numeric_limits
|
||||
#include <string> // string
|
||||
|
||||
#include <nlohmann/detail/macro_scope.hpp>
|
||||
|
||||
@@ -29,8 +32,9 @@
|
||||
|
||||
// This file contains the value-conversion helpers used by the lexer to turn an
|
||||
// already-validated number token into a value, without the locale/errno
|
||||
// overhead of std::strtoull/std::strtod. They are free functions so the lexer
|
||||
// stays focused on scanning; see lexer::convert_number().
|
||||
// overhead of std::strtoull/std::strtod where possible. They are free functions
|
||||
// so the lexer stays focused on scanning (see lexer::convert_number()) and so
|
||||
// that other parsers of JSON text can convert tokens exactly like it does.
|
||||
|
||||
NLOHMANN_JSON_NAMESPACE_BEGIN
|
||||
namespace detail
|
||||
@@ -293,5 +297,183 @@ bool parse_float_from_chars(const char* first, const char* last, FloatType& out)
|
||||
#endif
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief check whether Clinger's fast path can still succeed for a float token
|
||||
|
||||
parse_float_fast() needs a significand below 2^53. A mantissa with 17 or
|
||||
more significant digits is at least 10^16 and therefore always exceeds it,
|
||||
so calling the fast path would walk the token one extra time only to
|
||||
decline before strtod has to run anyway.
|
||||
|
||||
Significant digits are the mantissa's digits from the first nonzero one on;
|
||||
the sign, the decimal point, leading zeros, and the exponent do not count.
|
||||
The answer is derived from indices - the digits are not scanned again - so
|
||||
this stays off the hot path of the number scanners.
|
||||
|
||||
@param[in] token the validated number token ('.' as decimal point)
|
||||
@param[in] decimal_point_position index of the '.' in @a token, or
|
||||
std::string::npos if there is none
|
||||
@param[in] mantissa_end offset just past the last mantissa byte
|
||||
@return false if parse_float_fast() is guaranteed to decline
|
||||
*/
|
||||
inline bool mantissa_fits_clinger(const char* token, std::size_t decimal_point_position, std::size_t mantissa_end) noexcept
|
||||
{
|
||||
// 10^16 already exceeds 2^53, so 17 digits can never fit
|
||||
constexpr std::size_t limit = 17;
|
||||
|
||||
const std::size_t neg = (token[0] == '-') ? 1u : 0u;
|
||||
const std::size_t has_dot = (decimal_point_position != std::string::npos) ? 1u : 0u;
|
||||
// the JSON grammar restricts the integer part to "0" or [1-9][0-9]*, so
|
||||
// a leading zero can only be a lone "0", which is not significant
|
||||
const std::size_t lead_zero = (token[neg] == '0') ? 1u : 0u;
|
||||
JSON_ASSERT(mantissa_end >= neg + has_dot + lead_zero);
|
||||
std::size_t digits = mantissa_end - neg - has_dot - lead_zero;
|
||||
|
||||
if (JSON_HEDLEY_LIKELY(digits < limit))
|
||||
{
|
||||
return true;
|
||||
}
|
||||
|
||||
// Only a number below 1 can carry further insignificant zeros, and only
|
||||
// while the count stays at the limit does removing them change the
|
||||
// answer - so this loop is skipped for all but a few tokens. The
|
||||
// fraction is located through decimal_point_position rather than by
|
||||
// searching '.'.
|
||||
if (lead_zero != 0)
|
||||
{
|
||||
JSON_ASSERT(has_dot != 0); // an integer "0" cannot reach the limit
|
||||
for (std::size_t i = decimal_point_position + 1;
|
||||
digits >= limit && i < mantissa_end && token[i] == '0'; ++i)
|
||||
{
|
||||
--digits;
|
||||
}
|
||||
}
|
||||
|
||||
return digits < limit;
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief convert a validated float token without the C library, if possible
|
||||
|
||||
Tries std::from_chars (when available) and then Clinger's exact fast path
|
||||
(double only), skipping the latter when it cannot succeed.
|
||||
|
||||
@param[in] first pointer to the first character of the token
|
||||
@param[in] last pointer past the last character
|
||||
@param[in] decimal_point_position index of the '.' in the token, or
|
||||
std::string::npos if there is none
|
||||
@param[in] mantissa_end offset just past the last mantissa byte (the
|
||||
index of 'e'/'E', or the token length)
|
||||
@param[out] value the converted value on success
|
||||
@return true if the value was converted; false if convert_float_locale_aware()
|
||||
must convert it
|
||||
*/
|
||||
template<typename FloatType>
|
||||
bool convert_float_fast(const char* first, const char* last, std::size_t decimal_point_position,
|
||||
std::size_t mantissa_end, FloatType& value) noexcept
|
||||
{
|
||||
if (parse_float_from_chars(first, last, value))
|
||||
{
|
||||
return true;
|
||||
}
|
||||
// Skipping a fast path that cannot succeed is lossless and saves a full
|
||||
// extra pass over the token's bytes, which otherwise shows up on
|
||||
// high-precision inputs such as canada.json
|
||||
return mantissa_fits_clinger(first, decimal_point_position, mantissa_end)
|
||||
&& parse_float_fast(first, last, value);
|
||||
}
|
||||
|
||||
/// std::strtof, std::strtod, or std::strtold, chosen by the type of @a f
|
||||
JSON_HEDLEY_NON_NULL(2)
|
||||
inline void strtof_by_type(float& f, const char* str, char** endptr) noexcept
|
||||
{
|
||||
f = std::strtof(str, endptr);
|
||||
}
|
||||
|
||||
/// std::strtof, std::strtod, or std::strtold, chosen by the type of @a f
|
||||
JSON_HEDLEY_NON_NULL(2)
|
||||
inline void strtof_by_type(double& f, const char* str, char** endptr) noexcept
|
||||
{
|
||||
f = std::strtod(str, endptr);
|
||||
}
|
||||
|
||||
/// std::strtof, std::strtod, or std::strtold, chosen by the type of @a f
|
||||
JSON_HEDLEY_NON_NULL(2)
|
||||
inline void strtof_by_type(long double& f, const char* str, char** endptr) noexcept
|
||||
{
|
||||
f = std::strtold(str, endptr);
|
||||
}
|
||||
|
||||
/// return the decimal point of the current locale
|
||||
inline char get_decimal_point() noexcept
|
||||
{
|
||||
const auto* loc = localeconv();
|
||||
JSON_ASSERT(loc != nullptr);
|
||||
return (loc->decimal_point == nullptr) ? '.' : *(loc->decimal_point);
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief convert a validated float token with strtof/strtod/strtold
|
||||
|
||||
These functions expect the decimal point of the *current* locale, so it is
|
||||
looked up right before the conversion instead of once when the lexer is
|
||||
constructed: a locale change in between (by a parser callback, a SAX
|
||||
handler, or another thread) must not truncate the value (#5198). The
|
||||
token has been validated before, so if the conversion stops early and the
|
||||
decimal point changed in the meantime, the locale changed between the
|
||||
lookup and the call, and the conversion is repeated with the new decimal
|
||||
point. If the decimal point did not change, a retry cannot succeed: the
|
||||
locale's decimal point is not a single character (e.g., the two-byte
|
||||
U+066B of ar_EG.UTF-8 or fa_IR.UTF-8) and cannot be substituted in place.
|
||||
The value strtod parsed up to that point is kept, as before this change.
|
||||
|
||||
Note that changing the locale in another thread *while* strtod runs is
|
||||
undefined behavior of the C library, which this function cannot prevent.
|
||||
|
||||
@param[in,out] token the token with '.' as decimal point; its
|
||||
decimal point is replaced during the
|
||||
conversion and restored afterwards
|
||||
(data() must be NUL-terminated)
|
||||
@param[in] decimal_point_position index of the '.' in @a token, or
|
||||
std::string::npos if there is none
|
||||
@param[out] value the converted value
|
||||
*/
|
||||
template<typename StringType, typename FloatType>
|
||||
void convert_float_locale_aware(StringType& token, std::size_t decimal_point_position, FloatType& value)
|
||||
{
|
||||
const bool has_dot = decimal_point_position != std::string::npos;
|
||||
char decimal_point = get_decimal_point();
|
||||
for (;;)
|
||||
{
|
||||
const bool substitute = has_dot && decimal_point != '.';
|
||||
if (substitute)
|
||||
{
|
||||
token[decimal_point_position] = static_cast<typename StringType::value_type>(decimal_point);
|
||||
}
|
||||
|
||||
char* endptr = nullptr; // NOLINT(misc-const-correctness,cppcoreguidelines-pro-type-vararg,hicpp-vararg)
|
||||
strtof_by_type(value, token.data(), &endptr);
|
||||
|
||||
if (substitute)
|
||||
{
|
||||
// the caller hands the token on (e.g. to the SAX interface) with '.'
|
||||
token[decimal_point_position] = '.';
|
||||
}
|
||||
|
||||
if (JSON_HEDLEY_LIKELY(endptr == token.data() + token.size()))
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
// retry only if the locale changed; otherwise, this would loop forever
|
||||
const char current_decimal_point = get_decimal_point();
|
||||
if (current_decimal_point == decimal_point)
|
||||
{
|
||||
return;
|
||||
}
|
||||
decimal_point = current_decimal_point;
|
||||
}
|
||||
}
|
||||
|
||||
} // namespace detail
|
||||
NLOHMANN_JSON_NAMESPACE_END
|
||||
|
||||
+188
-153
@@ -8472,10 +8472,8 @@ NLOHMANN_JSON_NAMESPACE_END
|
||||
|
||||
|
||||
#include <array> // array
|
||||
#include <clocale> // localeconv
|
||||
#include <cstddef> // size_t
|
||||
#include <cstdio> // snprintf
|
||||
#include <cstdlib> // strtof, strtod, strtold, strtoll, strtoull
|
||||
#include <initializer_list> // initializer_list
|
||||
#include <string> // char_traits, string
|
||||
#include <utility> // move
|
||||
@@ -8496,9 +8494,12 @@ NLOHMANN_JSON_NAMESPACE_END
|
||||
|
||||
#include <array> // array
|
||||
#include <cfloat> // FLT_EVAL_METHOD
|
||||
#include <clocale> // localeconv
|
||||
#include <cstddef> // size_t
|
||||
#include <cstdint> // int64_t, uint64_t
|
||||
#include <cstdlib> // strtof, strtod, strtold
|
||||
#include <limits> // numeric_limits
|
||||
#include <string> // string
|
||||
|
||||
// #include <nlohmann/detail/macro_scope.hpp>
|
||||
|
||||
@@ -8516,8 +8517,9 @@ NLOHMANN_JSON_NAMESPACE_END
|
||||
|
||||
// This file contains the value-conversion helpers used by the lexer to turn an
|
||||
// already-validated number token into a value, without the locale/errno
|
||||
// overhead of std::strtoull/std::strtod. They are free functions so the lexer
|
||||
// stays focused on scanning; see lexer::convert_number().
|
||||
// overhead of std::strtoull/std::strtod where possible. They are free functions
|
||||
// so the lexer stays focused on scanning (see lexer::convert_number()) and so
|
||||
// that other parsers of JSON text can convert tokens exactly like it does.
|
||||
|
||||
NLOHMANN_JSON_NAMESPACE_BEGIN
|
||||
namespace detail
|
||||
@@ -8780,6 +8782,184 @@ bool parse_float_from_chars(const char* first, const char* last, FloatType& out)
|
||||
#endif
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief check whether Clinger's fast path can still succeed for a float token
|
||||
|
||||
parse_float_fast() needs a significand below 2^53. A mantissa with 17 or
|
||||
more significant digits is at least 10^16 and therefore always exceeds it,
|
||||
so calling the fast path would walk the token one extra time only to
|
||||
decline before strtod has to run anyway.
|
||||
|
||||
Significant digits are the mantissa's digits from the first nonzero one on;
|
||||
the sign, the decimal point, leading zeros, and the exponent do not count.
|
||||
The answer is derived from indices - the digits are not scanned again - so
|
||||
this stays off the hot path of the number scanners.
|
||||
|
||||
@param[in] token the validated number token ('.' as decimal point)
|
||||
@param[in] decimal_point_position index of the '.' in @a token, or
|
||||
std::string::npos if there is none
|
||||
@param[in] mantissa_end offset just past the last mantissa byte
|
||||
@return false if parse_float_fast() is guaranteed to decline
|
||||
*/
|
||||
inline bool mantissa_fits_clinger(const char* token, std::size_t decimal_point_position, std::size_t mantissa_end) noexcept
|
||||
{
|
||||
// 10^16 already exceeds 2^53, so 17 digits can never fit
|
||||
constexpr std::size_t limit = 17;
|
||||
|
||||
const std::size_t neg = (token[0] == '-') ? 1u : 0u;
|
||||
const std::size_t has_dot = (decimal_point_position != std::string::npos) ? 1u : 0u;
|
||||
// the JSON grammar restricts the integer part to "0" or [1-9][0-9]*, so
|
||||
// a leading zero can only be a lone "0", which is not significant
|
||||
const std::size_t lead_zero = (token[neg] == '0') ? 1u : 0u;
|
||||
JSON_ASSERT(mantissa_end >= neg + has_dot + lead_zero);
|
||||
std::size_t digits = mantissa_end - neg - has_dot - lead_zero;
|
||||
|
||||
if (JSON_HEDLEY_LIKELY(digits < limit))
|
||||
{
|
||||
return true;
|
||||
}
|
||||
|
||||
// Only a number below 1 can carry further insignificant zeros, and only
|
||||
// while the count stays at the limit does removing them change the
|
||||
// answer - so this loop is skipped for all but a few tokens. The
|
||||
// fraction is located through decimal_point_position rather than by
|
||||
// searching '.'.
|
||||
if (lead_zero != 0)
|
||||
{
|
||||
JSON_ASSERT(has_dot != 0); // an integer "0" cannot reach the limit
|
||||
for (std::size_t i = decimal_point_position + 1;
|
||||
digits >= limit && i < mantissa_end && token[i] == '0'; ++i)
|
||||
{
|
||||
--digits;
|
||||
}
|
||||
}
|
||||
|
||||
return digits < limit;
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief convert a validated float token without the C library, if possible
|
||||
|
||||
Tries std::from_chars (when available) and then Clinger's exact fast path
|
||||
(double only), skipping the latter when it cannot succeed.
|
||||
|
||||
@param[in] first pointer to the first character of the token
|
||||
@param[in] last pointer past the last character
|
||||
@param[in] decimal_point_position index of the '.' in the token, or
|
||||
std::string::npos if there is none
|
||||
@param[in] mantissa_end offset just past the last mantissa byte (the
|
||||
index of 'e'/'E', or the token length)
|
||||
@param[out] value the converted value on success
|
||||
@return true if the value was converted; false if convert_float_locale_aware()
|
||||
must convert it
|
||||
*/
|
||||
template<typename FloatType>
|
||||
bool convert_float_fast(const char* first, const char* last, std::size_t decimal_point_position,
|
||||
std::size_t mantissa_end, FloatType& value) noexcept
|
||||
{
|
||||
if (parse_float_from_chars(first, last, value))
|
||||
{
|
||||
return true;
|
||||
}
|
||||
// Skipping a fast path that cannot succeed is lossless and saves a full
|
||||
// extra pass over the token's bytes, which otherwise shows up on
|
||||
// high-precision inputs such as canada.json
|
||||
return mantissa_fits_clinger(first, decimal_point_position, mantissa_end)
|
||||
&& parse_float_fast(first, last, value);
|
||||
}
|
||||
|
||||
/// std::strtof, std::strtod, or std::strtold, chosen by the type of @a f
|
||||
JSON_HEDLEY_NON_NULL(2)
|
||||
inline void strtof_by_type(float& f, const char* str, char** endptr) noexcept
|
||||
{
|
||||
f = std::strtof(str, endptr);
|
||||
}
|
||||
|
||||
/// std::strtof, std::strtod, or std::strtold, chosen by the type of @a f
|
||||
JSON_HEDLEY_NON_NULL(2)
|
||||
inline void strtof_by_type(double& f, const char* str, char** endptr) noexcept
|
||||
{
|
||||
f = std::strtod(str, endptr);
|
||||
}
|
||||
|
||||
/// std::strtof, std::strtod, or std::strtold, chosen by the type of @a f
|
||||
JSON_HEDLEY_NON_NULL(2)
|
||||
inline void strtof_by_type(long double& f, const char* str, char** endptr) noexcept
|
||||
{
|
||||
f = std::strtold(str, endptr);
|
||||
}
|
||||
|
||||
/// return the decimal point of the current locale
|
||||
inline char get_decimal_point() noexcept
|
||||
{
|
||||
const auto* loc = localeconv();
|
||||
JSON_ASSERT(loc != nullptr);
|
||||
return (loc->decimal_point == nullptr) ? '.' : *(loc->decimal_point);
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief convert a validated float token with strtof/strtod/strtold
|
||||
|
||||
These functions expect the decimal point of the *current* locale, so it is
|
||||
looked up right before the conversion instead of once when the lexer is
|
||||
constructed: a locale change in between (by a parser callback, a SAX
|
||||
handler, or another thread) must not truncate the value (#5198). The
|
||||
token has been validated before, so if the conversion stops early and the
|
||||
decimal point changed in the meantime, the locale changed between the
|
||||
lookup and the call, and the conversion is repeated with the new decimal
|
||||
point. If the decimal point did not change, a retry cannot succeed: the
|
||||
locale's decimal point is not a single character (e.g., the two-byte
|
||||
U+066B of ar_EG.UTF-8 or fa_IR.UTF-8) and cannot be substituted in place.
|
||||
The value strtod parsed up to that point is kept, as before this change.
|
||||
|
||||
Note that changing the locale in another thread *while* strtod runs is
|
||||
undefined behavior of the C library, which this function cannot prevent.
|
||||
|
||||
@param[in,out] token the token with '.' as decimal point; its
|
||||
decimal point is replaced during the
|
||||
conversion and restored afterwards
|
||||
(data() must be NUL-terminated)
|
||||
@param[in] decimal_point_position index of the '.' in @a token, or
|
||||
std::string::npos if there is none
|
||||
@param[out] value the converted value
|
||||
*/
|
||||
template<typename StringType, typename FloatType>
|
||||
void convert_float_locale_aware(StringType& token, std::size_t decimal_point_position, FloatType& value)
|
||||
{
|
||||
const bool has_dot = decimal_point_position != std::string::npos;
|
||||
char decimal_point = get_decimal_point();
|
||||
for (;;)
|
||||
{
|
||||
const bool substitute = has_dot && decimal_point != '.';
|
||||
if (substitute)
|
||||
{
|
||||
token[decimal_point_position] = static_cast<typename StringType::value_type>(decimal_point);
|
||||
}
|
||||
|
||||
char* endptr = nullptr; // NOLINT(misc-const-correctness,cppcoreguidelines-pro-type-vararg,hicpp-vararg)
|
||||
strtof_by_type(value, token.data(), &endptr);
|
||||
|
||||
if (substitute)
|
||||
{
|
||||
// the caller hands the token on (e.g. to the SAX interface) with '.'
|
||||
token[decimal_point_position] = '.';
|
||||
}
|
||||
|
||||
if (JSON_HEDLEY_LIKELY(endptr == token.data() + token.size()))
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
// retry only if the locale changed; otherwise, this would loop forever
|
||||
const char current_decimal_point = get_decimal_point();
|
||||
if (current_decimal_point == decimal_point)
|
||||
{
|
||||
return;
|
||||
}
|
||||
decimal_point = current_decimal_point;
|
||||
}
|
||||
}
|
||||
|
||||
} // namespace detail
|
||||
NLOHMANN_JSON_NAMESPACE_END
|
||||
|
||||
@@ -9309,18 +9489,6 @@ class lexer : public lexer_base<BasicJsonType>
|
||||
~lexer() = default;
|
||||
|
||||
private:
|
||||
/////////////////////
|
||||
// locales
|
||||
/////////////////////
|
||||
|
||||
/// return the decimal point of the current locale
|
||||
static char get_decimal_point() noexcept
|
||||
{
|
||||
const auto* loc = localeconv();
|
||||
JSON_ASSERT(loc != nullptr);
|
||||
return (loc->decimal_point == nullptr) ? '.' : *(loc->decimal_point);
|
||||
}
|
||||
|
||||
/////////////////////
|
||||
// scan functions
|
||||
/////////////////////
|
||||
@@ -10128,24 +10296,6 @@ class lexer : public lexer_base<BasicJsonType>
|
||||
}
|
||||
}
|
||||
|
||||
JSON_HEDLEY_NON_NULL(2)
|
||||
static void strtof(float& f, const char* str, char** endptr) noexcept
|
||||
{
|
||||
f = std::strtof(str, endptr);
|
||||
}
|
||||
|
||||
JSON_HEDLEY_NON_NULL(2)
|
||||
static void strtof(double& f, const char* str, char** endptr) noexcept
|
||||
{
|
||||
f = std::strtod(str, endptr);
|
||||
}
|
||||
|
||||
JSON_HEDLEY_NON_NULL(2)
|
||||
static void strtof(long double& f, const char* str, char** endptr) noexcept
|
||||
{
|
||||
f = std::strtold(str, endptr);
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief scan a number literal
|
||||
|
||||
@@ -10185,7 +10335,7 @@ class lexer : public lexer_base<BasicJsonType>
|
||||
@note The scanner is independent of the current locale: token_buffer
|
||||
always holds `.`. Only the std::strtod fallback of convert_number()
|
||||
depends on the locale, and it looks up the decimal point right
|
||||
before converting (see convert_float_locale_aware()).
|
||||
before converting (see detail::convert_float_locale_aware()).
|
||||
*/
|
||||
token_type scan_number() // lgtm [cpp/use-of-goto] `goto` is used in this function to implement the number-parsing state machine described above. By design, any finite input will eventually reach the "done" state or return token_type::parse_error. In each intermediate state, 1 byte of the input is appended to the token_buffer vector, and only the already initialized variables token_buffer, number_type, and error_message are manipulated.
|
||||
{
|
||||
@@ -10516,59 +10666,6 @@ scan_number_done:
|
||||
return token_type::uninitialized;
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief check whether Clinger's fast path can still succeed for this token
|
||||
|
||||
parse_float_fast() needs a significand below 2^53. A mantissa with 17 or
|
||||
more significant digits is at least 10^16 and therefore always exceeds it,
|
||||
so calling the fast path would walk the token one extra time only to
|
||||
decline before strtod has to run anyway.
|
||||
|
||||
Significant digits are the mantissa's digits from the first nonzero one on;
|
||||
the sign, the decimal point, leading zeros, and the exponent do not count.
|
||||
The answer is derived from indices - the digits are not scanned again - so
|
||||
this stays off the hot path of the number scanners.
|
||||
|
||||
@param[in] mantissa_end offset just past the last mantissa byte in
|
||||
token_buffer
|
||||
@return false if parse_float_fast() is guaranteed to decline
|
||||
*/
|
||||
bool mantissa_fits_clinger(std::size_t mantissa_end) const
|
||||
{
|
||||
// 10^16 already exceeds 2^53, so 17 digits can never fit
|
||||
constexpr std::size_t limit = 17;
|
||||
|
||||
const std::size_t neg = (!token_buffer.empty() && token_buffer[0] == '-') ? 1u : 0u;
|
||||
const std::size_t has_dot = (decimal_point_position != std::string::npos) ? 1u : 0u;
|
||||
// the JSON grammar restricts the integer part to "0" or [1-9][0-9]*, so
|
||||
// a leading zero can only be a lone "0", which is not significant
|
||||
const std::size_t lead_zero = (token_buffer[neg] == '0') ? 1u : 0u;
|
||||
JSON_ASSERT(mantissa_end >= neg + has_dot + lead_zero);
|
||||
std::size_t digits = mantissa_end - neg - has_dot - lead_zero;
|
||||
|
||||
if (JSON_HEDLEY_LIKELY(digits < limit))
|
||||
{
|
||||
return true;
|
||||
}
|
||||
|
||||
// Only a number below 1 can carry further insignificant zeros, and only
|
||||
// while the count stays at the limit does removing them change the
|
||||
// answer - so this loop is skipped for all but a few tokens. The
|
||||
// fraction is located through decimal_point_position rather than by
|
||||
// searching '.'.
|
||||
if (lead_zero != 0)
|
||||
{
|
||||
JSON_ASSERT(has_dot != 0); // an integer "0" cannot reach the limit
|
||||
for (std::size_t i = decimal_point_position + 1;
|
||||
digits >= limit && i < mantissa_end && token_buffer[i] == '0'; ++i)
|
||||
{
|
||||
--digits;
|
||||
}
|
||||
}
|
||||
|
||||
return digits < limit;
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief convert the number text in token_buffer to its value and token type
|
||||
|
||||
@@ -10582,7 +10679,7 @@ scan_number_done:
|
||||
token_buffer (the index of 'e'/'E', or
|
||||
token_buffer.size() when there is no exponent);
|
||||
used to skip Clinger's fast path when it cannot
|
||||
possibly succeed - see mantissa_fits_clinger()
|
||||
possibly succeed - see detail::mantissa_fits_clinger()
|
||||
*/
|
||||
token_type convert_number(token_type number_type, std::size_t mantissa_end)
|
||||
{
|
||||
@@ -10655,77 +10752,15 @@ scan_number_done:
|
||||
// (Eisel-Lemire, locale-independent, correctly rounded) when available;
|
||||
// otherwise the exact Clinger fast path (double only); otherwise the
|
||||
// locale-aware strtof/strtod/strtold.
|
||||
if (parse_float_from_chars(num_begin, num_end, value_float))
|
||||
{
|
||||
return token_type::value_float;
|
||||
}
|
||||
// Skipping a fast path that cannot succeed is lossless and saves a full
|
||||
// extra pass over the token's bytes, which otherwise shows up on
|
||||
// high-precision inputs such as canada.json
|
||||
if (mantissa_fits_clinger(mantissa_end)
|
||||
&& parse_float_fast(num_begin, num_end, value_float))
|
||||
if (convert_float_fast(num_begin, num_end, decimal_point_position, mantissa_end, value_float))
|
||||
{
|
||||
return token_type::value_float;
|
||||
}
|
||||
|
||||
convert_float_locale_aware();
|
||||
convert_float_locale_aware(token_buffer, decimal_point_position, value_float);
|
||||
return token_type::value_float;
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief convert the float in token_buffer with strtof/strtod/strtold
|
||||
|
||||
These functions expect the decimal point of the *current* locale, so it is
|
||||
looked up right before the conversion instead of once when the lexer is
|
||||
constructed: a locale change in between (by a parser callback, a SAX
|
||||
handler, or another thread) must not truncate the value (#5198). The
|
||||
token has been validated before, so if the conversion stops early and the
|
||||
decimal point changed in the meantime, the locale changed between the
|
||||
lookup and the call, and the conversion is repeated with the new decimal
|
||||
point. If the decimal point did not change, a retry cannot succeed: the
|
||||
locale's decimal point is not a single character (e.g., the two-byte
|
||||
U+066B of ar_EG.UTF-8 or fa_IR.UTF-8) and cannot be substituted in place.
|
||||
The value strtod parsed up to that point is kept, as before this change.
|
||||
|
||||
Note that changing the locale in another thread *while* strtod runs is
|
||||
undefined behavior of the C library, which this function cannot prevent.
|
||||
*/
|
||||
void convert_float_locale_aware()
|
||||
{
|
||||
const bool has_dot = decimal_point_position != std::string::npos;
|
||||
char decimal_point = get_decimal_point();
|
||||
for (;;)
|
||||
{
|
||||
const bool substitute = has_dot && decimal_point != '.';
|
||||
if (substitute)
|
||||
{
|
||||
token_buffer[decimal_point_position] = static_cast<typename string_t::value_type>(decimal_point);
|
||||
}
|
||||
|
||||
char* endptr = nullptr; // NOLINT(misc-const-correctness,cppcoreguidelines-pro-type-vararg,hicpp-vararg)
|
||||
strtof(value_float, token_buffer.data(), &endptr);
|
||||
|
||||
if (substitute)
|
||||
{
|
||||
// get_string() hands the token to the SAX interface with '.'
|
||||
token_buffer[decimal_point_position] = '.';
|
||||
}
|
||||
|
||||
if (JSON_HEDLEY_LIKELY(endptr == token_buffer.data() + token_buffer.size()))
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
// retry only if the locale changed; otherwise, this would loop forever
|
||||
const char current_decimal_point = get_decimal_point();
|
||||
if (current_decimal_point == decimal_point)
|
||||
{
|
||||
return;
|
||||
}
|
||||
decimal_point = current_decimal_point;
|
||||
}
|
||||
}
|
||||
|
||||
/*!
|
||||
@brief contiguous fast path for scanning a number
|
||||
|
||||
|
||||
@@ -8,3 +8,6 @@ The following changes have been made to the code with respect to <https://github
|
||||
- membership check
|
||||
- made function from `_is_within`
|
||||
- removed unused variable `actual_path`
|
||||
- Added the optional config key `external`: include paths listed there are kept as
|
||||
`#include` directives instead of being inlined (the first directive per path; the
|
||||
repeated ones are commented out).
|
||||
|
||||
@@ -57,6 +57,11 @@ Python v.2.7.0 or higher is required.
|
||||
amalgamation. Have a look at `test/source.c.json` and `test/include.h.json`
|
||||
to see two examples.
|
||||
|
||||
The optional `external` list names include paths that are kept as `#include`
|
||||
directives instead of being inlined, e.g. `["nlohmann/json.hpp"]` for a header
|
||||
that includes another amalgamated header. Only the first directive for each
|
||||
of these paths is kept; the repeated ones are commented out.
|
||||
|
||||
* The `-s, --source` option should specify the path to the source directory.
|
||||
This is useful for supporting separate source and build directories.
|
||||
|
||||
|
||||
@@ -62,6 +62,10 @@ class Amalgamation(object):
|
||||
return None
|
||||
|
||||
def __init__(self, args):
|
||||
# include paths that are kept as #include directives instead of
|
||||
# being inlined (e.g. a header amalgamated on its own)
|
||||
self.external = []
|
||||
self.included_external = []
|
||||
with open(args.config, 'r') as f:
|
||||
config = json.loads(f.read())
|
||||
for key in config:
|
||||
@@ -220,11 +224,14 @@ class TranslationUnit(object):
|
||||
while include_match:
|
||||
if not _is_within(include_match, skippable_contexts):
|
||||
include_path = include_match.group("path")
|
||||
search_same_dir = include_match.group(1) == '"'
|
||||
found_included_path = self.amalgamation.find_included_file(
|
||||
include_path, self.file_dir if search_same_dir else None)
|
||||
if found_included_path:
|
||||
includes.append((include_match, found_included_path))
|
||||
if include_path in self.amalgamation.external:
|
||||
includes.append((include_match, None))
|
||||
else:
|
||||
search_same_dir = include_match.group(1) == '"'
|
||||
found_included_path = self.amalgamation.find_included_file(
|
||||
include_path, self.file_dir if search_same_dir else None)
|
||||
if found_included_path:
|
||||
includes.append((include_match, found_included_path))
|
||||
|
||||
include_match = self.include_pattern.search(self.content,
|
||||
include_match.end())
|
||||
@@ -235,6 +242,17 @@ class TranslationUnit(object):
|
||||
for include in includes:
|
||||
include_match, found_included_path = include
|
||||
tmp_content += self.content[prev_end:include_match.start()]
|
||||
if found_included_path is None:
|
||||
# an external header: keep the first directive and comment
|
||||
# out the repeated ones
|
||||
include_path = include_match.group("path")
|
||||
if include_path in self.amalgamation.included_external:
|
||||
tmp_content += "// {0}".format(include_match.group(0))
|
||||
else:
|
||||
self.amalgamation.included_external.append(include_path)
|
||||
tmp_content += include_match.group(0)
|
||||
prev_end = include_match.end()
|
||||
continue
|
||||
tmp_content += "// {0}\n".format(include_match.group(0))
|
||||
if found_included_path not in self.amalgamation.included_files:
|
||||
t = TranslationUnit(found_included_path, self.amalgamation, False)
|
||||
|
||||
@@ -0,0 +1,9 @@
|
||||
{
|
||||
"project": "JSON for Modern C++",
|
||||
"target": "single_include/nlohmann/json_view.hpp",
|
||||
"sources": [
|
||||
"include/nlohmann/json_view.hpp"
|
||||
],
|
||||
"include_paths": ["include"],
|
||||
"external": ["nlohmann/json.hpp"]
|
||||
}
|
||||
Reference in New Issue
Block a user