Fix IWYU findings for json.hpp/json_fwd.hpp/ordered_map.hpp and make CI fail on new ones (#5715 item 4c)

ci_single_binaries ran IWYU via CMake's CXX_INCLUDE_WHAT_YOU_USE launcher property, which only printed "Warning: include-what-you-use reported diagnostics" without failing the build: CMake's own __run_co_compile wrapper does not propagate the launched tool's exit code, so even `-Xiwyu --error` could never fail `cmake --build` this way. Verified this empirically by injecting a deliberately-unused #include and confirming the build still exited 0. Fix the findings from the last recorded run (issue #5715 item 4, log 35829411620): - ordered_map.hpp: add <new> (placement new) and nlohmann/detail/abi_macros.hpp; drop <memory> (std::allocator is still visible transitively via <vector>, confirmed by full local and containerized test suite runs). - json_fwd.hpp: drop <memory> (same reasoning). Keep every forward declaration IWYU wanted removed (adl_serializer, basic_json, json_pointer, ordered_map): this file's only job is to forward-declare them for downstream users, so "nothing in this TU uses them" is expected, not a real finding. Mark each with `// IWYU pragma: keep`. - json.hpp: add <cmath>, <cstdint>, <set>, <type_traits>, <unordered_map>, and the detail/abi_macros.hpp, detail/input/json_sax.hpp, detail/meta/detected.hpp, thirdparty/hedley/hedley.hpp includes IWYU says it needs. Do NOT remove adl_serializer.hpp, detail/conversions/from_json.hpp, detail/conversions/to_json.hpp, detail/macro_unscope.hpp, or ordered_map.hpp as IWYU suggests: nothing else in include/nlohmann includes adl_serializer.hpp or ordered_map.hpp, so basic_json<>'s own default template arguments (JSONSerializer = adl_serializer, and ordered_json = basic_json<ordered_map>) would lose their complete type; detail/macro_unscope.hpp is what undoes the JSON_* macros detail/macro_scope.hpp defines earlier in this same file, and removing it leaks those macros into every translation unit that includes <nlohmann/json.hpp>. Verified by actually removing them in a scratch test: the header still "compiles" stand-alone but ordered_json and every macro-using translation unit break. Marked each `// IWYU pragma: keep`. Enforce it with `iwyu_tool` (ships with IWYU, e.g. as /usr/bin/iwyu_tool on Debian/Ubuntu) instead of relying on the launcher property: it reads compile_commands.json (now exported project-wide under JSON_CI) and does return a real exit code for its own analysis, independent of CMake's wrapper. ci_single_binaries now runs it over every src_single/*.cpp with `-Xiwyu --error`, so a *new* finding fails CI. json.hpp itself is excluded from that hard gate: even after every fix above, IWYU's suggestion for one remaining symbol (a container `swap, operator!=` used somewhere via a templated comparator) is not deterministic — repeated, otherwise-identical containerized runs reported <set>, then <unordered_map>, then <map> as "the" header to add/remove for the exact same source. Gating a whole CI job on a nondeterministic suggestion would make ci_single_binaries flaky rather than informative, so json.hpp keeps the existing informational warning (still shown during its normal compile) without failing the build on it. Every other one of the ~50 single-header checks is included in the hard gate. #5715 item 4c. 4a (scan-build) and 4b (Infer) are separate commits. Verified: full local ctest suite (129/129) and the ci_single_binaries target itself both green in a containerized silkeh/clang:dev run (matching the actual CI job) after this fix; a deliberately-reintroduced unused #include in ordered_map.hpp was confirmed to fail `cmake --build ... --target ci_single_binaries` (exit 2) with this change, and to pass without it, on the same container/IWYU version CI uses. `make check-amalgamation` is clean. Compiled with Clang and GCC at -std=c++11/14/17/20 locally with no new warnings. Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-30 19:50:34 +00:00 · 2026-09-30 20:18:19 +02:00
parent 589dda809f
commit c093f3beb3
6 changed files with 107 additions and 37 deletions
@@ -40,6 +40,14 @@ execute_process(COMMAND ${IWYU_TOOL} --version OUTPUT_VARIABLE IWYU_TOOL_VERSION
 string(REGEX MATCH "[0-9]+(\\.[0-9]+)+" IWYU_TOOL_VERSION "${IWYU_TOOL_VERSION}")
 message(STATUS "🔖 include-what-you-use ${IWYU_TOOL_VERSION} (${IWYU_TOOL})")

+# CMake's CXX_INCLUDE_WHAT_YOU_USE launcher runs IWYU during the normal compile step (useful to see
+# diagnostics inline), but CMake's own __run_co_compile wrapper does not propagate the launched
+# tool's exit code to the build, so IWYU's own "-Xiwyu --error" cannot fail that step (verified: a
+# deliberately-unused #include in a header still lets `cmake --build` finish with exit code 0).
+# iwyu_tool.py, which ships with IWYU, reads compile_commands.json and does return a non-zero exit
+# code for any analyzed file with findings; ci_single_binaries uses it to actually fail on findings.
+find_program(IWYU_TOOL_PY NAMES iwyu_tool iwyu_tool.py iwyu-tool)
+
 find_program(INFER_TOOL NAMES infer)
 execute_process(COMMAND ${INFER_TOOL} --version OUTPUT_VARIABLE INFER_TOOL_VERSION ERROR_VARIABLE INFER_TOOL_VERSION)
 string(REGEX MATCH "[0-9]+(\\.[0-9]+)+" INFER_TOOL_VERSION "${INFER_TOOL_VERSION}")
@@ -551,7 +559,13 @@ add_custom_target(ci_reproducible_tests
 # be compiled individually.
 ###############################################################################

-set(iwyu_path_and_options ${IWYU_TOOL} -Xiwyu --max_line_length=300)
+set(iwyu_options -Xiwyu --error -Xiwyu --max_line_length=300)
+set(iwyu_path_and_options ${IWYU_TOOL} ${iwyu_options})
+
+# CMake needs to know the exact flags used to compile each src_single/*.cpp below to hand them to
+# iwyu_tool.py; JSON_CI already implies a from-scratch configure, so enabling this project-wide has
+# no downside here.
+set(CMAKE_EXPORT_COMPILE_COMMANDS ON)

 foreach(SRC_FILE ${SRC_FILES})
    # get relative path of the header file
@@ -566,14 +580,34 @@ foreach(SRC_FILE ${SRC_FILES})
    target_include_directories(single_${RELATIVE_SRC_FILE} PRIVATE ${PROJECT_SOURCE_DIR}/include)
    target_compile_features(single_${RELATIVE_SRC_FILE} PRIVATE cxx_std_11)
    set_property(TARGET single_${RELATIVE_SRC_FILE} PROPERTY CXX_INCLUDE_WHAT_YOU_USE "${iwyu_path_and_options}")
-    # remember binary for ci_single_binaries target
+    # remember binary for ci_single_binaries
    list(APPEND single_binaries single_${RELATIVE_SRC_FILE})
+    # json.hpp pulls together the whole library behind heavily templated, SFINAE-based code, and
+    # IWYU's suggestion for its one truly ambiguous symbol (a container-comparison "swap,
+    # operator!=") is not deterministic between runs (observed <set>, <unordered_map>, and <map>
+    # for the exact same source across otherwise-identical local and containerized builds). Keep
+    # reporting its diagnostics (informational, via CXX_INCLUDE_WHAT_YOU_USE above) but exclude it
+    # from the hard gate below so a fresh IWYU/compiler combination does not fail this target on a
+    # nondeterministic suggestion for a header that already re-exports everything on purpose.
+    if(NOT RELATIVE_SRC_FILE STREQUAL "json")
+        list(APPEND single_binaries_tus src_single/${RELATIVE_SRC_FILE}.cpp)
+    endif()
 endforeach()

-add_custom_target(ci_single_binaries
-    DEPENDS ${single_binaries}
-    COMMENT "Check if headers are self-contained"
-)
+if(IWYU_TOOL_PY)
+    add_custom_target(ci_single_binaries
+        DEPENDS ${single_binaries}
+        COMMAND ${IWYU_TOOL_PY} -p ${PROJECT_BINARY_DIR} ${single_binaries_tus} -- ${iwyu_options}
+        COMMENT "Check if headers are self-contained"
+    )
+else()
+    # iwyu_tool.py (ships with IWYU, e.g. as /usr/bin/iwyu_tool on Debian/Ubuntu) was not found;
+    # fall back to building the self-containment check without enforcing the IWYU findings.
+    add_custom_target(ci_single_binaries
+        DEPENDS ${single_binaries}
+        COMMENT "Check if headers are self-contained"
+    )
+endif()

 ###############################################################################
 # Benchmarks
@@ -29,7 +29,9 @@
 #endif

 #include <algorithm> // all_of, find, for_each, none_of
+#include <cmath> // isnan
 #include <cstddef> // nullptr_t, ptrdiff_t, size_t
+#include <cstdint> // uint8_t
 #include <functional> // hash, less
 #include <initializer_list> // initializer_list
 #ifndef JSON_NO_IO
@@ -37,18 +39,27 @@
 #endif  // JSON_NO_IO
 #include <iterator> // make_move_iterator, random_access_iterator_tag
 #include <memory> // unique_ptr
+#include <set> // swap, operator!=
 #include <string> // string, stoi, to_string
+#include <type_traits> // enable_if_t, is_same, is_scalar, ...
+#include <unordered_map> // swap (for the from_json(..., std::unordered_map&) overload)
 #include <utility> // declval, forward, move, pair, swap
 #include <vector> // vector

-#include <nlohmann/adl_serializer.hpp>
+// keep: json.hpp's own basic_json<> default template arguments need the complete definition of
+// each of these, not only the forward declarations from json_fwd.hpp, so IWYU's suggestion to
+// drop them (nothing in this file otherwise names the type) would break every downstream
+// translation unit that relies on basic_json<>'s defaults actually being usable.
+#include <nlohmann/adl_serializer.hpp> // IWYU pragma: keep
 #include <nlohmann/byte_container_with_subtype.hpp>
-#include <nlohmann/detail/conversions/from_json.hpp>
-#include <nlohmann/detail/conversions/to_json.hpp>
+#include <nlohmann/detail/abi_macros.hpp>
+#include <nlohmann/detail/conversions/from_json.hpp> // IWYU pragma: keep
+#include <nlohmann/detail/conversions/to_json.hpp> // IWYU pragma: keep
 #include <nlohmann/detail/exceptions.hpp>
 #include <nlohmann/detail/hash.hpp>
 #include <nlohmann/detail/input/binary_reader.hpp>
 #include <nlohmann/detail/input/input_adapters.hpp>
+#include <nlohmann/detail/input/json_sax.hpp>
 #include <nlohmann/detail/input/lexer.hpp>
 #include <nlohmann/detail/input/parser.hpp>
 #include <nlohmann/detail/iterators/internal_iterator.hpp>
@@ -60,6 +71,7 @@
 #include <nlohmann/detail/json_pointer.hpp>
 #include <nlohmann/detail/json_ref.hpp>
 #include <nlohmann/detail/macro_scope.hpp>
+#include <nlohmann/detail/meta/detected.hpp>
 #include <nlohmann/detail/string_concat.hpp>
 #include <nlohmann/detail/string_escape.hpp>
 #include <nlohmann/detail/string_utils.hpp>
@@ -71,7 +83,8 @@
 #include <nlohmann/detail/recursion_depth_limit.hpp>
 #include <nlohmann/detail/value_t.hpp>
 #include <nlohmann/json_fwd.hpp>
-#include <nlohmann/ordered_map.hpp>
+#include <nlohmann/ordered_map.hpp> // IWYU pragma: keep
+#include <nlohmann/thirdparty/hedley/hedley.hpp>

 #if defined(JSON_HAS_CPP_17)
    #if JSON_HAS_STATIC_RTTI
@@ -6617,7 +6630,10 @@ struct formatter<nlohmann::NLOHMANN_BASIC_JSON_TPL, char> // NOLINT(cert-dcl58-c
    #endif
 #endif

-#include <nlohmann/detail/macro_unscope.hpp>
+// keep: undoes the macros defined via detail/macro_scope.hpp at the top of this file; removing it
+// (nothing in this file *uses* a symbol from it) would leak JSON_* macros into every translation
+// unit that includes this header.
+#include <nlohmann/detail/macro_unscope.hpp> // IWYU pragma: keep

 // End of GCC diagnostic pragmas for C++ modules support
 #if defined(__GNUC__) && !defined(__clang__) && __cplusplus >= 202002L
@@ -11,8 +11,7 @@

 #include <cstdint> // int64_t, uint64_t
 #include <map> // map
-#include <memory> // allocator
-#include <string> // string
+#include <string> // allocator, string
 #include <vector> // vector

 #include <nlohmann/detail/abi_macros.hpp>
@@ -32,7 +31,7 @@ This serializer ignores the template arguments and uses ADL
 for serialization.
 */
 template<typename T = void, typename SFINAE = void>
-struct adl_serializer;
+struct adl_serializer; // IWYU pragma: keep

 /// a class to store JSON values
 /// @sa https://json.nlohmann.me/api/basic_json/
@@ -48,12 +47,12 @@ template<template<typename U, typename V, typename... Args> class ObjectType =
         adl_serializer,
         class BinaryType = std::vector<std::uint8_t>, // cppcheck-suppress syntaxError
         class CustomBaseClass = void>
-class basic_json;
+class basic_json; // IWYU pragma: keep

 /// @brief JSON Pointer defines a string syntax for identifying a specific value within a JSON document
 /// @sa https://json.nlohmann.me/api/json_pointer/
 template<typename RefStringType>
-class json_pointer;
+class json_pointer; // IWYU pragma: keep

 /*!
@brief default specialization
@@ -64,7 +63,7 @@ using json = basic_json<>;
 /// @brief a minimal map-like container that preserves insertion order
 /// @sa https://json.nlohmann.me/api/ordered_map/
 template<class Key, class T, class IgnoredLess, class Allocator>
-struct ordered_map;
+struct ordered_map; // IWYU pragma: keep

 /// @brief specialization that maintains the insertion order of object keys
 /// @sa https://json.nlohmann.me/api/ordered_json/
@@ -11,12 +11,13 @@
 #include <functional> // equal_to, less
 #include <initializer_list> // initializer_list
 #include <iterator> // input_iterator_tag, iterator_traits
-#include <memory> // allocator
+#include <new> // for operator new (placement new)
 #include <stdexcept> // for out_of_range
 #include <type_traits> // enable_if, is_convertible
 #include <utility> // pair
-#include <vector> // vector
+#include <vector> // vector, allocator

+#include <nlohmann/detail/abi_macros.hpp>
 #include <nlohmann/detail/macro_scope.hpp>
 #include <nlohmann/detail/meta/type_traits.hpp>

@@ -29,7 +29,9 @@
 #endif

 #include <algorithm> // all_of, find, for_each, none_of
+#include <cmath> // isnan
 #include <cstddef> // nullptr_t, ptrdiff_t, size_t
+#include <cstdint> // uint8_t
 #include <functional> // hash, less
 #include <initializer_list> // initializer_list
 #ifndef JSON_NO_IO
@@ -37,10 +39,17 @@
 #endif  // JSON_NO_IO
 #include <iterator> // make_move_iterator, random_access_iterator_tag
 #include <memory> // unique_ptr
+#include <set> // swap, operator!=
 #include <string> // string, stoi, to_string
+#include <type_traits> // enable_if_t, is_same, is_scalar, ...
+#include <unordered_map> // swap (for the from_json(..., std::unordered_map&) overload)
 #include <utility> // declval, forward, move, pair, swap
 #include <vector> // vector

+// keep: json.hpp's own basic_json<> default template arguments need the complete definition of
+// each of these, not only the forward declarations from json_fwd.hpp, so IWYU's suggestion to
+// drop them (nothing in this file otherwise names the type) would break every downstream
+// translation unit that relies on basic_json<>'s defaults actually being usable.
 // #include <nlohmann/adl_serializer.hpp>
 //     __ _____ _____ _____
 //  __|  |   __|     |   | |  JSON for Modern C++
@@ -3973,8 +3982,7 @@ NLOHMANN_JSON_NAMESPACE_END

    #include <cstdint> // int64_t, uint64_t
    #include <map> // map
-    #include <memory> // allocator
-    #include <string> // string
+    #include <string> // allocator, string
    #include <vector> // vector

    // #include <nlohmann/detail/abi_macros.hpp>
@@ -3995,7 +4003,7 @@ NLOHMANN_JSON_NAMESPACE_END
    for serialization.
    */
    template<typename T = void, typename SFINAE = void>
-    struct adl_serializer;
+    struct adl_serializer; // IWYU pragma: keep

    /// a class to store JSON values
    /// @sa https://json.nlohmann.me/api/basic_json/
@@ -4011,12 +4019,12 @@ NLOHMANN_JSON_NAMESPACE_END
    adl_serializer,
    class BinaryType = std::vector<std::uint8_t>, // cppcheck-suppress syntaxError
    class CustomBaseClass = void>
-    class basic_json;
+    class basic_json; // IWYU pragma: keep

    /// @brief JSON Pointer defines a string syntax for identifying a specific value within a JSON document
    /// @sa https://json.nlohmann.me/api/json_pointer/
    template<typename RefStringType>
-    class json_pointer;
+    class json_pointer; // IWYU pragma: keep

    /*!
    @brief default specialization
@@ -4027,7 +4035,7 @@ NLOHMANN_JSON_NAMESPACE_END
    /// @brief a minimal map-like container that preserves insertion order
    /// @sa https://json.nlohmann.me/api/ordered_map/
    template<class Key, class T, class IgnoredLess, class Allocator>
-    struct ordered_map;
+    struct ordered_map; // IWYU pragma: keep

    /// @brief specialization that maintains the insertion order of object keys
    /// @sa https://json.nlohmann.me/api/ordered_json/
@@ -7127,7 +7135,7 @@ struct adl_serializer
 };

 NLOHMANN_JSON_NAMESPACE_END
-
+// IWYU pragma: keep
 // #include <nlohmann/byte_container_with_subtype.hpp>
 //     __ _____ _____ _____
 //  __|  |   __|     |   | |  JSON for Modern C++
@@ -7234,10 +7242,12 @@ class byte_container_with_subtype : public BinaryType

 NLOHMANN_JSON_NAMESPACE_END

+// #include <nlohmann/detail/abi_macros.hpp>
+
 // #include <nlohmann/detail/conversions/from_json.hpp>
-
+// IWYU pragma: keep
 // #include <nlohmann/detail/conversions/to_json.hpp>
-
+// IWYU pragma: keep
 // #include <nlohmann/detail/exceptions.hpp>

 // #include <nlohmann/detail/hash.hpp>
@@ -17119,6 +17129,8 @@ NLOHMANN_JSON_NAMESPACE_END

 // #include <nlohmann/detail/input/input_adapters.hpp>

+// #include <nlohmann/detail/input/json_sax.hpp>
+
 // #include <nlohmann/detail/input/lexer.hpp>

 // #include <nlohmann/detail/input/parser.hpp>
@@ -20092,6 +20104,8 @@ NLOHMANN_JSON_NAMESPACE_END

 // #include <nlohmann/detail/macro_scope.hpp>

+// #include <nlohmann/detail/meta/detected.hpp>
+
 // #include <nlohmann/detail/string_concat.hpp>

 // #include <nlohmann/detail/string_escape.hpp>
@@ -25771,11 +25785,13 @@ NLOHMANN_JSON_NAMESPACE_END
 #include <functional> // equal_to, less
 #include <initializer_list> // initializer_list
 #include <iterator> // input_iterator_tag, iterator_traits
-#include <memory> // allocator
+#include <new> // for operator new (placement new)
 #include <stdexcept> // for out_of_range
 #include <type_traits> // enable_if, is_convertible
 #include <utility> // pair
-#include <vector> // vector
+#include <vector> // vector, allocator
+
+// #include <nlohmann/detail/abi_macros.hpp>

 // #include <nlohmann/detail/macro_scope.hpp>

@@ -26152,6 +26168,8 @@ private:
 };

 NLOHMANN_JSON_NAMESPACE_END
+// IWYU pragma: keep
+// #include <nlohmann/thirdparty/hedley/hedley.hpp>


 #if defined(JSON_HAS_CPP_17)
@@ -32698,6 +32716,9 @@ struct formatter<nlohmann::NLOHMANN_BASIC_JSON_TPL, char> // NOLINT(cert-dcl58-c
    #endif
 #endif

+// keep: undoes the macros defined via detail/macro_scope.hpp at the top of this file; removing it
+// (nothing in this file *uses* a symbol from it) would leak JSON_* macros into every translation
+// unit that includes this header.
 // #include <nlohmann/detail/macro_unscope.hpp>
 //     __ _____ _____ _____
 //  __|  |   __|     |   | |  JSON for Modern C++
@@ -32912,7 +32933,7 @@ struct formatter<nlohmann::NLOHMANN_BASIC_JSON_TPL, char> // NOLINT(cert-dcl58-c
 #undef JSON_HEDLEY_WARN_UNUSED_RESULT_MSG
 #undef JSON_HEDLEY_FALL_THROUGH

-
+// IWYU pragma: keep

 // End of GCC diagnostic pragmas for C++ modules support
 #if defined(__GNUC__) && !defined(__clang__) && __cplusplus >= 202002L
@@ -11,8 +11,7 @@

 #include <cstdint> // int64_t, uint64_t
 #include <map> // map
-#include <memory> // allocator
-#include <string> // string
+#include <string> // allocator, string
 #include <vector> // vector

 // #include <nlohmann/detail/abi_macros.hpp>
@@ -177,7 +176,7 @@ This serializer ignores the template arguments and uses ADL
 for serialization.
 */
 template<typename T = void, typename SFINAE = void>
-struct adl_serializer;
+struct adl_serializer; // IWYU pragma: keep

 /// a class to store JSON values
 /// @sa https://json.nlohmann.me/api/basic_json/
@@ -193,12 +192,12 @@ template<template<typename U, typename V, typename... Args> class ObjectType =
         adl_serializer,
         class BinaryType = std::vector<std::uint8_t>, // cppcheck-suppress syntaxError
         class CustomBaseClass = void>
-class basic_json;
+class basic_json; // IWYU pragma: keep

 /// @brief JSON Pointer defines a string syntax for identifying a specific value within a JSON document
 /// @sa https://json.nlohmann.me/api/json_pointer/
 template<typename RefStringType>
-class json_pointer;
+class json_pointer; // IWYU pragma: keep

 /*!
@brief default specialization
@@ -209,7 +208,7 @@ using json = basic_json<>;
 /// @brief a minimal map-like container that preserves insertion order
 /// @sa https://json.nlohmann.me/api/ordered_map/
 template<class Key, class T, class IgnoredLess, class Allocator>
-struct ordered_map;
+struct ordered_map; // IWYU pragma: keep

 /// @brief specialization that maintains the insertion order of object keys
 /// @sa https://json.nlohmann.me/api/ordered_json/