amorfo77/json
1
0
Fork 0
mirror of https://github.com/nlohmann/json.git synced 2026-10-03 21:20:30 +00:00
Code Issues Packages Projects Releases Wiki Activity

Extend tools/macro_builder to also generate DOUBLE_PASTE and TYPE_BODY

Browse Source 
tools/macro_builder only emitted NLOHMANN_JSON_EXPAND..PASTE64, so two
other tables that scale with the same max_args stayed hand-maintained
with nothing checking them: NLOHMANN_JSON_DOUBLE_PASTE (added by hand
in #4563 for the *_WITH_NAMES macros) and the 64-slot dispatch table of
NLOHMANN_JSON_TYPE_BODY (the #4041 zero-member/one-or-more-member
switch). Both tables pass one macro name per slot to the same
NLOHMANN_JSON_GET_MACRO dispatch as PASTE, so they can drift out of
sync with max_args exactly the way _dp did in the ABI tag list fixed
by #5544.

Extend main.cpp with build_double_paste_code() (same recursive-doubling
shape as build_paste_code(), but DOUBLE_PASTE consumes two arguments
per member, so an even slot index falls back to the next lower odd
DOUBLE_PASTE<N>) and build_type_body_table() (max_args - 1 MEMBERS
slots and one trailing EMPTY slot, 8 per line, matching how it is
written by hand today). Add a "type_body" argument that selects the
TYPE_BODY block, since it lives at a separate location in
macro_scope.hpp from the EXPAND..DOUBLE_PASTE63 block; plain invocation
is unchanged apart from covering the extended range. No longer emit
the tool's old trailing blank line, so its output is directly diffable
without post-processing.

Verified with c++ -std=c++11: running the tool (with and without
"type_body") and piping the raw output through the pinned astyle
reproduces both blocks of the current macro_scope.hpp byte for byte.
tests/src/unit-udt_macro.cpp (all NLOHMANN_DEFINE_TYPE_*/_WITH_NAMES/
zero-member variants) passes unchanged under -std=c++11 and -std=c++17
with -fsanitize=address,undefined.

Add a "macro_builder_check" Makefile target that builds main.cpp,
regenerates both blocks into a scratch directory inside the repository
(astyle's --project lookup needs the target files under the same tree
as .astylerc, unlike an external /tmp directory), and diffs them
against the corresponding ranges of macro_scope.hpp; wire it into
check-amalgamation next to the natvis check. Wire the same regeneration
into check_amalgamation.yml, splicing the (still unindented) generated
blocks back into the PR's own macro_scope.hpp before the existing
astyle/amalgamation step runs, so that step's own tree-wide astyle
pass both indents them and folds any drift into the amalgamation
patch/diff the workflow already produces.

Unlike amalgamate.py and generate_natvis.py, this step builds
tools/macro_builder/main.cpp from the pull request's own checkout
($MAIN_DIR) rather than a separate checkout of tools/ at develop: this
tool has no independent source of truth to regenerate against (its
README documents that it must reproduce macro_scope.hpp byte for
byte), so a develop-pinned copy would only reproduce the
generate_natvis.py trap fixed in a previous commit on this branch,
where a PR that teaches the tool to cover more of the file fails its
own CI until that PR merges and updates the develop copy.

Add tools/macro_builder/README.md documentation for both new tables
and the two-invocation usage, and a short pointer comment above
NLOHMANN_JSON_TYPE_BODY (the EXPAND pointer already covered the first
block; extended its wording to include DOUBLE_PASTE63).

Closes #5717 item 5 in full, completing what the documentation-only
"Document tools/macro_builder..." commit already on this branch left
open (that commit's README/pointer-comment half stands; it also covers
item 7).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
This commit is contained in:
Niels Lohmann
2026-09-30 18:35:18 +02:00
parent a436ba2775
commit 7aeb98414e
6 changed files with 167 additions and 27 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tools/macro_builder/README.md
+21 -13
View File
@@ -7,9 +7,16 @@ macros in [`include/nlohmann/detail/macro_scope.hpp`](../../include/nlohmann/det
- `NLOHMANN_JSON_GET_MACRO`, which selects a macro by the number of its arguments (64 slots)
- `NLOHMANN_JSON_PASTE`, which calls a function-like macro for each member, and its helpers `NLOHMANN_JSON_PASTE2`
to `NLOHMANN_JSON_PASTE64`
- `NLOHMANN_JSON_DOUBLE_PASTE`, which the `*_WITH_NAMES` macros use to call a function-like macro for each
(JSON name, member) pair, and its helpers `NLOHMANN_JSON_DOUBLE_PASTE3` to `NLOHMANN_JSON_DOUBLE_PASTE63`
- the slot table of `NLOHMANN_JSON_TYPE_BODY`, which dispatches `NLOHMANN_DEFINE_TYPE_*(Type)` (no further
arguments) to the zero-member implementation and every other argument count to the one-or-more-member
implementation
The number of slots (`max_args` in [`main.cpp`](main.cpp)) sets the member limit of these macros.
`NLOHMANN_JSON_PASTE` takes the function as its first argument, so 64 slots allow 63 members.
`NLOHMANN_JSON_PASTE` and `NLOHMANN_JSON_TYPE_BODY` take the function/prefix as their first argument, so 64 slots
allow 63 members; `NLOHMANN_JSON_DOUBLE_PASTE` additionally consumes its members two at a time (name, member), so
it only defines the odd helpers up to `NLOHMANN_JSON_DOUBLE_PASTE63`.
## Usage
@@ -18,26 +25,27 @@ From the project root:
```shell
c++ -std=c++11 tools/macro_builder/main.cpp -o macro_builder
./macro_builder
./macro_builder type_body
```
1. In `include/nlohmann/detail/macro_scope.hpp`, replace the lines from `#define NLOHMANN_JSON_EXPAND( x ) x` to the
`#define NLOHMANN_JSON_PASTE64(...)` line with the output, without its trailing empty line.
2. Run `make amalgamate`. It updates `single_include/nlohmann/json.hpp` and runs `make pretty`, which indents the
continuation lines of `NLOHMANN_JSON_PASTE` that the tool writes unindented.
1. Run `./macro_builder` (no arguments). In `include/nlohmann/detail/macro_scope.hpp`, replace the lines from
`#define NLOHMANN_JSON_EXPAND( x ) x` to the `#define NLOHMANN_JSON_DOUBLE_PASTE63(...)` line with the output,
without its trailing empty line.
2. Run `./macro_builder type_body`. Replace the lines from `#define NLOHMANN_JSON_TYPE_BODY(Prefix, ...)` to the
`NLOHMANN_JSON_TYPE_BODY_SENTINEL))` line with the output.
3. Run `make amalgamate`. It updates `single_include/nlohmann/json.hpp` and runs `make pretty`, which indents the
continuation lines that the tool writes unindented.
With an unchanged `main.cpp`, these steps reproduce the header byte for byte.
With an unchanged `main.cpp`, these steps reproduce both blocks of `macro_scope.hpp` byte for byte. `make
macro_builder_check` (also run by CI, see `.github/workflows/check_amalgamation.yml`) automates this: it builds
`main.cpp`, regenerates both blocks, and fails on a diff against the checked-in header.
## Maintained by hand
The tool does not generate everything that depends on the number of slots. When changing `max_args`, also update:
- the `NLOHMANN_JSON_DOUBLE_PASTE` table right after the generated block (`NLOHMANN_JSON_DOUBLE_PASTE` and
`NLOHMANN_JSON_DOUBLE_PASTE3`, `NLOHMANN_JSON_DOUBLE_PASTE5`, ..., `NLOHMANN_JSON_DOUBLE_PASTE63`), which the
`*_WITH_NAMES` macros use
- the slot table of `NLOHMANN_JSON_TYPE_BODY`, which chooses between the implementations for zero members and for one
or more members
- the documented limit of 63 members in `docs/mkdocs/docs` and the tests at that limit in
`tests/src/unit-udt_macro.cpp`
Both tables pass one macro name per slot to `NLOHMANN_JSON_GET_MACRO`, so they need exactly as many entries as it has
slots.
All three tables pass one macro name per slot to `NLOHMANN_JSON_GET_MACRO`, so they need exactly as many entries as
it has slots.
tools/macro_builder/main.cpp
+76 -11
View File
@@ -1,10 +1,14 @@
#include <cstdlib>
#include <iostream>
#include <sstream>
#include <string>
using namespace std;
void build_code(int max_args)
// Builds NLOHMANN_JSON_EXPAND, NLOHMANN_JSON_GET_MACRO, and the
// NLOHMANN_JSON_PASTE / NLOHMANN_JSON_PASTE2..PASTE<max_args> dispatch table
// and recursive definitions.
string build_paste_code(int max_args)
{
stringstream ss;
ss << "#define NLOHMANN_JSON_EXPAND( x ) x" << endl;
@@ -12,32 +16,93 @@ void build_code(int max_args)
for (int i = 0 ; i < max_args ; i++)
ss << "_" << i + 1 << ", ";
ss << "NAME,...) NAME" << endl;
ss << "#define NLOHMANN_JSON_PASTE(...) NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_GET_MACRO(__VA_ARGS__, \\" << endl;
for (int i = max_args ; i > 1 ; i--)
ss << "NLOHMANN_JSON_PASTE" << i << ", \\" << endl;
ss << "NLOHMANN_JSON_PASTE1)(__VA_ARGS__))" << endl;
ss << "#define NLOHMANN_JSON_PASTE2(func, v1) func(v1)" << endl;
for (int i = 3 ; i <= max_args ; i++)
{
ss << "#define NLOHMANN_JSON_PASTE" << i << "(func, ";
ss << "#define NLOHMANN_JSON_PASTE" << i << "(func, ";
for (int j = 1 ; j < i -1 ; j++)
ss << "v" << j << ", ";
ss << "v" << j << ", ";
ss << "v" << i-1 << ") NLOHMANN_JSON_PASTE2(func, v1) NLOHMANN_JSON_PASTE" << i-1 << "(func, ";
for (int j = 2 ; j < i-1 ; j++)
ss << "v" << j << ", ";
ss << "v" << i-1 << ")" << endl;
}
cout << ss.str() << endl;
return ss.str();
}
int main(int argc, char** argv)
// Builds the NLOHMANN_JSON_DOUBLE_PASTE dispatch table and recursive
// definitions used by the *_WITH_NAMES macros. Its GET_MACRO dispatch reuses
// the same max_args slots as NLOHMANN_JSON_PASTE, but DOUBLE_PASTE consumes
// its arguments two at a time (name, member), so an even slot count falls
// back to the next lower odd NLOHMANN_JSON_DOUBLE_PASTE<N>.
string build_double_paste_code(int max_args)
{
stringstream ss;
ss << "#define NLOHMANN_JSON_DOUBLE_PASTE(...) NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_GET_MACRO(__VA_ARGS__, \\" << endl;
for (int i = max_args ; i > 1 ; i--)
{
int k = (i % 2 == 1) ? i : i - 1;
ss << "NLOHMANN_JSON_DOUBLE_PASTE" << k << ", \\" << endl;
}
ss << "NLOHMANN_JSON_DOUBLE_PASTE1)(__VA_ARGS__))" << endl;
ss << "#define NLOHMANN_JSON_DOUBLE_PASTE3(func, v1, v2) func(v1, v2)" << endl;
for (int k = 5 ; k <= max_args - 1 ; k += 2)
{
ss << "#define NLOHMANN_JSON_DOUBLE_PASTE" << k << "(func, ";
for (int j = 1 ; j < k - 1 ; j++)
ss << "v" << j << ", ";
ss << "v" << k - 1 << ") NLOHMANN_JSON_DOUBLE_PASTE3(func, v1, v2) NLOHMANN_JSON_DOUBLE_PASTE" << k - 2 << "(func, ";
for (int j = 3 ; j < k - 1 ; j++)
ss << "v" << j << ", ";
ss << "v" << k - 1 << ")" << endl;
}
return ss.str();
}
// Builds the NLOHMANN_JSON_TYPE_BODY dispatch table: max_args - 1 slots
// selecting the *_MEMBERS implementation and a final slot selecting
// *_EMPTY, so NLOHMANN_DEFINE_TYPE_*(Type) with no further arguments still
// resolves (issue #4041).
string build_type_body_table(int max_args)
{
stringstream ss;
ss << "#define NLOHMANN_JSON_TYPE_BODY(Prefix, ...) NLOHMANN_JSON_EXPAND(NLOHMANN_JSON_GET_MACRO(__VA_ARGS__, \\" << endl;
const int per_line = 8;
for (int i = 1 ; i <= max_args ; i++)
{
ss << (i == max_args ? "Prefix ## EMPTY" : "Prefix ## MEMBERS") << ", ";
if (i % per_line == 0)
ss << "\\" << endl;
}
ss << "NLOHMANN_JSON_TYPE_BODY_SENTINEL))" << endl;
return ss.str();
}
int main(int argc, char** argv)
{
int max_args = 64;
build_code(max_args);
// With "type_body", print only the NLOHMANN_JSON_TYPE_BODY dispatch
// table (a separate insertion point in macro_scope.hpp); otherwise
// print the EXPAND/GET_MACRO/PASTE/DOUBLE_PASTE block that precedes it.
if (argc > 1 && string(argv[1]) == "type_body")
{
cout << build_type_body_table(max_args);
}
else
{
cout << build_paste_code(max_args) << build_double_paste_code(max_args);
}
return 0;
}