# MessagePack MessagePack is an efficient binary serialization format. It lets you exchange data among multiple languages like JSON. But it's faster and smaller. Small integers are encoded into a single byte, and typical short strings require only one extra byte in addition to the strings themselves. !!! abstract "References" - [MessagePack website](https://msgpack.org) - [MessagePack specification](https://github.com/msgpack/msgpack/blob/master/spec.md) ## Serialization The library uses the following mapping from JSON values types to MessagePack types according to the MessagePack specification: | JSON value type | value/range | MessagePack type | first byte | |-----------------|------------------------------------------|------------------|------------| | null | `null` | nil | 0xC0 | | boolean | `true` | true | 0xC3 | | boolean | `false` | false | 0xC2 | | number_integer | -9223372036854775808..-2147483649 | int64 | 0xD3 | | number_integer | -2147483648..-32769 | int32 | 0xD2 | | number_integer | -32768..-129 | int16 | 0xD1 | | number_integer | -128..-33 | int8 | 0xD0 | | number_integer | -32..-1 | negative fixint | 0xE0..0xFF | | number_integer | 0..127 | positive fixint | 0x00..0x7F | | number_integer | 128..255 | uint 8 | 0xCC | | number_integer | 256..65535 | uint 16 | 0xCD | | number_integer | 65536..4294967295 | uint 32 | 0xCE | | number_integer | 4294967296..18446744073709551615 | uint 64 | 0xCF | | number_unsigned | 0..127 | positive fixint | 0x00..0x7F | | number_unsigned | 128..255 | uint 8 | 0xCC | | number_unsigned | 256..65535 | uint 16 | 0xCD | | number_unsigned | 65536..4294967295 | uint 32 | 0xCE | | number_unsigned | 4294967296..18446744073709551615 | uint 64 | 0xCF | | number_float | *any value representable by a float* | float 32 | 0xCA | | number_float | *any value NOT representable by a float* | float 64 | 0xCB | | string | *length*: 0..31 | fixstr | 0xA0..0xBF | | string | *length*: 32..255 | str 8 | 0xD9 | | string | *length*: 256..65535 | str 16 | 0xDA | | string | *length*: 65536..4294967295 | str 32 | 0xDB | | array | *size*: 0..15 | fixarray | 0x90..0x9F | | array | *size*: 16..65535 | array 16 | 0xDC | | array | *size*: 65536..4294967295 | array 32 | 0xDD | | object | *size*: 0..15 | fix map | 0x80..0x8F | | object | *size*: 16..65535 | map 16 | 0xDE | | object | *size*: 65536..4294967295 | map 32 | 0xDF | | binary | *size*: 0..255 | bin 8 | 0xC4 | | binary | *size*: 256..65535 | bin 16 | 0xC5 | | binary | *size*: 65536..4294967295 | bin 32 | 0xC6 | !!! success "Complete mapping" The mapping is **complete** in the sense that any JSON value type can be converted to a MessagePack value. Any MessagePack output created by `to_msgpack` can be successfully parsed by `from_msgpack`. !!! warning "Size constraints" The following values can **not** be converted to a MessagePack value: - strings with more than 4294967295 bytes - byte strings with more than 4294967295 bytes - arrays with more than 4294967295 elements - objects with more than 4294967295 elements Serializing such a value throws [`out_of_range.412`](../../home/exceptions.md#jsonexceptionout_of_range412). !!! info "NaN/infinity handling" `NaN`, `Infinity`, and `-Infinity` are serialized as a MessagePack float 32 (type 0xCA, 5 bytes total), regardless of magnitude, in contrast to the [dump](../../api/basic_json/dump.md) function which serializes NaN or Infinity to `null`. !!! note Prior to version 3.13.0, NaN and Infinity were instead serialized as a MessagePack float 64 (type 0xCB, 9 bytes total), because the check used to select the smaller float 32 encoding compared magnitudes with NaN, which is always `false` and caused the float 32 path to be skipped. ??? example "Example: serialize a JSON value to MessagePack" ```cpp --8<-- "examples/to_msgpack.cpp" ``` Output: ```c --8<-- "examples/to_msgpack.output" ``` ### Byte layout Consider the following JSON value (shown formatted for readability): ```json { "id": 300, "name": "json", "pi": 3.14, "tags": [ "a", "b" ] } ``` `to_msgpack` encodes it to 39 bytes (shown in hexadecimal; the encoding contains no whitespace). The colors mark the role of each byte: type/marker length/count/size data.
JSONBytesMeaning
{84map with 4 key/value pairs
"id": 300,A2 69 64 CD 01 2Ckey: fixstr of length 2, then its UTF-8 bytes; value: uint 16 (big-endian)
"name": "json",A4 6E 61 6D 65 A4 6A 73 6F 6Ekey: fixstr of length 4; value: fixstr of length 4
"pi": 3.14,A2 70 69 CB 40 09 1E B8 51 EB 85 1Fkey: fixstr of length 2; value: float 64 (IEEE 754, big-endian)
"tags": [A4 74 61 67 73 92key: fixstr of length 4; value: fixarray with 2 elements
"a",A1 61fixstr of length 1
"b"A1 62fixstr of length 1
]no end marker: the element count was given up front
}no end marker: the member count was given up front
## Deserialization The library maps MessagePack types to JSON value types as follows: | MessagePack type | JSON value type | first byte | |------------------|-----------------|------------| | positive fixint | number_unsigned | 0x00..0x7F | | fixmap | object | 0x80..0x8F | | fixarray | array | 0x90..0x9F | | fixstr | string | 0xA0..0xBF | | nil | `null` | 0xC0 | | false | `false` | 0xC2 | | true | `true` | 0xC3 | | float 32 | number_float | 0xCA | | float 64 | number_float | 0xCB | | uint 8 | number_unsigned | 0xCC | | uint 16 | number_unsigned | 0xCD | | uint 32 | number_unsigned | 0xCE | | uint 64 | number_unsigned | 0xCF | | int 8 | number_integer | 0xD0 | | int 16 | number_integer | 0xD1 | | int 32 | number_integer | 0xD2 | | int 64 | number_integer | 0xD3 | | str 8 | string | 0xD9 | | str 16 | string | 0xDA | | str 32 | string | 0xDB | | array 16 | array | 0xDC | | array 32 | array | 0xDD | | map 16 | object | 0xDE | | map 32 | object | 0xDF | | bin 8 | binary | 0xC4 | | bin 16 | binary | 0xC5 | | bin 32 | binary | 0xC6 | | ext 8 | binary | 0xC7 | | ext 16 | binary | 0xC8 | | ext 32 | binary | 0xC9 | | fixext 1 | binary | 0xD4 | | fixext 2 | binary | 0xD5 | | fixext 4 | binary | 0xD6 | | fixext 8 | binary | 0xD7 | | fixext 16 | binary | 0xD8 | | negative fixint | number_integer | 0xE0-0xFF | !!! info Any MessagePack output created by `to_msgpack` can be successfully parsed by `from_msgpack`. !!! warning "Object keys" MessagePack allows map keys of any type, whereas JSON only allows strings as keys in object values. Like the JSON-compatible [profile](https://github.com/msgpack/msgpack/blob/master/spec.md#profile) sketched in the MessagePack specification, this library restricts map keys to `str` values. Maps with keys of any other type are rejected with a [`parse_error.113`](../../home/exceptions.md#jsonexceptionparse_error113) exception (or, with `allow_exceptions` set to `false`, a discarded value) naming the type of the key that was found, for instance: ``` [json.exception.parse_error.113] parse error at byte 2: syntax error while parsing MessagePack object key: only string keys are supported, but found nil; last byte: 0xC0 ``` This applies to the [SAX interface](../parsing/sax_interface.md) as well, as the key is read before it is passed on. Such input needs a general-purpose MessagePack library instead. !!! warning "Ill-formed UTF-8 in string values" The MessagePack specification explicitly allows a `str` value (`fixstr`, `str 8`, `str 16`, `str 32`) to contain a byte sequence that is not valid UTF-8, and expects a deserializer to hand the original bytes back unchanged. This library follows that by default: with its [`error_handler`](../../api/basic_json/from_msgpack.md) parameter left at `keep` (the default), `from_msgpack()` reads `str` bytes (object keys included) as-is, without validating them, so such a value round-trips through `from_msgpack(to_msgpack(j))` byte for byte. Passing `error_handler_t::strict` makes `from_msgpack()` check anyway and throw [`parse_error.113`](../../home/exceptions.md#jsonexceptionparse_error113) for ill-formed UTF-8, and `replace`/`ignore` sanitize the string instead of keeping it. `to_msgpack()` also writes `str` bytes as-is by default, since the specification permits it; its [`error_handler`](../../api/basic_json/to_msgpack.md) parameter can be set to `strict` to throw [`type_error.316`](../../home/exceptions.md#jsonexceptiontype_error316) instead, or to `replace`/`ignore` to sanitize the string, for instance for a decoder that rejects ill-formed UTF-8. However, [`dump()`](../../api/basic_json/dump.md) still requires valid UTF-8 and throws [`type_error.316`](../../home/exceptions.md#jsonexceptiontype_error316) for a value read this way with the default `keep` handler, unless an error handler is passed that replaces or ignores the ill-formed bytes. ??? example "Example: deserialize a JSON value from MessagePack" ```cpp --8<-- "examples/from_msgpack.cpp" ``` Output: ```json --8<-- "examples/from_msgpack.output" ```