One document, read and written
The four verbs the whole family shares. The s is on
the form that takes or answers a string.
let config = @json.loads("{\"host\":\"localhost\",\"port\":8080,\"tls\":false}")
@json.dumps(config)
@json.dumps(config, indent=2)
@json.dump(config) // the same, as the bytes of a body
@json.load(@json.dump(config)[:])
{"host":"localhost","port":8080,"tls":false}
{
"host": "localhost",
"port": 8080,
"tls": false
}
A document is one value and nothing else: anything after it other
than whitespace is Trailing, not quietly ignored. A
parser that stops at the first value turns a truncated file into
a plausible one.
| text → tree | bytes → tree | tree → text | tree → bytes | |
|---|---|---|---|---|
| one document | loads |
load |
dumps |
dump |
| a stream of them | loads_all |
load_all |
dumps_all |
dump_all |
The plural set exists exactly where the format defines a
stream: lines is a stream and has only
those, while json, jsonc and
json5 define one document and have only the
singular set.
Not from_str and to_string:
to_string already means
Show::to_string in MoonBit, and
@json.to_string(x) beside x.to_string()
is a sentence that reads two ways.
Writing is the strict grammar whatever the document was read as, and its three settings are arguments because a call site is where a diff or a transport is known:
| Setting | Default | What it does |
|---|---|---|
indent |
0 | Zero is the compact form; two is over several lines. |
ascii |
off | Escape everything above U+007E, for a transport that is not eight-bit clean. |
sort |
off | Members in order of name instead of the order they were read in, which is what a diff wants. |
Changing it
Three ways, for three kinds of change. One place is named with a pointer (RFC 6901); a set of changes that must all hold or none is a patch (RFC 6902); and a change that looks like the document itself is a merge patch (RFC 7396).
// One field, and nothing else changes.
let cfg = @pointer.set(config, "/port", Json::number(9090))
@pointer.get(cfg, "/port") // 9090
@pointer.has(cfg, "/port") // true
// Six operations, applied in order — or, if one fails, not at all.
let ops = @patch.loads(
"[{\"op\":\"replace\",\"path\":\"/port\",\"value\":9090},{\"op\":\"add\",\"path\":\"/tls\",\"value\":true}]"
)
@patch.apply(config, ops)
// A patch that looks like the document: a member that is not there is added,
// one that is there is replaced, and a null takes one away.
@merge.merge(config, @json.loads("{\"port\":9090,\"tls\":null}"))
{"host":"localhost","port":9090,"tls":true}
{"host":"localhost","port":9090}
A patch is read into Operations rather than left as
a tree of objects, so a misspelled op is an error
rather than a document that quietly does nothing. A patch that
fails partway applies nothing — the document it was given is
never changed.
@pointer's. The syntax, the escapes and
the refusals come from there, said about the member that named
them, so one implementation of RFC 6901 serves both packages.
The notations
One tree, several ways of writing it. Each dialect is a package with the same four faces, so learning one is learning all of them, and they read the same document to the same tree.
let plain = @json.loads("{\"host\":\"localhost\",\"port\":8080,\"tls\":false}")
let source =
#|{
#| "host": "localhost", // where to bind
#| "port": 8080,
#| "tls": false, // a certificate is not set up yet
#|}
@jsonc.loads(source[:]) == plain // true
@json5.loads("{host:'localhost',port:8080,tls:false,}") == plain // true
jsonc == json -> true json5 == json -> true
json
JSON, exactly — RFC 8259 and ECMA-404. The grammar and nothing more: no comments, no trailing comma, no leading zero, no unquoted name, no control character inside a string. Everything the grammar refuses is an error naming the line and the column.
jsonc
JSON with comments and trailing commas — what editors and their
configuration files have settled on, tsconfig.json
being the best-known.
json5
JSON5: unquoted names, single quotes, hexadecimal,
Infinity, NaN, and the whitespace
ECMAScript recognises. It carries values JSON has no syntax for;
those go back out as null, as every JavaScript
implementation writes them.
@json5.dumps(@json5.loads("[Infinity,NaN,0xFF]"))
[null,null,255]
lines
JSON Lines, also called NDJSON: one document to a line, separated by a line feed, each one complete. It is what a log file or a stream of records is written as, because a reader can start anywhere and a writer can append without rewriting. Every line it writes ends in a feed, including the last.
@lines.loads_all("{\"n\":1}\n{\"n\":2}\n{\"n\":3}\n").length()
3
jcs
JSON Canonicalisation Scheme, RFC 8785: one document, one
sequence of bytes, so that two parties signing the same values
sign the same bytes. Members sort by UTF-16 code unit — not by
code point, which differs above U+FFFF — numbers are written the
way ECMAScript writes them, and nothing above U+001F is escaped.
A number that is not finite has no canonical form and is refused
rather than replaced with null.
hjson
Hjson, "a user interface for JSON": comments of three kinds, the
braces around an object and the brackets around an array both
optional, keys that need not be quoted, commas optional, and
multiline strings opened with ''' or
""". The reader is real and lenient; the writer emits
strict JSON, because Hjson has no canonical text form.
cson
CSON, CoffeeScript Object Notation: indentation instead of braces,
a line beginning with - for an array element,
# comments. A real indentation parser; the writer is
strict JSON.
the JSON-shaped dialects
Several formats are JSON itself under another name, so their
packages read and write the same tree through @moonjson:
superjson, ntv, rjson,
json11, jsonz, zson and
zjson. Their writers emit strict JSON, because that
is what each of them is.
pretty
A formatter, not a dialect: the same writer with the knobs a
formatter wants — spaces or tabs, LF or CRLF, sorted members —
and reformat, which reads a document and writes it
back, the way python -m json.tool does.
Querying it
A pointer names one place; JSONPath asks for a set of them, with
filters. RFC 9535's grammar, and its five functions
length, count, match,
search and value.
let shelf = @json.loads(
"{\"store\":{\"book\":[{\"title\":\"A\",\"price\":8.95},{\"title\":\"B\",\"price\":12.99},{\"title\":\"C\",\"price\":8.99}]}}"
)
@path.select(shelf, "$..book[?@.price<10].title")
// A query is compiled once and run against any number of documents, because a
// filter is evaluated once per node.
let query = @path.compile("$.store.book[*].title")
query.select(shelf)
["A","C"]
match and search take I-Regexp,
RFC 9485's restricted form — no backreferences, no lookaround,
no flags, and \p{…} for the Unicode general
categories. It is a small backtracking matcher over characters,
in the package, and a pattern that is not one is refused before
it runs.
How to read is a value
Flavor carries both what syntax a dialect accepts
and what it does at the edges — how deep the nesting may go,
what a lone surrogate means, what a repeated member name means —
and every reader takes one.
@json.loads(text) // strict JSON
@jsonc.loads(text) // the same, plus comments
@json.loads(text, flavor=@moonjson.Flavor::new(depth=100))
@json.loads(text, flavor={ ..@moonjson.strict, duplicates: Reject })
Two layers, the later overriding the earlier:
the dialect's preset < the flavor you
pass. The two ways of building one — the constructor
and the record update — are the same thing.
| Setting | Default | Why that one |
|---|---|---|
depth |
500 | A hundred thousand open brackets is a denial of service, not a document. |
surrogates |
refuse | The case is implementation-defined, so the safer side is the default: a lone half cannot be written back out. |
duplicates |
Last |
The behaviour most readers already have. |
bom |
skipped | RFC 8259 §8.1 forbids writing one and says nothing about reading one; refusing would reject half of what Windows tooling produces. |
Core's Json, not one of our own
Anything that already speaks @json.Json speaks
this, with no conversion.
A number keeps the text it was written as, so a document read
and written again comes back unchanged — including a
twenty-digit integer and 1e400, which a double
cannot hold and which a parser that goes through one silently
rewrites.
let wide = "[1e400,12345678901234567890,0.1]" @json.dumps(@json.loads(wide)) == wide // true
A number JSON has no syntax for, which JSON5 can produce, is
written as null, as every JavaScript
implementation writes it. Members keep their order.
What went wrong, and where
Reading raises Refused, which says what went wrong
and where — offset, line and column. One type for the whole
family, so there is one thing to catch.
try @json.loads("{\n \"a\": 1,\n \"b\": ?\n}") catch {
e => match e.at() {
Some(spot) => println("line \{spot.line}, column \{spot.column}")
None => println(e) // a pointer or a patch, which has a path instead
}
}
line 3, column 8
A caller that only wants to show the failure to a person does not
have to walk the variants: message() writes the
sentence, in the same place, with the character that was found.
try @json.loads(text) catch {
e => println(e.message())
}
line 3, column 8: unexpected '?'
The variants carry the scanner's refusals, the pointer's
Missing and BadIndex, the patch's
TestFailed and IntoChild, and
Unwritable for a value a writer cannot put on
paper. A failure in a binary format answers None
from at() and carries a byte offset in its variant
instead, because a line number invented for a byte string would
be worse than none.
Nesting is refused past 500 levels rather than taken to the stack: a document of a hundred thousand open brackets is a denial of service, not a document. A lone half of a surrogate pair is refused, because it is not a character and any value put in its place would be a guess.
The binary formats
Ten encodings of the same tree. CBOR, MessagePack and BSON are
the three everybody meets; UBJSON, Smile and BJSON are the next
tier; KRSON and CompactBinaryData are compact newcomers; and the
two database-native ones are SQLite's jsonb and
PostgreSQL's jsonb. Each has its own package and its
own specification, each with the two verbs that take bytes.
let body = @cbor.dump(config) // the same tree, as bytes
@hex.encode(body[:])
@cbor.load(body[:]) == config // true
@json.dumps(@msgpack.load(@msgpack.dump(config)))
@bson.dump(@json.loads("{\"$oid\":\"507f1f77bcf86cd799439011\"}"))
a363746c73f464686f7374696c6f63616c686f737464706f7274191f90
Writing fills in the shortest form the format defines; reading keeps everything. What JSON has no syntax for travels as a marker, so a value that came from one of them survives a trip through JSON:
| Value | Marker |
|---|---|
| byte string | {"$bytes": "0102"} |
| CBOR tag | {"$tag": 1, "value": …} |
| MessagePack extension | {"$ext": 1, "data": "0102"} |
undefined | {"$undefined": true} |
| non-string map key | {"$map": [[1, 2]]} |
| ObjectId | {"$oid": "507f…"} |
| UTC datetime | {"$date": 1356350400000} |
A marker name is reserved: an object whose only member is
$oid is an ObjectId. That is the same bargain
MongoDB's Extended JSON makes, and for the same reason — a tree
with one object type needs one reserved name. The root package
holds the scanner every dialect shares, the writer, the
Flavor record and Refused.
Every one of the seven extra binaries — UBJSON, Smile, BJSON,
KRSON, CompactBinaryData, SQLite jsonb and
PostgreSQL jsonb — keeps the same promise: a tree
read from one and written to another is unchanged, and a byte
string crosses them all as $bytes. The streaming
pair, sink and source, carries a
sequence of documents one at a time in the NDJSON shape that
lines reads and writes.