how it works
Four packages,
one direction of dependence.
Nothing here imports upward. The event vocabulary knows nothing; the tree builder knows the vocabulary; the parsers know the tree; the chain knows all of it and is known by none.
The shape of the module
A selector engine generic over a trait view of the tree would be a second representation of the tree, and two representations drift. So the engines live beside the tree, and the parsers sit above it.
HTML: the spec's algorithm, written down
The tokenizer folds states that share a shape — there are no separate states for every attribute-value flavour — but the tree construction is the specification's list of insertion modes, in the specification's order, with the specification's repairs.
Why it is not "lenient XML"
A lenient XML parser produces one repaired tree. WHATWG produces the tree: a specific one, which is why two browsers agree about <table><div>x<div> and a lenient parser does not.
What "repair" means concretely
Implied <html>, <head>, <body> and <tbody>; cells that close themselves; stray content fostered in front of a table; formatting reopened after a block closed it; misnested markup rearranged by the adoption agency.
Reading without building
The tree builder is one consumer of the tokenizer, not the tokenizer itself. @html.scan(text, sink) hands the tokens — a run of characters, a start tag with its attributes, an end tag, a comment, a doctype — to any Sink, before a tree exists. An end tag that matches nothing is still an end tag there, and a repeated attribute keeps the first: the tree builder is what would have dealt with them.
The algorithm has one switch, and it shows in one place. With scripting on, a noscript's content is what a browser would run, so it is raw text; with it off, it is markup a reader can walk. A document remembers which way it was read, and the writer asks, because text read as raw text has to be written without escaping.
scripts/gen_html5lib.py writes the fixture: the inputs, the contexts, the flag each case asks for, and the trees they are expected to produce.The chain: one shape, narrowing
A Selection is the nodes and the document they came from. Every query takes a selection and answers a selection, so the type never changes and the chain never stops.
Identity, not equality
Nodes are compared with physical_equal, never by value: two <li>coffee</li> elements are different nodes, and a set that contains both must keep both.
Inside the XPath engine a node is named by where it sits — its parent and its index — rather than by what it holds. A text node carries no pointer of its own, and two texts that spell the same thing are still two nodes: //. answering the whole document, and a node reached twice counting once, both depend on that.
Attributes and namespaces are nodes of the same engine. An attribute knows the element it belongs to and its own name; a namespace knows its owner and its prefix. Neither is a child, so the writer never walks them, but //a/@href answers something with an identity and attribute::x/self::node() has an answer. A name test keeps the axis's principal node type, so attribute::* is every attribute while self::* on one is nothing.
XPath's two answers
XPath can answer a number. A chain cannot: it is always a set of nodes. So xpath() answers the node-set, and xpath_value() answers the typed XPathValue — nodes, string, number or boolean — for the queries that are questions rather than selections.
Writing back
XML: exact
Attribute order, CDATA as CDATA, comments, processing instructions with their raw data, the doctype including an internal subset, <a/> versus <a></a>. Names are Clark-spelled in the tree and prefixed on the way out, using the prefix the document declared; a namespace nothing declared gets ns1 and a declaration on the element that needed it.
HTML: by the rules
Void elements take no close tag, raw-text elements are written raw, & becomes & and a no-break space becomes — WHATWG's own serialization, so what you write is what a browser would read back as the same tree.
What the tests hold down
| Suite | What it pins |
|---|---|
| Tree and chain | Every verb, on both sides — the node it creates and the parent pointers it leaves behind. |
| CSS engine | Every combinator, every attribute operator, every pseudo-class implemented — and the parse errors for the ones refused. |
| XPath 1.0 | Axes including the reverse ones, predicate positions per context node, the comparison rules of §3.4, number formatting of §4.2. |
| XML | Byte-for-byte round trips, namespaces, and the well-formedness errors with their offsets. |
| HTML | html5lib's tree-construction corpus — every case of it — and a hand-written case for every repair above. |
| htmx | Triggers and swaps as they are spelled, inheritance and hx-disinherit, the values a form sends, and the answer landing where the page said. |
All of it runs on all four backends — wasm, wasm-gc, js, native — in CI, with --deny-warn and a formatting check that fails the build.
❦