demo
Two panels, one library
Everything below is computed by moonbitstack/moonrag compiled to JavaScript
from the same sources the package ships. There is no second implementation behind these
controls: the page imports docs/engine/moonrag.js, built from
src/examples/browser/ when the site is published.
Cut a document
Paste anything. The three modes are the three splitters: markdown walks the
syntax tree, text tries separators from coarse to fine, and
token measures in words — a stand-in for the counter your model really uses.
Code points, not bytes: a Chinese sentence and an English one of the same length cost the same budget. Overlap is capped below the budget, because a chunk cannot carry more of the previous one than it holds.
Rerank a corpus
Seven passages, four numbers each, written down in the engine. Switch between a plain
cosine search and MMR, then move
lambda: at 1.0 the two orders agree, and as it falls the list
starts choosing passages that say something new over passages that say the same thing
again.
| passage | score |
|---|
The score is the cosine to the query. MMR uses it and the similarity between candidates to decide the order; the number shown stays the one a reader can check.
The page cannot drift from the library
The engine is rebuilt from these sources whenever the site is published, so the page cannot drift from the library. A page that reimplemented the splitter in JavaScript would not break; it would quietly disagree, which is worse.
The two things this page cannot show are the ones that need a database: the SQL store and its filters. Those are exercised against SQLite and against PostgreSQL with pgvector, on the same standard cases the in-memory store passes.