moonmd markdown

the guide

A document, whole and in pieces.

Install the package, read one document, look at what came out, then read one that is still arriving. Every output below is what the library printed for the code above it.

§01

Install

shell
moon add moonbitstack/moonmd

The import alias is a convention, not a requirement. parse takes a StringView, so a String is handed over with [:].

moonbit
import {
  "moonbitstack/moonmd" @moonmd,
}

§02

A document

One call reads the text and returns a Document. It holds the blocks and the flavor it was read with; nothing is rendered until asked.

moonbit
let document = @moonmd.parse("# Notes\n\nA *small* page.\n"[:])
println(document.html(safe=false))
output
<h1>Notes</h1>
<p>A <em>small</em> page.</p>

A document that is empty, or holds only blank lines, has no blocks at all. A last line without a line ending is still a line.

§03

The tree

Blocks hold blocks; a paragraph, a heading and a table cell hold inlines. There are nine block kinds and ten inline kinds, and walk visits each node once, containers before their contents.

moonbit
let document = @moonmd.parse("# Notes\n\n- one\n- two\n"[:])
@moonmd.walk(document, fn(node) {
  match node {
    @moonmd.Block(@moonmd.Heading(heading)) =>
      println("heading, level \{heading.level}")
    @moonmd.Block(@moonmd.List(list)) =>
      println("list of \{list.items.length()}")
    @moonmd.Inline(@moonmd.Text(text)) => println("text: \{text}")
    _ => ()
  }
})
output
heading, level 1
text: Notes
list of 2
text: one
text: two

A list item carries a task marker of its own, so a checkbox is data rather than a string that has to be recognised again later.

§04

HTML

html takes one flag. With the default safe=true, raw HTML is escaped and a destination that would run script keeps its text and loses its link.

moonbit
let page = @moonmd.parse("<b>bold</b> [x](javascript:alert(1))\n"[:])
println(page.html())
println(page.html(safe=false))
output
<p>&lt;b&gt;bold&lt;/b&gt; x</p>
<p><b>bold</b> <a href="javascript:alert(1)">x</a></p>

The flag is a rendering choice, not a parsing one: the tree is the same either way, so a viewer can offer both without reading the document twice.

safe=false reproduces the HTML the CommonMark examples expect, character for character. Keep it for text you control.

§05

Markdown

The same tree writes itself back out. The writer escapes what would otherwise start a block, and grows a fence past any run of its own characters inside the code, so the result reads back to the same HTML.

moonbit
let document = @moonmd.parse("# Notes\n\nA *small* page.\n"[:])
println(document.markdown())
output
# Notes

A *small* page.

The output is not the input: a table written with uneven pipes comes back aligned, an indented code block becomes a fence. What survives is the document, not its whitespace.

§06

Footnotes

gfm also reads footnotes: a [^label] in the text is a reference, and a line that opens [^label]: is its definition. The reference becomes a numbered marker; the definitions gather at the end as a notes list, each with a link back to where it was cited.

moonbit
let document = @moonmd.parse("A claim[^note]\n\n[^note]: the support\n"[:])
println(document.html(safe=false))
output
<p>A claim<sup class="footnote-ref"><a href="#fn-note" id="fnref-note" data-footnote-ref>1</a></sup></p>
<section class="footnotes" data-footnotes>
<ol>
<li id="fn-note">
<p>the support <a href="#fnref-note" class="footnote-backref" data-footnote-backref data-footnote-backref-idx="1" aria-label="Back to reference 1">↩</a></p>
</li>
</ol>
</section>

The number follows the first reference, not the definition, so a document can define its notes in any order. markdown writes both the reference and the definition back out, and a reference with no definition stays the text it was written as.

§07

Streaming

feed takes the next chunk and returns the blocks that no later text can change. finish returns what is left. A chunk may end anywhere, including in the middle of a line.

moonbit
let stream = @moonmd.Stream::new()
println(stream.feed("# Notes\n\nA *sm"[:]).length())
println(stream.feed("all* page.\n\n"[:]).length())
println(stream.finish().length())
output
1
1
0

The heading settles when the blank line after it arrives; the paragraph settles when its own blank line does. Nothing is handed out twice and nothing is taken back.

defer_links is on by default: a block that mentions a link label waits for finish, because a definition written later still applies to it. Pass defer_links=false to Stream::new to get such blocks earlier, and accept that a forward reference then stays text.

§08

Flavors

gfm is the default: CommonMark plus tables, strikethrough, bare links, task list markers, footnotes and the tag filter. commonmark turns all six off, and a Flavor literal turns them off one at a time.

moonbit
let plain = @moonmd.parse("~~gone~~\n"[:], flavor=@moonmd.commonmark)
println(plain.html(safe=false))
output
<p>~~gone~~</p>

A flavor is a plain record, so a caller can take gfm and clear one field without copying the rest by hand.

§09

API reference

Generated from pkg.generated.mbti and the doc comments in the source, so this list is the package's interface, not a copy of it.

values

pub let commonmark : Flavor

CommonMark without any of the extensions.

pub let flavor : Flavor

The preset parse uses when the caller does not pass one: gfm.

pub let gfm : Flavor

CommonMark as GitHub reads it: tables, strikethrough, bare links, task list items, the tag filter and footnotes. This is the default.

functions

pub fn parse(StringView, flavor? : Flavor) -> Document

Parse a Markdown document.

test {
    let document = @moonmd.parse("# Title")
    assert_eq(document.html(safe=false), "<h1>Title</h1>\n")
  }
pub fn walk(Document, (Node) -> Unit) -> Unit

Visit every node of the document, each container before its children and siblings in order.

types

pub(all) enum Align

How a table column lines up its cells, taken from the delimiter row.

Default Left Center Right
pub(all) enum Block

A block level node.

Heading(Heading) Paragraph(Array[Inline]) Code(Code) Quote(Array[Block]) List(ListBlock) Table(Table) Html(String) Break Footnote(Footnote)
pub(all) enum Break

A line break inside a paragraph.

Soft Hard
pub(all) struct Code

A code block, fenced or indented. info is the word after the opening fence, empty for indented code.

info : String text : String fenced : Bool
pub struct Document

A parsed document: the blocks in order, and the flavor it was read with, which the HTML renderer needs for the tag filter.

pub fn Document::blocks(Self) -> Array[Block] pub fn Document::flavor(Self) -> Flavor pub fn Document::html(Self, safe? : Bool) -> String pub fn Document::markdown(Self) -> String pub fn Document::of_blocks(Array[Block], flavor? : Flavor) -> Self
pub(all) struct Flavor

What a parser accepts beyond the CommonMark core. The presets commonmark and gfm cover the two dialects this library is asked for; the fields are public so a caller can take one and drop what it does not want.

tables : Bool strikethrough : Bool autolinks : Bool task_lists : Bool tag_filter : Bool footnotes : Bool
pub(all) struct Footnote

A footnote, the body of one [^label]: definition. label is the label as it was written. number is the order the footnote is shown with, assigned in the order its first reference appears in the text; backrefs is how many references point at it.

label : String blocks : Array[Block] number : Int32 backrefs : Int32
pub(all) struct FootnoteRef

A footnote reference, [^label], and where it lands: the definition's label, the number that definition is shown with, and ref_ix, the order of this reference among the ones that point at the definition.

label : String number : Int32 ref_ix : Int32
pub(all) struct Heading

A heading, level counting the ATX # marks or the setext underline.

level : Int32 content : Array[Inline]
pub(all) enum Inline

An inline level node.

Text(String) Code(String) Emphasis(Array[Inline]) Strong(Array[Inline]) Link(Link) Image(Link) Html(String) Strike(Array[Inline]) Break(Break) FootnoteRef(FootnoteRef)
pub(all) struct Item

One item of a list, with the task marker that introduced it if any.

blocks : Array[Block] task : Task
pub(all) struct Link

The destination, title and content of a link or image. For an image the content is the alternative text.

dest : String title : String content : Array[Inline]
pub(all) struct ListBlock

A list, with its items and whether it is tight (no blank lines between items, so their paragraphs are rendered without <p>).

kind : ListKind tight : Bool items : Array[Item]
pub(all) enum ListKind

What kind of list a ListBlock is, together with the marks needed to write it back unchanged.

Bullet(Char) Ordered(Int32, Char)
pub(all) enum Node

A node as walk hands it to its visitor: either a block or an inline.

Block(Block) Inline(Inline)
pub struct Stream

A parser that reads a document in pieces. Text that has arrived is parsed again on every call, but only the part that has not been handed out yet, so the work per chunk is proportional to the unfinished tail rather than to the whole document.

pub fn Stream::feed(Self, StringView) -> Array[Block] pub fn Stream::finish(Self) -> Array[Block] pub fn Stream::new(flavor? : Flavor, defer_links? : Bool) -> Self
pub(all) struct Table

A table: the alignment of each column, the header row, then the body rows. A row holds one cell per column, and a cell holds the inlines inside it.

align : Array[Align] head : Array[Array[Inline]] body : Array[Array[Array[Inline]]]
pub(all) enum Task

Whether a list item is a task list item, and if so whether it is done.

Plain Unchecked Checked