Cross-language serialization

Your value,
in the bytes
eleven languages
already read.

Apache Fory fixes its wire format apart from any one implementation. This is that format for MoonBit: the xlang binary, the row layout, and the type table between them — written from the specification, proved against pyfory.

Read the docs See it run Source

What is here

Two formats, one type table.

A serialization format is a promise about bytes. Fory keeps two: the cross-language binary that moves a value between runtimes, and the row layout that lets a reader take one field without rebuilding the object. Both sit on the same mapping from MoonBit types to Fory's type IDs.

The cross-language binary

A one-byte header, a type ID, then the body. Varints where the width varies, little-endian where it does not, and a reference flag in front of every value that may be shared.

The row format

A null bitmap, eight bytes per field, then the variable data. Field i is always at bitmap + i × 8, so a reader takes one field and leaves the rest unread.

Shared and circular values

A value written twice is written once and pointed at afterwards. MoonBit has no object identity to track, so sharing is marked with a key you choose; a cycle that cannot be resolved is refused, not hung on.

Time through moondate

Date, Moment and Span travel as DATE, TIMESTAMP and DURATION. Nanoseconds stay the sub-second part, in [0, 1e9), never a negative remainder.

The shape of the API

Four verbs, and the bytes.

dumps and loads move a whole message. dump and load do the same over a buffer or a reader you already hold. A type joins in by implementing Codec — one method to write the body, one to read it.

Call	Result
`dumps(v)`	one message as `Bytes`
`loads(b)`	the value, or a `Refused`
`dump(v, buf)`	writes into your buffer
`load(reader)`	the next value

let bytes : Bytes = @moonfory.dumps(42)
// 01 ff 04 2a 00 00 00

let back : Int = @moonfory.loads(bytes)
// 42

pub impl Codec for UserId with fn ty(_) {
  TypeId::Int64
}

pub impl Codec for UserId with fn write_data(self, w) {
  w.write_i64(self.0)
}

Row format

One field, without the rest.

Give the schema once. Set fields by index, finish, and open a cursor over the bytes. Reading field 1 never touches field 0.

let schema = @moonfory.RowSchema::new([
  { name: "id",   ty: @moonfory.RowType::Int64,  nullable: false },
  { name: "name", ty: @moonfory.RowType::String, nullable: false },
  { name: "age",  ty: @moonfory.RowType::Int32,  nullable: true  },
])

let row = @moonfory.Row::new(schema)
row.set(0, @moonfory.RowValue::Int64(7L))         // id
row.set(1, @moonfory.RowValue::String("ada"))    // name
row.set(2, @moonfory.RowValue::Null)               // age

let cur = @moonfory.RowCursor::new(row.finish()[:], schema)
cur.get_field(1)   // "ada" — field 0 never read
cur.is_null(2)     // true

—

Proved, not asserted

89 tests, every one a byte fixture: our own, and the ones pyfory writes. Every refusal has a test that provokes it.

—

Four backends

wasm, wasm-gc, js and native, checked with --deny-warn in CI on Linux, macOS and Windows.