Install
One package holds the five formats and the packager. The formats are imported by their own paths, and the alias is the last part of the path.
moon add moonbitstack/moonmedia
Nothing is pulled in beyond the standard library, apart from the base encodings that turn a sample description's bytes into the hexadecimal a codec string quotes. A program that reads playlists and never opens a container carries only the playlist code.
Playlists
A playlist is text, and Playlist::parse takes it as it arrived
in a response. The result is either a master playlist or a media playlist.
The segments of a media playlist carry the key and the initialisation section
that were in force where they appear, so a caller looking for "what encrypts
this segment" does not have to replay the file to find out.
test {
let text =
#|#EXTM3U
#|#EXT-X-VERSION:7
#|#EXT-X-TARGETDURATION:6
#|#EXT-X-PLAYLIST-TYPE:VOD
#|#EXT-X-MAP:URI="init.mp4"
#|#EXTINF:6,
#|segment0.m4s
#|#EXTINF:4.004,
#|segment1.m4s
#|#EXT-X-ENDLIST
let playlist = @m3u8.Playlist::parse(text)
println(if playlist.is_master() { "master" } else { "media" })
let media = playlist.media().unwrap()
println("target: " + media.target_duration.to_string() + " s")
// The map in force for a segment is resolved for you.
println("map: " + media.segments[0].map.unwrap().uri)
for segment in media.segments {
println(segment.uri + " " + segment.duration.to_string())
}
}Writing goes the other way: build the model, then encode it. An empty media playlist takes the target duration, because every other tag can be defaulted but that one cannot.
test {
let media = @m3u8.MediaPlaylist::new(6)
media.version = Some(7)
media.playlist_type = Some(@m3u8.PlaylistType::Vod)
media.end_list = true
let init = @m3u8.InitSection::new("init.mp4")
for i = 0; i < 2; i = i + 1 {
let segment = @m3u8.Segment::new("segment" + i.to_string() + ".m4s", 6.0)
segment.map = Some(init)
media.segments.push(segment)
}
println(media.encode())
}6, not 6.0 — and
tags come out in the order RFC 8216 lists them, not the order the file
happened to use. Reading a playlist and writing it back gives the same model,
and a player sees the same thing; the bytes may differ.Boxes
An MP4 is a tree of boxes. Mp4::parse reads the tree, and a box
the package has no name for keeps its payload and its header form, so writing
the file back gives the file that was read.
test {
let samples : Array[@mp4.SampleData] = [
{
data: Bytes::from_array([b'\x00', b'\x00', b'\x00', b'\x01', b'\x65']),
duration: 1024L,
composition_offset: 0L,
sync: true,
},
{
data: Bytes::from_array([b'\x00', b'\x00', b'\x00', b'\x01', b'\x41']),
duration: 1024L,
composition_offset: 512L,
sync: false,
},
]
let fragment = @mp4.MovieFragment::new(1, [
{ track_id: 1, base_decode_time: 0L, samples },
])
let data = fragment.encode()
let file = @mp4.Mp4::parse(data[:])
for box in file.boxes {
println(box.type_ + " " + box.byte_length().to_string() + " bytes")
}
let read = file.fragments()[0]
println("fragment " + read.sequence.to_string() + ", " + read.start.to_string() +
".." + read.end.to_string())
for run in read.runs {
for sample in run.samples {
println("sample " + sample.size.to_string() + " bytes at " +
sample.offset.to_string() + ", key frame: " + sample.sync.to_string())
}
}
println("round trip: " + (file.encode() == data).to_string())
}The offsets in a run are absolute, measured from the start of the bytes that
were parsed; fragment.start and fragment.end are the
part of that buffer the fragment occupies, media data included. A caller
serving byte ranges needs both.
tracks() reads the sample table when a file has one, and the
fragments when it has those instead — a file written by a packager and a file
written by an encoder both come back as tracks with samples, and the caller
does not have to know which kind of file it opened.
The times on those samples are the media's, and a track says where the
presentation starts with edit_time. An encoder that primes its
decoder leaves the priming samples in the file and asks a player to begin
later; subtract edit_time from a sample's decode time and the
result is the instant it is played at.
Cut a movie into HLS
Packager takes a parsed movie and returns what a player asks
for: one initialisation section, then one fragment per key frame interval. The
audio that falls in an interval travels with the video that starts it.
The file below is 698 bytes of a 64 × 64 test pattern: eight frames a quarter of a second apart, the first and the fifth of them key frames. A caller passes whatever bytes it has; this page carries one so that the block runs as written.
test {
let movie =
#|AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1wNDEAAAJqbW9vdgAAAGxtdmhkAAAAAAAAAAAAAAAAAAAD6AAAB9AAAQAAAQAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAgAAAfZ0cmFrAAAAXHRraGQAAAAHAAAAAAAAAAAAAAABAAAAAAAAB9AAAAAAAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAABAAAAAAEAAAABAAAAAAAGSbWRpYQAAACBtZGhkAAAAAAAAAAAAAAAAAAAyAAAAB9BVxAAAAAAALWhkbHIAAAAAAAAAAHZpZGUAAAAAAAAAAAAAAABWaWRlb0hhbmRsZXIAAAABPW1pbmYAAAAUdm1oZAAAAAEAAAAAAAAAAAAAASFzdGJsAAAAhXN0c2QAAAAAAAAAAQAAAHVhdmMxAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAEAAQABIAAAASAAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAGP//AAAAH2F2Y0MBQsAK/+EACWdCwArZAPA8AQAEaM48gAAAABhzdHRzAAAAAAAAAAEAAAAIAAAMgAAAABxzdHNjAAAAAAAAAAEAAAABAAAACAAAAAEAAAA0c3RzegAAAAAAAAAAAAAACAAAAAUAAAAFAAAABQAAAAUAAAAFAAAABQAAAAUAAAAFAAAAGHN0c3MAAAAAAAAAAgAAAAEAAAAFAAAAFHN0Y28AAAAAAAAAAQAAApIAAAAwbWRhdAAAAAFlAAAAAUEAAAABQQAAAAFBAAAAAWUAAAABQQAAAAFBAAAAAUE=
let file = @mp4.Mp4::parse(@base64.decode(movie, whitespace=true)[:])
let packager = @moonmedia.Packager::new(file, target_duration=0.5)
let init = packager.init_segment()
println("init " + init.length().to_string() + " bytes")
for i, segment in packager.segments() {
println("segment" + i.to_string() + ".m4s " + segment.duration.to_string() +
" s " + segment.data.length().to_string() + " bytes")
}
println(packager.media_playlist().encode())
println(packager.master_playlist(uri="index.m3u8").encode())
}Each segment is a movie fragment of its own: a moof and the
mdat it points at, so a player can fetch one without the others.
The initialisation section is ftyp and moov, with
the sample tables stripped and the codec description kept.
The target duration is a hint, not a rule: the packager cuts where the
key frames are, and a segment can be shorter or longer than the target. The
playlist's own EXT-X-TARGETDURATION is the longest segment that
came out, rounded up.
Transport streams
A transport stream is a sequence of 188-byte packets. Some carry tables that say which programme is on which PID and what each programme is made of; the rest carry pieces of elementary streams, timed by a clock that rides in the adaptation field of one of them.
test {
let pat = @ts.Pat::new(transport_stream=1, programs=[@ts.Program::new(1, 0x1000)])
let packet = pat.section().packets(@ts.pat_pid)[0]
let stream = packet.encode()
println("packet " + stream.length().to_string() + " bytes")
let packets = @ts.parse(stream[:])
let table = @ts.sections(packets, @ts.pat_pid)[0]
let read = @ts.Pat::of(table)
println("programme " + read.programs[0].number.to_string() +
" on pid " + read.programs[0].pid.to_string())
println("intact: " + table.is_intact().to_string())
let pes = @ts.Pes::new(0xE0, Bytes::from_array([b'\x00', b'\x00', b'\x00', b'\x01', b'\x09']),
pts=90000L)
println("pts " + pes.pts().unwrap().to_string())
}Reading a whole stream is one call, and then the tables and the elementary
streams are questions asked of the packets: sections puts a PID's
payloads back into tables, pes_packets puts them back into frames.
The packets themselves are kept as they arrived, so writing them out again is
byte for byte the stream that came in.
Section::is_intact recomputes the CRC-32. A table with a bit error
in it is still parsed — a caller may want to see what it said — but it can tell
that the table is damaged before acting on it.Ingest a publish stream
RTMP is a conversation with an order to it: a handshake, a chunk size, the
commands that set up a stream, then media. Session is that
conversation as a state machine. Feed it the bytes a connection read; it
returns the bytes to write back.
test {
let session = @rtmp.Session::new()
let answer = session.push(@rtmp.client_start(0)[:])
let _ = session.push(@rtmp.client_finish(answer[0][:])[:])
let writer = @rtmp.ChunkWriter::new()
let _ = session.push(writer.write(@rtmp.set_chunk_size(4096))[:])
writer.chunk_size = 4096
let connect = @rtmp.Message::new(
@rtmp.msg_command,
0,
@rtmp.encode_all([
@rtmp.Amf0::String("connect"),
@rtmp.Amf0::Number(1.0),
@rtmp.Amf0::Object([("app", @rtmp.Amf0::String("live"))]),
]),
)
let replies = session.push(writer.write(connect)[:])
println("replies " + replies.length().to_string() + ", app " + session.app())
let publish = @rtmp.Message::new(
@rtmp.msg_command,
1,
@rtmp.encode_all([
@rtmp.Amf0::String("publish"),
@rtmp.Amf0::Number(2.0),
@rtmp.Amf0::Null,
@rtmp.Amf0::String("camera"),
@rtmp.Amf0::String("live"),
]),
)
let _ = session.push(writer.write(publish)[:])
println("stream " + session.stream_name())
let video = @rtmp.Message::new(@rtmp.msg_video, 1,
Bytes::from_array([b'\x17', b'\x00', b'\x00']))
let _ = session.push(writer.write(video)[:])
println("media " + session.take_media().length().to_string())
}What comes back from push is what the peer is owed right now:
the handshake answer, a window acknowledgement size, the reply to each
command. take_media empties the queue of audio and video messages
in the order they arrived, each with its timestamp, and
metadata holds the properties from the publisher's
onMetaData.
A session that is only halfway through a message keeps the pieces and waits for the rest, so a caller can hand it whatever a socket returned — one byte or sixty kilobytes.
DASH manifests
An MPD is XML. The model is built, encoded, and parsed back; XML never reaches the caller, and a manifest written here carries the attributes the specification requires even when the model left them out.
test {
let manifest = @mpd.Manifest::new(kind=@mpd.Kind::Static)
manifest.min_buffer_time = Some(2.0)
manifest.duration = Some(8.0)
let period = @mpd.Period::new(id="1", start=0.0)
let video = @mpd.AdaptationSet::new(content_type="video", mime_type="video/mp4")
let template = @mpd.Template::new()
template.duration = Some(4000L)
template.timescale = Some(1000L)
template.media = Some("segment-$Number$.m4s")
video.template = Some(template)
video.representations.push(@mpd.Representation::new("v0", bandwidth=800_000))
period.adaptations.push(video)
manifest.periods.push(period)
println(manifest.encode())
println("read back: " +
@mpd.Manifest::parse(manifest.encode()).periods.length().to_string() + " period")
}Durations are handled as the format writes them: PT2S reads as
two seconds, PT1H2M3.5S as an hour, two minutes and three and a
half seconds, and a value that is not a duration is an error rather than a
silent zero.
A manifest addresses its segments one of three ways: a template with the numbers filled in, a list of URLs, or one file with a byte range per segment. Filling a template in is the library's job, because the padding is part of the name. A list and a base are what a packager produces when it writes a whole rendition into one file, and both are read into byte ranges a player can ask for.
test {
let template = @mpd.Template::new()
template.timescale = Some(16384L)
template.duration = Some(8192L)
template.initialization = Some("init-$RepresentationID$.m4s")
template.media = Some("chunk-$RepresentationID$-$Number%05d$.m4s")
println(template.segment_url(number=7L, representation_id="v0"))
println(template.initialization_url(representation_id="v0"))
println("segments in one second: " + template.count(1.0).to_string())
let base = @mpd.SegmentBase::new()
base.index_range = @mpd.Range::of("1409-1476")
base.index_range_exact = Some(true)
let header = @mpd.Initialization::new()
header.range = @mpd.Range::of("0-1380")
base.initialization = Some(header)
println("index " + base.index_range.unwrap().text() + ", " +
base.index_range.unwrap().length().unwrap().to_string() + " bytes")
println("init " + base.initialization.unwrap().range.unwrap().text())
let list = @mpd.SegmentList::new()
list.timescale = Some(16384L)
list.duration = Some(8192L)
let first = @mpd.SegmentUrl::new()
first.media_range = @mpd.Range::of("1389-15304")
list.urls.push(first)
println("a list of " + list.length().to_string() + " segment, " +
list.urls[0].media_range.unwrap().length().unwrap().to_string() + " bytes")
}Protected content is signalled in the manifest, not in the media: an entry names the key system and carries the header that system is handed, base64 in the document. The header arrives decoded, and leaves the model encoded, so a manifest that is read and written again still says which key opens it.
test {
let manifest = @mpd.Manifest::new(kind=@mpd.Kind::Static)
manifest.duration = Some(8.0)
let period = @mpd.Period::new(id="1")
let video = @mpd.AdaptationSet::new(content_type="video")
video.protection.push(@mpd.ContentProtection::new(
"urn:uuid:edef8ba9-79d6-4ace-a3c8-27dcd51d21ed",
pssh=b"psshpayload",
))
period.adaptations.push(video)
manifest.periods.push(period)
let xml = manifest.encode()
println(xml)
let header = @mpd.Manifest::parse(xml)
.periods[0].adaptations[0].protection[0].pssh.unwrap()
println("header: " + header.length().to_string() + " bytes")
}Checked against other implementations
The bytes this library reads in its examples are bytes other programs wrote: ffmpeg's muxers and GPAC's packager. Each format is replayed from a payload one of them produced, and the reading is compared line for line with what a probe says about the same file — sample counts, decode times, key frames, the elementary bytes of a stream, the length of the presentation.
The other direction is checked too. A movie is packaged into HLS, the segments and playlists are written out as a server would hold them, and a player is asked to play the result: the frame counts, the key frame positions and the length it reads agree with what the packager said it made, and every frame of both streams decodes.
Where the library stops
The library reads and writes formats. It does not open sockets, read files, or decode anything.
- No I/O. A playlist arrives as bytes and leaves as bytes; serving them is the caller's job, whether that is a range request or a file on disk.
- No codecs. Samples are carried, not inspected. The codec string from a sample description is passed through, and an H.264 access unit is bytes like any other.
- No live window.
Packagercuts a file that has an end. A rolling playlist for a live stream is a media playlist the caller maintains, dropping the segment that fell out and adding the one that just finished — the playlist type writes it, and the low-latency tags are read and written the same way. - A manifest is read into a model, not kept as text. What the model has no name for — an event stream, a clock hint, a role — is not written back when the manifest is encoded again. Protection is named, because a player cannot open a rendition without it.
Next: the reference lists every function and type, and the anatomy takes one second of video apart three ways.