API reference — moonward

01The breaker

Sre

struct derives Debug, Eq

The constants of probability-based client-side throttling.

The preset is sre. Build another with Sre::new, or update one in place with { ..sre, k: 2.0 }.

field	type	what it is
`k`	`Double`	How many calls each success pays for. At 1.5 a backend is left alone until more than a third of its calls fail.
`floor`	`Double`	What `k` decays to while every bucket at the head of the window is failing, so past successes count for less the longer an outage lasts.
`protection`	`Double`	Calls every backend gets regardless, which keeps a quiet one from throttling itself over two bad calls.
`probe`	`@moondate.Span`	After this long with nothing let through, one call is let through anyway, so a backend that has recovered is noticed.
`size`	`Int32`	Buckets in the window.
`window`	`@moondate.Span`	How far back the window reaches.

pub fn Sre::new(k~ : Double = sre.k, floor~ : Double = sre.floor, protection~ : Double = sre.protection, probe~ : @moondate.Span = sre.probe, size~ : Int32 = sre.size, window~ : @moondate.Span = sre.window) -> Sre

A policy by name, every knob with the preset's value.

parameter	type	default	what it is
`k`~	`Double`	`sre.k`	how many calls each success pays for; the preset's `1.5` leaves a backend alone until more than a third of its calls fail
`floor`~	`Double`	`sre.floor`	what `k` decays to during an outage, `1.1` in the preset
`protection`~	`Double`	`sre.protection`	calls every backend gets regardless, `5` in the preset
`probe`~	`@moondate.Span`	`sre.probe`	how long with nothing let through before one call is let through anyway
`size`~	`Int32`	`sre.size`	buckets in the window, `40` in the preset
`window`~	`@moondate.Span`	`sre.window`	how far back the window reaches, `10 s` in the preset

Breaker

struct

A circuit breaker that sheds a fraction of calls rather than all of them.

There is no open, half-open and closed: the breaker keeps a rolling window of outcomes and refuses each new call with a probability that grows with the failures in it. A struggling backend keeps receiving as much as it can still serve, instead of being cut off and then flooded again when it recovers.

With accepts successes out of total calls in the window, failing the run of all-failure buckets at its head and working the run of all-success ones:

w     = max(k - (k - floor) × failing / size, floor)
drop  = (total - protection - w × accepts) / (total + 1)
drop ×= (size - working) / size

A drop at or below zero lets everything through, which is the whole healthy case.

Held by the caller. Its fields are private; the methods below are the way in.

pub fn Breaker::allow(self : Breaker, now : Int64, roll : Double) -> Promise?

Ask to make one call at now.

roll is a draw from [0, 1); the call is refused when it falls under the drop fraction. Some means go ahead, and the caller settles the promise once the call has finished. A refusal is recorded as a Drop, so a breaker that is refusing never reads back as idle and throttles itself off.

parameter	type	default	what it is
`now`	`Int64`	—	the caller's clock reading, in nanoseconds
`roll`	`Double`	—	a draw from `[0, 1)`

pub fn Breaker::new(sre~ : Sre = sre, now~ : Int64 = 0L) -> Breaker

A breaker with an empty window opening at now.

parameter	type	default	what it is
`sre`~	`Sre`	`sre`	the policy; the `sre`(#sre) preset by default
`now`~	`Int64`	`0L`	the clock reading the window opens at, `0L` by default

pub fn Breaker::ratio(self : Breaker, now : Int64) -> Double

The fraction of calls the breaker would refuse at now, zero while the window is healthy.

The probe is left out: this is the standing throttle, which is what a dashboard or an alert wants.

parameter	type	default	what it is
`now`	`Int64`	—	the caller's clock reading, in nanoseconds

Promise

struct

A call the breaker let through. Exactly one of pass and fail settles it; until then the window holds no record of how the call went.

Held by the caller. Its fields are private; the methods below are the way in.

pub fn Promise::fail(self : Promise, now : Int64) -> Unit

The call failed.

parameter	type	default	what it is
`now`	`Int64`	—	the clock reading the call finished at

pub fn Promise::pass(self : Promise, now : Int64) -> Unit

The call succeeded.

parameter	type	default	what it is
`now`	`Int64`	—	the clock reading the call finished at

02Load shedding

Shed

struct derives Debug, Eq

The constants of adaptive load shedding.

The preset is shed. Build another with Shed::new, or update one in place with { ..shed, cpu: 800 }.

field	type	what it is
`size`	`Int32`	Buckets in the window the throughput and response times are read from.
`window`	`@moondate.Span`	How far back that window reaches.
`cpu`	`Int32`	CPU use in thousandths at and above which the server counts as overloaded.
`latency`	`@moondate.Span`	The response time assumed while the window has none, long enough that a server with no history is not shed.
`beta`	`Double`	Weight of the old value in the moving average of calls in flight. Updated only as calls finish, so it lags a surge and admits a little more of it.
`cool`	`@moondate.Span`	How long the server stays hot after the CPU falls back under the threshold, if it has been shedding.
`floor`	`Double`	The least share of the estimated capacity admitted however high the CPU is.

pub fn Shed::new(size~ : Int32 = shed.size, window~ : @moondate.Span = shed.window, cpu~ : Int32 = shed.cpu, latency~ : @moondate.Span = shed.latency, beta~ : Double = shed.beta, cool~ : @moondate.Span = shed.cool, floor~ : Double = shed.floor) -> Shed

A policy by name, every knob with the preset's value.

parameter	type	default	what it is
`size`~	`Int32`	`shed.size`	buckets in the window, `50` in the preset
`window`~	`@moondate.Span`	`shed.window`	how far back that window reaches, `5 s` in the preset
`cpu`~	`Int32`	`shed.cpu`	CPU use in thousandths at or above which the server counts as hot, `900` in the preset
`latency`~	`@moondate.Span`	`shed.latency`	response time assumed while the window has none, `1 s` in the preset
`beta`~	`Double`	`shed.beta`	weight of the old value in the moving average of calls in flight, `0.9` in the preset
`cool`~	`@moondate.Span`	`shed.cool`	how long the server stays hot after the CPU falls back, `1 s` in the preset
`floor`~	`Double`	`shed.floor`	the least share of capacity admitted however high the CPU is, `0.1` in the preset

Shedder

struct

Load shedding that refuses work only when the server is both hot and full.

Hot means the CPU is at or over the threshold, or was within the cooling time and the shedder has refused since. Full means more calls are in flight than the server has shown it can finish: the busiest bucket's passes per millisecond times the fastest bucket's mean response time, scaled down as the CPU nears 100%. Both the moving average and the current count of calls in flight must be over that figure.

A CPU spike with nothing queued sheds nothing, and neither does a queue on an idle CPU. CPU use is read by the caller, in thousandths, and passed to each Shedder::allow.

Held by the caller. Its fields are private; the methods below are the way in.

pub fn Shedder::allow(self : Shedder, now : Int64, cpu : Int32) -> Ticket?

Ask to start one call at now, with the CPU at cpu thousandths.

Some means go ahead, and the caller settles the ticket once the call has finished.

parameter	type	default	what it is
`now`	`Int64`	—	the caller's clock reading, in nanoseconds
`cpu`	`Int32`	—	CPU use in thousandths, read by the caller

pub fn Shedder::flying(self : Shedder) -> Int64

How many calls are in flight.

pub fn Shedder::new(shed~ : Shed = shed, now~ : Int64 = 0L) -> Shedder

A shedder with empty windows opening at now.

parameter	type	default	what it is
`shed`~	`Shed`	`shed`	the policy; the `shed`(#shed) preset by default
`now`~	`Int64`	`0L`	the clock reading both windows open at, `0L` by default

Ticket

struct

A call the shedder let through. Exactly one of pass and fail settles it.

Held by the caller. Its fields are private; the methods below are the way in.

pub fn Ticket::fail(self : Ticket) -> Unit

The call failed. It leaves flight but says nothing about capacity.

pub fn Ticket::pass(self : Ticket, now : Int64) -> Unit

The call succeeded at now. Its response time counts towards what the server has shown it can do.

parameter	type	default	what it is
`now`	`Int64`	—	the clock reading the call finished at; its response time counts towards capacity

03Rate and quota

Bucket

struct

A token bucket: rate tokens accrue per second up to burst, and each request takes one.

It is the rate limiter that allows a burst after a quiet period, which is what a limit on a human-facing API wants — nobody should be refused for making two requests in the same second after an hour of making none.

Time is nanoseconds from the caller's clock, as everywhere in this package.

Held by the caller. Its fields are private; the methods below are the way in.

pub fn Bucket::new(rate : Double, burst~ : Double = 0.0, now~ : Int64 = 0L) -> Bucket

A bucket that starts full, which is what lets the first burst through.

rate is tokens per second and burst is the most it will ever hold. A burst smaller than one means nothing ever passes, so it is raised to one.

parameter	type	default	what it is
`rate`	`Double`	—	tokens accrued per second
`burst`~	`Double`	`0.0`	the most the bucket will ever hold; with none given, the rate, and never under one
`now`~	`Int64`	`0L`	the clock reading the bucket starts at, `0L` by default

pub fn Bucket::next(self : Bucket, now : Int64, n~ : Double = 1.0) -> @moondate.Span

How long until n tokens would be available, without taking them.

Zero when they are available now. This is what a caller sleeps for rather than spinning, and what a Retry-After header is computed from.

parameter	type	default	what it is
`now`	`Int64`	—	the caller's clock reading, in nanoseconds
`n`~	`Double`	`1.0`	how many tokens are wanted, one by default

pub fn Bucket::take(self : Bucket, now : Int64, n~ : Double = 1.0) -> Bool

Whether n tokens are available now, taking them if they are.

parameter	type	default	what it is
`now`	`Int64`	—	the caller's clock reading, in nanoseconds
`n`~	`Double`	`1.0`	how many tokens the call costs, one by default

pub fn Bucket::tokens(self : Bucket, now : Int64) -> Double

How many tokens are in the bucket, for a caller that reports its own headroom.

parameter	type	default	what it is
`now`	`Int64`	—	the caller's clock reading, in nanoseconds

Sliding

struct

A sliding window counter: at most quota events in any window long stretch.

Where a bucket smooths, a window counts. A quota of "a thousand a day" is a window, not a rate, and answering it with a bucket would let a client spend the whole day's allowance in the first minute.

The stamps of the events in the window are kept, so the count is exact rather than the approximation a two-bucket scheme gives.

Held by the caller. Its fields are private; the methods below are the way in.

pub fn Sliding::left(self : Sliding, now : Int64) -> Int32

How many events remain in the current window.

parameter	type	default	what it is
`now`	`Int64`	—	the caller's clock reading, in nanoseconds

pub fn Sliding::new(quota : Int32, window : @moondate.Span) -> Sliding

An empty window.

parameter	type	default	what it is
`quota`	`Int32`	—	events allowed in any `window`-long stretch
`window`	`@moondate.Span`	—	the length of that stretch

pub fn Sliding::next(self : Sliding, now : Int64) -> @moondate.Span

When the window will next have room, which is when its oldest event falls out.

parameter	type	default	what it is
`now`	`Int64`	—	the caller's clock reading, in nanoseconds

pub fn Sliding::take(self : Sliding, now : Int64) -> Bool

Whether an event is within the quota now, counting it if it is.

parameter	type	default	what it is
`now`	`Int64`	—	the caller's clock reading, in nanoseconds

Period

struct

A fixed window counter: at most quota events per period, the count starting again when a period ends.

Cheaper than Sliding, which keeps every stamp, and coarser: a client can spend one quota at the end of a period and another at the start of the next, so forty events can pass in a second that allows twenty. That is the trade a fixed window makes, and the reason it is the cheap one.

A period opens on the first event after the last one closed. With align it opens on a multiple of period on the caller's clock instead, so with a Unix clock a daily quota resets at midnight UTC rather than a day after first use.

Held by the caller. Its fields are private; the methods below are the way in.

pub fn Period::left(self : Period, now : Int64) -> Int32

How many events remain in the current period.

parameter	type	default	what it is
`now`	`Int64`	—	the caller's clock reading, in nanoseconds

pub fn Period::new(quota : Int32, period : @moondate.Span, align~ : Bool = false) -> Period

A counter no event has touched. A period shorter than a nanosecond is raised to one.

parameter	type	default	what it is
`quota`	`Int32`	—	events allowed per period
`period`	`@moondate.Span`	—	how long a period is; anything under a nanosecond is raised to one
`align`~	`Bool`	`false`	open periods on multiples of `period` on the caller's clock instead of a period after first use

pub fn Period::next(self : Period, now : Int64) -> @moondate.Span

How long until the current period ends. Zero when no period is open, because the next event would open one.

parameter	type	default	what it is
`now`	`Int64`	—	the caller's clock reading, in nanoseconds

pub fn Period::take(self : Period, now : Int64) -> Quota

Count one event at now and say where the quota stands.

parameter	type	default	what it is
`now`	`Int64`	—	the caller's clock reading, in nanoseconds

Quota

enum derives Debug, Eq

Where a Period stands after an event: under the quota, exactly on it, or past it.

The event that hits the quota is still admitted. It is told apart because it is the last one, which is when a caller would warn that the next will be refused.

Under
Hit
Over

04Concurrency

Gate

struct

A count with a ceiling: the concurrency limit a server puts on itself.

Unlike a rate, this does not depend on time at all — it is how many things are happening at once. A server past its limit answers rather than queues, which is what keeps a queue from becoming the outage.

Held by the caller. Its fields are private; the methods below are the way in.

pub fn Gate::enter(self : Gate) -> Bool

Whether there is room, taking it if there is.

pub fn Gate::held(self : Gate) -> Int32

How many are inside.

pub fn Gate::leave(self : Gate) -> Unit

Give the room back.

pub fn Gate::new(limit : Int32) -> Gate

A gate that nothing has entered.

parameter	type	default	what it is
`limit`	`Int32`	—	how many calls may be inside at once; zero or less means no ceiling

05Retry

Backoff

struct derives Debug, Eq

When to try again, and how long to wait first.

The preset is backoff. Build another with Backoff::new, or update one in place with { ..backoff, attempts: 5 }.

field	type	what it is
`base`	`@moondate.Span`
`factor`	`Double`
`cap`	`@moondate.Span`
`attempts`	`Int32`
`jitter`	`Jitter`

pub fn Backoff::ceiling(self : Backoff, retry : Int32) -> @moondate.Span

The undelayed ceiling before the retry-th retry, the first retry being 1: min(base × factor^(retry-1), cap).

This is what a jitter-free client sleeps and what every other strategy draws within, so it is public: a caller comparing this library against a policy written elsewhere compares this.

parameter	type	default	what it is
`retry`	`Int32`	—	which retry, the first being one; zero or less returns nothing

pub fn Backoff::delay(self : Backoff, retry : Int32, random~ : Double = 0.0, previous~ : @moondate.Span = @moondate.Span::new()

How long to wait before the retry-th retry.

random is a value in [0, 1) that the caller draws; passing the same one twice gives the same delay twice, which is what makes a retry sequence testable. previous is what the last delay turned out to be, which only Decorrelated reads — it grows from where it landed rather than from the attempt number.

parameter	type	default	what it is
`retry`	`Int32`	—	which retry this is, the first being one
`random`~	`Double`	`0.0`	a draw from `[0, 1)` that the strategy scales; the same draw gives the same wait
`previous`~	`@moondate.Span`	`@moondate.Span::new(`	what the last wait turned out to be; only `Decorrelated` reads it

pub fn Backoff::more(self : Backoff, made : Int32) -> Bool

Whether a call that has made made attempts may make another.

The first attempt is 1, so a policy of three attempts allows two retries.

parameter	type	default	what it is
`made`	`Int32`	—	attempts made so far, the first attempt being one

pub fn Backoff::new(base~ : @moondate.Span = backoff.base, factor~ : Double = backoff.factor, cap~ : @moondate.Span = backoff.cap, attempts~ : Int32 = backoff.attempts, jitter~ : Jitter = backoff.jitter) -> Backoff

A backoff by name, every knob with the preset's value.

parameter	type	default	what it is
`base`~	`@moondate.Span`	`backoff.base`	the wait before the first retry, `100 ms` in the preset
`factor`~	`Double`	`backoff.factor`	what each wait is multiplied by, `2.0` in the preset
`cap`~	`@moondate.Span`	`backoff.cap`	the ceiling that multiplication is held to, `1 s` in the preset
`attempts`~	`Int32`	`backoff.attempts`	calls allowed in total, the first one included; `3` in the preset
`jitter`~	`Jitter`	`backoff.jitter`	how much of that schedule is randomised, `Spread(0.2)` in the preset

pub fn Backoff::schedule(self : Backoff, randoms~ : ArrayView[Double] = [][:]) -> Array[@moondate.Span]

Every delay a run of retries would wait, for a caller that wants the whole sequence rather than one step — a test, or a log line explaining a plan.

randoms supplies the draws; when it runs short the remaining steps use zero, which is the low end of whatever strategy is in force.

parameter	type	default	what it is
`randoms`~	`ArrayView[Double]`	`[][:]`	the draws to use, one per retry; when it runs short the rest use zero

Jitter

enum derives Debug, Eq

How much randomness goes into a delay.

A retry with no jitter makes every client that failed together try again together, which is how one outage becomes a second one. Full, Equal and Decorrelated are the three shapes the literature names and measures; Spread keeps the wait within a fraction of the ceiling instead, which is the easier one to reason about when the schedule is fixed by something else:


`Rigid`	the computed delay, unchanged. Reproducible, and it synchronises clients
`Full`	uniform over `[0, cap]`. The one that spreads clients furthest
`Equal`	half the cap plus uniform over `[0, cap/2]`. Keeps a floor under the wait
`Decorrelated`	uniform over `[base, previous × 3]`. Grows from where it last landed rather than from the attempt number
`Spread(f)`	uniform over `[cap × (1 − f), cap × (1 + f)]`, `f` held to `[0, 1]`. So the wait can pass the cap by `f`

Rigid
Full
Equal
Decorrelated
Spread(Double)

06Time

Deadline

struct

A time budget measured from the instant it started.

A budget of zero or less never expires, which is how most servers spell "no timeout": a configuration that leaves the timeout unset should not time everything out at once.

Held by the caller. Its fields are private; the methods below are the way in.

pub fn Deadline::expired(self : Deadline, now : Int64) -> Bool

Whether the budget is spent at now.

parameter	type	default	what it is
`now`	`Int64`	—	the caller's clock reading, in nanoseconds

pub fn Deadline::remaining(self : Deadline, now : Int64) -> @moondate.Span?

What is left of the budget at now, never negative. None when there is no budget, which has no finite remainder.

parameter	type	default	what it is
`now`	`Int64`	—	the caller's clock reading, in nanoseconds

pub fn Deadline::start(budget : @moondate.Span, now : Int64) -> Deadline

A deadline of budget starting at now.

parameter	type	default	what it is
`budget`	`@moondate.Span`	—	how long the deadline lasts; zero or less never expires
`now`	`Int64`	—	the clock reading it starts at

07Windows

Window

struct

A rolling window: size buckets that together cover the last window of time, oldest dropping off as time moves on.

The bucket type is the caller's. A breaker keeps a Tally of outcomes per bucket, a shedder a Total of response times; anything with a Default can be a bucket, and the default is what a bucket is cleared back to.

Time never moves on its own: every call carries the now it happens at. Writing at a later now clears the buckets the gap swept past; reading at a later now skips them without clearing, so a window nobody writes to still ages out, and two reads at the same now agree.

Held by the caller. Its fields are private; the methods below are the way in.

pub fn[B] Window::each(self : Window[B], now : Int64, f : (B) -> Unit, current~ : Bool = true) -> Unit

Visit the buckets still inside the window at now, oldest first.

With current false the bucket now falls in is left out: it has seen only part of its time, and a reading that takes the busiest or the fastest bucket should not compare a partial one against whole ones.

parameter	type	default	what it is
`now`	`Int64`	—	the caller's clock reading, in nanoseconds
`f`	`(B) -> Unit`	—	called once per live bucket, oldest first
`current`~	`Bool`	`true`	whether to include the bucket `now` falls in, `true` by default

pub fn[B : Default] Window::new(size~ : Int32, window~ : @moondate.Span, now~ : Int64 = 0L) -> Window[B]

An empty window of size buckets spanning window, its first bucket opening at now.

A size below one is raised to one, and so is a bucket narrower than a nanosecond.

parameter	type	default	what it is
`size`~	`Int32`	—	how many buckets; anything under one is raised to one
`window`~	`@moondate.Span`	—	how much time the buckets together cover
`now`~	`Int64`	`0L`	the clock reading the first bucket opens at, `0L` by default

pub fn[B] Window::size(self : Window[B]) -> Int32

How many buckets the window has.

pub fn[B : Default] Window::slot(self : Window[B], now : Int64) -> B

The bucket now falls in, for the caller to add to.

The buckets the gap since the last write swept past are cleared first. The head is snapped back to a bucket boundary, so buckets keep their width rather than drifting with the arrival time of each write.

parameter	type	default	what it is
`now`	`Int64`	—	the caller's clock reading, in nanoseconds

pub fn[B] Window::width(self : Window[B]) -> @moondate.Span

How much time one bucket covers.

Outcome

enum derives Debug, Eq

How a call ended, as a breaker records it.

A Drop is a call refused before it ran: traffic, but neither a success nor a failure of whatever is downstream.

Success
Failure
Drop

Tally

struct derives Debug, Default, Eq

Outcomes counted in one bucket.

total counts every call including the dropped ones, so a bucket being throttled still reads as busy rather than idle.

Held by the caller. Its fields are private; the methods below are the way in.

pub fn Tally::add(self : Tally, o : Outcome) -> Unit

Count one call ending as o.

parameter	type	default	what it is
`o`	`Outcome`	—	how the call ended

Total

struct derives Debug, Default, Eq

A sum and how many values went into it, which is a mean once count is positive.

Held by the caller. Its fields are private; the methods below are the way in.

pub fn Total::add(self : Total, v : Int64) -> Unit

Add one value.

parameter	type	default	what it is
`v`	`Int64`	—	the value to fold into the sum

08Presets

sre

preset let sre : Sre

The preset: k 1.5 decaying to 1.1, five calls of protection, a probe every second, and ten seconds of history in forty buckets.

k and protection are the slope and the floor of the standard client-side throttling formula; the decay, the probe and the recovery scaling are what keep a long outage from hurting past successes, and a short one from being remembered for long.

shed

preset let shed : Shed

The preset: five seconds in fifty buckets, overloaded at 90% CPU, a one-second assumed response time, an average weighted 0.9, one second to cool and never less than a tenth of capacity.

The memory is short on purpose: a five-second window follows the load it is in rather than averaging a burst away, and the floor keeps a server already at full CPU from being told it can serve nobody.

backoff

preset let backoff : Backoff

The preset: three attempts, a hundred milliseconds doubling to a one-second ceiling, spread by a fifth either way.

Each wait is the computed ceiling spread by a fifth below or above it, so a plan of 0.8 to 1.2 of the schedule is easy to explain to whoever asks why the second retry waited 168 ms and not 200.