Breaker
Refuses a growing fraction of calls as a backend fails, and eases off as it recovers. Keeps a rolling window of outcomes; never opens, never closes.
MoonBit · resilience primitives
Keep a failing dependency from taking the rest of the system down with it. Nine primitives that decide from the numbers you hand them, so a ten-second outage replays in a loop that takes no time at all.
moon add moonbitstack/moonward@0.2.1
Breaker, shedder, bucket, sliding window, period, gate, backoff, deadline, rolling window.
Every call takes now in nanoseconds from your clock. Nothing here samples one.
A breaker's roll and a backoff's jitter ask you for a value in [0, 1).
Forty buckets of outcomes. A backend that recovers is noticed within one of them.
Four places a request can be stopped, cheapest first. Each check is a struct you hold in your own code and a call that returns Option: a call that was never going to succeed is turned away before it costs a socket.
A count with a ceiling. It has nothing to do with time: it is how many calls are inside at this instant. Past the limit the answer is no, not later.
Refuses when held has reached limit
let gate = @moonward.Gate::new(64)
if gate.enter() {
// the call runs, then
gate.leave()
}
Not a switch. The breaker keeps ten seconds of outcomes and refuses each call with a probability that grows with the failures in that window, so a struggling backend keeps getting what it can still serve.
Refuses when a draw falls under ratio(now)
let b = @moonward.Breaker::new()
match b.allow(now, roll) {
Some(p) => p.pass(now) // or p.fail(now)
None => refused
}
Three answers to three different questions. A bucket smooths a rate and lets a burst through. A sliding window counts events in any window-long stretch. A period counts them per fixed period, which is cheaper and coarser.
Refuses when the token, the count or the quota is spent
let rate = @moonward.Bucket::new(100.0) let quota = @moonward.Sliding::new(1000, day) let daily = @moonward.Period::new(1000, day, align=true) rate.take(now) // true while tokens last
Both, or nothing is refused. A CPU spike with nothing queued sheds nothing, and neither does a long queue on an idle CPU. The threshold is what the server has shown it can finish.
Refuses when cpu is over the threshold and calls in flight are over capacity
let s = @moonward.Shedder::new()
match s.allow(now, cpu) {
Some(t) => t.pass(now) // or t.fail()
None => "overloaded"
}
Nine types, one policy record each where a policy makes sense. Every preset can be copied and changed with two braces: { ..sre, k: 2.0 }.
Refuses a growing fraction of calls as a backend fails, and eases off as it recovers. Keeps a rolling window of outcomes; never opens, never closes.
Refuses work only when the CPU is hot and the calls in flight are over what this server has shown it can finish.
A token bucket. Accrues at a rate, holds up to a burst, spends one token a call.
At most a quota of events in any window-long stretch. Keeps each stamp, so the count is exact.
A quota per fixed period, reset at a multiple of the period when aligned. Cheaper than a slide, coarser.
A concurrency limit. Counts what is inside now, and does not know what time it is.
When to try again and how long to wait first. Five jitter strategies, each drawing from the value you pass.
A budget measured from the instant it started. A budget of zero never expires, so an unset timeout cannot time everything out at once.
A rolling window of buckets you define. Anything with a Default can be a bucket, and the default is what a bucket is cleared back to.
Three runs from the example program, unchanged. Nothing is rounded for the camera: these are the lines it writes.
let b = @moonward.Breaker::new()
for i in 0L..<500L {
let now = i * 20L * 1_000_000L
match b.allow(now, roll(i)) {
Some(p) if down(now) => p.fail(now)
Some(p) => p.pass(now)
None => refused += 1
}
}
3500 ms sent 23 refused 2 ratio 15% 4000 ms sent 19 refused 6 ratio 26% 4500 ms sent 18 refused 7 ratio 35% 5000 ms sent 15 refused 10 ratio 43% 5500 ms sent 13 refused 12 ratio 49% 6000 ms sent 12 refused 13 ratio 53% 6500 ms sent 14 refused 11 ratio 43% 7000 ms sent 15 refused 10 ratio 39%
let bucket = @moonward.Bucket::new(2.0, burst=3.0) let burst = [0, 1, 2, 3] .map(_ => bucket.take(0L)) let quota = @moonward.Period::new(2, day, align=true) [0, 1, 2].map(_ => quota.take(0L))
bucket [true, true, true, false], next in 0.5 s period [Under, Hit, Over] gate true false backoff [0.1, 0.2] s deadline false true
let s = @moonward.Shedder::new()
// one call every millisecond, in flight
// for 50 ms, then settled
match s.allow(now, cpu) {
Some(t) =>
open.push((t, now + 50L * 1_000_000L))
None => refused += 1
}
in flight 26 refused 960 of 3000
No clock read.
No dice thrown.
A breaker that reads the clock itself can only be tested by waiting, and one that draws its own numbers cannot be tested at all. Here the time is a number you pass and the roll is a number you draw, so an hour of traffic is a loop over now.