moonzero · control plane v0.15.0 · 49 packages · 4 targets · Apache-2.0 docs / resilience / limiting GitHub

05 / RESILIENCE

limiting

The concurrency limiter that sheds load past a ceiling — releasing its permit with defer, so cancellation cannot wedge it shut — and the period limiter that counts requests per window.

struct MaxConns

structA permit pool of max concurrent slots, moonward's Gate under names. A max of zero or less admits everything, as server does when MaxConns is unset.

fn MaxConns::new(max : Int32) -> MaxConns

fnA pool admitting at most max requests at once.

fn MaxConns::try_acquire(self : MaxConns) -> Bool

fnTake a permit if one is free (TryBorrow), returning whether it was taken.

fn MaxConns::release(self : MaxConns) -> Unit

fnReturn a permit taken by try_acquire (Return).

fn MaxConns::in_flight(self : MaxConns) -> Int32

fnThe number of requests currently holding a permit.

enum PeriodResult

enum

struct PeriodLimit

structA fixed-window rate limiter kept in this process: each key may make up to quota requests per period, counted by one moonward Period per key. A key's window opens on its first request, or with align on a multiple of the period), and starts again once the period has passed. The window is local, so N replicas admit N quotas. RedisPeriodLimit` is the same limiter with its window in redis and is what a fleet should run; this one is for a single process, and for the fallback a caller wants when redis is unreachable.

fn PeriodLimit::new( period_secs~ : Int32, quota~ : Int32, align~ : Bool = false) -> PeriodLimit

fnA limiter admitting quota requests per period_secs seconds, per key.

fn PeriodLimit::take( self : PeriodLimit, key : String, now : Int64) -> PeriodResult

fnAccount for one request under key at now, in milliseconds, and report its outcome: PeriodAllowed below quota, PeriodHitQuota at exactly the quota (the last admitted request), and PeriodOverQuota beyond it.

let period_script : String =

letcore/limit/periodscript.lua, verbatim: INCRBY the key, hang the window on it the first time it appears, and answer 1 below quota, 2 exactly at it, 0 past it. Public so a caller can pre-load it, and so a redis double can evaluate the very text the limiter ships.

struct RedisPeriodLimit

structA fixed-window rate limiter whose window lives in redis. One INCRBY-ed counter per key, given a period-second expiry the first time it appears, so every replica pointed at the same redis and key draws on one quota instead of each getting its own.

fn RedisPeriodLimit::new( client : @moonredis.Client, period_secs~ : Int32, quota~ : Int32, prefix~ : String = "") -> RedisPeriodLimit

fnA limiter admitting quota requests per period_secs seconds per key, counting in client's redis under prefix-prefixed keys.

async fn RedisPeriodLimit::take( self : RedisPeriodLimit, key : String) -> PeriodResult raise @moonredis.RedisError

itemAccount for one request under key and report its outcome, in the same three states the local limiter reports. A code the script cannot have returned, or a reply that is not an integer at all, raises — answers Unknown with ErrUnknownCode there, and a caller has to decide what an undecided limiter means.

extend PeriodResult with Debug::

item

extend PeriodResult with Eq::

item

fn max_conns( limit : @limit.MaxConns, refusal~ : Refusal = unavailable) -> Middleware

fnMax-connections middleware: hold a permit for the wrapped app's duration, or answer 503 Service Unavailable when every permit is taken. Non-HTTP scopes (lifespan, websocket) pass through untouched.

fn rate_limit( bucket : @limit.TokenBucket, clock : @clock.Clock, refusal~ : Refusal = too_many) -> Middleware

fnRate-limit middleware over the process-local bucket: admit each HTTP request against a shared TokenBucket read at clock.now(), answering 429 Too Many Requests when the bucket is empty and otherwise delegating to the wrapped app. The bucket is captured once per assembly, so its state is shared across every request this layer serves — but only within this process; use redis_rate_limit to share it across replicas. Non-HTTP scopes (lifespan, websocket) pass through untouched.

fn redis_rate_limit( limiter : @limit.RedisTokenLimit, clock : @clock.Clock, refusal~ : Refusal = too_many) -> Middleware

fnRate-limit middleware over a shared redis: rate_limit with the bucket in redis, so replicas behind one redis spend from one bucket. A redis that cannot be reached leaves the limiter running on its local bucket, so the layer keeps limiting either way. Non-HTTP scopes pass through untouched.

fn period_limit( limiter : @limit.PeriodLimit, clock : @clock.Clock, key~ : String = "global", refusal~ : Refusal = too_many) -> Middleware

fnFixed-window rate-limit middleware over the process-local limiter: account for each HTTP request under key and answer 429 Too Many Requests once the key is over quota for the current window, otherwise delegating to the wrapped app. The limiter is captured once per assembly, so its windows are shared across every request this layer serves — but only within this process; use redis_period_limit to share them across replicas. Non-HTTP scopes pass through untouched.

fn redis_period_limit( limiter : @limit.RedisPeriodLimit, key~ : String = "global", refusal~ : Refusal = too_many) -> Middleware

fnFixed-window rate-limit middleware over a shared redis: period_limit with the window in redis, so replicas behind one redis spend one quota between them. A redis that cannot be reached admits the request — the limiter is a guard on the service, not a dependency that should be able to close it.