Guide

Six answers, one caller

The pool is a bookkeeper. It says what to do with a resource; you open it, close it, and wait. These chapters walk the whole surface in that order, and every output below was printed by the code above it.

01The first take

Add the module and import it. The date package is where a span comes from, and the ward chapter adds one more.

moon add moonbitstack/moonpool

import {
  "moonbitstack/moonpool",
  "moonbitstack/moondate",
}

A pool of Int32, with the preset limits. The helper below turns an answer into a line, and stays the same through the guide.

/// One answer, as the text this guide prints.
fn describe(t : @moonpool.Taken[Int32]) -> String {
  match t {
    Ready(id) => "Ready(\{id})"
    Stale(id) => "Stale(\{id})"
    Make => "Make"
    Wait => "Wait"
    Later(span) => "Later(\{span.nanos / 1_000_000} ms)"
    Shed => "Shed"
  }
}

let pool : @moonpool.Pool[Int32] = @moonpool.Pool::new()
println(pool.live())
println(describe(pool.take(0L)))
pool.made(1, 0L)
println(describe(pool.take(0L)))
pool.made(2, 0L)
println(pool.live())
println(pool.give(1, 0L))
println(pool.idle())
println(describe(pool.take(1L)))

An empty pool has nothing to lend, so the first two takes both answer Make: the preset allows ten live, and only two have been made. made registers the one you opened as lent, which is why the pool counts two live and nothing idle afterwards.

give puts one back and answers true — it kept it. The next take hands it out as Ready(1): the resource returned most recently comes back first.

02Limits, and what happens at the ceiling

Four numbers. size is how many may be live at once, counting the ones being made; when the pool is full it answers Wait rather than borrowing against the future.

let small : @moonpool.Pool[Int32] = @moonpool.Pool::new(
  limits=@moonpool.Limits::new(size=2),
)
ignore(small.take(0L))
small.made(1, 0L)
ignore(small.take(0L))
small.made(2, 0L)
println(small.live())
println(small.busy())
println(describe(small.take(0L)))

Wait is not a queue. The pool has no runtime, so it cannot park a caller; it says that this one cannot be served now and leaves the decision above it — retry, time out, or try another pool.

idle is the other ceiling: how many may sit unused. A pool that grew to twenty under load and may keep ten idle will close the rest as they are returned, which is what give answering false means.

03Retiring: idle age and lifetime

keep retires a resource that has sat unused; life retires one that has lived too long, however busy it has been. Both are spans, and both are measured against the clock you pass in.

let p : @moonpool.Pool[Int32] = @moonpool.Pool::new(
  limits=@moonpool.Limits::new(size=4, keep=@moondate.Span::new(seconds=1)),
)
ignore(p.take(0L))
p.made(7, 0L)
ignore(p.give(7, 0L))
println(p.evict(500_000_000L).length())
println(p.evict(1_500_000_000L).length())
println(p.idle())

evict is the timer's job: it takes everything older than the limit and hands it back for you to close. At half a second nothing is due; a second later the one idle resource is.

A timer can be late, so take checks the age itself. An idle resource past its limit comes back as Stale instead of being skipped — you close it and ask again, and nothing is stranded in the pool waiting for a timer that never ran.

ignore(p.take(0L))
p.made(8, 0L)
ignore(p.give(8, 0L))
println(describe(p.take(2_000_000_000L)))

Lifetime is the same test against a different stamp. A resource lent and returned repeatedly keeps the moment it was made, so life still retires it: the pool recognises a returned resource by identity, not by its arrival at the door.

let aged : @moonpool.Pool[Int32] = @moonpool.Pool::new(
  limits=@moonpool.Limits::new(size=2, life=@moondate.Span::new(seconds=4)),
)
ignore(aged.take(0L))
aged.made(5, 0L)
ignore(aged.give(5, 3_000_000_000L))
println(aged.idle())
println(describe(aged.take(5_000_000_000L)))

It was returned a moment ago and the pool is holding it, but it was made five seconds ago and the limit is four. The answer is Stale, not Ready.

04A ward in front

The ward is what stands between the pool and a backend that is failing. Every part is optional; here it is a breaker, which starts with nothing against the backend and learns from the makes that fail.

let guarded : @moonpool.Pool[Int32] = @moonpool.Pool::new(
  limits=@moonpool.Limits::new(size=4),
  ward=@moonpool.Ward::new(breaker=@moonward.Breaker::new()),
)
println(guarded.refusal(0L))
for i in 0..<8 {
  match guarded.take(i.to_int64() * 100_000_000L, roll=0.99) {
    Make => guarded.unmade(i.to_int64() * 100_000_000L, roll=0.5)
    _ => ()
  }
}
println(guarded.refusal(800_000_000L))
println(describe(guarded.take(900_000_000L, roll=0.1)))
println(describe(guarded.take(900_000_000L, roll=0.99)))

Eight makes, all failed, and the breaker now refuses a third of what it is asked. The refusal is a rate, and roll is the draw that decides this request: at 0.1 it is inside the refused share and the answer is Shed; at 0.99 it is outside and a make is allowed. The same two numbers always give the same answer, which is what makes this testable.

made and unmade are what keep that rate honest. A make that succeeded tells the breaker the backend is alive; one that failed counts against it.

A backoff part holds makes back after a failure instead of refusing them. It answers Later with the span still to wait:

let held : @moonpool.Pool[Int32] = @moonpool.Pool::new(
  limits=@moonpool.Limits::new(size=4),
  ward=@moonpool.Ward::new(backoff=@moonward.backoff),
)
ignore(held.take(0L))
held.unmade(0L, roll=0.5)
println(describe(held.take(1_000_000L)))
println(describe(held.take(5_000_000_000L)))

One failure is enough to hold the next make for forty-nine milliseconds; the delay grows with each failure in a row. Five seconds later the hold has passed and the pool is willing to try again.

05Picking between pools

When one service is reached through several pools, pick chooses among them: two are drawn, and of those the one whose breaker refuses less wins, then the one with fewer in use.

let a : @moonpool.Pool[Int32] = @moonpool.Pool::new(limits=@moonpool.Limits::new(size=4))
let b : @moonpool.Pool[Int32] = @moonpool.Pool::new(limits=@moonpool.Limits::new(size=4))
ignore(a.take(0L))
a.made(1, 0L)
println(@moonpool.pick([a, b][:], 0L, first=0.0, second=0.9))
println(@moonpool.pick([][:], 0L, first=0.0, second=0.0))

The two draws land on slot 0 and slot 1; the second pool has nothing in use, so it is the one to ask. No pools at all answers -1.

06A client loop

All six answers in one place. This is the whole of what a caller has to write; the opening, the closing and the queueing at Wait belong to whatever runtime is underneath.

type Conn = Int32

fn use(_c : Conn) -> Unit { () }
fn close(_c : Conn) -> Unit { () }
fn open() -> Conn? { Some(1) }
fn queue(_now : Int64) -> Unit { () }
fn later(_span : @moondate.Span) -> Unit { () }
fn fail_fast() -> Unit { () }

fn serve(pool : @moonpool.Pool[Conn], now : Int64, roll : Double) -> Unit {
  let conn = match pool.take(now, roll=roll) {
    Ready(conn) => Some(conn)
    Stale(conn) => {
      close(conn)
      None
    }
    Make =>
      match open() {
        Some(conn) => {
          pool.made(conn, now)
          Some(conn)
        }
        None => {
          pool.unmade(now, roll=roll)
          None
        }
      }
    Wait => {
      queue(now)
      None
    }
    Later(span) => {
      later(span)
      None
    }
    Shed => {
      fail_fast()
      None
    }
  }
  if conn is Some(c) {
    use(c)
    if !pool.give(c, now) {
      close(c)
    }
  }
}

Three things in there are easy to get wrong, and the pool is shaped around them. give answering false means the pool already had enough idle and did not keep this one: close it. Stale is a resource you now own and the pool no longer counts: close it, and take again. And a resource that breaks goes to drop, not give — a pool that kept a broken connection would hand it to the next caller.