cache

structA cache in front of a store: take answers from the store when the value is there, and otherwise runs the loader once — not once per waiting caller — and writes the answer back with a lifetime that is jittered so a fleet warmed at the same moment does not expire together. A value is stored as bytes: the cache is over the codec, so it works for a record read out of SQL and for a counter read out of redis alike.

itemA cache over store, reading and writing V through encode and decode, and keeping a loader's answer for ttl_ms milliseconds. jitter spreads an expiry over a window; the default answers a value between a tenth below and a tenth above the lifetime, which is what keeps a fleet from stampeding the database one second at a time.

itemThe configured lifetime.

itemThe value under key, loading and caching it when the store does not have it. One loader runs per key however many callers arrive while it is working, and all of them receive its answer. None from the loader is not cached: an absent row may appear a moment later, and caching the absence is how a cache turns a transient gap into a minute of wrong answers.

itemDrop key from the store, so the next take loads it again.

itemHow many loads are in flight, which is also how many keys are being shared.

itemA store failure: a backend that is down, a value that will not decode.

itemWhere cached values live: a key, the value's bytes, and how long the value stays valid. Values cross this seam as bytes so one store serves any type — the codec belongs to the cache above it, not to the store. Everything a cache does (the single flight, the jitter, the ordering of reads and writes) sits above this trait, so a process-local map and a remote redis are interchangeable behind it.

structA store in this process, over the bounded LRU cache: an entry lives until its lifetime is up or the cache needs its room.

fnA memory store holding at most limit entries.

item

item

item

item