moonsd service discovery

Backend · etcd

moonetcd

A complete etcd client, and moonetcd/sd: the contract on keys attached to leases. The key goes when the lease does, and the revision etcd stamps on every write is where a watcher picks up.

key/moonsd/greeter/10.0.0.11:12000
value{"host":"10.0.0.11","port":12000}
registergrant(ttl), then put the key on that lease
renewone keep_alive; a reply with no time left means gone
watchresumes from the revision it last saw
lapsethe lease is revoked or lost, and the key goes with it
i

What it is

A key that cannot outlive its lease

etcd already has the two mechanisms a registry needs: keys that vanish with a lease, and a monotonically increasing revision on every write. The registry puts /moonsd/{service}/{id} on a granted lease and lets etcd do the forgetting. A service name is escaped on the way into the key, so a prefix read of service a cannot pick up service a/b.

ii

The lease

Seconds on the wire, a token that is the lease

01

register

lease().grant(seconds) then kv().put(key, value, lease=id). A lease lasts whole seconds and the server enforces a minimum, so a millisecond ttl is rounded up rather than down — rounding down would end the lease early.

02

renew

One keep_alive round. etcd answers an acknowledgement with no time left rather than an error when the lease is gone, and that is what becomes Refused::Expired.

03

deregister

Revoking the lease takes every key on it with it, which is why the token can be a string in the contract and still be enough to clean up.

iii

The watcher

Resume from a revision, recover once

The first snapshot is a range read; each later one is a watch from the last revision seen. A connection that drops, or a history that has been compacted past the revision, is answered by reading the whole service once and watching from there — the caller sees another snapshot and nothing else, which is the promise the watcher made.

prefix /moonsd ttl rounded up to seconds token the lease id verified against etcd 3.5, three nodes, members killed
iv

The client behind it

Both transports, both directions

sd sits on a client that speaks etcd's gRPC service and its JSON gateway, over TLS with or without a client certificate. KV, watch, lease, cluster, auth, maintenance, lock and election are all there, along with the concurrency helpers built on them, because a service that elects a leader already has the client.