jepsen.control.retry

SSH client libraries appear to be near universally-flaky. Maybe race conditions, maybe underlying network instability, maybe we’re just doing it wrong. For whatever reason, they tend to throw errors constantly. The good news is we can almost always retry their commands safely! This namespace provides a Remote which wraps an underlying Remote in a jepsen.reconnect wrapper, catching certain exception classes and ensuring they’re automatically retried.

backoff-time

Roughly how long should we back off when retrying, in ms?

remote

(remote remote)

Constructs a new Remote by wrapping another Remote in one which automatically catches and retries any exception of the form {:type :jepsen.control/ssh-failed}.

retries

How many times should we retry exceptions before giving up and throwing?

with-retry

macro

(with-retry & body)

Takes a body. Evaluates body, retrying SSH exceptions.