jepsen.util

Kitchen sink

*relative-time-origin*

dynamic

A reference point for measuring time in a test run.

all-jdk-loggers

(all-jdk-loggers)

arities

(arities c)

The arities of a function class.

await-fn

(await-fn f)(await-fn f opts)

Invokes a function (f) repeatedly. Blocks until (f) returns, rather than throwing. Returns that return value. Catches Exceptions (except for InterruptedException) and retries them automatically. Options:

:retry-interval How long between retries, in ms. Default 1s. :log-interval How long between logging that we’re still waiting, in ms. Default `retry-interval. :log-message What should we log to the console while waiting? :timeout How long until giving up and throwing :type :timeout, in ms. Default 60 seconds.

buf-size

chunk-vec

(chunk-vec n v)

Partitions a vector into reducibles of size n (somewhat like partition-all) but uses subvec for speed.

(chunk-vec 2 [1])     ; => ([1])
(chunk-vec 2 [1 2 3]) ; => ([1 2] [3])

coll

(coll thing-or-things)

Wraps non-collection things into singleton lists, and leaves colls as themselves. Useful when you can take either a single thing or a sequence of things.

compare<

(compare< a b)

Like <, but works on any comparable objects, not just numbers.

concat-files!

(concat-files! out fs)

Appends contents of all fs, writing to out. Returns fs.

contains-many?

(contains-many? m & ks)

Takes a map and any number of keys, returning true if all of the keys are present. Ex. (contains-many? {:a 1 :b 2 :c 3} :a :b :c) => true

deepfind

(deepfind pred haystack)(deepfind pred path haystack)

Finds things that match a predicate in a nested structure. Returns a lazy sequence of matching things, each represented by a vector path which denotes how to access that object, ending in the matching thing itself. Path elements are:

  • keys for maps
  • integers for sequentials
  • :member for sets
  • :deref for deref-ables.

(deepfind string? :a {:b “foo”} :c) ; => (1 :b “foo”)

default

(default m k v)

Like assoc, but only fills in values which are NOT present in the map.

drop-common-proper-prefix

(drop-common-proper-prefix cs)

Given a collection of sequences, removes the longest common proper prefix from each one.

ex-root-cause

(ex-root-cause t)

Unwraps throwables to return their original cause.

exception?

(exception? x)

Is x an Exception?

fast-last

(fast-last coll)

Like last, but O(1) on counted collections.

fcatch

(fcatch f)

Takes a function and returns a version of it which returns, rather than throws, exceptions.

fixed-point

(fixed-point f x)

Applies f repeatedly to x until it converges.

forget!

(forget! this)

Allows this forgettable reference to be reclaimed by the GC at some later time. Future attempts to dereference it may throw. Returns self.

forgettable

(forgettable x)

Constructs a deref-able reference to x which can be explicitly forgotten. Helpful for controlling access to infinite seqs (e.g. the generator) when you don’t have firm control over everyone who might see them.

fraction

(fraction a b)

a/b, but if b is zero, returns unity.

get-named-lock!

(get-named-lock! locks name)

Given a pool of locks, and a lock name, returns the object used for locking in that pool. Creates the lock if it does not already exist.

history->latencies

(history->latencies history)

Takes a history–a sequence of operations–and returns a new history where operations have two new keys:

:latency the time in nanoseconds it took for the operation to complete. :completion the next event for that process

inc*

(inc* x)

Like inc, but (inc nil) => 1.

integer-interval-set-str

(integer-interval-set-str set)

Takes a set of integers and yields a sorted, compact string representation.

lazy-atom

(lazy-atom f)

An atom with lazy state initialization. Calls (f) on first use to provide the initial value of the atom. Only supports swap/reset/deref. Reset bypasses lazy initialization. If f throws, behavior is undefined (read: proper fucked).

letr

macro

(letr bindings & body)

Let bindings, plus early return.

You want to do some complicated, multi-stage operation assigning lots of variables–but at different points in the let binding, you need to perform some conditional check to make sure you can proceed to the next step. Ordinarily, you’d intersperse let and if statements, like so:

(let [res (network-call)]
  (if-not (:ok? res)
    :failed-network-call

    (let [people (:people (:body res))]
      (if (zero? (count people))
        :no-people

        (let [res2 (network-call-2 people)]
          ...

This is a linear chain of operations, but we’re forced to nest deeply because we have no early-return construct. In ruby, we might write

res = network_call
return :failed_network_call if not x.ok?

people = res[:body][:people]
return :no-people if people.empty?

res2 = network_call_2 people
...

which reads the same, but requires no nesting thanks to Ruby’s early return. Clojure’s single-return is usually a boon to understandability, but deep linear branching usually means something like

  • Deep nesting (readability issues)
  • Function chaining (lots of arguments for bound variables)
  • Throw/catch (awkward exception wrappers)
  • Monadic interpreter (slow, indirect)

This macro lets you write:

(letr [res    (network-call)
       _      (when-not (:ok? res) (return :failed-network-call))
       people (:people (:body res))
       _      (when (zero? (count people)) (return :no-people))
       res2   (network-call-2 people)]
  ...)

letr works like let, but if (return x) is ever returned from a binding, letr returns x, and does not evaluate subsequent expressions.

If something other than (return x) is returned from evaluating a binding, letr binds the corresponding variable as normal. Here, we use _ to indicate that we’re not using the results of (when …), but this is not mandatory. You cannot use a destructuring bind for a return expression.

letr is not a true early return–(return x) must be a terminal expression for it to work–like (recur). For example,

(letr [x (do (return 2) 1)]
  x)

returns 1, not 2, because (return 2) was not the terminal expression.

return only works within letr’s bindings, not its body.

letr-let-if

(letr-let-if groups body)

Takes a sequence of binding groups and a body expression, and emits a let for the first group, an if statement checking for a return, and recurses; ending with body.

letr-partition-bindings

(letr-partition-bindings bindings)

Takes a vector of bindings sym expr, sym’ expr, …. Returns binding-groups: a sequence of vectors of bindgs, where the final binding in each group has an early return. The final group (possibly empty!) contains no early return.

letr-rewrite-return

(letr-rewrite-return expr)

Rewrites (return x) to (Return. x) in expr. Returns a pair of changed? expr, where changed is whether the expression contained a return.

linear-time-nanos

(linear-time-nanos)

A linear time source in nanoseconds.

local-time

(local-time)

Local time.

log

(log & things)

log-op

(log-op op)

Logs an operation and returns it.

log-print

(log-print _ & things)

logger

longest-common-prefix

(longest-common-prefix cs)

Given a collection of sequences, finds the longest sequence which is a prefix of every sequence given.

majority

(majority n)

Given a number, returns the smallest integer strictly greater than half.

map-keys

(map-keys f m)

Maps keys in a map.

map-kv

(map-kv f m)

Takes a function (f k v) which returns k v, and builds a new map by applying f to every pair.

map-vals

(map-vals f m)

Maps values in a map.

max-by

(max-by f coll)

Finds the maximum element of a collection based on some (f element), which returns Comparables. If coll is empty, returns nil.

maybe-number

(maybe-number s)

Tries reading a string as a long, then double, then string. Passes through nil. Useful for getting nice values out of stats APIs that just dump a bunch of heterogenously-typed strings at you.

meh

macro

(meh & body)

Returns, rather than throws, exceptions.

min-by

(min-by f coll)

Finds the minimum element of a collection based on some (f element), which returns Comparables. If coll is empty, returns nil.

minority-third

(minority-third n)

Given a number, returns the largest integer strictly less than 1/3rd. Helpful for testing byzantine fault-tolerant systems.

ms->nanos

(ms->nanos ms)

mute

macro

(mute & body)

mute-jdk

macro

(mute-jdk & body)

name+

(name+ x)

Tries name, falls back to pr-str.

named-locks

(named-locks)

Creates a mutable data structure which backs a named locking mechanism.

Named locks are helpful when you need to coordinate access to a dynamic pool of resources. For instance, you might want to prohibit multiple threads from executing a command on a remote node at once. Nodes are uniquely identified by a string name, so you could write:

(defonce node-locks (named-locks))

...
(defn start-db! [node]
  (with-named-lock node-locks node
    (c/exec :service :meowdb :start)))

Now, concurrent calls to start-db! will not execute concurrently.

The structure we use to track named locks is an atom wrapping a map, where the map’s keys are any object, and the values are canonicalized versions of that same object. We use standard Java locking on the canonicalized versions. This is basically an arbitrary version of string interning.

nanos->ms

(nanos->ms nanos)

nanos->secs

(nanos->secs nanos)

nemesis-intervals

(nemesis-intervals history)(nemesis-intervals history opts)

Given a history where a nemesis goes through :f :start and :f :stop type transitions, constructs a sequence of pairs of start and stop ops. Since a nemesis usually goes :start :start :stop :stop, we construct pairs of the first and third, then second and fourth events. Where no :stop op is present, we emit a pair like start nil. Optionally, a map of start and stop sets may be provided to match on user-defined :start and :stop keys.

Multiple starts are ended by the same pair of stops, so :start1 :start2 :start3 :start4 :stop1 :stop2 yields:

start1 stop1 start2 stop2 start3 stop1 start4 stop2

op->str

(op->str op)

Format an operation as a string.

parse-long

(parse-long s)

Parses a string to a Long. Look, we use this a lot, okay?

poly-compare

(poly-compare a b)

Comparator function for sorting heterogenous collections.

polysort

(polysort coll)

Sort, but on heterogenous collections.

pprint-str

(pprint-str x)

print-history

(print-history history)(print-history printer history)

Prints a history to the console.

prn-op

(prn-op op)

Prints an operation to the console.

processors

(processors)

How many processors on this platform?

pwrite-history!

(pwrite-history! f history)(pwrite-history! f printer history)

Writes history, taking advantage of more cores.

rand-distribution

(rand-distribution)(rand-distribution distribution-map)

Generates a random value with a distribution (default :uniform) of:

; Uniform distribution from min (inclusive, default 0) to max (exclusive, default Long/MAX_VALUE). 
{:distribution :uniform, :min 0, :max 1024}

; Geometric distribution with mean 1/p.
{:distribution :geometric, :p 1e-3}

; Select a value from a sequence with equal probability.
{:distribution :one-of, :values [-1, 4097, 1e+6]}

; Select a value based on weights. :weights are {value weight ...}
{:distribution :weighted :weights {1e-3 1 1e-4 3 1e-5 1}}

rand-exp

(rand-exp lambda)

Generates a exponentially distributed random value with rate parameter lambda.

rand-nth-empty

(rand-nth-empty coll)

Like rand-nth, but returns nil if the collection is empty.

random-nonempty-subset

(random-nonempty-subset coll)

A randomly selected, randomly ordered, non-empty subset of the given collection. Returns nil if collection is empty.

real-pmap

(real-pmap f coll)

Like pmap, but runs a thread per element, which prevents deadlocks when work elements have dependencies. The dom-top real-pmap throws the first exception it gets, which might be something unhelpful like InterruptedException or BrokenBarrierException. This variant works like that real-pmap, but throws more interesting exceptions when possible.

relative-time-nanos

(relative-time-nanos)

Time in nanoseconds since relative-time-origin

retry

macro

(retry dt & body)

Evals body repeatedly until it doesn’t throw, sleeping dt seconds.

secs->nanos

(secs->nanos s)

sequential

(sequential thing-or-things)

Wraps non-sequential things into singleton lists, and leaves sequential things or nil as themselves. Useful when you can take either a single thing or a sequence of things.

sh

(sh & args)

A wrapper around clojure.java.shell’s sh which throws on nonzero exit.

sleep

(sleep dt)

High-resolution sleep; takes a (possibly fractional) time in ms.

spy

(spy x)

test->str

(test->str test)

Pretty-prints a test to a string. This binds print-length to avoid printing infinite sequences for generators.

time-

macro

(time- & body)

timeout

macro

(timeout millis timeout-val & body)

Times out body after n millis, returning timeout-val.

uninteresting-exceptions

Exceptions which are less interesting; used by real-pmap and other cases where we want to pick a meaningful exception.

with-named-lock

macro

(with-named-lock locks name & body)

Given a lock pool, and a name, locks that name in the pool for the duration of the body.

with-relative-time

macro

(with-relative-time & body)

Binds relative-time-origin at the start of body.

with-retry

macro

(with-retry initial-bindings & body)

It’s really fucking inconvenient not being able to recur from within (catch) expressions. This macro wraps its body in a (loop bindings(try …)). Provides a (retry & new bindings) form which is usable within (catch) blocks: when this form is returned by the body, the body will be retried with the new bindings.

with-thread-name

macro

(with-thread-name thread-name & body)

Sets the thread name for duration of block.

write-history!

(write-history! f history)(write-history! f printer history)

Writes a history to a file.