Jepsen 0.2.5

Released under the Eclipse Public License

Distributed systems testing framework.


To install, add the following dependency to your project or build file:

[jepsen "0.2.5"]



Moved to jepsen.tests.adya.

Public variables and functions:


    Validates that a history is correct with respect to some model.


    Helps analyze clock skew over time.

    Public variables and functions:


    Supporting functions for performance analysis.


    Renders an HTML timeline of a history.


    Command line interface. Provides a default main method for common Jepsen functions (like the web interface), and utility functions for Jepsen tests to create their own test runners.


    Applies operations to a database.

    Public variables and functions:


    Serializes and deserializes objects to/from bytes.

    Public variables and functions:


    Provides control over a remote node. There’s a lot of dynamically bound state in this namespace because we want to make it as simple as possible for scripts to open connections to various nodes.


    A CLJ-SSH powered implementation of the Remote protocol.


    Provides the base protocol for running commands on remote nodes, as well as common functions for constructing and evaluating shell commands.

    Public variables and functions:


    The recommended way is to use SSH to setup and teardown databases. It’s however sometimes conveniet to be able to setup and teardown the databases using docker exec and docker cp instead, which is what this namespace helps you do.

    Public variables and functions:


    The recommended way is to use SSH to setup and teardown databases. It’s however sometimes conveniet to be able to setup and teardown the databases using kubectl instead, which is what this namespace helps you do. Use at your own risk, this is an unsupported way of running Jepsen.

    Public variables and functions:

    Network control functions.

    Public variables and functions:


    SSH client libraries appear to be near universally-flaky. Maybe race conditions, maybe underlying network instability, maybe we’re just doing it wrong. For whatever reason, they tend to throw errors constantly. The good news is we can almost always retry their commands safely! This namespace provides a Remote which wraps an underlying Remote in a jepsen.reconnect wrapper, catching certain exception classes and ensuring they’re automatically retried.

    Public variables and functions:


    Built-in JDK SSH libraries can be orders of magnitude slower than plain old SCP for copying even medium-sized files of a few GB. This provides a faster implementation of a Remote which shells out to SCP.


    An sshj-backed control Remote. Experimental; I’m considering replacing jepsen.control’s use of clj-ssh with this instead.


    Utility functions for scripting installations.


    Entry point for all Jepsen tests. Coordinates the setup of servers, running tests, creating and resolving failures, and interpreting results.


    Allows Jepsen to set up and tear down databases.

    Public variables and functions:


    Libfaketime is useful for making clocks run at differing rates! This namespace provides utilities for stubbing out programs with faketime.

    Public variables and functions:


    Some systems Jepsen tests are expensive or time-consuming to set up. They might involve lengthy compilation processes, large packages which take a long time to download, or allocate large files on initial startup.


    In a Nutshell


    This namespace interprets operations from a pure generator, handling worker threads, spawning processes for interacting with clients and nemeses, and recording a history.


    This namespace contains functions for testing generators. See the jepsen.generator-test namespace in the test/ directory for a concrete example of how these functions can be used.


    Some tests are expensive to check–for instance, linearizability–which requires we verify only short histories. But if histories are short, we may not be able to sample often or long enough to reveal concurrency errors. This namespace supports splitting a test into independent components–for example taking a test of a single register and lifting it to a map of keys to registers.



    A nemesis which combines common operations on nodes and processes: clock skew, crashes, pauses, and partitions. So far, writing these sorts of nemeses has involved lots of special cases. I expect that the API for specifying these nemeses is going to fluctuate as we figure out how to integrate those special cases appropriately. Consider this API unstable.


    EXPERIMENTAL: provides standardized support for nemeses which add and remove nodes from a cluster.


    This namespace defines the protocol for nemesis membership state machines—how to find the current view from a node, how to merge node views together, how to generate, apply, and complete operations, etc.

    Public variables and functions:


    Functions for messing with time and clocks.

    Controls network manipulation.

    Public variables and functions:

    Protocols for network manipulation. High-level functions live in

    Public variables and functions:


    Controls operating system setup and teardown.

    Public variables and functions:


    Common tasks for CentOS boxes.


    Common tasks for Debian boxes.


    Common tasks for SmartOS boxes.


    Common tasks for Ubuntu boxes. Tested against Ubuntu 18.04.

    Public variables and functions:


    Stateful wrappers for automatically reconnecting network clients.


    Helper functions for mucking around with tests!

    Public variables and functions:

    Prints out stuff.

    Public variables and functions:

    Persistent storage for test runs and later analysis.


    Provide utilities for writing tests using jepsen.

    Public variables and functions:


    Generators and checkers for tests of Adya’s proscribed behaviors for weakly-consistent systems. See

    Public variables and functions:

    Helper functions for doing bank tests, where you simulate transfers between accounts, and verify that reads always show the same balance. The test map should have these additional options:


    Public variables and functions:


    Checks for a strict serializability anomaly in which T1 < T2, but T2 is visible without T1.

    Public variables and functions:


    Tests based on transactional cycle detection via Elle. If you’re looking for code that used to be here, see elle.core.

    Public variables and functions:


    Detects cycles in histories where operations are transactions over named lists lists, and operations are either appends or reads. See elle.list-append for docs.

    Public variables and functions:


    A test which looks for cycles in write/read transactions. Writes are assumed to be unique, but this is the only constraint. See for docs.

    Public variables and functions:


    Common generators and checkers for linearizability over a set of independent registers. Clients should understand three functions, for writing a value, reading a value, and compare-and-setting a value from v to v’. Reads receive nil, and replace it with the value actually read.

    Public variables and functions:


    Tests for an anomaly in parallel snapshot isolation (but which is prohibited in normal snapshot isolation). In long-fork, concurrent write transactions are observed in conflicting order. For example:


    Kitchen sink


    Web server frontend for browsing test results.