Jepsen 0.2.5

Released under the Eclipse Public License

Distributed systems testing framework.

Installation

To install, add the following dependency to your project or build file:

[jepsen "0.2.5"]

Namespaces

jepsen.adya

Moved to jepsen.tests.adya.

Public variables and functions:

    jepsen.checker

    Validates that a history is correct with respect to some model.

    jepsen.checker.clock

    Helps analyze clock skew over time.

    Public variables and functions:

    jepsen.checker.perf

    Supporting functions for performance analysis.

    jepsen.checker.timeline

    Renders an HTML timeline of a history.

    jepsen.cli

    Command line interface. Provides a default main method for common Jepsen functions (like the web interface), and utility functions for Jepsen tests to create their own test runners.

    jepsen.client

    Applies operations to a database.

    Public variables and functions:

    jepsen.codec

    Serializes and deserializes objects to/from bytes.

    Public variables and functions:

    jepsen.control

    Provides control over a remote node. There’s a lot of dynamically bound state in this namespace because we want to make it as simple as possible for scripts to open connections to various nodes.

    jepsen.control.clj-ssh

    A CLJ-SSH powered implementation of the Remote protocol.

    jepsen.control.core

    Provides the base protocol for running commands on remote nodes, as well as common functions for constructing and evaluating shell commands.

    Public variables and functions:

    jepsen.control.docker

    The recommended way is to use SSH to setup and teardown databases. It’s however sometimes conveniet to be able to setup and teardown the databases using docker exec and docker cp instead, which is what this namespace helps you do.

    Public variables and functions:

    jepsen.control.k8s

    The recommended way is to use SSH to setup and teardown databases. It’s however sometimes conveniet to be able to setup and teardown the databases using kubectl instead, which is what this namespace helps you do. Use at your own risk, this is an unsupported way of running Jepsen.

    Public variables and functions:

    jepsen.control.net

    Network control functions.

    Public variables and functions:

    jepsen.control.retry

    SSH client libraries appear to be near universally-flaky. Maybe race conditions, maybe underlying network instability, maybe we’re just doing it wrong. For whatever reason, they tend to throw errors constantly. The good news is we can almost always retry their commands safely! This namespace provides a Remote which wraps an underlying Remote in a jepsen.reconnect wrapper, catching certain exception classes and ensuring they’re automatically retried.

    Public variables and functions:

    jepsen.control.scp

    Built-in JDK SSH libraries can be orders of magnitude slower than plain old SCP for copying even medium-sized files of a few GB. This provides a faster implementation of a Remote which shells out to SCP.

    jepsen.control.sshj

    An sshj-backed control Remote. Experimental; I’m considering replacing jepsen.control’s use of clj-ssh with this instead.

    jepsen.control.util

    Utility functions for scripting installations.

    jepsen.core

    Entry point for all Jepsen tests. Coordinates the setup of servers, running tests, creating and resolving failures, and interpreting results.

    jepsen.db

    Allows Jepsen to set up and tear down databases.

    Public variables and functions:

    jepsen.faketime

    Libfaketime is useful for making clocks run at differing rates! This namespace provides utilities for stubbing out programs with faketime.

    Public variables and functions:

    jepsen.fs-cache

    Some systems Jepsen tests are expensive or time-consuming to set up. They might involve lengthy compilation processes, large packages which take a long time to download, or allocate large files on initial startup.

    jepsen.generator

    In a Nutshell

    jepsen.generator.interpreter

    This namespace interprets operations from a pure generator, handling worker threads, spawning processes for interacting with clients and nemeses, and recording a history.

    jepsen.generator.test

    This namespace contains functions for testing generators. See the jepsen.generator-test namespace in the test/ directory for a concrete example of how these functions can be used.

    jepsen.independent

    Some tests are expensive to check–for instance, linearizability–which requires we verify only short histories. But if histories are short, we may not be able to sample often or long enough to reveal concurrency errors. This namespace supports splitting a test into independent components–for example taking a test of a single register and lifting it to a map of keys to registers.

    jepsen.nemesis

    jepsen.nemesis.combined

    A nemesis which combines common operations on nodes and processes: clock skew, crashes, pauses, and partitions. So far, writing these sorts of nemeses has involved lots of special cases. I expect that the API for specifying these nemeses is going to fluctuate as we figure out how to integrate those special cases appropriately. Consider this API unstable.

    jepsen.nemesis.membership

    EXPERIMENTAL: provides standardized support for nemeses which add and remove nodes from a cluster.

    jepsen.nemesis.membership.state

    This namespace defines the protocol for nemesis membership state machines—how to find the current view from a node, how to merge node views together, how to generate, apply, and complete operations, etc.

    Public variables and functions:

    jepsen.nemesis.time

    Functions for messing with time and clocks.

    jepsen.net

    Controls network manipulation.

    Public variables and functions:

    jepsen.net.proto

    Protocols for network manipulation. High-level functions live in jepsen.net.

    Public variables and functions:

    jepsen.os

    Controls operating system setup and teardown.

    Public variables and functions:

    jepsen.os.centos

    Common tasks for CentOS boxes.

    jepsen.os.debian

    Common tasks for Debian boxes.

    jepsen.os.smartos

    Common tasks for SmartOS boxes.

    jepsen.os.ubuntu

    Common tasks for Ubuntu boxes. Tested against Ubuntu 18.04.

    Public variables and functions:

    jepsen.reconnect

    Stateful wrappers for automatically reconnecting network clients.

    jepsen.repl

    Helper functions for mucking around with tests!

    Public variables and functions:

    jepsen.report

    Prints out stuff.

    Public variables and functions:

    jepsen.store

    Persistent storage for test runs and later analysis.

    jepsen.tests

    Provide utilities for writing tests using jepsen.

    Public variables and functions:

    jepsen.tests.adya

    Generators and checkers for tests of Adya’s proscribed behaviors for weakly-consistent systems. See http://pmg.csail.mit.edu/papers/adya-phd.pdf

    Public variables and functions:

    jepsen.tests.bank

    Helper functions for doing bank tests, where you simulate transfers between accounts, and verify that reads always show the same balance. The test map should have these additional options:

    jepsen.tests.causal

    Public variables and functions:

    jepsen.tests.causal-reverse

    Checks for a strict serializability anomaly in which T1 < T2, but T2 is visible without T1.

    Public variables and functions:

    jepsen.tests.cycle

    Tests based on transactional cycle detection via Elle. If you’re looking for code that used to be here, see elle.core.

    Public variables and functions:

    jepsen.tests.cycle.append

    Detects cycles in histories where operations are transactions over named lists lists, and operations are either appends or reads. See elle.list-append for docs.

    Public variables and functions:

    jepsen.tests.cycle.wr

    A test which looks for cycles in write/read transactions. Writes are assumed to be unique, but this is the only constraint. See elle.rw-register for docs.

    Public variables and functions:

    jepsen.tests.linearizable-register

    Common generators and checkers for linearizability over a set of independent registers. Clients should understand three functions, for writing a value, reading a value, and compare-and-setting a value from v to v’. Reads receive nil, and replace it with the value actually read.

    Public variables and functions:

    jepsen.tests.long-fork

    Tests for an anomaly in parallel snapshot isolation (but which is prohibited in normal snapshot isolation). In long-fork, concurrent write transactions are observed in conflicting order. For example:

    jepsen.util

    Kitchen sink

    jepsen.web

    Web server frontend for browsing test results.