jepsen.control.util
Utility functions for scripting installations.
await-tcp-port
(await-tcp-port port)(await-tcp-port port opts)Blocks until a local TCP port is bound. Options:
:retry-interval How long between retries, in ms. Default 1s. :log-interval How long between logging that we’re still waiting, in ms. Default `retry-interval. :timeout How long until giving up and throwing :type :timeout, in ms. Default 60 seconds.
cached-wget!
(cached-wget! url)(cached-wget! url opts)Downloads a string URL to the Jepsen wget cache directory, and returns the full local filename as a string. Skips if the file already exists. Local filenames are base64-encoded URLs, as opposed to the name of the file–this is helpful when you want to download a package like https://foo.com/v1.2/foo.tar; since the version is in the URL but not a part of the filename, downloading a new version could silently give you the old version instead.
Options:
:force? Even if we have this cached, download the tarball again anyway. :user? User for wget authentication. If provided, valid pw must also be provided. :pw? Password for wget authentication.
daemon-running?
(daemon-running? pidfile)Given a pidfile, returns true if the pidfile is present and the process it contains is alive, nil if the pidfile is absent, false if it’s present and the process doesn’t exist.
Strictly this doesn’t mean the process is RUNNING; it could be asleep or a zombie, but you know what I mean. ;-)
grepkill!
(grepkill! pattern)(grepkill! signal pattern)Kills processes by grepping for the given string. If a signal is given, sends that signal instead. Signals may be either numbers or names, e.g. :term, :hup, …
install-archive!
(install-archive! url dest)(install-archive! url dest opts)Gets the given tarball URL, caching it in /tmp/jepsen/, and extracts its sole top-level directory to the given dest directory. Deletes current contents of dest. Supports both zip files and tarballs, compressed or raw. Returns dest.
URLs can be HTTP, HTTPS, or file://, in which case they are interpreted as a file path on the remote node.
Standard practice for release tarballs is to include a single directory, often named something like foolib-1.2.3-amd64, with files inside it. If only a single directory is present, its contents will be moved to dest, so foolib-1.2.3-amd64/my.file becomes dest/my.file. If the tarball includes multiple files, those files are moved to dest, so my.file becomes dest/my.file.
Options:
:force? Even if we have this cached, download the tarball again anyway. :user? User for wget authentication. If provided, valid pw must also be provided. :pw? Password for wget authentication.
ls
(ls)(ls dir)(ls dir opts)A seq of directory entries (not including . and ..). Options:
{:recursive? If set, lists entries recursively :types A collection like :file :dir, filtering what kinds of entries are returned :full-path? Return the full path, rather than the path within dir}
TODO: escaping for control chars in filenames (if you do this, WHO ARE YOU???)
ls-full
(ls-full)(ls-full dir)(ls-full dir opts)Like ls, but prepends dir to each entry. TODO: deprecate this in favor of ls {:full-path? true}.
signal!
(signal! process-name signal)Sends a signal to a named process by signal number or name.
start-daemon!
(start-daemon! opts bin & args)Starts a daemon process, logging stdout and stderr to the given file. Invokes bin with args. Options are:
:env Environment variables for the invocation of start-stop-daemon. Should be a Map of env var names to string values, like {:SEEDS “flax, cornflower”}. See jepsen.control/env for alternative forms. :background? :chdir :exec Sets a custom executable to check for. :logfile :make-pidfile? :match-executable? Helpful for cases where the daemon is a wrapper script that execs another process, so that pidfile management doesn’t work right. When this option is true, we ask start-stop-daemon to check for any process running the given executable program: either :exec or the bin argument. :match-process-name? Helpful for cases where the daemon is a wrapper script that execs another process, so that pidfile management doesn’t work right. When this option is true, we ask start-stop-daemon to check for any process with a COMM field matching :process-name (or the name of the bin). :pidfile Where should we write (and check for) the pidfile? If nil, doesn’t use the pidfile at all. :process-name Overrides the process name for :match-process-name?
Returns :started if the daemon was started, or :already-running if it was already running, or throws otherwise.
stop-daemon!
(stop-daemon! pidfile)(stop-daemon! cmd pidfile)Kills a daemon process by pidfile, or, if given a command name, kills all processes with that command name, and cleans up pidfile. Pidfile may be nil in the two-argument case, in which case it is ignored.
tarball!
(tarball! path)(tarball! path tarball)Takes a path and creates a .tar.gz file of it, stored in a Jepsen temporary directory. Returns the path to the tarball. Especially useful for tarring up data directories on a DB node. In the 2-arity form, takes the tarball filename as a second argument.
Ignores failed reads–these happen often when tarring up data and log files from nodes that are crashing, and getting some data is better than crashing the test.
tmp-file!
(tmp-file!)(tmp-file! ext)Creates a random, temporary file under tmp-dir-base, and returns its path. Optionally takes an extension–default is “.tmp”
wget!
(wget! url)(wget! url opts)Downloads a string URL and returns the filename as a string. Skips if the file already exists.
Options:
:force? Even if we have this cached, download the tarball again anyway. :user? User for wget authentication. If provided, valid pw must also be provided. :pw? Password for wget authentication.
wget-helper!
(wget-helper! & args)A helper for wget! and cached-wget!. Calls wget with options; catches name resolution and other network errors, and retries them. EC2 name resolution can be surprisingly flaky.