distssh ~dynamic-poll

Distributed load balance over SSH


To use this package, run the following command in your project's root directory:

Manual usage
Put the following dependency into your project's dependences section:


This package provides sub packages which can be used individually:

distssh:distssh_common - Distributed load balance over SSH

distssh:colorlog - std.experimental.logger with color support

distssh:d2sqlite3 - A thin wrapper around SQLite 3

distssh:miniorm - A minimalistic ORM for SQLite

distssh:msgpack-ll - A low level @nogc, nothrow, @safe, pure and betterC compatible MessagePack library

distssh:sumtype - A sum type with pattern matching

distssh:unit-threaded - Advanced multi-threaded unit testing framework with minimal to no boilerplate using built-in unittest blocks

distssh:proc - Library for managing running processes and trees of them

distssh:my - A library collection of algorithms and data structures for personal use

distssh

Build Status

distssh is a frontend to ssh that find the least loaded host in a cluster and execute commands on it.

It can alternatively be used to find and run an interactive shell on the least loaded host.

Getting Started

distssh depends on the following software packages:

Download the D compiler of your choice, extract it and add to your PATH shell variable.

# example with an extracted DMD
export PATH=/path/to/dmd/linux/bin64/:$PATH

Once the dependencies are installed it is time to download the source code to install distssh.

git clone https://github.com/joakim-brannstrom/distssh.git
cd distssh
dub build -b release

Copy the file in build/ to wherever you want to install it. When you have placed it at where you want the run the install command to setup the needed symlinks:

/my/install/path/distssh --install

Done! Have fun. Don't be shy to report any issue that you find.

Usage

For distssh to be useful the environment variable DISTSSH_HOSTS has to be set. The ; is used to separate hosts. Example:

export DISTSSH_HOSTS='foo;bar;wun'

When that is done it is now ready to use!

Remote Shell

This is the simplest usage. It gives you a shell at the cluster.

distshell
# or
distssh shell

Remote Command

This will execute the command on the cluster.

distssh cmd -- ls

Export the Environemnt to Remote Host

This is useful for those development environments where it is heavy to reload the shell with the correct modules. By exporting and then importing the environment on the remote host this can be bypassed/sped up.

Note that this basically requires them to be equivalent.

# store an export of the env
distssh env -e
# now the env is reused on the remote hosts
distcmd ls

It is also possible to clone the current environment without first exporting it to a file.

distssh distcmd --clone-env -- ls

The exported environment can also be used to run commands locally.

distssh localrun -- env

Cluster Load

Distssh can display the current load of the cluster. The servers that are listed are those that distssh is currently able to connect to and that respond within the --timeout limit.

The load is normalised to what is considered "best practice" when comparing the value from loadavg by dividing it by the number of cores that are available on the server. A server is most probably overloaded if it is above 1.0.

distssh measurehosts

Purge

Even though distssh try to do a good job of cleaning up stale processes and such when it logout of a server there are circumstances wherein a process can be left dangling on a server in the cluster and take up resources. What resource is obviously dependent on what type of process that is left.

distssh can purge these processes from the cluster. This can be executed either manually or by the daemon in the background.

# print all processes that have escaped sshd, from all users
distssh purge -p
# print those that have escaped the current user
distssh purge -p --user-filter
# kill them
distssh purge -p --user-filter -k

Configuration

These environment variables control the behavior of distssh.

  • DISTSSH_HOSTS: the hosts to load balance among. Hosts are separated by ;. Example:
export DISTSSH_HOSTS='localhost;some_remove'
distcmd ls
  • DISTSSH_IMPORT_ENV: filename to load the environment from. Example:
export DISTSSH_IMPORT_ENV="$HOME/foo.export"
distcmd ls
  • DISTSSH_ENV_EXPORT_FILTER: environment keys to remove when creating an export. Keys are separated by ;. Example:
export DISTSSH_ENV_EXPORT_FILTER='PWD;USER;USERNAME;_'
distssh --export-env
  • DISTSSH_AUTO_PURGE: set the variable to 1 to activate automatic purge of processes that isn't part of a process subtree which contains a whitelisted process when the daemon is running. Example:
export DISTSSH_AUTO_PURGE=1
  • DISTSSH_PURGE_WLIST: a list of case insensitive regex separated by ;. Used both by the manual purge sub-command and the daemon mode. Example:
# will purge all processes that have "escaped" an interactive sshd instance
export DISTSSH_PURGE_WLIST='.*sshd'
Authors:
  • Joakim Brännström
Sub packages:
distssh:distssh_common, distssh:colorlog, distssh:d2sqlite3, distssh:miniorm, distssh:msgpack-ll, distssh:sumtype, distssh:unit-threaded, distssh:proc, distssh:my
Dependencies:
distssh:sumtype, distssh:colorlog, distssh:proc, distssh:distssh_common, distssh:msgpack-ll, distssh:miniorm, distssh:d2sqlite3
Versions:
3.1.1 2020-Sep-23
3.1.0 2020-Sep-06
3.0.0 2020-Apr-30
2.0.1 2019-Oct-06
2.0.0 2019-Sep-30
Show all 14 versions
Download Stats:
  • 0 downloads today

  • 0 downloads this week

  • 0 downloads this month

  • 31 downloads total

Score:
1.2
Short URL:
distssh.dub.pm