Subsystems are a layer of abstraction for defining and running remote commands via SSH.^[83] Normally remote commands are specified ad hoc on the client command line. For example, the following command runs a script to perform tape backups:

    $ ssh server.example.com /usr/local/sbin/tape-backups

Subsystems are a set of remote commands predefined on the server machine, with simple names so that they can be executed conveniently.

The syntax to define subsystems in the server configuration file is slightly different for OpenSSH and Tectia. A subsystem for the preceding backup command is:

    # OpenSSH
    Subsystem backups   /usr/local/sbin/tape-backups

    # Tectia
    Subsystem-backups   /usr/local/sbin/tape-backups

Note that OpenSSH uses the keyword Subsystem with a separate value for the subsystem name, whereas Tectia uses a keyword of the form Subsystem- name. This Tectia syntax is quite odd and unlike anything else in its configuration language; we don’t know how it ended up that way.

To run this tape backup script on the server machine, use the ssh -s option:

    $ ssh server.example.com -s backups

This command behaves identically to the previous one in which the script was specified explicitly.

Subsystems are mainly a convenience feature to predefine commands for SSH clients to invoke easily. The additional level of abstraction is useful for system administrators, who can hide (and therefore easily change) details for the subsystem commands. For example, the backups subsystem could be changed to use a completely different script, without any changes in the ssh client command that operators run to perform tape backups.

System administrators can also define and advertise more generally useful subsystems. Suppose your users run the Pine email reader to connect to your IMAP server to secure the connection. [11.3] Instead of telling everyone to use the command:

    $ ssh server.example.com /usr/sbin/imapd

and revealing the path to the IMAP daemon, imapd, you can define an imap subsystem to hide the path in case it changes in the future:

    # OpenSSH
    Subsystem imap  /usr/sbin/imapd

    # Tectia
    Subsystem-imap  /usr/sbin/imapd

Now users can run the command:

    $ ssh server.example.com -s imap

to establish secure IMAP connections via the subsystem.

Subsystems are especially useful for tunneling other protocols. If clients refer only to a subsystem, the corresponding server implementation can be changed without modifying (and redeploying) the clients, which might be numerous and widely scattered.

The best example is the sftp subsystem, which provides secure file transfers. [2.7.1] The sftp client runs ssh -s sftp to launch an sftp-server program and set up a secure tunnel for communication between the client and server.^[84] The default server configuration file for both OpenSSH and Tectia contains a definition of the sftp subsystem, with the correct, absolute pathname for sftp-server. Tectia also provides an internal implementation of the sftp subsystem that is built into the SSH server itself. This can be selected by using a special syntax for the command:

    # Tectia
    Subsystem-sftp  internal://sftp-server

The internal sftp subsystem is much more convenient than the default (external) sftp-server command for accounts that are subject to chroot restrictions. [5.5.7]

Subsystem commands are executed by each user’s shell, and they can be affected by environment variables set by the user (if permitted by the server [5.6.2]), shell start-up scripts, etc. OpenSSH avoids running the ~/.ssh/rc script for subsystems, but Tectia always runs ~/.ssh2/rc. If a subsystem server command uses a special token to mark the start of its output, clients can ignore unexpected output from user scripts. Of course, the token must be defined as part of the protocol that’s understood and used by the client and server.

OpenSSH requires that subsystem commands use absolute filenames, since no PATH search is performed. If a relative filename is used, e.g.:

    # OpenSSH: this does not work
    Subsystem backups   tape-backups

then no error occurs when the server configuration file is read, but on subsequent attempts to use the subsystem, clients fail silently, and the server emits syslog warnings:

    Dec 20 14:14:47 server.example.com sshd[1554]: error: subsystem: cannot stat 
tape-backups: No such file or directory

Furthermore, OpenSSH doesn’t permit command-line arguments for subsystem commands:

    # OpenSSH: this does not work
    Subsystem backups   /usr/local/sbin/tape-backups --full --filesystem=/home

This restriction is enforced when the server configuration file is read:

    /etc/ssh/sshd_config line 99: garbage at end of line; "--full".

Tectia is more permissive. The server searches for simple commands (i.e., relative filenames and no command-line arguments) in the libexec and bin subdirectories of the Tectia install directory, and then searches each directory in the PATH. Absolute filenames are still recommended, however, since the PATH can be redefined or modified by each user, and (if not set explicitly) defaults to the value inherited when the server was started.

Tectia also allows extra arguments or even shell metacharacters in subsystem commands:

    # Tectia
    Subsystem-backups   /usr/local/sbin/tape-backups --full 2>&1 | tee /var/log/backups

This is usually a bad idea, because various shells for individual users differ in their interpretation of metacharacters (e.g., the 2>&1 notation in the previous example is understood only by Bourne-style shells). The SSH server configuration file is the wrong place for this complexity: a better approach is to wrap the details in a separate script, and use the name of that script as the subsystem command.

Subsystem keywords can be repeated to define multiple, independent subsystems. OpenSSH can define a maximum of 256 subsystems; there is no limit for Tectia. OpenSSH refuses to allow subsystem names to be reused:

    /etc/ssh/sshd_config line 98: Subsystem 'backups' already defined.

Tectia uses later subsystem definitions with the same name to override the commands from earlier definitions. This can be useful in conjunction with subconfiguration files. [11.6.2]

OpenSSH subsystem names are case-sensitive. In contrast, Tectia maps subsystem names to lowercase when the configuration file is read, but then uses case-sensitive comparisons to look up the subsystems specified by clients. This unfortunate and confusing behavior effectively restricts Tectia subsystem names to be all lowercase.^[85]

The IETF SECSH draft only defines the “sftp” subsystem name and mandates that other, nonstandard names use an @ suffix to identify the domain that defined the subsystem:

    # OpenSSH
    Subsystem smail@example.com     /usr/local/sbin/secure-mail-server

    # Tectia
    Subsystem-smail@example.com     /usr/local/sbin/secure-mail-server

This convention should be followed to avoid name clashes for software that is widely used, but the domain suffix is commonly omitted for subsystems that are used only within a single organization, and the convention is not enforced.