Job Control
Running, suspending, and switching between multiple commands within a single shell session - foreground and background jobs, and what happens to them when you log out.
Job control is what lets one terminal session run several commands at once without opening several terminals - starting things in the background, pausing a foreground command, and bringing things back to the foreground as needed.
Foreground vs. background
A command runs in the foreground by default, occupying the shell
until it finishes. Append & to start it in the background instead,
returning control of the shell immediately:
long-task.sh &
[1] 4821
The [1] is a job number (specific to this shell session); 4821 is the
process's actual PID (see Process Lifecycle).
Suspending and resuming
Ctrl+Z suspends whatever's running in the foreground, handing control
back to the shell without killing it - the process is paused, not
terminated:
some-command
# press Ctrl+Z
[1]+ Stopped some-command
From there:
jobs # list jobs in this shell, with their state
fg # resume the most recent job in the foreground
fg %1 # resume job number 1 specifically, in the foreground
bg %1 # resume job number 1, but in the background
kill %1 # send SIGTERM to job number 1
See Signals for what's actually happening
underneath Ctrl+Z (SIGTSTP), fg/bg (SIGCONT), and kill.
Jobs die when the shell does - unless you say otherwise
By default, a background job is tied to the shell session that started
it: closing the terminal (or losing an SSH connection) sends SIGHUP to
every job still attached to it, which terminates them. Two ways around
this:
nohup - makes a command ignore SIGHUP specifically, so it survives
the shell exiting:
nohup long-task.sh &
disown - detaches an already-running background job from the
shell's job table, after the fact:
long-task.sh &
disown
Neither of these keeps a process usable interactively after you log out - for that, you want a session multiplexer.
For anything you need to reattach to: tmux or screen
nohup and disown solve "don't kill my job when I disconnect," but
they don't let you reconnect to see its output later. tmux (or the
older screen) runs an entire persistent terminal session on the server,
independent of any one SSH connection - you can detach from it, log out,
log back in later, and reattach to find everything exactly as you left
it. This is the standard tool for long-running interactive work (a build,
a database migration, a training job) on a remote machine, and is
generally preferable to nohup for anything you might need to check back
in on.
tmux new -s mysession # start a named session
# ... do work, then detach with Ctrl+B, D ...
tmux attach -t mysession # reattach later, from any SSH connection