Rescuing Running Jobs

GNU Screen & Linux Job Control

Chapter 8 ยท Moving an Already-Running Job Into Screen

It happens to everyone: you start a long job straight in your SSH session, and only then realise you should have started it inside screen. This chapter covers the two different things "adding a job to a screen" can mean — rescuing a job that's already running outside screen, and starting a new job inside a session that's already running — and is honest about what each method can and can't do.

The Honest Answer First

Screen itself has no command to adopt a running process. A process's controlling terminal is set when it starts, and ordinary tools can't change it afterwards. That leaves three real options:

OptionJob survives logout?Can you interact with it again?Needs
Ctrl-Z, bg, disownYes (Chapter 7)NoNothing extra
reptyrYesYes — it becomes a normal screen windowThe reptyr tool and ptrace permission
Stop it and restart it in screenYesYesA job that's safe to restart

Method 1: Protect It With disown

If you only need the job to keep running, Chapter 7's sequence is enough, and it needs no extra software:

^Z # suspend the foreground job $ bg # resume it in the background $ disown %1 # stop the shell sending it SIGHUP

The job survives, but it's still writing to your old terminal, and you can never get it back to type into. For anything you want to watch or control later, you need reptyr.

Method 2: reptyr

reptyr, written by Nelson Elhage, attaches a running program to a new terminal. It uses the kernel's debugging interface, ptrace, to make the process switch its controlling terminal to the one you run reptyr in. Run it inside a screen window, and the job becomes part of that window: its output appears there, and Ctrl-C and Ctrl-Z reach it normally.

# Install it (Debian/Ubuntu; also packaged for Fedora and Arch) sudo apt install reptyr # 1. In the original terminal: suspend, background, find the PID, disown ^Z $ bg $ jobs -l [1]+ 82417 Running ./import.sh & $ disown %1 # 2. Start (or attach to) a screen session $ screen -S import # 3. Inside the screen window: grab the process $ reptyr 82417

The window now shows the job's output. You can detach with Ctrl-a d, log out, and reattach later exactly as if you'd started the job inside screen in the first place.

Why disown before reptyr?
The original shell still considers the job one of its own. Disowning it means that when you close the original terminal, the shell won't send it SIGHUP or complain about running jobs. And if you've lost the original terminal already, find the PID with pgrep -af (Chapter 6).

The ptrace permission problem

Because ptrace is also what debuggers use to inspect other programs, many distributions restrict it for security, through the Yama setting /proc/sys/kernel/yama/ptrace_scope:

ValueMeaningEffect on reptyr
0Classic: you can trace any of your own processesWorks
1Restricted: you can only trace your own descendantsFails for normal users, since the job isn't reptyr's child
2Only administrators can traceOnly works with sudo
3No tracing at allDoesn't work
# Check the current setting cat /proc/sys/kernel/yama/ptrace_scope # Allow it temporarily (until reboot), as root echo 0 | sudo tee /proc/sys/kernel/yama/ptrace_scope # Or run reptyr itself with sudo, when scope is 1 or 2 sudo reptyr 82417
Loosening ptrace is a security trade-off
The restriction exists to stop a compromised program from reading the memory of your other programs, such as passwords in an SSH agent or browser. Change it only temporarily, only if you're allowed to, and on shared or work systems ask an administrator first. If the file doesn't exist, your kernel doesn't use Yama and this restriction doesn't apply.

reptyr's limits

  • It works most reliably on a single process. A pipeline, or a program that has started its own child processes on the same terminal, may not move cleanly; newer versions have a -T ("steal") mode designed for those cases.
  • It can only take your own processes (or anyone's, with sudo).
  • Output the job wrote to the old terminal before the move isn't copied over — you only see what it writes afterwards.
  • Programs that draw full-screen interfaces may need a redraw (often Ctrl-L) after the move.

Method 3: Restart It Properly

If the job can be safely stopped and started again — it hasn't done much yet, or it can resume from where it left off, as rsync can — the simplest and most reliable fix is to stop it and start it again inside screen. Don't reach for reptyr just to avoid a restart that costs nothing.

Adding a New Job to a Session That's Already Running

The other meaning of "add a job to a screen" is starting a new command inside an existing session, often without attaching. Screen's -X option sends a command to a session, and screen is itself one of screen's commands: it opens a new window.

# Open a new window titled "logs" in the session "work", running tail screen -S work -X screen -t logs tail -f /var/log/syslog # Start a whole new session in the background, running one job screen -dmS import ./import.sh # Check it worked screen -ls

From inside the session you can do the same thing with screen -t title command (Chapter 4). Chapter 9 covers -dm and -X in depth, including typing commands into an existing window.

The best rescue is not needing one
The habit that avoids this whole chapter: before starting anything that might take more than a few minutes on a remote machine, run screen -S name first.

Hands-On Exercises

Exercise 1

Start top directly in a terminal (not in screen). Move it into a new screen session using reptyr, then detach, close the original terminal, and reattach to prove it survived. Record every command you used and any permission problem you hit.

๐Ÿ“„ View solution
Exercise 2

A colleague has a two-hour pg_dump running in their SSH session and must leave now. The server has no reptyr and they have no sudo. What would you advise, and what would they lose?

๐Ÿ“„ View solution
Exercise 3

With a detached session called work already running, add two new windows to it from outside — one running htop (or top) and one running watch df -h — without attaching. Then attach and confirm they're there.

๐Ÿ“„ View solution

Chapter 8 Quick Reference

  • Screen can't adopt a running process by itself
  • To keep a job alive only: Ctrl-Z, bg, disown
  • To move it into screen: Ctrl-Z, bg, jobs -l, disown, start screen, then reptyr PID inside it
  • reptyr needs ptrace permission: check /proc/sys/kernel/yama/ptrace_scope (0 works; 1 or 2 needs sudo)
  • reptyr works best on single processes; -T steal mode is for harder cases; earlier output isn't copied
  • If a restart is cheap, just restart the job inside screen
  • Add a window to a running session: screen -S name -X screen -t title cmd; start a detached session: screen -dmS name cmd