Process Substitution and Advanced Redirection
🔀 Intermediate Topic 2 — Process Substitution and Advanced Redirection
The beginner course covered >, >>, <, 2>&1, and pipelines. That's enough to get things done, but the moment you need to compare two command outputs, write to a log file and the terminal at the same time, or redirect inside a long-lived loop without closing and reopening files, you hit the limits of the basics. This chapter covers process substitution, named pipes, exec-based persistent file descriptors, and every redirection form bash supports — the machinery that serious scripts are built on.
1 — How Redirection Works Under the Hood
Every process inherits three open file descriptors: stdin (0), stdout (1), and stderr (2). A redirection operator doesn't move data — it rewires file descriptor numbers before the command starts, using the kernel's dup2() system call. Understanding this makes every redirection form predictable.
Without redirection:
script ──FD 0──▶ /dev/pts/0 (keyboard)
──FD 1──▶ /dev/pts/0 (terminal)
──FD 2──▶ /dev/pts/0 (terminal)
After: cmd > file.txt 2>&1
Step 1: open file.txt → FD 1 (stdout now points to file.txt)
Step 2: dup FD 1 to FD 2 (stderr now also points to file.txt)
cmd ──FD 0──▶ /dev/pts/0 (keyboard, unchanged)
──FD 1──▶ file.txt
──FD 2──▶ file.txt (FD 2 is a copy of FD 1's target)
Order matters! cmd 2>&1 > file.txt is WRONG — copies stderr→terminal THEN redirects stdout
The classic ordering mistake:
cmd 2>&1 >file.txt does not send both streams to the file. Redirections are processed left-to-right. At the moment 2>&1 is evaluated, FD 1 still points to the terminal, so stderr ends up on the terminal and only stdout goes to the file. Always write cmd >file.txt 2>&1 (redirect stdout first, then copy that to stderr).Every redirection form at a glance
| Syntax | What it does |
|---|---|
cmd > file | Redirect stdout to file (truncate) |
cmd >> file | Redirect stdout to file (append) |
cmd < file | Read stdin from file |
cmd 2> file | Redirect stderr to file |
cmd &> file | Redirect both stdout and stderr to file (bash shorthand) |
cmd &>> file | Append both stdout and stderr to file |
cmd > file 2>&1 | Redirect stdout to file, then copy that to stderr (POSIX-safe form) |
cmd 2>&1 > file | Copy stderr to terminal, then redirect stdout to file (usually a bug) |
cmd > /dev/null | Discard stdout |
cmd 2> /dev/null | Discard stderr |
cmd &> /dev/null | Discard all output |
cmd >&2 | Send stdout to wherever stderr currently points |
cmd << EOF ... EOF | Here-document: feed multi-line string as stdin |
cmd <<< "string" | Here-string: feed single string as stdin |
cmd n> file | Open file on file descriptor n for writing |
cmd n< file | Open file on file descriptor n for reading |
cmd n>&m | Duplicate FD m to FD n |
cmd n>&- | Close file descriptor n |
cmd {var}> file | Open file on a bash-chosen FD; store number in var (bash 4.1+) |
2 — Here-Documents and Here-Strings
Here-documents are the clean way to feed multi-line text to a command without a temporary file. There are four variations, each with different quoting and indentation behaviour.
🐧 The four here-doc forms
# ── 1. Standard: variable expansion IS performed ──────────────
name="Alice"
cat << EOF
Hello, $name!
Today is $(date +%A).
EOF
Hello, Alice!
Today is Monday.
# ── 2. Quoted delimiter: NO expansion (literal text) ──────────
cat << 'EOF'
Hello, $name! <- this prints literally, no substitution
EOF
Hello, $name!
# ── 3. Indented form (<<-): strips leading TABS only ──────────
# Use tabs (not spaces) for the heredoc lines
if true; then
cat <<- EOF
This line is indented with a tab — the tab is stripped.
So is this one.
EOF
fi
This line is indented with a tab — the tab is stripped.
So is this one.
# ── 4. Here-string: single value, no newlines ─────────────────
# Avoids echo | cmd which spawns a subshell
read -r first rest <<< "one two three"
echo "first=$first rest=$rest"
first=one rest=two three
# bc without a pipe
result=$(bc <<< "scale=4; 355/113")
echo "$result" → 3.1415
# Passing a variable as stdin to a command expecting a file
json='{"key":"value"}'
jq '.key' <<< "$json" → "value"
Here-strings (
<<<) always append a newline to the string before feeding it to the command — most commands that read stdin expect this, but be aware if you're doing byte-level work.Here-docs for generating files and scripts
🐧 Writing multi-line content to a file
# Write an entire config file in one heredoc
db_host="db.example.com"
db_port=5432
cat > /etc/myapp/config.ini << EOF
[database]
host = ${db_host}
port = ${db_port}
name = myapp
[server]
port = 8080
workers = 4
EOF
# Generate a script with literal dollar signs (quoted delimiter)
cat > /tmp/generated.sh << 'SCRIPT'
#!/usr/bin/env bash
echo "Today is $(date)"
echo "User: $USER"
SCRIPT
chmod +x /tmp/generated.sh
# The $USER and $(date) will expand when generated.sh runs, not now
# Append to a file using tee (allows both write and stdout if needed)
tee -a /var/log/myapp.log << EOF
[$(date -Is)] Application started
[$(date -Is)] PID: $$
EOF
3 — Process Substitution: <(cmd) and >(cmd)
Process substitution is one of bash's most powerful features and one of the least understood. It lets you use a command's output as if it were a file, without creating a temporary file. Bash runs the command in the background, connects it to a named file descriptor under /dev/fd/, and passes that path to the outer command.
How diff <(sort a.txt) <(sort b.txt) works:
bash creates two processes:
[sort a.txt] ──writes──▶ /dev/fd/63 (a kernel pipe)
[sort b.txt] ──writes──▶ /dev/fd/62
diff is called as:
diff /dev/fd/63 /dev/fd/62
diff reads from those paths like files.
No temp files created. All three processes run concurrently.
Process substitution is bash-specific. It requires
#!/usr/bin/env bash — it is not available in POSIX sh, dash, or BusyBox shells. If portability matters, use named pipes (covered in section 5) instead.Input process substitution — <(cmd)
🐧 Using command output as a file argument
# ── Compare two sorted files without modifying them ───────────
diff <(sort old.txt) <(sort new.txt)
# ── Find lines present in one file but not another ────────────
comm -23 <(sort list_a.txt) <(sort list_b.txt)
# comm requires sorted input — process substitution sorts on-the-fly
# ── Join two CSVs on a common field ───────────────────────────
join -t, -1 1 -2 1 \
<(sort -t, -k1,1 users.csv) \
<(sort -t, -k1,1 orders.csv)
# ── Read from a process substitution with while ───────────────
# This is the key fix for the "pipe loop loses variables" problem
# (covered in depth in Topic 4 — Subshells)
count=0
while IFS= read -r line; do
(( count++ ))
done < <(find /var/log -name '*.log')
echo "Found $count log files"
# Compare the broken version: find ... | while ... done ← $count resets to 0
# ── Read output of two commands with a single while loop ──────
while IFS= read -r -u3 a && IFS= read -r -u4 b; do
echo "$a | $b"
done 3< <(cat file1.txt) \
4< <(cat file2.txt)
The space in
< <(cmd) is required — the first < is the stdin redirect; the second is the start of the process substitution. Without the space, bash sees << which starts a here-document.Output process substitution — >(cmd)
Less commonly used, output process substitution feeds a command's stdin from another command's stdout — allowing a command that only writes to stdout to be "directed" into multiple consumers simultaneously.
🐧 Writing to multiple destinations with tee + >(cmd)
# ── Write to a file AND pass through to another command ───────
# tee sends its stdin to both the file and the process substitution
generate_report | tee >(gzip > report.gz) | mail -s "Report" boss@example.com
# One pipeline: compress to file, AND email — data read only once
# ── Split stderr and stdout to different log files ─────────────
cmd="./my_script.sh"
$cmd \
1> >(tee -a stdout.log) \
2> >(tee -a stderr.log >&2)
# stdout goes to stdout.log AND terminal; stderr to stderr.log AND terminal
# ── Log with timestamps in real time ──────────────────────────
LOGFILE="app.log"
timestamp_logger() {
while IFS= read -r line; do
printf '[%s] %s\n' "$(date '+%H:%M:%S')" "$line"
done
}
./my_app > >(timestamp_logger > "$LOGFILE")
# ── Fan-out: send output to three places simultaneously ────────
./my_app | tee \
>(grep ERROR > errors.log) \
>(gzip > full_output.gz) \
> stdout.log
4 — exec and Persistent File Descriptors
When you write cmd > file, the file is opened for the duration of that one command. If you're inside a loop that writes thousands of times, each iteration opens and closes the file. Using exec to redirect a file descriptor in the current shell keeps the file open for the entire script — dramatically faster for heavy I/O, and essential for any pattern that needs to interleave reading and writing.
🐧 exec redirects — persistent FDs for the current shell
# ── Redirect all stdout for the rest of the script ────────────
exec > output.log # everything printed after this goes to output.log
echo "This goes to the file"
ls -la # stdout of ls also goes to the file
# ── Redirect stderr to a log file for the rest of the script ──
exec 2> errors.log
echo "normal output" # still to terminal (stdout unchanged)
ls /nonexistent # error message goes to errors.log
# ── Save and restore FDs — the classic pattern ─────────────────
# Save current stdout to FD 3, redirect stdout to file, then restore
exec 3>&1 # save stdout (1) into FD 3
exec > capture.log # redirect stdout to file
echo "in the file"
exec 1>&3 # restore stdout from FD 3
exec 3>&- # close FD 3 (no longer needed)
echo "back to terminal"
# ── Open a file for reading on a custom FD ────────────────────
exec 4< data.txt # open data.txt for reading on FD 4
while IFS= read -r -u4 line; do
# read -u4 reads from FD 4 instead of stdin
# stdin is still free for interactive prompts inside the loop
read -r -p "Process '$line'? [y/n] " ans < /dev/tty
[[ "$ans" == "y" ]] && echo "Processing: $line"
done
exec 4<&- # close FD 4 when done
Auto-assigned file descriptors — {var}> file
🐧 Let bash choose the FD number (bash 4.1+)
# Hard-coding FD numbers risks collision with FDs opened by bash itself
# {var} lets bash pick a free number and stores it in var
exec {logfd}> app.log # bash chooses FD (typically 10+), stores in $logfd
echo "Log FD is: $logfd" → Log FD is: 10
# Write to the log using the variable
echo "Application starting" >&$logfd
echo "Another log line" >&$logfd
# Close when done
exec {logfd}>&-
# Practical: a logging function using a persistent FD
LOGFILE="run_$(date +%Y%m%d_%H%M%S).log"
exec {LOG}>> "$LOGFILE" # append mode
log() {
printf '[%s] %s\n' "$(date '+%T')" "$*" >&$LOG
}
log "Script started"
log "Running as: $USER"
# ... script body ...
exec {LOG}>&- # close log at end
Using
{var} is safer than picking a fixed number like 3 or 4 because bash uses higher-numbered FDs internally and won't assign one that's already in use.Swapping stdout and stderr
🐧 The three-step FD swap
# Swap stdout and stderr (send stdout to stderr and vice-versa)
# Uses the same save/swap/restore pattern as a variable swap
{ some_command; } 3>&1 1>&2 2>&3
# Step 1: save stdout to FD 3 (3>&1)
# Step 2: point stdout at stderr (1>&2)
# Step 3: point stderr at saved FD 3 (2>&3)
# Practical: capture stderr into a variable while stdout passes through
stderr_output=$( { some_command; } 2>&1 1>&3 ) 3>&1
# $() captures FD 1; the swap routes stderr there and stdout to the terminal
echo "Captured stderr: $stderr_output"
# Capture stdout AND stderr into separate variables simultaneously
exec {stdout_fd}>/tmp/stdout.$$
exec {stderr_fd}>/tmp/stderr.$$
some_command >&$stdout_fd 2>&$stderr_fd
rc=$?
exec {stdout_fd}>&-
exec {stderr_fd}>&-
out=$(cat /tmp/stdout.$$)
err=$(cat /tmp/stderr.$$)
rm -f /tmp/stdout.$$ /tmp/stderr.$$
echo "Exit: $rc | Out: $out | Err: $err"
5 — Named Pipes (FIFOs)
A named pipe (FIFO — First In, First Out) is like a regular pipe but with a name in the filesystem. Unlike a | pipeline, a FIFO can connect processes that aren't related, or that start at different times. A write to a FIFO blocks until a reader opens it, and vice versa — this built-in synchronisation makes FIFOs useful for coordinating producer/consumer workflows.
🐧 mkfifo — creating and using named pipes
# Create a named pipe
mkfifo /tmp/mypipe
ls -l /tmp/mypipe
prw-r--r-- 1 alice alice 0 Jun 9 09:00 /tmp/mypipe
# The 'p' in 'prw-' means "pipe"
# Terminal 1 — writer (blocks until reader connects)
echo "hello from the other side" > /tmp/mypipe
# Terminal 2 — reader
cat /tmp/mypipe
hello from the other side
# Remove when done — FIFOs persist until deleted
rm /tmp/mypipe
🐧 FIFO as a portable alternative to process substitution
# Process substitution equivalent using a FIFO (POSIX-portable)
fifo=$(mktemp -u) # generate a unique path without creating the file
mkfifo "$fifo"
trap 'rm -f "$fifo"' EXIT
# Start the producer in the background, writing to the FIFO
sort input.txt > "$fifo" &
# Consumer reads from the FIFO
uniq "$fifo"
# The FIFO synchronises them — uniq waits for sort to write
# ── Multi-producer / single-consumer ──────────────────────────
# Several background jobs write to the same FIFO; one reader collects
results=$(mktemp -u)
mkfifo "$results"
trap 'rm -f "$results"' EXIT
for host in host1 host2 host3; do
( ssh "$host" 'uptime'; echo "$host done" ) > "$results" &
done
# Collect all results — read until all writers are done
cat "$results"
wait
A key difference from anonymous pipes: FIFOs have names so they can be opened multiple times by independent processes. Each open/close pair is independent, but data flows one way and in order, just like any pipe.
Using a FIFO as a persistent input channel (log sink)
🐧 A log aggregator — multiple writers, one reader
#!/usr/bin/env bash
# log_server.sh — reads from a FIFO and writes timestamped lines to a log
set -euo pipefail
FIFO=/tmp/app_log.fifo
LOGFILE=/var/log/app_combined.log
mkfifo -m 600 "$FIFO"
trap 'rm -f "$FIFO"' EXIT INT TERM
# Open the FIFO for reading — keep it open even when no writer is connected
# Opening with O_RDWR (/dev/stdin trick) prevents EOF when last writer leaves
exec {fd}<> "$FIFO" # <> opens for both read and write
echo "Log server started. FIFO: $FIFO"
while IFS= read -r -u"$fd" line; do
printf '[%s] %s\n' "$(date -Iseconds)" "$line" >> "$LOGFILE"
done
# Writers send to the FIFO like this:
# echo "worker A: job complete" > /tmp/app_log.fifo
6 — Special Paths: /dev/stdin, /dev/fd/N, /dev/tcp
🐧 Shell-special paths for redirection
# /dev/stdin, /dev/stdout, /dev/stderr
# Symbolic names for FDs 0, 1, 2 — useful when a command expects a filename
wc -l /dev/stdin <<< "line 1\nline 2\nline 3"
3 /dev/stdin
# Useful when a command requires a filename but you have a string
md5sum /dev/stdin <<< "hash this"
# /dev/null — the bitbucket
# Redirect stderr to /dev/null to suppress error messages
ls /nonexistent 2> /dev/null
# Run a command silently (discard all output)
some_noisy_cmd &> /dev/null
# /dev/fd/N — access any open file descriptor by number
# Used automatically by process substitution: <(cmd) → /dev/fd/63
ls -la /dev/fd/ # see which FDs are open in the current shell
# /dev/tcp/HOST/PORT — raw TCP connection (bash built-in, no netcat needed)
# Opens a TCP socket; bash handles it as a regular file descriptor
exec 3<> /dev/tcp/example.com/80
# Send an HTTP request
printf 'GET / HTTP/1.0\r\nHost: example.com\r\n\r\n' >&3
# Read the response
cat <&3
exec 3<&-
# Simple port-open check using /dev/tcp
port_open() {
local host="$1" port="$2"
(exec 3<> /dev/tcp/"${host}/${port}") 2>/dev/null
}
if port_open "db.example.com" 5432; then
echo "Database port is open"
else
echo "Database unreachable"
fi
/dev/tcp is a bash feature, not a real filesystem path. It only works in bash (not sh, dash, or zsh). Some security-hardened systems disable it at compile time.7 — tee and Multi-Destination Output
tee reads from stdin and writes to both stdout and one or more files simultaneously. Combined with process substitution and output FDs, it's the glue that makes fan-out pipelines possible.
🐧 tee patterns — display AND capture, branching pipelines
# Basic: show output on terminal and save to a file
./long_running_job | tee job.log
# Append to existing file
./job | tee -a job.log
# ── Capture output into a variable while still showing it ─────
output=$(./job | tee /dev/tty)
# /dev/tty is the controlling terminal — writes bypass stdout redirect
# ── Fan out to multiple files ──────────────────────────────────
./job | tee copy1.log copy2.log copy3.log > /dev/null
# Writes to all three files; /dev/null discards the tee stdout
# ── Pipeline inspection — "T" into the middle of a pipe ───────
# See what's flowing through a pipeline at any point
cat input.txt \
| tee /tmp/after_cat.log \
| grep 'ERROR' \
| tee /tmp/after_grep.log \
| sort -u
# ── Use tee to copy a download to multiple destinations ────────
curl -sL https://example.com/bigfile.tar.gz \
| tee /backups/bigfile.tar.gz \
| tar -xz -C /opt/app/
# Saves the archive AND extracts it in one download pass
# ── Fan-out with different processing per branch ───────────────
./my_app | tee \
>(grep -i 'error\|warn' >> alerts.log) \
>(gzip -9 > full.log.gz) \
>(wc -l > line_count.txt) \
> /dev/null
8 — Quick Reference
Process substitution forms
| Syntax | Description | Where cmd runs |
|---|---|---|
<(cmd) | Provide cmd's stdout as a readable file path | Background subshell |
>(cmd) | Provide a writable file path that feeds cmd's stdin | Background subshell |
cmd1 | tee >(cmd2) | Send cmd1 output to both cmd2 and stdout | cmd2 in background |
Key exec redirection patterns
| Pattern | Effect |
|---|---|
exec > file | Redirect all script stdout to file from this point |
exec 2> file | Redirect all script stderr to file from this point |
exec 3>&1 | Save FD 1 (stdout) into FD 3 |
exec 1>&3 | Restore FD 1 from FD 3 |
exec 3>&- | Close FD 3 |
exec 4< file | Open file for reading on FD 4 |
exec {fd}> file | Open file on auto-chosen FD; number stored in $fd |
exec 3<> file | Open file for both reading and writing on FD 3 |
FIFO quick reference
| Command | Description |
|---|---|
mkfifo /path/to/fifo | Create a named pipe |
mkfifo -m 600 /path/to/fifo | Create with restricted permissions |
mktemp -u | Generate a unique path without creating a file (use with mkfifo) |
exec {fd}<> fifo | Keep FIFO open (prevents EOF when last writer disconnects) |
✏️ Exercises
These exercises require combining multiple techniques from this chapter. Resist reaching for temporary files unless explicitly building one — the goal is fluency with pipelines, process substitution, and persistent file descriptors.
Exercise 1
Write a script called
compare_dirs.sh that accepts two directory paths as arguments and reports three things: (1) files present in the first directory but not the second, (2) files present in the second but not the first, and (3) files present in both. Use comm with process substitution — no temporary files. Handle the case where either path doesn't exist.Hint:
comm requires sorted input. find DIR -maxdepth 1 -type f -printf '%f\n' | sort gives sorted filenames. Use comm -23 for only-in-first, comm -13 for only-in-second, comm -12 for in-both.Sample Solution
#!/usr/bin/env bash
# compare_dirs.sh DIR1 DIR2
set -euo pipefail
dir1="${1:?Usage: compare_dirs.sh DIR1 DIR2}"
dir2="${2:?Usage: compare_dirs.sh DIR1 DIR2}"
[[ -d "$dir1" ]] || { echo "Error: '$dir1' is not a directory" >&2; exit 1; }
[[ -d "$dir2" ]] || { echo "Error: '$dir2' is not a directory" >&2; exit 1; }
# Helper: sorted filenames in a directory (one per line)
sorted_files() {
find "$1" -maxdepth 1 -type f -printf '%f\n' | sort
}
printf '\033[1mComparing:\033[0m\n %s\n %s\n\n' "$dir1" "$dir2"
# Only in dir1
printf '\033[33mOnly in %s:\033[0m\n' "$dir1"
comm -23 <(sorted_files "$dir1") <(sorted_files "$dir2") \
| sed 's/^/ /'
# Only in dir2
printf '\033[33mOnly in %s:\033[0m\n' "$dir2"
comm -13 <(sorted_files "$dir1") <(sorted_files "$dir2") \
| sed 's/^/ /'
# In both
printf '\033[32mIn both:\033[0m\n'
comm -12 <(sorted_files "$dir1") <(sorted_files "$dir2") \
| sed 's/^/ /'
Exercise 2
Write a function called
run_logged() that executes any command passed to it and: (a) shows stdout in the terminal in real time, (b) simultaneously writes stdout to a timestamped log file, (c) captures stderr separately and prints a summary at the end if there were any errors, and (d) returns the original exit code of the command. Do not use temporary files for the main stdout flow — use process substitution and tee.Hint: use
tee >(timestamp_fn >> logfile) to simultaneously print and log. For stderr, redirect to a temp file (mktemp) so you can display it at the end. Use a subshell group to capture the exit code: { cmd; } ... ; rc=$?.Sample Solution
#!/usr/bin/env bash
set -uo pipefail
run_logged() {
local logfile="run_$(date +%Y%m%d_%H%M%S)_$$.log"
local errfile
errfile=$(mktemp)
local rc=0
# Timestamper: prepend HH:MM:SS to each line
_stamp() {
while IFS= read -r line; do
printf '[%s] %s\n' "$(date '+%T')" "$line"
done
}
printf '\033[36m[RUN]\033[0m %s\n' "$*"
printf '\033[2m[LOG] → %s\033[0m\n' "$logfile"
# Run command: stdout fans out to terminal + timestamped logfile
# stderr goes to errfile
"$@" \
2> "$errfile" \
| tee >(_stamp >> "$logfile") \
|| rc=$?
# If there was stderr output, show it
if [[ -s "$errfile" ]]; then
printf '\n\033[31m[STDERR]\033[0m\n' >&2
cat "$errfile" >&2
# Also append stderr to log
_stamp < "$errfile" >> "$logfile"
fi
rm -f "$errfile"
if (( rc == 0 )); then
printf '\033[32m[OK]\033[0m Exit 0\n'
else
printf '\033[31m[FAIL]\033[0m Exit %d\n' "$rc" >&2
fi
return $rc
}
# Test
run_logged ls -la /etc/hosts /nonexistent
Exercise 3
Write a script called
multi_log.sh that uses exec with auto-assigned file descriptors ({var}> file) to open three log files at the start — info.log, warn.log, error.log — and a function log LEVEL MESSAGE that writes to the appropriate file descriptor based on the level. The script should process a list of simulated events, close all file descriptors when done, and print a summary of how many lines went to each log. Do not reopen the files inside the loop.Hint: store the FD numbers in variables
INFO_FD, WARN_FD, ERROR_FD using exec {INFO_FD}> info.log. In the log function, use printf ... >&$INFO_FD etc. Use a case statement to dispatch on level. For the count, use wc -l on each file after closing the FDs.Sample Solution
#!/usr/bin/env bash
# multi_log.sh
set -euo pipefail
LOGDIR="$(mktemp -d)"
trap 'echo "Logs in: $LOGDIR"' EXIT
# Open three log files on auto-assigned FDs
exec {INFO_FD}> "${LOGDIR}/info.log"
exec {WARN_FD}> "${LOGDIR}/warn.log"
exec {ERROR_FD}> "${LOGDIR}/error.log"
log() {
local level="${1^^}" # uppercase the level
local msg="$2"
local ts
ts="$(date '+%H:%M:%S')"
case "$level" in
INFO) printf '[%s] INFO %s\n' "$ts" "$msg" >&$INFO_FD ;;
WARN) printf '[%s] WARN %s\n' "$ts" "$msg" >&$WARN_FD ;;
ERROR) printf '[%s] ERROR %s\n' "$ts" "$msg" >&$ERROR_FD ;;
*) printf '[%s] ?? %s\n' "$ts" "$msg" >&$INFO_FD ;;
esac
}
# Simulated events — in a real script this would be a processing loop
log INFO "Application starting"
log INFO "Loading configuration"
log WARN "Config file not found, using defaults"
log INFO "Connected to database"
log INFO "Processing 1000 records"
log WARN "Record 42 has missing field 'email'"
log ERROR "Failed to insert record 99: duplicate key"
log INFO "Batch complete"
log ERROR "Connection reset during final commit"
# Close all file descriptors
exec {INFO_FD}>&-
exec {WARN_FD}>&-
exec {ERROR_FD}>&-
# Summary
echo
printf "%-12s %s\n" "Log" "Lines"
printf '%.0s─' {1..20}; echo
for f in info warn error; do
n=$(wc -l < "${LOGDIR}/${f}.log")
printf "%-12s %d\n" "${f}.log" "$n"
done
Exercise 4
Write a script called
health_check.sh that checks whether a set of services are reachable using /dev/tcp. It should accept a list of host:port pairs (either as arguments or from a here-string), check each one with a configurable timeout, and produce a report showing each service as UP or DOWN with response time in milliseconds. Use a FIFO or process substitution to feed the list if it comes from a here-doc.Hint: use
( exec 3<> /dev/tcp/"$host"/"$port" ) 2>/dev/null in a subshell so the socket closes on exit. Capture $SECONDS or date +%s%N before and after for timing. For timeout, wrap the subshell in a background job and wait with a timeout loop or use timeout 2 bash -c "...".Sample Solution
#!/usr/bin/env bash
# health_check.sh [host:port ...]
set -uo pipefail
TIMEOUT=${TIMEOUT:-3}
check_service() {
local host="$1" port="$2"
local start_ns end_ns ms
start_ns=$(date '+%s%N')
if timeout "$TIMEOUT" bash -c \
"exec 3<> /dev/tcp/${host}/${port}" 2>/dev/null; then
end_ns=$(date '+%s%N')
ms=$(( (end_ns - start_ns) / 1000000 ))
printf ' \033[32m%-6s\033[0m %-30s %dms\n' "UP" "${host}:${port}" "$ms"
else
printf ' \033[31m%-6s\033[0m %-30s (timeout ${TIMEOUT}s)\n' "DOWN" "${host}:${port}"
fi
}
printf '\033[1mService Health Check\033[0m (timeout: %ss)\n' "$TIMEOUT"
printf '%.0s─' {1..50}; echo
if (( $# > 0 )); then
# Args mode: each arg is host:port
for svc in "$@"; do
host="${svc%:*}"
port="${svc##*:}"
check_service "$host" "$port"
done
else
# Stdin mode: read host:port from process substitution / pipe
while IFS=: read -r host port; do
[[ -z "$host" || "$host" == '#'* ]] && continue
check_service "$host" "$port"
done
fi
# Usage examples:
# ./health_check.sh google.com:443 github.com:22 localhost:5432
#
# Or feed a list via here-string:
# ./health_check.sh <<< $'google.com:80\ngoogle.com:443\nlocalhost:5432'
#
# Or via process substitution:
# ./health_check.sh < <(grep -v '^#' services.txt)