Regular Expressions in Bash

Chapter 8 — sed and Regex in Depth

The sed Substitute Command

The s command is the heart of sed. It finds a regex match on a line and replaces it with a replacement string. Understanding every part of its syntax — address, pattern, replacement, and flags — is what makes sed a powerful text transformation engine.

• Address — which lines to operate on (optional; if omitted, all lines)
• Pattern — BRE by default; ERE with sed -E
• Replacement — literal text plus special tokens (&, \1–\9, case modifiers)
• Flags — modify how the substitution behaves (g, i, p, w, Nth)
• Delimiter — / is conventional but any character works: s|/usr|/opt|g

BRE vs ERE in sed

By default sed uses BRE. The -E flag (GNU sed; also -r on some systems) switches to ERE — the same extended syntax that grep -E uses.

Copy# BRE (default) — grouping and + need backslashes
sed 's/\(foo\)\+/bar/g' file.txt

# ERE with -E — cleaner syntax, + and () work directly
sed -E 's/(foo)+/bar/g' file.txt

# BRE alternation needs \|
sed 's/cat\|dog/pet/g' file.txt

# ERE alternation — just |
sed -E 's/cat|dog/pet/g' file.txt

Substitution Flags

Copy# g: replace all occurrences (default replaces only the first)
echo "aaa bbb aaa" | sed 's/aaa/xxx/'
xxx bbb aaa        # only first replaced
echo "aaa bbb aaa" | sed 's/aaa/xxx/g'
xxx bbb xxx        # all replaced

# i: case-insensitive replacement
echo "Error ERROR error" | sed 's/error/FAULT/gi'
FAULT FAULT FAULT

# 2: replace only the second occurrence
echo "cat and cat and cat" | sed 's/cat/dog/2'
cat and dog and cat

# 2g: replace from the second occurrence onwards
echo "cat and cat and cat" | sed 's/cat/dog/2g'
cat and dog and dog

# p with -n: print ONLY lines where a substitution was made
sed -n 's/ERROR/FAULT/p' app.log

# w: write substituted lines to a separate file
sed 's/ERROR/FAULT/gw /tmp/faults.txt' app.log

# e: replace line with output of a shell command (GNU sed only)
echo "date" | sed 's/.*/date/e'         # replaces "date" with the output of date(1)

Replacement Special Tokens

Copy# &: wrap the entire match in brackets
echo "hello world" | sed 's/[a-z]*/[&]/g'
[hello] [world]

# \1 \2: reorder captured groups (date reformatting)
echo "2026-06-11" | sed -E 's/([0-9]{4})-([0-9]{2})-([0-9]{2})/\3\/\2\/\1/'
11/06/2026

# Swap first and last name
echo "Smith, John" | sed -E 's/([A-Za-z]+), ([A-Za-z]+)/\2 \1/'
John Smith

# Quote every word
echo "one two three" | sed 's/[^ ]*/"\&"/g'
"one" "two" "three"

# Insert a newline: split "key=value" into two lines
echo "name=Alice" | sed 's/=/\n/'
name
Alice

# Escape & and \ when they must be literal in replacement
echo "foo" | sed 's/foo/a\&b/'     # output: a&b  (\& = literal &)
echo "foo" | sed 's/foo/a\\b/'     # output: a\b  (\\ = literal \)

Case Modifier Sequences (GNU sed)

GNU sed supports case-conversion sequences in the replacement string. These are not available in POSIX sed or BSD sed.

Copy# Title-case: capitalise the first letter of every word (BRE)
echo "the quick brown fox" | sed 's/\b\([a-z]\)/\u\1/g'
The Quick Brown Fox

# Title-case with ERE
echo "the quick brown fox" | sed -E 's/\b([a-z])/\u\1/g'
The Quick Brown Fox

# SCREAMING_SNAKE_CASE from lowercase
echo "my variable name" | sed 's/ /_/g; s/.*/\U&/'
MY_VARIABLE_NAME

# Normalise HTTP method names (mixed case → uppercase)
echo "method: Get" | sed -E 's/(GET|POST|PUT|DELETE|PATCH)/\U\1/gi'
method: GET

# camelCase to snake_case: insert _ before each uppercase letter then lowercase all
echo "camelCaseVariable" | sed 's/\([A-Z]\)/_\L\1/g'
camel_case_variable

Address-Targeted Substitution

Addresses restrict which lines a sed command applies to. This is one of sed's most powerful features — you can combine regex addresses with substitution commands to perform context-aware replacements.

Copy# Line number addresses
sed '1s/^/# File header\n/' file.txt          # add comment before line 1
sed '$s/$/\n# End of file/' file.txt           # add comment after last line
sed '2,5s/^/    /' file.txt                   # indent lines 2–5

# Pattern address: substitute only on lines matching a condition
sed '/^Port/s/22/2222/' /etc/ssh/sshd_config  # change port only on Port lines
sed '/ERROR/s/$/  <<< ALERT/' app.log          # append ALERT to error lines

# Pattern range: substitute within a block
sed '/\[section\]/,/\[end\]/s/^ *//' config.ini # strip leading spaces in section
sed '/^---/,/^---/s/\bFoo\b/Bar/g' file.md    # replace only inside front-matter

# Negated address: substitute on every line EXCEPT those matching
sed '/^#/!s/foo/bar/g' script.sh             # skip comment lines
sed '/^$/!s/^/  /' file.txt                  # indent all non-empty lines

# Step address: number every other line (GNU sed)
sed '1~2s/^/> /' file.txt                   # prefix odd lines with "> "

# Combined: two addresses + substitution in a block
sed '/^BEGIN/,/^END/ { s/\bold\b/new/g; s/foo/bar/g }' file.txt

Alternate Delimiters

The delimiter does not have to be /. Any punctuation character can be used. This avoids the "leaning toothpick" problem when patterns or replacements contain slashes.

Copy# Path substitution — using | as delimiter avoids escaping every /
sed 's|/usr/local|/opt|g' Makefile

# Compare: using / would require escaping all the slashes
sed 's/\/usr\/local/\/opt/g' Makefile   # harder to read

# URL rewriting with # as delimiter
sed 's#https://old.example.com#https://new.example.com#g' links.txt

# Pattern address with alternate delimiter — use \%..%
sed '\%/usr/local%s/local/share/' config.txt

# The rule: whatever character follows s is the delimiter
sed 's,foo,bar,g' file.txt     # comma as delimiter
sed 's@foo@bar@g' file.txt     # @ as delimiter

Practical sed Recipes

Config file management

Copy# Set a key to a new value (handles both "key value" and "key=value" forms)
sed -E 's/^(MaxSessions\s*)=?\s*.*/\1= 20/' sshd_config

# Uncomment a setting: remove leading # from lines matching a key
sed '/^#.*Port /s/^#//' sshd_config

# Comment out a setting
sed 's/^PermitRootLogin/# PermitRootLogin/' sshd_config

# In-place edit with backup (GNU sed)
sed -i.bak 's/^Port 22/Port 2222/' /etc/ssh/sshd_config

Data reformatting

Copy# ISO date (YYYY-MM-DD) to US format (MM/DD/YYYY)
sed -E 's/([0-9]{4})-([0-9]{2})-([0-9]{2})/\2\/\3\/\1/g' dates.txt

# CSV: quote unquoted fields (add quotes around each comma-separated value)
sed -E 's/([^,]+)/"\1"/g' data.csv

# Remove ANSI colour escape codes from terminal output
sed -E 's/\x1B\[[0-9;]*[mK]//g' coloured.log

# Normalise multiple spaces to one
sed 's/  */ /g' file.txt           # BRE: two or more spaces → one space
sed -E 's/ {2,}/ /g' file.txt     # ERE: cleaner syntax

# Strip leading and trailing whitespace
sed 's/^[[:space:]]*//; s/[[:space:]]*$//' file.txt

# Extract value from "key: value" format (print only the value)
sed -n '/^Name:/s/^Name:[[:space:]]*//'p record.txt

Log and source code manipulation

Copy# Mask sensitive data: replace password values
sed -E 's/(password=)[^ &]*/\1[REDACTED]/gi' app.log

# Add line numbers as prefix
sed = file.txt | sed 'N; s/\n/\t/'

# Delete blank lines
sed '/^[[:space:]]*$/d' file.txt

# Delete comments (lines starting with #)
sed '/^[[:space:]]*#/d' config.txt

# Rename a variable across a source file
sed -i 's/\bconnection_timeout\b/connect_timeout/g' config.py

Portability — GNU sed vs BSD sed

Quick Reference — Chapter 8