Regular Expressions in Bash

Chapter 6 — Grouping, Alternation, and Backreferences

Grouping with Parentheses

A group is a portion of a regex wrapped in parentheses (...). Groups serve two purposes: they allow a quantifier to apply to more than one character at a time, and they capture the text matched by the group so it can be reused later.

Groups as a unit for quantifiers

Without a group, a quantifier applies only to the immediately preceding character or class. With a group, it applies to everything inside the parentheses.

Copy# Without grouping: + applies only to 'p'
echo -e "hap\nhapp\nhappp\nhappy" | grep -E 'hap+'
hap
happ
happp
happy   # all match — + only requires one or more 'p'

# With grouping: + applies to the whole "ha" unit
echo -e "ha\nhaha\nhahaha\nhello" | grep -E '(ha)+'
ha
haha
hahaha
# "hello" has no match — does not contain "ha"

# Repeated two-character pattern
grep -E '([0-9]{1,3}\.){3}[0-9]{1,3}' file.txt   # IPv4: three "NNN." groups then NNN

# Optional suffix as a group
grep -E 'colou(r|rs|red|ring)' text.txt
# cleaner: "colour" optionally followed by a suffix group
grep -E 'colour(s|ed|ing)?' text.txt

BRE requires backslashes around parentheses

Copy# ERE: ( ) are grouping metacharacters
grep -E '(foo)+' file.txt

# BRE: ( ) are LITERALS — you need \( \) for grouping
grep '\(foo\)\+' file.txt

# BRE: a bare ( ) matches the literal characters ( and )
grep '(foo)' file.txt   # matches the string "(foo)" literally!

Alternation with |

The pipe character | means OR — the pattern matches if either the left side or the right side matches. It has the lowest precedence of all regex operators: everything to the left of | is one alternative, everything to the right is another. Use groups to limit the scope of alternation.

Copy# Match "cat" or "dog"
grep -E 'cat|dog' file.txt

# Match "ERROR" or "FATAL" or "CRITICAL" log levels
grep -E 'ERROR|FATAL|CRITICAL' app.log

# Alternation without grouping: ^ applies only to left alternative
grep -E '^cat|dog' file.txt
# Matches: lines STARTING with "cat"  OR  lines containing "dog" anywhere
# NOT: lines starting with "cat" or "dog"

# Fix: use a group to limit the alternation scope
grep -E '^(cat|dog)' file.txt
# NOW: lines starting with "cat" OR starting with "dog"

Copy# Alternation inside a larger pattern
grep -E '(Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec) [0-9]{1,2}' dates.txt

# Match http or https or ftp URLs
grep -E '(https?|ftp)://[^[:space:]]+' file.txt

# Match common image extensions
grep -E '\.(jpg|jpeg|png|gif|webp|svg)$' filelist.txt

# BRE alternation uses \| (GNU extension — not POSIX)
grep 'cat\|dog' file.txt
# On strictly POSIX systems without GNU grep, alternation requires -E

Capturing Groups and Backreferences

Every group enclosed in (...) — or \(...\) in BRE — is a capturing group. The regex engine stores the text matched by each group and makes it available as a numbered backreference: \1 for the first group, \2 for the second, and so on up to \9.

Backreferences in sed replacements

Copy# Reformat ISO date YYYY-MM-DD → DD/MM/YYYY
sed -E 's/([0-9]{4})-([0-9]{2})-([0-9]{2})/\3\/\2\/\1/' <<< "2026-06-11"
11/06/2026

# Swap first and last name (comma-separated)
sed -E 's/([^,]+), ([^,]+)/\2 \1/' <<< "Smith, John"
John Smith

# Wrap every number in brackets
sed -E 's/([0-9]+)/[\1]/g' <<< "port 8080 timeout 30"
port [8080] timeout [30]

# Extract just the filename from a full path
sed -E 's|.*/([^/]+)$|\1|' <<< "/home/user/documents/report.pdf"
report.pdf

# Surround a key=value pair: key → "key"
sed -E 's/^([^=]+)=/"\1"=/' <<< "username=alice"
"username"=alice

# BRE version of the date reformat (backslashes on groups)
sed 's/\([0-9]\{4\}\)-\([0-9]\{2\}\)-\([0-9]\{2\}\)/\3\/\2\/\1/' <<< "2026-06-11"
11/06/2026

Backreferences in patterns — matching repeated text

Backreferences can also appear inside the pattern itself (not just the replacement). This forces the engine to match the same text a second time.

Copy# Match a line where the same word appears twice consecutively
echo -e "the the cat\nthe cat\nhello hello world" | grep -E '\b(\w+) \1\b'
the the cat
hello hello world

# Find duplicate words and remove them
sed -E 's/\b(\w+) \1\b/\1/g' <<< "the the quick brown fox fox"
the quick brown fox

# Match a repeated character (e.g. doubled letters)
grep -E '([a-z])\1' words.txt   # words with a doubled letter: "book", "free"

# Match a string that starts and ends with the same character
grep -E '^(.).*\1$' words.txt   # e.g. "radar", "level", "noon"

# Match an HTML tag and its matching closing tag (simplified)
grep -E '<([a-z]+)>.*</\1>' index.html   # <b>...</b>, <em>...</em>

Backreferences in Bash [[ =~ ]] with BASH_REMATCH

When you use [[ =~ ]] in Bash, the shell populates the BASH_REMATCH array with the captured groups. BASH_REMATCH[0] holds the entire match; BASH_REMATCH[1] holds group 1; BASH_REMATCH[2] holds group 2; and so on.

Copy# Extract year, month, day from a date string
date_str="2026-06-11"
if [[ "$date_str" =~ ^([0-9]{4})-([0-9]{2})-([0-9]{2})$ ]]; then
    echo "Full match : ${BASH_REMATCH[0]}"
    echo "Year       : ${BASH_REMATCH[1]}"
    echo "Month      : ${BASH_REMATCH[2]}"
    echo "Day        : ${BASH_REMATCH[3]}"
fi
Full match : 2026-06-11
Year       : 2026
Month      : 06
Day        : 11

# Parse a key=value pair
line="max_connections=100"
[[ "$line" =~ ^([^=]+)=(.+)$ ]] && {
    echo "Key:   ${BASH_REMATCH[1]}"
    echo "Value: ${BASH_REMATCH[2]}"
}
Key:   max_connections
Value: 100

# Extract protocol and host from a URL
url="https://api.example.com/v2/users"
[[ "$url" =~ ^(https?)://([^/]+)(/.*)$ ]] && {
    echo "Protocol : ${BASH_REMATCH[1]}"
    echo "Host     : ${BASH_REMATCH[2]}"
    echo "Path     : ${BASH_REMATCH[3]}"
}
Protocol : https
Host     : api.example.com
Path     : /v2/users

Nested and Numbered Groups

Groups are numbered left to right by the position of their opening parenthesis. Nested groups count from the outermost in.

Copy# Groups numbered by opening parenthesis, left to right
# Pattern: ((a)(b)) — three groups
#           ^  ^  ^
#           1  2  3
str="ab"
[[ "$str" =~ ((a)(b)) ]]
echo "${BASH_REMATCH[0]}"  # ab  — full match
echo "${BASH_REMATCH[1]}"  # ab  — group 1: outer (ab)
echo "${BASH_REMATCH[2]}"  # a   — group 2: inner (a)
echo "${BASH_REMATCH[3]}"  # b   — group 3: inner (b)

# Practical nested groups: match a version like "v1.2.3"
ver="v1.2.3"
[[ "$ver" =~ ^v([0-9]+)(\.([0-9]+))?(\.([0-9]+))?$ ]]
# Group 1: major (1)   Group 3: minor (2)   Group 5: patch (3)
echo "major=${BASH_REMATCH[1]} minor=${BASH_REMATCH[3]} patch=${BASH_REMATCH[5]}"
major=1 minor=2 patch=3

Non-Capturing Groups (?:...)

Sometimes you need a group for alternation or quantification but don't want it to capture — perhaps because you are already using \1–\9 for other groups and don't want to waste a slot. Non-capturing groups use (?:...) syntax. They work in ERE and PCRE but are not available in BRE.

Copy# Capturing group wastes \1 on the protocol — we only care about the host
sed -E 's|(https?://)([^/]+)|\2|' <<< "https://example.com/path"
example.com/path   # \2 is the host

# Non-capturing group: (?:https?://) groups but doesn't capture
# so the host becomes \1 instead of \2
sed -E 's|(?:https?://)([^/]+)|\1|' <<< "https://example.com/path"
example.com/path   # \1 is now the host — cleaner numbering

# Non-capturing group for alternation without wasting a capture slot
grep -E '(?:cat|dog) food' file.txt   # group needed for alternation, not capture

# In [[ =~ ]] — non-capturing groups work fine
[[ "$url" =~ ^(?:https?|ftp)://(.+)$ ]]
echo "rest of URL: ${BASH_REMATCH[1]}"   # \1 is the path, not the protocol

The & Replacement Special

In sed replacements, & (ampersand) refers to the entire matched string — the full match, not just a captured group. It is a shorthand for "whatever was matched".

Copy# Wrap the entire match in brackets
sed -E 's/[0-9]+/[&]/g' <<< "order 42 total 199"
order [42] total [199]

# Quote every word
sed -E 's/[[:alpha:]]+/"&"/g' <<< "hello world"
"hello" "world"

# Prepend a label to every matched IP address
sed -E 's/[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}/IP:&/g' log.txt

# & is equivalent to \0 (the whole match) in some tools
# In sed: & means the whole match; \1 means group 1
sed -E 's/([0-9]+)/num=\1 raw=&/' <<< "value 42"
value num=42 raw=42   # \1 and & are the same here (only one group)

BRE vs ERE — Grouping and Alternation Summary

Practical Real-World Recipes

Log and data extraction

Copy# Extract the HTTP status code from an Apache log line
sed -E 's/.*" ([0-9]{3}) .*/\1/' access.log

# Extract timestamp and log level from structured log
# Input: [2026-06-11 14:32:05] ERROR Something went wrong
[[ "$line" =~ ^\[([0-9-]+ [0-9:]+)\] ([A-Z]+) (.+)$ ]] && {
    echo "time=${BASH_REMATCH[1]} level=${BASH_REMATCH[2]}"
    echo "msg=${BASH_REMATCH[3]}"
}

# Reformat CSV date column: MM/DD/YYYY → YYYY-MM-DD
sed -E 's|([0-9]{2})/([0-9]{2})/([0-9]{4})|\3-\1-\2|g' report.csv

Code manipulation

Copy# Rename a function: getUser → fetchUser (whole word, all files)
sed -i -E 's/\bgetUser\b/fetchUser/g' src/*.js

# Convert snake_case identifiers to camelCase
# Approach: loop sed until no more underscores-with-next-letter remain
echo "get_user_name" | sed -E 's/_([a-z])/\u\1/g'
getUserName   # \u uppercases the next char (GNU sed extension)

# Add quotes around all unquoted values in key=value format
sed -E 's/^([^=]+)=([^"].*)$/\1="\2"/' config.env

Parsing shell script output into variables

Copyparse_df_line() {
    # Parse: /dev/sda1   50G   20G   30G  40%  /home
    local line="$1"
    if [[ "$line" =~ ^([^[:space:]]+)[[:space:]]+[^[:space:]]+[[:space:]]+[^[:space:]]+[[:space:]]+[^[:space:]]+[[:space:]]+([0-9]+)%[[:space:]]+(.+)$ ]]; then
        echo "device=${BASH_REMATCH[1]}"
        echo "use%=${BASH_REMATCH[2]}"
        echo "mount=${BASH_REMATCH[3]}"
    fi
}
# Alert if any filesystem is over 90% used
df -h | grep -E '([0-9]+)%' | while read -r line; do
    [[ "$line" =~ ([0-9]+)% ]] && [[ "${BASH_REMATCH[1]}" -gt 90 ]] &&
        echo "WARNING: $line"
done

Quick Reference — Chapter 6