Regular Expressions in Bash

Chapter 12 — Your Regex Toolkit: Patterns, Testing, and Real Pipelines

Building a Personal Pattern Library

The most productive regex habit is maintaining a library of tested, annotated patterns you reach for repeatedly. Instead of rewriting "what was that IPv4 pattern again?" from scratch, you keep one authoritative version that you know works correctly — including the edge cases that bit you before.

Below is a complete regex_lib.sh — source it in any script with source regex_lib.sh.

#!/usr/bin/env bash
# regex_lib.sh — tested, reusable regex patterns and validation functions
# Usage: source regex_lib.sh

# ── Anchored validation patterns ─────────────────────────────────────────
RE_INTEGER='^-?[0-9]+$'
RE_POS_INT='^[1-9][0-9]*$'
RE_FLOAT='^-?[0-9]+(\.[0-9]+)?$'
RE_HEX='^(0[xX])?[0-9a-fA-F]+$'

RE_IPV4='^([0-9]{1,3}\.){3}[0-9]{1,3}$'
RE_IPV6='^([0-9a-fA-F]{0,4}:){2,7}[0-9a-fA-F]{0,4}$'
RE_CIDR='^([0-9]{1,3}\.){3}[0-9]{1,3}/([0-9]|[1-2][0-9]|3[0-2])$'
RE_MAC='^([0-9a-fA-F]{2}[:-]){5}[0-9a-fA-F]{2}$'

RE_EMAIL='^[[:alnum:]._%+-]+@[[:alnum:].-]+\.[[:alpha:]]{2,}$'
RE_URL='^https?://[[:alnum:]._~:/?#\[\]@!$&()*+,;=%-]+'
RE_HOSTNAME='^[a-zA-Z0-9]([a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?(\.[a-zA-Z0-9]([a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?)*$'

RE_ISO_DATE='^[0-9]{4}-(0[1-9]|1[0-2])-(0[1-9]|[12][0-9]|3[01])$'
RE_TIME_24='^([01][0-9]|2[0-3]):[0-5][0-9](:[0-5][0-9])?$'
RE_SEMVER='^[0-9]+\.[0-9]+\.[0-9]+(-[[:alnum:].-]+)?(\+[[:alnum:].-]+)?$'

RE_SLUG='^[a-z0-9]+(-[a-z0-9]+)*$'
RE_HEX_COLOUR='^#([0-9a-fA-F]{3}|[0-9a-fA-F]{6})$'
RE_UUID='^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$'
RE_BASE64='^[A-Za-z0-9+/]+={0,2}$'

# ── Validation functions ──────────────────────────────────────────────────
# Each returns 0 (true) if valid, 1 (false) if not

is_integer()    { [[ "$1" =~ $RE_INTEGER   ]]; }
is_pos_int()    { [[ "$1" =~ $RE_POS_INT   ]]; }
is_float()      { [[ "$1" =~ $RE_FLOAT     ]]; }
is_email()      { [[ "$1" =~ $RE_EMAIL     ]]; }
is_url()        { [[ "$1" =~ $RE_URL       ]]; }
is_hostname()   { [[ "$1" =~ $RE_HOSTNAME  ]]; }
is_iso_date()   { [[ "$1" =~ $RE_ISO_DATE ]]; }
is_semver()     { [[ "$1" =~ $RE_SEMVER    ]]; }
is_uuid()       { [[ "$1" =~ $RE_UUID      ]]; }
is_hex_colour() { [[ "$1" =~ $RE_HEX_COLOUR ]]; }
is_slug()       { [[ "$1" =~ $RE_SLUG      ]]; }

is_ipv4() {
    [[ "$1" =~ $RE_IPV4 ]] || return 1
    local IFS='.' octet
    for octet in $1; do
        (( octet >= 0 && octet <= 255 )) || return 1
    done
}

A Regex Test Harness

Every pattern in your library should have tests. This harness lets you declare positive cases (must match) and negative cases (must not match), then run them all and report failures — just like a unit test suite, but for regex.

#!/usr/bin/env bash
# test_regex.sh — regex unit test harness
source regex_lib.sh

PASS=0; FAIL=0

assert_match() {       # assert_match "label" pattern "value"
    local label="$1" re="$2" val="$3"
    if [[ "$val" =~ $re ]]; then
        printf "  \e[32mPASS\e[0m  %s: '%s'\n" "$label" "$val"
        (( PASS++ ))
    else
        printf "  \e[31mFAIL\e[0m  %s: '%s' should match /%s/\n" \
               "$label" "$val" "$re"
        (( FAIL++ ))
    fi
}

assert_no_match() {   # assert_no_match "label" pattern "value"
    local label="$1" re="$2" val="$3"
    if ! [[ "$val" =~ $re ]]; then
        printf "  \e[32mPASS\e[0m  %s: '%s' correctly rejected\n" "$label" "$val"
        (( PASS++ ))
    else
        printf "  \e[31mFAIL\e[0m  %s: '%s' should NOT match /%s/\n" \
               "$label" "$val" "$re"
        (( FAIL++ ))
    fi
}

# ── Tests ─────────────────────────────────────────────────────────────────
echo "=== RE_ISO_DATE ==="
assert_match    "valid date"         "$RE_ISO_DATE"  "2026-06-11"
assert_match    "leap day"           "$RE_ISO_DATE"  "2024-02-29"
assert_no_match "month 13"          "$RE_ISO_DATE"  "2026-13-01"
assert_no_match "day 00"            "$RE_ISO_DATE"  "2026-06-00"
assert_no_match "US format"         "$RE_ISO_DATE"  "06/11/2026"
assert_no_match "partial"           "$RE_ISO_DATE"  "2026-06"

echo "=== RE_EMAIL ==="
assert_match    "simple"             "$RE_EMAIL"     "user@example.com"
assert_match    "plus tag"           "$RE_EMAIL"     "user+tag@example.co.uk"
assert_no_match "no @"              "$RE_EMAIL"     "userexample.com"
assert_no_match "no TLD"            "$RE_EMAIL"     "user@example"
assert_no_match "spaces"            "$RE_EMAIL"     "user @example.com"

echo "=== is_ipv4 (with octet check) ==="
assert_match    "valid"              "$RE_IPV4"      "192.168.1.100"
assert_no_match "octet 256 (regex)" "$RE_IPV4"      "999.1.1.1"

# Summary
echo
printf "Results: \e[32m%d passed\e[0m  \e[31m%d failed\e[0m\n" "$PASS" "$FAIL"
(( FAIL == 0 ))   # exit 0 if all passed, 1 if any failed

Sample output:

=== RE_ISO_DATE ===
  PASS  valid date: '2026-06-11'
  PASS  leap day: '2024-02-29'
  PASS  month 13 correctly rejected
  PASS  day 00 correctly rejected
  PASS  US format correctly rejected
  PASS  partial correctly rejected
=== RE_EMAIL ===
  PASS  simple: 'user@example.com'
  PASS  plus tag: 'user+tag@example.co.uk'
  PASS  no @ correctly rejected
  PASS  no TLD correctly rejected
  FAIL  spaces: 'user @example.com' should NOT match /^[[:alnum:]...

Results: 10 passed  1 failed

Tool Picker — Choosing the Right Tool

Do you need to match/filter lines from a file or stream?

→ yes, single pattern

grep 'pattern' file

→ yes, ERE syntax (|, +, ?)

grep -E 'pat1|pat2' file

→ yes, extract only the matched text

grep -oE 'pattern' file

→ yes, PCRE (lookahead, lazy, named groups)

grep -P 'pattern' file — or — perl -ne 'print if /pat/'

Do you need to transform / substitute text?

→ simple find-and-replace on every line

sed 's/old/new/g' file

→ ERE with capture groups, case modifiers

sed -E 's/(group)/\U\1/g' file

→ PCRE substitution (lazy, lookahead in pattern)

perl -pe 's/pattern/replacement/g' file

Do you need to process fields, aggregate, or do arithmetic?

→ split on delimiter, match per-field

awk -F, '$3 ~ /pattern/ { print $1 }'

→ count, sum, group matches

awk '/ERROR/ { count++ } END { print count }'

Are you validating a single variable inside a script?

→ no subprocess needed

[[ "$var" =~ ^pattern$ ]]

Real Pipeline Scripts

Script 1 — Log report: errors by hour

#!/usr/bin/env bash
# hourly_errors.sh — count ERROR lines per hour from an app log
# Input format: "2026-06-11 14:32:00 ERROR message..."

awk '$3 == "ERROR" {
    match($2, /^([0-9]{2})/, t)
    hours[t[1]]++
}
END {
    print "Hour  Errors"
    print "----  ------"
    for (h = 0; h < 24; h++) {
        if (h in hours)
            printf "%02d:00  %d\n", h, hours[h]
    }
}' "${1:-app.log}"

Script 2 — Deploy guard: validate all inputs before proceeding

#!/usr/bin/env bash
# deploy.sh — validate arguments before running deployment
source regex_lib.sh

die() { echo "ERROR: $*" >&2; exit 1; }

VERSION="$1"
HOST="$2"
PORT="${3:-8080}"

is_semver  "$VERSION" || die "'$VERSION' is not a valid version (need MAJOR.MINOR.PATCH)"
is_hostname "$HOST"   || is_ipv4 "$HOST" || die "'$HOST' is not a valid host"
is_pos_int "$PORT"   || die "'$PORT' is not a valid port"
(( PORT <= 65535 ))  || die "Port $PORT exceeds 65535"

echo "Deploying v$VERSION to $HOST:$PORT ..."
# ... rest of deployment

Script 3 — Config auditor: scan for insecure settings

#!/usr/bin/env bash
# audit_config.sh — grep multiple files for security red flags

TARGET_DIR="${1:-/etc}"
ISSUES=0

check() {
    local label="$1" pattern="$2"
    shift 2
    local hits
    hits=$(grep -rnE --include='*.conf' --include='*.cfg' \
              "$pattern" "$@" 2>/dev/null)
    if [[ -n "$hits" ]]; then
        printf "\e[33mWARN\e[0m  %s\n%s\n\n" "$label" "$hits"
        (( ISSUES++ ))
    fi
}

check "PermitRootLogin enabled"     '^PermitRootLogin\s+yes'        "$TARGET_DIR"
check "PasswordAuthentication on"   '^PasswordAuthentication\s+yes' "$TARGET_DIR"
check "Plaintext password in config" '(?i)password\s*=\s*\S+'        "$TARGET_DIR"
check "Hardcoded API key"            '(?i)api.?key\s*=\s*["\x27]\S+'  "$TARGET_DIR"

(( ISSUES == 0 )) && echo "No issues found."
exit $(( ISSUES > 0 ))

Script 4 — Access log analyser

#!/usr/bin/env bash
# access_summary.sh — parse nginx/apache combined log format
# Format: IP - - [date] "METHOD /path HTTP/x.x" STATUS bytes

awk '{
    # Extract status code (field 9) and request path
    status = $9
    match($7, /^\/[^?]*/, m)
    path = m[0]

    status_count[status]++
    if (status ~ /^[45]/) {
        errors[path]++
    }
    bytes += $10
}
END {
    print "=== Status codes ==="
    for (s in status_count)
        printf "  %s  %d\n", s, status_count[s]

    print "\n=== Top error paths ==="
    for (p in errors)
        printf "  %d  %s\n", errors[p], p
    | "sort -rn | head -10"

    printf "\n=== Total bytes: %.1f MB ===\n", bytes / 1024 / 1024
}' "${1:-/var/log/nginx/access.log}"

Complete Course Reference Card

Anchors

^Start of line

$End of line

\bWord boundary (GNU)

\< \>Word boundary (POSIX)

^...$Whole-line / whole-string match

Quantifiers

* + ?0+, 1+, 0-or-1 (greedy)

{n} {n,} {n,m}Exact, at-least, between

*? +? ??Lazy (PCRE only)

BRE: \+ \?ERE + and ? in BRE context

Character Classes

[abc]Literal set

[a-z]Range

[^abc]Negated set

[:alpha:] [:digit:]POSIX classes (inside [])

\w \d \sGNU/PCRE shorthands

.Any char except newline

Groups and Alternation

(...)Capturing group — ERE/PCRE

$...$Capturing group — BRE

(?:...)Non-capturing group — PCRE

(?P<n>...)Named group — PCRE

| vs \|Alternation ERE / BRE

PCRE Assertions

(?=pat)Positive lookahead

(?!pat)Negative lookahead

(?<=pat)Positive lookbehind

(?<!pat)Negative lookbehind

(?>...)Atomic group — no backtrack

Back-references

\1 … \9Group back-ref in pattern or replacement

&Whole match in sed/awk replacement

\u \U \l \L \ECase modifiers in GNU sed replacement

BASH_REMATCH[N]Capture group N after [[ =~ ]]

grep Flags

-E -F -PERE / fixed-string / PCRE mode

-o -n -cOnly match / line nums / count

-l -L -vFiles with / without / invert

-r -i -wRecursive / case-insensitive / word

-q -A -B -CQuiet / after / before / context

sed / awk / Bash

sed -E 's/pat/rep/g'ERE substitution, all occurrences

sed -i.bak '...'In-place edit with backup

awk '$N ~ /pat/'Field N matches regex

awk 'gsub(/p/,"r")'Replace all matches in $0

[[ "$v" =~ ^pat$ ]]Native Bash ERE test, no subprocess

Flavour Comparison — The One Table

Feature	BRE	ERE	PCRE
One or more	`\+`	`+`	`+`
Zero or one	`\?`	`?`	`?`
Grouping	`$...$`	`(...)`	`(...)`
Alternation	`\\|`	`\|`	`\|`
Intervals `{n,m}`	`\{n,m\}`	`{n,m}`	`{n,m}`
Lazy quantifiers	No	No	`*? +? ??`
Non-capturing group	No	No	`(?:...)`
Lookahead / Lookbehind	No	No	`(?=) (?<=)`
Named groups	No	No	`(?P<n>...)`
Tools (typical)	grep, sed (default)	grep -E, sed -E, awk	grep -P, perl

Chapter Index

01What Are Regular Expressions — engines, flavours, first patterns

02Literals, the Dot, and Escaping — shell quoting, grep -F

03Anchors — ^, $, \b, \<\>, whole-line matching

04Character Classes — ranges, POSIX, \w \d \s shorthands

05Quantifiers — *, +, ?, {n,m}, greedy rules, ERE vs BRE

06Groups, Alternation, Back-references — BASH_REMATCH, &, \1–\9

07grep and egrep in Depth — all flags, -o, -F, exit codes, pipelines

08sed and Regex — s flags, case modifiers, address targeting

09Regex in awk — ~, !~, match(), gsub(), gensub(), ERE

10Bash [[ =~ ]] — BASH_REMATCH, quoting rules, validation library

11PCRE — lookahead, lookbehind, lazy, named groups, perl -ne

12Your Toolkit — pattern library, test harness, real pipeline scripts

Regular Expressions in Bash — Complete 12 chapters · grep · sed · awk · [[ =~ ]] · PCRE BRE · ERE · PCRE · anchors · quantifiers · character classes · groups · back-references · lookahead · lookbehind · lazy matching · named groups · BASH_REMATCH · validation library · test harness