Regular Expressions in Bash

Chapter 11 — PCRE: Lookahead, Lookbehind, and Advanced Features

What Is PCRE?

PCRE — Perl-Compatible Regular Expressions — is a regex engine originally written to match Perl 5's regex behaviour. It extends ERE with features that are impossible in BRE or ERE: lookahead and lookbehind assertions, lazy quantifiers, named capture groups, non-capturing groups, and more.

In the Bash toolkit, PCRE is available via:

• grep -P — GNU grep with PCRE engine (Linux; not available in BSD/macOS grep)
• perl -ne — inline Perl one-liners, available everywhere Perl is installed
• pcregrep — standalone PCRE grep (installable on macOS via Homebrew)

Lookahead and Lookbehind Assertions

Assertions test what is around the match position without consuming characters — they are zero-width. The matched text does not include the assertion's content.

Copy# Positive lookahead: extract keys (words before a colon)
echo "host: localhost  port: 8080" | grep -oP '\w+(?=:)'
host
port

# Positive lookbehind: extract values (digits after $)
echo "price $42 or $199" | grep -oP '(?<=\$)[0-9]+'
42
199

# Negative lookahead: match "foo" not followed by "bar"
echo "foobar foobaz fooqux" | grep -oP 'foo(?!bar)\w*'
foobaz
fooqux

# Negative lookbehind: match digits NOT preceded by a minus
echo "count=42 temp=-7 size=100" | grep -oP '(?<!-)[0-9]+'
42
100

# Lookahead + lookbehind together: extract value between delimiters
echo 'name="Alice"' | grep -oP '(?<=")[^"]+'
Alice

# Validate a password: must contain a digit AND an uppercase letter
pw="Secret42"
if echo "$pw" | grep -qP '^(?=.*[0-9])(?=.*[A-Z]).{8,}$'; then
    echo "password strong"
fi

Lazy Quantifiers

All ERE/BRE quantifiers are greedy — they match as much as possible. PCRE adds lazy (minimal) quantifiers by appending ? to any quantifier. A lazy quantifier matches as little as possible while still allowing the overall pattern to succeed.

Copy# Greedy: matches from first < to LAST > on the line
echo "<b>bold</b> and <i>italic</i>" | grep -oP '<.+>'
<b>bold</b> and <i>italic</i>   # one match — greedy consumed everything

# Lazy: matches each individual tag
echo "<b>bold</b> and <i>italic</i>" | grep -oP '<.+?>'
<b>
</b>
<i>
</i>

# Lazy: extract content of the FIRST quoted string
echo '"Alice" and "Bob"' | grep -oP '".*?"'
"Alice"
"Bob"

# Lazy: match shortest balanced {...} block
echo "{a} text {b} more {c}" | grep -oP '\{.+?\}'
{a}
{b}
{c}

# All lazy quantifier forms
# *?   zero or more, lazy
# +?   one or more, lazy
# ??   zero or one, lazy
# {n,m}?  between n and m, lazy

Named Capture Groups

PCRE allows capture groups to be named, making patterns self-documenting and the code that uses them easier to maintain. Names are referenced in the pattern and in replacement strings.

Copy# Named group syntax: (?P<name>...) or (?<name>...) in PCRE

# grep -oP: extract named match (prints the whole match — use perl for the name)
echo "2026-06-11" | grep -oP '(?P<year>[0-9]{4})-(?P<month>[0-9]{2})-(?P<day>[0-9]{2})'
2026-06-11

# perl -ne: access named groups via %+ hash
echo "2026-06-11" | perl -ne '
    if (/(?P<year>\d{4})-(?P<month>\d{2})-(?P<day>\d{2})/) {
        print "year=$+{year} month=$+{month} day=$+{day}\n"
    }
'
year=2026 month=06 day=11

# perl: named group in replacement with \k<name>
echo "Smith, John" | perl -pe 's/(?P<last>\w+), (?P<first>\w+)/$+{first} $+{last}/'
John Smith

# Parse a log line with named groups for readability
echo "2026-06-11 14:32:00 ERROR connection failed" | perl -ne '
    if (/(?P<date>\S+) (?P<time>\S+) (?P<level>\S+) (?P<msg>.+)/) {
        printf "LEVEL=%-8s MSG=%s\n", $+{level}, $+{msg}
    }
'

Group Types — Complete Reference

Copy# Non-capturing group: group for alternation without a capture slot
echo "colour color" | grep -oP 'colo(?:u?r)'
colour
color

# Contrast: capturing group uses a slot even if you don't need it
# Non-capturing keeps group numbers clean when you only care about \1
echo "2026-06-11" | grep -oP '(?:[0-9]{4})-([0-9]{2})-([0-9]{2})'
# \1 = month (06), \2 = day (11) — year group not numbered

# Inline modifier: case-insensitive only within a group
echo "HTTP https FTP" | grep -oP '(?i:https?)'
HTTP
https

# Atomic group: prevent backtracking into a completed group
# Useful for performance; prevents runaway backtracking on long strings
echo "aaaaab" | grep -P '(?>a+)b'   # a+ consumed, no backtrack needed

perl -ne — PCRE in Shell Pipelines

perl -ne runs a Perl script line-by-line over stdin or a file. It is the most portable way to use full PCRE in shell pipelines, available on virtually every Unix system.

Copy# -n: loop over lines, no auto-print; -e: inline script
# Basic filter: print matching lines (equivalent to grep -P)
perl -ne 'print if /ERROR/' app.log

# -p: loop and auto-print (like sed); substitution with s///
perl -pe 's/foo/bar/g' file.txt

# Extract only the matched portion (like grep -oP)
perl -ne 'print "$1\n" while /([0-9]{4}-[0-9]{2}-[0-9]{2})/g' log.txt

# Named groups + formatted output
perl -ne '
    if (/(?P<ip>\d{1,3}(?:\.\d{1,3}){3}).*?"(?P<method>GET|POST|PUT|DELETE)/) {
        print "$+{ip}  $+{method}\n"
    }
' access.log

# Lazy match: extract each JSON string value
echo '{"name":"Alice","city":"Paris"}' | \
perl -ne 'print "$1\n" while /"([^"]+)":\s*"(.+?)"/g and print "key=$1 val=$2\n" and 0'

# Cleaner version for key:value JSON extraction
echo '{"name":"Alice","city":"Paris"}' | \
perl -ne 'while (/"(\w+)":\s*"([^"]+)"/g) { print "$1 = $2\n" }'
name = Alice
city = Paris

# Lookahead in perl: extract words before colons
echo "host: localhost  port: 8080" | \
perl -ne 'print "$1\n" while /(\w+)(?=:)/g'
host
port

# Multi-line mode: . matches newlines with /s, ^ and $ match line ends with /m
perl -0777 -ne 'print "$1\n" while /BEGIN(.*?)END/gs' file.txt
# -0777: slurp entire file into $_; /s: dot matches newline; /g: all matches

Practical PCRE Recipes

Extraction

Copy# Extract all IPv4 addresses from a file
grep -oP '\b(?:[0-9]{1,3}\.){3}[0-9]{1,3}\b' access.log | sort -u

# Extract all URLs (http and https)
grep -oP 'https?://[^\s">]+' page.html

# Extract email addresses
grep -oP '[\w.+-]+@[\w-]+\.[\w.]+' mail.txt | sort -u

# Extract value of a specific JSON field (not for production — use jq)
echo '{"user": "alice", "role": "admin"}' | \
grep -oP '(?<="user": ")[^"]+'
alice

# Extract all words after a specific keyword
echo "imported module requests as req" | grep -oP '(?<=as )\w+'
req

Validation and filtering

Copy# Password policy: min 8 chars, must have digit + uppercase + special
grep -P '^(?=.*[0-9])(?=.*[A-Z])(?=.*[!@#$%^&*]).{8,}$' passwords.txt

# Match lines that do NOT start with a comment or blank line
grep -P '^(?!#|$)' config.txt

# Find duplicate consecutive words
grep -P '\b(\w+)\s+\1\b' document.txt

# Validate that a version string follows MAJOR.MINOR.PATCH-PRERELEASE format
echo "2.31.0-beta.1" | grep -P '^[0-9]+\.[0-9]+\.[0-9]+(?:-[\w.]+)?$'

Substitution with perl -pe

Copy# Reformat date using named groups
echo "2026-06-11" | perl -pe 's/(?P<y>\d{4})-(?P<m>\d{2})-(?P<d>\d{2})/$+{d}\/$+{m}\/$+{y}/'
11/06/2026

# Mask credit card numbers: keep last 4 digits
echo "card: 4111-1111-1111-1234" | \
perl -pe 's/\b\d{4}(?:-\d{4}){2}(?=-\d{4}\b)/****-****-****/g'
card: ****-****-****-1234

# Add a thousands separator to large numbers
echo "1234567" | perl -pe 's/(\d)(?=(\d{3})+(?!\d))/$1,/g'
1,234,567

# Remove ANSI colour codes from terminal output
cat coloured.log | perl -pe 's/\x1b\[[0-9;]*m//g'

# Normalise line endings: CRLF to LF
perl -pi -e 's/\r\n/\n/g' file.txt    # in-place edit

Catastrophic Backtracking — and How to Avoid It

Poorly written PCRE patterns on adversarial input can cause the regex engine to run for an exponentially long time — a ReDoS (Regular Expression Denial of Service) attack. Understanding the cause is essential for writing safe patterns.

Copy# DANGEROUS: nested quantifiers on overlapping patterns
# Pattern (a+)+ on input "aaaaaaaaaaaab" causes exponential backtracking
# Each 'a' can be assigned to inner or outer group in 2^N ways
# grep -P '(a+)+b' <<< "aaaaaaaaaaaab"  — hangs!

# SAFE alternatives:
# 1. Atomic group — commit to match, no backtracking
grep -P '(?>a+)b'  # atomic: once a+ matched, don't re-try assignments

# 2. Possessive quantifiers (PCRE2 / newer Perl)
grep -P 'a++b'     # ++ = possessive: no backtrack

# 3. Rewrite to eliminate ambiguity
grep -P 'a+b'      # the nested group was unnecessary here

PCRE Feature Availability

Quick Reference — Chapter 11