Regular Expressions in Bash

Chapter 1 — What Are Regular Expressions?

The Core Idea

A regular expression (regex) is a pattern that describes a set of strings. Instead of saying "find the exact text error", a regex lets you say "find any line that contains the word error followed by a colon and then one or more digits." It is a mini-language for describing text shapes.

Regexes appear constantly in Bash work — in grep for filtering, in sed for substitution, in awk for field processing, and directly inside Bash's own [[ =~ ]] conditional. Learning regex once unlocks all of them simultaneously.

The regex engine reads the pattern and the input text, then reports one of three outcomes: a match (the pattern was found), no match, or — when using capturing groups — the matched text and any captured sub-strings.

Where Bash Uses Regular Expressions

You will encounter regex in four main places when working at the Bash command line:

Copy# grep: print lines containing a pattern
grep '[Ee]rror' app.log

# sed: substitute text matching a pattern
sed 's/[0-9]\+/NUM/g' data.txt

# awk: process lines where a field matches a pattern
awk '/^ERROR/ { print $0 }' app.log

# [[ =~ ]]: test a string against a regex in a script
if [[ "$input" =~ ^[0-9]{4}-[0-9]{2}-[0-9]{2}$ ]]; then
    echo "Valid date format"
fi

The Three Flavours You Will Meet

Not all regex is the same. The Linux/Bash ecosystem has three distinct flavours, each with slightly different rules for which characters are "special". Understanding which tool uses which flavour is one of the most important things you can learn.

How the Regex Engine Works

Understanding the engine's behaviour prevents a large class of surprises. The engine follows two simple rules:

1. Left to right: the engine tries to match the pattern starting at the leftmost character of the input. If it fails, it moves one character to the right and tries again.
2. Greedy by default: quantifiers like * and + consume as many characters as possible while still allowing the overall match to succeed.

Copy# Leftmost match wins — engine finds the first position that works
echo "the cat sat on the mat" | grep -oE '[a-z]at'
cat
sat
mat   # -o prints each match on its own line

# Greedy: .* consumes as much as possible
echo "<b>bold</b> and <i>italic</i>" | grep -oE '<.*>'
<b>bold</b> and <i>italic</i>   # matched the whole thing!

# Fix: use a negated class to stop at the first >
echo "<b>bold</b> and <i>italic</i>" | grep -oE '<[^>]*>'
<b>
</b>
<i>
</i>

Your First Regex Patterns

Let's build familiarity with the most important regex building blocks. These work in all three flavours.

Literal characters

The simplest regex is just the text you want to find. Every letter and digit matches itself.

Copy# Find all lines containing "error" (lowercase only)
grep 'error' app.log

# Find lines containing "404"
grep '404' access.log

# Most punctuation is also literal — but some are special (see below)
grep 'v1.0' changelog.txt   # . is special — matches any char here
grep 'v1\.0' changelog.txt  # \. matches a literal dot

Special (metacharacter) characters

These characters have special meaning in regex. To match them literally, escape with \:

A quick tasting menu

Copy# . — match any single character
echo "cat bat hat" | grep -oE '.at'
cat
bat
hat

# ^ — match only at the start of the line
echo -e "apple\nbanana\napricot" | grep '^a'
apple
apricot

# $ — match only at the end of the line
echo -e "run\nfun\nbun\nsun" | grep 'un$'
run
fun
bun
sun

# [abc] — match one character from the set
echo "cat dog bat" | grep -oE '[cb]at'
cat
bat

# * — zero or more of the preceding
echo -e "ac abc abbc abbbc" | grep -oE 'ab*c'
ac
abc
abbc
abbbc

Testing Regex at the Command Line

The fastest way to learn regex is to experiment interactively. Here are the best testing patterns to use at a Bash prompt:

Copy# Pattern 1: pipe echo into grep -E to test a pattern instantly
echo "test string here" | grep -E 'your_pattern'

# Pattern 2: use -o to show only the matched text (not the whole line)
echo "order #12345 total $99.50" | grep -oE '[0-9]+'
12345
99
50

# Pattern 3: use --color to highlight matches in context
grep --color=always -E '[0-9]+' data.txt

# Pattern 4: test [[ =~ ]] in the shell directly
str="hello123"
[[ "$str" =~ [0-9]+ ]] && echo "contains digits" || echo "no digits"
contains digits

# Pattern 5: use a here-string to test sed substitutions
sed -E 's/[0-9]+/NUM/g' <<< "There are 42 items and 7 errors"
There are NUM items and NUM errors

A Comparison: Shell Globbing vs Regex

New Bash users sometimes confuse shell glob patterns (used in filename expansion) with regular expressions (used in text tools). They look similar but mean completely different things:

Copy# Shell glob: * means "any filename characters"
ls *.log           # lists all .log files — * is a glob wildcard

# Regex: * means "zero or more of the preceding element"
grep 'log*' file   # matches "lo", "log", "logg", "loggg" etc.
                   # NOT "anything ending in log"!

# To match "anything then .log" in a regex:
grep '.*\.log' file  # .* = any chars, \. = literal dot, log = literal

Quick Reference — Chapter 1