Faster Alternatives

grep is installed almost everywhere and does one thing well. For searching a project, though, other tools have the sensible defaults built in: they know about git, skip the files you would have excluded by hand, and are noticeably faster. This chapter looks at git grep and ripgrep (rg), and then at what to use on Windows when grep is not around.

A Practice Repository

Paste this into an empty folder. It creates a small git repository, proj, with two commits, and then adds files that git does not track. The git commands are given your user name locally, so nothing outside the folder is changed.

What you now have, as git sees it:

(git status lists only the two untracked files: ignored files are not shown.) Every file above contains the word TODO. The earlier commit also had a line, TODO old note about caching, that the second commit removed.

What Plain grep Finds

With the options from Chapter 2 (-r, -I), grep finds every file that contains TODO:

Nine files, and only three of them are files git tracks. The rest are the files .gitignore says you do not care about, the hidden file, the untracked scratch note, and some sample scripts inside the .git folder (git puts them there when a repository is created). Chapter 7 dealt with this by naming each folder to skip. These tools do it from what git already knows.

git grep

git grep is part of git, so it is there wherever git is. It searches the files git tracks:

Four files: the three text files and the binary. No .git folder, no ignored files, no untracked ones. It takes many of grep's options (here I used -n -l -e -I -q and -P, which worked on both machines; the manual lists the rest):

The binary file is reported with git's own message, the same on both versions. -I skips it, as in grep, and the exit status is 1 when nothing is found (as in grep):

Untracked files, and files that are ignored

A file you have just made but not yet added is not searched. --untracked adds the files that are neither tracked nor ignored, which is usually what you want while you work:

--no-index goes the other way: it searches the working folder the way grep -r would (ignored files included, .git still left alone):

Old versions, and the working tree

Name a commit and git grep searches the files as they were then. The note that was removed in the second commit is still in the first:

By default it reads the files in your working folder, including changes you have not committed or staged. --cached searches what has been staged. I added a line to README.md without staging it:

Combining conditions, and choosing paths

--and, --or and --not combine patterns on the same line, which in plain grep takes two greps in a pipe:

After -- come pathspecs, git's own file patterns. '*.js' picks files by name and ':!...' removes some:

Run from a subfolder, it searches that folder and prints paths relative to it:

Outside a repository

git grep needs a repository, and says so with status 128 (not grep's 2). --no-index makes it work anywhere:

ripgrep (rg)

ripgrep searches the current folder recursively by default, and by default it skips what you would usually skip: files matched by .gitignore (its documentation names other ignore files too), hidden files and folders (so .git too), and binary files. Here is the same search with no options at all:

The same files as git grep --untracked, except two: rg skipped .hidden-notes because it is hidden, and the binary file because it is binary. scratch.txt was found although git does not track it, because rg works from ignore files, not from git's list of tracked files. Three things about this output:

• It looks like grep's, file:text. That is the form when the output goes to a pipe or a file. On a terminal rg instead groups results under a file heading, with line numbers and colour. I could only capture piped output here, so I have not shown the terminal form.
• Line numbers are off in that form: add -n.
• The files come out in any order, because rg searches several at once. I sorted the output above with | sort (Chapter 4's habit again).

On Windows, rg printed paths with a backslash, as in src\app.js (I expect slashes on Linux and macOS but could not check).

The -u switches: how much to skip

--files lists the files rg would search, without searching: the best way to see what its defaults leave out.

Each -u (“unrestricted”) switches off one more kind of skipping: -u stops honouring ignore files, -uu also searches hidden files and folders, -uuu also searches binary files. The number of files grows at every step (mostly .git internals at -uu):

With -uuu a binary match is reported, with a note of where the zero byte was:

A claim worth testing: rg -uu (nothing ignored, hidden files searched, binary files still skipped) should behave like Chapter 2's grep -rnI --exclude-dir=.git. Run both, normalise the paths and sort, and compare:

Choosing files

-t chooses a file type (rg knows a long list: rg --type-list), -g a glob, and a leading ! in a glob excludes:

The same options, and a few new ones

-i -w -x -F -v -c -l -o -A -B -C -e -f and -n mean what they mean in grep. A new one is smart case: -S ignores case if the pattern is all lower case, and is case-sensitive as soon as it has a capital. Plain rg is case-sensitive, like grep, and exits with 1 for no match:

-c counts matching lines per file, as grep does:

-z searches inside compressed files, as zgrep (Chapter 7) did:

A different regular expression language

rg's patterns are written in the Rust regex syntax: close to Chapter 3's extended form, with \d \w \s and lazy repeats, but with no look-around and no back-references. The error message points to the way round:

-P switches to Perl-style patterns (Chapter 5), if rg was built with them; mine was:

There is one more difference worth knowing. rg's \w understands Unicode, so it keeps accented letters in a word, where grep -P (Chapter 5) breaks them up. A file containing café résumé:

Where rg gets it wrong for you

rg honours .gitignore only inside a git repository, which is easy to forget when you copy files somewhere else. Outside one, the folder build is searched. --no-require-git makes rg obey the ignore file anyway, and so does running git init:

And when a search finds less than you expect, the first suspect is one of the skipping rules, which -u, -uu and -uuu will confirm. A missing file gives status 2 and a different message from grep's:

The Silver Searcher (ag)

ag is a similar tool: from its documentation, it also searches recursively by default and respects ignore files. It is not installed here and I did not run it, so I will not describe its options. If you already use it, the ideas in this chapter apply: check what it skips by default.

How Much Faster?

A measurement, not a promise. The folder is this course's own content folder: 13,136 files, 12,470 of them tracked by git. Each command lists the files containing a word. First a word that occurs in 1,976 files, then one that occurs in none. Each ran three times; the numbers are seconds, as time measured them. The grep is the Windows build in Git Bash (3.0), the files are in a OneDrive folder, and I did not repeat this on Linux, so treat the figures as one data point from this machine:

Plain grep took about 3.6 seconds for each run once the files were cached, and 36.6 seconds on the very first run, when they had to be read from disk. git grep and rg took about half a second each, roughly seven times quicker than the warm grep, and all three found the same 1,976 files. The first run of a tool you have not used for a while is the slow one, which is why I ran each three times. On a project of a few dozen files, none of this matters.

grep on Windows

You have four ways to get grep-like searching on a Windows machine, and they are not the same:

Select-String

Select-String does the job in PowerShell. It differs from grep in ways that catch people out. First, it is case-insensitive by default: here todo matches TODO. Its output is a match object, which prints in a familiar file:line:text form (PowerShell adds a blank line before the first result, left out here):

-CaseSensitive gives grep's behaviour:

It does not recurse by itself. Feed it files from Get-ChildItem -Recurse. It knows nothing about .gitignore, so build\out.js is found:

The first shows -NotMatch (lines without TODO, with their line numbers); the second -Context 1,1, with > marking the match; the third pulls out just the matching text and counts the matching lines (case-sensitively).

findstr

findstr is old, but it is present on every Windows machine and works from cmd. /n numbers lines, /i ignores case, /s recurses, /m lists file names only, and /c:"..." searches for a phrase with spaces. It sets an exit status like grep (0, or 1 for no match):

Its regular-expression support is limited and I did not test it, so use grep, rg or Select-String when the pattern is more than words.

Which One Should I Use?

A caution that applies to every tool in this chapter: their defaults hide files. A search that finds nothing might mean nothing is there, or that the tool skipped it. When it matters, check with the “do not skip anything” form (grep -r, rg -uuu, git grep --no-index).

Hands-On Exercises

Course Complete

That is all eight chapters. You can now search text with grep from a single file up to a whole project, write patterns in basic, extended and Perl form, use grep safely in scripts, and pick a better tool when grep is not the best one. What each chapter left you able to do: