Advanced Masking

Raster Editing Fundamentals' Chapter 5 taught the layer mask: paint white to reveal, black to hide, on a mask attached to a layer. That works well for clean, simple edges — a straight horizon, a rectangular product shot. It breaks down completely for fine, tangled detail like hair, fur, or a semi-transparent veil, where painting every individual strand by hand is both impossibly slow and visibly imprecise. This chapter covers the professional alternative: building a selection from the image's own contrast, using its color channels, rather than painting one from scratch.

Why a Painted Mask Breaks Down for Fine Detail

A brush-painted mask is only as precise as your hand and your zoom level allow. Hair against a background has dozens or hundreds of individual strands, each a pixel or two wide, often semi-transparent where fine flyaway hairs let the background show through slightly. Painting each one is not just slow — it's the wrong tool for the job entirely, since a brush can't naturally reproduce that kind of soft, irregular, semi-transparent edge no matter how carefully it's used.

The Channels Panel: Using the Image's Own Contrast as a Selection Source

Every RGB image already contains three separate grayscale images — its Red, Green, and Blue channels — and one of them often shows far more contrast between your subject and the background than the others. The core technique:

1. Open the Channels panel and inspect the Red, Green, and Blue channels individually to find the one with the strongest contrast between subject and background.
2. Duplicate that channel — never edit the original, which is still needed for the final image's actual color.
3. Boost contrast on the duplicate (Levels or Curves) until the subject is close to solid white and the background close to solid black.
4. Paint over any remaining gray areas by hand to clean up the result, then load the duplicated channel as a selection.
5. Use that selection to build a precise layer mask back on the original layer.

Photoshop's Select and Mask Workspace

Photoshop packages a faster path to the same result: the Select and Mask workspace (accessible from any selection tool's options bar). Its Refine Edge Brush lets you brush directly over a hair-like edge, and Photoshop automatically detects fine detail within that brushed area using the same underlying channel-contrast logic described above — without manually building a duplicate channel yourself. Its Decontaminate Colors option additionally removes leftover background color fringing from semi-transparent edge pixels, replacing it with a plausible extension of the subject's own color.

GIMP's Channel-Based Approach

GIMP has no single packaged workspace equivalent to Select and Mask — the manual channel technique above is GIMP's primary approach for this kind of mask, done directly through its own Channels dialog, Curves, and the Threshold tool. GIMP's Foreground Select tool gets partway to Photoshop's automation for simpler cases (marking foreground and background roughly, then refining an initial matte), but for genuinely fine detail like hair, GIMP users lean on the manual channel workflow far more than Photoshop users typically need to.

Hands-On Exercises