🌐

Building a Web Browser Engine: Parsing & the DOM

A Real HTML Tokenizer, DOM Parser, CSS Engine & Style Tree — From Scratch

Topics covered:
A forgiving HTML tokenizer & stack-based DOM parser
A real CSS tokenizer, parser & selector matcher
Specificity, the cascade & inheritance
The style tree & a real user-agent stylesheet

Capstone: a real HTML+CSS document parsed end to end into a fully-resolved style tree
Exercises: 27 hands-on exercises with worked, verified solutions
Format: A4 · Dark-theme code examples

Philip Osztromok · Generated with Claude

Why Build a Browser Engine? HTML, CSS & the Rendering Pipeline
Tokenizing HTML: A Real, Forgiving Lexer
Parsing HTML into a DOM Tree
Tokenizing and Parsing CSS
Selectors & Selector Matching
Specificity & the Cascade
Inheritance & Computed Values
Building the Style Tree
Default (User-Agent) Stylesheets & Real-World Fidelity
Capstone — Parsing a Real HTML+CSS Document into a Style Tree

Chapter 1 of 10

Why Build a Browser Engine? HTML, CSS & the Rendering Pipeline

Building a Web Browser Engine: Parsing & the DOM

Chapter 1 · Why Build a Browser Engine? HTML, CSS & the Rendering Pipeline

Type a URL, and somewhere between pressing Enter and seeing a page, raw bytes of HTML and CSS text turn into an arrangement of colored rectangles and text on a screen. This two-course project builds the part of that journey that starts once the HTML and CSS text already exist — no networking, no JavaScript, nothing borrowed from a real browser engine's own source code. Just the actual pipeline, built by hand, in Python, verified at every step the same way every other course on this site has been: by running it and checking the answer, not by asserting it.

The Four-Stage Pipeline: Parse → Style → Layout → Paint

Every real browser engine — however large and however optimized — does the same four things to turn markup into pixels. This project builds each one, split across two courses.

Stage	Input	Output	Built in
Parse	Raw HTML text, raw CSS text	A DOM tree, a stylesheet	This course, Chapters 2-5
Style	The DOM tree + the stylesheet	A style tree — every DOM node paired with its own fully resolved computed style	This course, Chapters 6-9
Layout	The style tree	A layout tree — every box's own real width, height, and position	Course 2, Chapters 1-6
Paint	The layout tree	An actual pixel buffer	Course 2, Chapters 7-10

Each stage's own output is exactly the next stage's own input — a real pipeline, not four unrelated topics loosely grouped under one course title. By the end of Course 2's own capstone, a real static web page will go in as two text files and come out as an actual rendered image.

This Project's Own Honest Scope

Stated up front, the same way every other course in this site's own "ambitious learning projects" tier states its scope before writing a line of code: this is build-to-understand, not build-to-ship.

No JavaScript. No <script> execution, no DOM mutation after the initial parse.
No networking. HTML and CSS arrive as plain Python strings — no HTTP, no fetching linked stylesheets or images.
No images. Layout and paint both work purely with boxes, text, and color.
No Flexbox, no Grid, no forms. Block and inline layout only — the two layout modes CSS actually started with.
No real font rendering. Course 2's own text layout uses a simplified, honest glyph-width model, not a real font-rasterization library.
A software-rasterized pixel buffer, not a real window. The final output is a real image file, not anything drawn to actual screen hardware.

None of these are apologized-for gaps discovered too late — they're the same kind of deliberate, stated-in-advance boundary this site's own compiler1/compiler2 pair drew around JavaScript-style dynamic typing extras, and the same kind every course in this tier draws to stay genuinely finishable while still being genuinely real.

Why Not Just Use Regex?

Before building a real parser, it's worth trying the shortcut everyone tries first — and seeing exactly where it breaks, concretely, rather than taking "you can't parse HTML with regex" on faith.

import re

def naive_extract(html, tag):
    pattern = f"<{tag}>(.*?)</{tag}>"
    return re.findall(pattern, html, re.DOTALL)
    

Verified directly — nesting breaks it, exactly as the well-known warning says, and gets worse the deeper it goes

naive_extract("<div><div>Inner</div></div>", "div") returns ['<div>Inner'] — not the true outer content, "<div>Inner</div>". The regex has no concept of nesting depth at all; it just scans forward from the first <div> and captures raw characters until it hits the first literal </div> it finds — completely oblivious that an inner <div> opened in between. Three levels of nesting makes it worse, not better: "<div><div><div>Deep</div></div></div>" returns ['<div><div>Deep'] — two stray, unclosed opening tags leaked straight into the "content."

This isn't a fixable regex bug — it's a real, structural limit

Regular expressions match regular languages — patterns with no memory of how deep they've gone. HTML's own nesting is a context-free structure — correctly matching an outer tag's own closing tag requires remembering how many same-named tags have opened since, which a regex engine has no mechanism to track at all. Switching to a greedy .* instead of .*? doesn't fix this — it trades the nesting bug for the opposite failure, matching all the way from the first opening tag to the very last closing tag in the document, swallowing unrelated sibling elements in between.

What a Real Parser Needs Instead: Tracking Depth

The fix a real parser needs is small enough to preview here, one paragraph before Chapter 2 builds the real thing properly: track how many tags deep the scan currently is, and only treat a closing tag as the match once that count returns to zero.

def depth_aware_extract_first(html, tag):
    open_tag, close_tag = f"<{tag}>", f"</{tag}>"
    start = html.find(open_tag)
    pos = start + len(open_tag)
    depth = 1
    while depth > 0:
        next_open = html.find(open_tag, pos)
        next_close = html.find(close_tag, pos)
        if next_open != -1 and next_open < next_close:
            depth += 1; pos = next_open + len(open_tag)     # one level deeper
        else:
            depth -= 1; content_end = next_close; pos = next_close + len(close_tag)
    return html[start + len(open_tag): content_end]
    

Verified directly — tracking depth gets the nested case exactly right

On the identical nested input, depth_aware_extract_first returns "<div>Inner</div>" — the true, complete content of the outer div. Given a harder input mixing nesting and trailing text before the real close — "<div><div>Inner</div>after inner</div><div>Second</div>" — it correctly returns "<div>Inner</div>after inner", stopping at the right closing tag even with a second, unrelated div sitting right after it in the source.

This is a genuine preview, not a toy — Chapter 2's own real tokenizer and parser are a more complete, more careful version of exactly this idea: never trust a closing tag in isolation, always track what's currently open.

A First Look at a Real DOM Tree

Before building a from-scratch tokenizer, it's worth seeing what the target actually looks like. Python's own standard library includes an HTML parser — used here purely to illustrate the shape of a DOM tree for this one chapter, not reused anywhere else in this course. Chapter 2 builds a real tokenizer and parser completely from scratch.

source = "<div>\n <p>Hello</p>\n</div>"

Verified directly — a real, if illustrative, DOM tree for a two-element document

Element('#document')
  Element('div')
    Element('p')
      Text('Hello')

div contains p, which contains the text "Hello" — a tree, not a flat list, with each element's own children nested directly inside it. A sibling structure, "<ul><li>One</li><li>Two</li></ul>", produces two separate li elements as direct children of the same parent — ul — rather than nested inside one another, confirming the tree correctly distinguishes "nested inside" from "next to."

From DOM to Pixels — Sketching the Rest of the Pipeline

The same tiny div/p/"Hello" example, carried conceptually through every remaining stage this two-course project builds — not yet computed for real, since the cascade, the box model, and the rasterizer don't exist yet, but concrete enough to see where each future chapter's own work actually lands.

Stage	What it adds to this example
DOM tree (above)	`div` → `p` → `"Hello"`, with no notion of color, size, or position at all
Style tree (Ch.6-9)	Every node paired with its own resolved values — e.g. `div` might resolve to `padding: 10px`, `p` to `color: blue`, inherited down from whatever the stylesheet and the browser's own defaults specify
Layout tree (Course 2, Ch.1-6)	Real numbers — `div`'s own box might resolve to position `(0, 0)`, size `300×40`; `p`'s own line box sits inside it, inset by that `10px` padding
Pixels (Course 2, Ch.7-10)	An actual image — the `div`'s own background color filled into its box's own rectangle, "Hello" rendered as real glyphs at the `p`'s own resolved position

Nothing here was computed — that's deliberate

This table is a map, not a result. Every specific number above (10px, (0, 0), 300×40) is illustrative, chosen to be plausible, not derived from any real cascade or layout algorithm — those don't exist in this course yet. The point is orientation: knowing what a "style tree" or a "layout tree" is, concretely, before spending several chapters building the machinery that actually produces one correctly.

Where Each Future Chapter Fits

Chapter	Builds
2	A real, hand-written HTML tokenizer — the depth-tracking idea above, generalized properly
3	A real HTML parser, building an actual DOM tree from the token stream
4	A CSS tokenizer and parser, building a real `Stylesheet` structure
5	Selectors and selector matching — deciding whether one CSS rule applies to one DOM node
6	Specificity and the cascade — deciding which rule wins when more than one matches
7	Inheritance and computed values
8	The style tree itself — the DOM tree, the cascade, and inheritance combined into one structure
9	A real default stylesheet — why `div` and `span` behave differently even with no CSS written at all
10	Capstone — a real document parsed all the way to a verified style tree

Hands-On Exercises

Exercise 1

Using this chapter's own naive_extract, test a four-level-deep nested input — "<div><div><div><div>Deepest</div></div></div></div>" — for the tag "div". Determine exactly what gets captured, and explain the pattern connecting nesting depth to how many stray opening tags leak into the broken result.

This chapter's finding	What it connects to
A dedicated `voidtag` token, distinct from `opentag`	Chapter 3's own parser — a void element never gets pushed onto the open-element stack at all, sidestepping the entire "when does this close" question for the 14 tags that never need it answered
A trailing `/` is ignored unless the tag is void	Chapter 3's own stack-based parser, which will nest `<div/><p>...</p></div>` exactly as deeply as this chapter's own token stream implies — the parser doesn't re-decide this; the tokenizer already did
Attribute values default to `None` for boolean attributes	A forward reference to real CSS attribute selectors and form-control defaults — out of this course's own stated scope, but the distinction (present-with-no-value vs. genuinely absent) is preserved here regardless, for free
Tag names normalized to lowercase during tokenizing	Chapter 5's own selector matching — a CSS selector like `div` can be compared directly against a DOM node's own tag name with a plain string equality check, no case-folding needed at match time

This chapter's finding	What it connects to
A stack of currently-open elements, generalized from Chapter 1's own single-tag depth counter	Chapter 1's own `depth_aware_extract_first` — the same underlying idea, now tracking an entire document's own nesting at once instead of one tag pair
`voidtag` tokens never get pushed onto the stack	Chapter 2's own dedicated `voidtag` token kind — this chapter is where that distinction actually pays off, sidestepping any need to guess whether a `br` or `img` is "closed"
The real DOM tree this chapter produces	Chapter 8's own style tree, which pairs every one of these exact `Element` nodes with a fully resolved computed style — nothing about this tree's own shape changes between now and then
The `li` gap, found and left honestly unfixed	A direct parallel to this site's own established practice (Course 1 and Course 2 of the compiler project both did this) of naming a real, verified limitation rather than quietly working around it or pretending it doesn't exist

This chapter's finding	What it connects to
A flat list of raw selector strings on each `Rule`	Chapter 5's own selector matching — this chapter deliberately stops at "here is the text," leaving "does this text match this DOM node" as a separate, dedicated problem
CSS comments stripped once, up front, before any rule parsing begins	Mirrors Chapter 3's own `closetag` handling being defensive by construction — both chapters choose to make one part of the pipeline robust early, so nothing downstream has to re-solve the same problem
A corrupted selector from a non-nested comment produces no error anywhere	A direct parallel to this course's own Chapter 1 finding about regex and HTML — a parser that silently produces a plausible-looking wrong answer is a harder failure mode to catch than one that crashes loudly
Property names lowercased during parsing	Chapter 3's own tag-name lowercasing — the same "normalize once, at the source, so every later consumer can assume it's already done" discipline, applied to CSS instead of HTML

This chapter's finding	What it connects to
Chapter 3's tree had no parent pointer until a descendant selector needed one	A direct parallel to Chapter 9's own `origin`/`base` finding in the compiler project — a data structure built correctly for its original purpose still needing an honest, later revision once a new consumer's own requirements arrive
Gap-tolerant ancestor search for multi-part descendant selectors	Chapter 6's own specificity and cascade — every rule whose selector matches at all becomes a real candidate for a given element, regardless of how deeply nested the match happened to be
`SimpleSelector`/`Selector` built here, from Chapter 4's own raw strings	Chapter 8's own style tree, which will need to ask "does this rule apply to this element" for every rule in the stylesheet, against every element in the DOM — the exact question this chapter's own `matches_selector` answers
id uniqueness left honestly unenforced	The same kind of stated scope boundary Chapter 3 drew around `<li>`'s own missing implied-closing rule — a real limitation, named directly rather than silently assumed away

This chapter's finding	What it connects to
Specificity as a compared tuple, never a summed number	Chapter 5's own compound-selector matching — the same "every part has to hold together" discipline, applied here to scoring rather than matching
Per-property merging across every matching rule	Chapter 8's own style tree, which needs a complete, final property set for every DOM node — exactly what `cascade()` produces for one element at a time
A comma-list rule needing its own highest-matching specificity	Chapter 5's own honest id-uniqueness gap — another case where a real, unglamorous edge case in how CSS selectors combine only surfaces once you actually test it, not when you reason about the "normal" case alone

This chapter's finding	What it connects to
Top-down tree walk carrying each parent's computed style downward	Chapter 5's own `assign_parents()` — the same DOM tree, now walked in the opposite direction its structure was originally built to support, exactly as the descendant-selector matcher already needed to
A property's own final computed value being unambiguous for every element	Chapter 8's own style tree, which pairs every DOM node with exactly this — its own complete, resolved set of computed values, the direct input the layout engine in Course 2 will consume
Inherit-from-direct-cascade being a genuine, easy-to-miss bug	Chapter 6's own comma-selector bug — another case where a single, non-recursive parent lookup looks completely correct until tested against a tree more than one level deep

This chapter's finding	What it connects to
One combined tree, every node carrying its own final computed style	The exact structure Course 2's own layout engine consumes first — a `StyledNode` tree is what gets converted into a layout tree, not the raw DOM tree and not a separate cascade result per element
Recursing only into element children silently drops all text	Chapter 3's own `TextNode`/`CommentNode` classes existing as genuinely distinct `kind`s in the first place — this chapter is the first to actually need that distinction to matter for something beyond just "don't confuse a comment for a tag"
display:none removing a whole subtree in one step	Course 2's own upcoming distinction between "not rendered at all" (this chapter's `display: none`) and "rendered but invisible" (a future `visibility: hidden`-style property, which still occupies layout space) — a real, separate concept this course deliberately doesn't build

This chapter's finding	What it connects to
A real, built-in stylesheet supplying every tag's own default appearance	Chapter 8's own style tree — `display: none` was already demonstrated there via an author rule; this chapter shows `display: block`/`inline` normally comes from the UA sheet instead, for every element, on every page, with or without any author CSS at all
Origin outranking specificity, checked as its own separate tier	Chapter 6's own cascade sort key, now needing a THIRD component ahead of the two it already had — `(origin, specificity, order)` instead of just `(specificity, order)` — the same "add one more tier to the same sorting idea" pattern that specificity itself was to source order
Chapter 7's INITIAL_VALUES being individually correct but incomplete without this chapter	A direct parallel to Chapter 5's own honest id-uniqueness gap and Chapter 3's `<li>` gap — a piece that was genuinely right for what it covered, revealed as incomplete only once a later chapter tests the case it never handled

This chapter's finding	What it connects to
A real integration bug, invisible to every individual chapter's own tests	The exact reason a capstone chapter exists at all in this course's own format — Chapters 2 through 9 were each genuinely correct in isolation; only running them together for the first time could ever have surfaced this
A fully-resolved style tree, built from real source text start to finish	Course 2, Building a Web Browser Engine: Layout & Rendering — this exact `StyledNode` tree, produced by this exact pipeline, is the direct input its own first chapter starts from
Whitespace-only text nodes surviving as real style-tree entries	Course 2's own inline-layout chapters, which are where a real browser engine actually decides what to do with whitespace between elements — collapsing it, preserving it, or treating it as insignificant depending on context

Building a Web Browser Engine: Parsing & the DOM

Table of Contents

Why Build a Browser Engine? HTML, CSS & the Rendering Pipeline

The Four-Stage Pipeline: Parse → Style → Layout → Paint

This Project's Own Honest Scope

Why Not Just Use Regex?

What a Real Parser Needs Instead: Tracking Depth

A First Look at a Real DOM Tree

From DOM to Pixels — Sketching the Rest of the Pipeline

Where Each Future Chapter Fits

Hands-On Exercises

Chapter 1 Quick Reference

Tokenizing HTML: A Real, Forgiving Lexer

Five Kinds of Token

Void Elements: Tags That Never Get a Closing Tag

The Surprising Part: a Trailing Slash on an Ordinary Element Does Nothing

Attributes: Three Shapes, One Parser

Comments: Safe From the Characters That Normally Matter

Case-Insensitivity

Where This Connects

Hands-On Exercises

Chapter 2 Quick Reference

Parsing HTML into a DOM Tree

Three Real Node Types

The Parser: a Stack of Currently-Open Elements

The Real Quirk: <p> Closes Itself When It Has To

A Real, Honest Limitation — Found by Testing It

Where This Connects

Hands-On Exercises

Chapter 3 Quick Reference

Tokenizing and Parsing CSS

Three Small Structures

Comments First, Before Anything Else

The Real Quirk: CSS Comments Don't Nest

The Rest of the Grammar

Where This Connects

Hands-On Exercises

Chapter 4 Quick Reference

Selectors & Selector Matching

Simple Selectors and the Selector They Combine Into

A Real Gap, Found the Moment Descendant Selectors Are Needed

Matching a Descendant Selector: Search Upward, Part by Part

An Honest Limitation: Nothing Here Enforces id Uniqueness

Where This Connects

Hands-On Exercises

Chapter 5 Quick Reference

Specificity & the Cascade

Specificity: a Tuple, Not a Single Number

The Cascade: Merging Per Property, Not Per Rule

A Real Bug: Comma-Separated Selectors in One Rule

Where This Connects

Hands-On Exercises

Chapter 6 Quick Reference

Inheritance & Computed Values

Two Fates for an Unset Property

A Naive First Attempt — and a Real Bug It Has

The Fix: Inherit From the Parent's Already-Computed Style

Where This Connects

Hands-On Exercises

Chapter 7 Quick Reference

Building the Style Tree

The StyledNode: a DOM Node Paired With Its Own Computed Style

A Naive First Attempt — and the Bug Testing It Reveals

The Fix: Three Cases, Handled Explicitly

Where This Connects

Hands-On Exercises

Chapter 8 Quick Reference

Default (User-Agent) Stylesheets & Real-World Fidelity

The Bug Hiding in Plain Sight Since Chapter 7

A Real (If Simplified) User-Agent Stylesheet

A Real Bug: Specificity Alone Isn't Enough Once Two Origins Exist

Where This Connects

Hands-On Exercises

Chapter 9 Quick Reference

Capstone — Parsing a Real HTML+CSS Document into a Style Tree

A Real Bug, Found Only by Actually Integrating Everything

The Real Document

Verifying the Result, Node by Node

Where This Connects

What This Course Doesn't Cover

The Real Quirk: `<p>` Closes Itself When It Has To