Parsing HTML into a DOM Tree

Building a Web Browser Engine: Parsing & the DOM

Chapter 3 · Parsing HTML into a DOM Tree

Chapter 2's tokenizer produces a flat list — every open tag, close tag, void tag, run of text, and comment, one after another, with no notion of nesting at all. This chapter turns that list into an actual tree: a real Element/Text/Comment hierarchy, built by a stack-based parser that tracks which elements are currently open, the same way Chapter 1's own preview scan tracked depth for one tag at a time — generalized here to the whole document at once.

Three Real Node Types

class DOMNode:
    def __init__(self, kind):
        self.kind = kind
        self.children = []

class Element(DOMNode):
    def __init__(self, tag, attrs):
        super().__init__('element')
        self.tag = tag
        self.attrs = attrs

class TextNode(DOMNode):
    def __init__(self, text):
        super().__init__('text')
        self.text = text

class CommentNode(DOMNode):
    def __init__(self, text):
        super().__init__('comment')
        self.text = text
    

Unlike Chapter 1's own illustrative tree, these are the real node types this course keeps using for the rest of both courses — including CommentNode, which the tree stores faithfully even though it will simply never be visible once painting exists.

The Parser: a Stack of Currently-Open Elements

def parse(tokens):
    root = Element('#document', {})
    stack = [root]
    for tok in tokens:
        if tok.kind == 'opentag':
            node = Element(tok.name, tok.attrs)
            stack[-1].children.append(node)
            stack.append(node)
        elif tok.kind == 'voidtag':
            node = Element(tok.name, tok.attrs)
            stack[-1].children.append(node)     # never pushed -- no close expected
        elif tok.kind == 'closetag':
            for i in range(len(stack) - 1, 0, -1):
                if getattr(stack[i], 'tag', None) == tok.name:
                    del stack[i:]; break
        elif tok.kind == 'text':
            stack[-1].children.append(TextNode(tok.text))
        elif tok.kind == 'comment':
            stack[-1].children.append(CommentNode(tok.text))
    return root
    

voidtag's own special treatment carries straight through from Chapter 2: it becomes a child of whatever's currently open, but is never itself pushed onto the stack — there's nothing to later pop, because nothing will ever try to close it. The closetag branch searches from the top of the stack downward rather than assuming the top is always the right match — a real, deliberate defense against malformed markup, not just the tidy case.

Verified directly — a stray, mismatched closing tag doesn't crash the parser

parse_html("<p>Hello</div>") — a closing </div> that was never opened — produces a perfectly valid tree with p still open and containing "Hello". The search for a matching div on the stack simply finds nothing and the token is silently ignored, exactly the way real browsers treat an orphaned closing tag.

The Real Quirk: `<p>` Closes Itself When It Has To

Real HTML lets a huge number of closing tags be omitted entirely — the parser is expected to work out where an element really ends from context. The single most common case: an open <p> is implicitly closed the moment certain other elements begin, even with no </p> anywhere in the source.

# tags that implicitly close an open <p>, per the real HTML5 spec
P_CLOSING_TAGS = frozenset({
    'address', 'article', 'aside', 'blockquote', 'details', 'div', 'dl',
    'fieldset', 'figcaption', 'figure', 'footer', 'form',
    'h1', 'h2', 'h3', 'h4', 'h5', 'h6', 'header', 'hr',
    'main', 'menu', 'nav', 'ol', 'p', 'pre', 'section', 'table', 'ul',
})

# inside the opentag branch, checked BEFORE pushing the new element:
if (tok.name in P_CLOSING_TAGS and
        getattr(stack[-1], 'tag', None) == 'p'):
    stack.pop()      # the open <p> is done, whether the source said so or not
    

Verified directly — two consecutive <p> tags with only one closing tag produce two real siblings

parse_html("<p>Hello<p>World</p>") produces two separate sibling p elements — one containing "Hello", one containing "World" — not one p nested inside another. The first <p> is popped off the stack the instant the second one begins, purely because 'p' is itself in P_CLOSING_TAGS. The exact same thing happens with no closing tag written anywhere at all: "<p>Hello<div>World</div>" also produces two top-level siblings, p and div — the div is never nested inside the p, even though the source text never once wrote </p>.

Not every element triggers this — only the ones on the list

parse_html("<p>Hello<span>World</span></p>") keeps span correctly nested inside p, because span was never a block-level element and was never added to P_CLOSING_TAGS. The rule isn't "any new tag closes an open p" — it's a specific, named list, and getting the list wrong in either direction (too broad or too narrow) produces a genuinely wrong tree.

A Real, Honest Limitation — Found by Testing It

<li> has an almost identical real-world rule: a new <li> is supposed to implicitly close a previous open <li>, the exact same shape as the p rule above. This chapter's own parser doesn't implement it.

Verified directly — a second <li> nests inside the first instead of closing it

parse_html("<ul><li>One<li>Two</li></ul>") — real, valid, extremely common HTML — produces the second li genuinely nested inside the first, not as a sibling. A real browser renders this as two separate list items; this chapter's own parser, as written, would not. P_CLOSING_TAGS only ever checks whether the currently-open element is a p — it has no equivalent check for an open li at all.

This isn't a bug hidden until an exercise reveals it — it's a genuine, stated scope boundary: this chapter builds the single most common implied-closing rule (p) to establish the technique, not an exhaustive table of every element's own omission rules HTML actually defines. A real browser engine's own parser has dozens of rules like this one; this course builds one, correctly and completely, as a real, working example of the pattern.

Where This Connects

This chapter's finding	What it connects to
A stack of currently-open elements, generalized from Chapter 1's own single-tag depth counter	Chapter 1's own `depth_aware_extract_first` — the same underlying idea, now tracking an entire document's own nesting at once instead of one tag pair
`voidtag` tokens never get pushed onto the stack	Chapter 2's own dedicated `voidtag` token kind — this chapter is where that distinction actually pays off, sidestepping any need to guess whether a `br` or `img` is "closed"
The real DOM tree this chapter produces	Chapter 8's own style tree, which pairs every one of these exact `Element` nodes with a fully resolved computed style — nothing about this tree's own shape changes between now and then
The `li` gap, found and left honestly unfixed	A direct parallel to this site's own established practice (Course 1 and Course 2 of the compiler project both did this) of naming a real, verified limitation rather than quietly working around it or pretending it doesn't exist

Hands-On Exercises

Exercise 1

Parse "<p>Hello<ul><li>x</li></ul>" and "<p>Hello<h1>Title</h1>" using this chapter's own parse_html. Confirm both produce two top-level siblings rather than nested elements, and explain why ul and h1 both trigger the same implied-closing behavior div does.

📄 View solution

Exercise 2

Parse "<p>A<p>B<p>C</p>" — three consecutive <p> tags, only the last one explicitly closed. Determine the exact number of top-level elements produced and confirm each one's own text content, then trace through the parser's own stack contents at each of the three opentag events to explain why chaining this rule works correctly without any special handling beyond the single check already shown in this chapter.

📄 View solution

Exercise 3

Add an analogous rule fixing this chapter's own <li> limitation — a new <li> should implicitly close a previously open <li>, the same shape as the existing p check. Verify parse_html("<ul><li>One<li>Two</li></ul>") now produces two sibling li elements instead of one nested inside the other, and confirm the existing p-closing tests from this chapter still pass unchanged.

📄 View solution

Chapter 3 Quick Reference

Three real node types: Element, TextNode, CommentNode — the actual tree the rest of this course keeps using
The parser: a stack of currently-open elements; opentag pushes, closetag pops (searching down the stack, not just checking the top), voidtag is never pushed at all
Verified: a stray, mismatched closing tag is silently ignored rather than crashing the parser
Verified — the real quirk: an open <p> is implicitly closed by a following block-level element (another p, a div, a heading, a ul) even with no </p> anywhere in the source
Verified — the contrast: non-block elements like span never trigger this rule; nesting stays intact
Verified — an honest limitation: <li> needs its own analogous rule this chapter's own parser doesn't implement; a second <li> nests incorrectly instead of closing the first
Next chapter: Tokenizing and Parsing CSS — building the stylesheet half of the pipeline this course's own style tree will need

Parsing HTML into a DOM Tree

Three Real Node Types

The Parser: a Stack of Currently-Open Elements

The Real Quirk: <p> Closes Itself When It Has To

A Real, Honest Limitation — Found by Testing It

Where This Connects

Hands-On Exercises

Chapter 3 Quick Reference

The Real Quirk: `<p>` Closes Itself When It Has To