Exercise 3: A String Value Containing Real Keyword Text — Possible Solution
====================================================================

THE TEST
------------------------------
    stmt = Parser(tokenize("INSERT INTO users VALUES (2, 'SELECT * FROM WHERE');")).parse_statement()
    stmt.values

RESULT
------------------------------
    [2, 'SELECT * FROM WHERE']

The second value comes back as one single, intact string -- exactly
the text between the quotes -- not as five separate keyword/punct
tokens (SELECT, *, FROM, WHERE) the way it would if the tokenizer
tried to re-scan the quoted content as ordinary SQL.

WHICH PART OF THE TOKENIZER GUARANTEES THIS
------------------------------
The string-literal branch is:

    if ch == "'":
        start = i
        i += 1
        while i < n and sql[i] != "'":
            i += 1
        tokens.append(Token('STRING', sql[start + 1:i]))
        i += 1
        continue

Once the tokenizer's own main scanning loop sees an opening single
quote, it enters this branch and does ONE thing only: advance `i`
character by character until it finds the matching CLOSING quote,
completely regardless of what characters appear in between. The inner
`while i < n and sql[i] != "'":` loop has no logic anywhere that
checks whether the characters it's skipping over happen to spell out
a real keyword, an operator, a number, or anything else meaningful --
it purely looks for one specific character (the closing quote) and
otherwise treats every byte in between as opaque content to be
included in the resulting STRING token's own text.

This is structurally identical to how the tokenizer's own keyword/
identifier branch never runs at all while inside this branch --
control flow only reaches the `ch.isalpha()` check (the one that would
recognize "SELECT" as a keyword) on the OUTER loop's own NEXT
iteration, and by the time that next iteration starts, `i` has already
been advanced all the way past the closing quote, to whatever real SQL
comes after the string literal in the source.

WHY THIS WORKS AS AN ANSWER
------------------------------
This confirms tokenizing a quoted string is a genuinely separate mode
from tokenizing ordinary SQL text -- the tokenizer doesn't tokenize
"inside" a string and then reinterpret the results; it simply
DELIMITS the string (find where it starts, find where it ends) and
takes the raw substring in between as-is. A real database absolutely
needs this property: a user might legitimately want to store a string
value that happens to contain SQL-looking text (a stored example
query, a piece of user-submitted content quoting SQL, and so on)
without that text being misinterpreted as part of the actual query
structure.