Exercise 1: Why Text Is Data Cleaning's "Most Extreme Case Yet" — Possible Solution ==================================================================== WHAT ds1-4 ORIGINALLY COVERED ------------------------------ Per this chapter, "ds1-4 covered missing values, duplicates, inconsistent formatting, and string cleaning — real problems, but confined to a handful of structured columns with a known, fixed shape." Every dataset ds1-4 worked with had a defined set of columns, each with an expected type and range of values — messy, but bounded and predictable in its own overall shape. WHY TEXT HAS ALL THE SAME PROBLEMS ------------------------------ A document full of raw text can still have missing content, duplicated passages, and inconsistent formatting (capitalization, spacing, punctuation) — exactly the same categories of problem ds1-4 already named. Nothing about moving to text makes those specific issues disappear. WHY TEXT ALSO HAS PROBLEMS ds1-4's OWN DATA NEVER HAD ------------------------------ Per this chapter, text has "none of the fixed structure: no defined length, no fixed vocabulary, genuine ambiguity (the same word meaning different things in different contexts), and grammar and word order that carry real information a plain column of numbers never has to represent at all." A numeric column has a known type and a bounded, predictable range of values; a block of text has no fixed length, no fixed set of possible "values" (an essentially unbounded vocabulary), and — unlike a number, which means exactly what it says — the same literal word can carry entirely different meanings depending on surrounding context, something no numeric or simple categorical column in ds1-4's own material ever needed to account for at all. WHY THIS JUSTIFIES "MOST EXTREME CASE," NOT MERELY "A DIFFERENT KIND" ------------------------------ Calling text simply "a different kind of data" would understate the relationship — it would imply text's own problems are unrelated to ds1-4's own material, requiring an entirely separate set of concerns. Instead, text problems are a strict superset: everything ds1-4 already covered (missing/duplicate/inconsistent data) still applies, PLUS a genuinely new layer of problems (unbounded vocabulary, ambiguity, meaningful order) that ds1-4's own structured-column material never had to address. This is precisely what makes "most extreme case yet" the accurate framing — an escalation and extension of the same underlying concerns, not a break from them. WHY THIS WORKS AS AN ANSWER ------------------------------ It shows precisely which of ds1-4's own named problems carry over unchanged into text, and precisely which new problems text adds on top that ds1-4's own structured data never had to handle — together justifying "most extreme case yet" as an escalation of the same underlying challenge, not an unrelated new one.