ds1-1 drew a line through the data science workflow — Collect → Clean → Explore → Model → Communicate — and marked "Model" out of scope for that entire course. ds1-10's own closing scope note named exactly why: "that's ml1's entire job." This is that course, starting exactly at that boundary.

Learned Rules vs. Hand-Coded Rules

historyai2-6 covered the 1980s expert-systems boom — DENDRAL, MYCIN, XCON/R1 — real, commercially deployed systems that made genuinely useful decisions (MYCIN recommended antibiotic treatments) using large sets of IF...THEN rules that human domain experts wrote out by hand, one rule at a time, encoding their own explicit knowledge directly into code.

A machine learning model does the opposite. Instead of a human writing "if mileage is high and the car is old, then price is low," an algorithm is shown many real examples — actual cars, with their actual mileage, age, and price — and it discovers a pattern connecting them on its own, without anyone ever writing that rule down explicitly. The "knowledge" ends up encoded in a set of learned numbers (later chapters call these coefficients or weights), not in a rule a person could read and have written themselves.

Three Branches

This Course's Own Two Running Case Studies

Rather than invented, disconnected examples, this course deliberately reuses two real datasets ds1 already built and genuinely didn't finish with:

• ds1-9's own used-car listings — its own Step 7 hypothesis, "does mileage predict price more strongly than year does?", was raised and explicitly left untested. ml1-3 tests it for real.
• ds1-10's own employee-attrition table — its own Step 7 hypothesis about salary, and its own unresolved department/salary confounding question, were both raised and explicitly left untested. ml1-5 tests both for real.

The General Workflow — This Course's Own Roadmap

Hands-On Exercises

ml1-1's own roadmap named this as the very first practical step, before any model gets trained at all. It has to come first — every technique in this course depends on it being done honestly.

Why You Can't Evaluate a Model on Its Own Training Data

Imagine grading a student using the exact same practice questions they studied from, answer key included. A perfect score wouldn't tell you whether they actually understand the material — only whether they memorized those specific answers. A machine learning model has exactly the same failure mode: shown enough examples, it can simply memorize the relationship between each specific input and its specific answer, rather than learning a pattern that generalizes to a new example it's never seen. Testing it on that same training data would make even a model that did nothing but memorize look flawless — and tell you nothing about whether it would be any good on a genuinely new used car it's never encountered.

The Train/Test Split

The practical fix: split the dataset into two pieces before training anything. The training set is what the model actually learns from. The test set is held back, completely untouched during training, and used only afterward, to check how the model performs on data it has genuinely never seen.

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

An 80/20 or 70/30 split is common — a convention, not a law. random_state fixes the random shuffle so the same split can be reproduced later, useful for genuinely comparing two different models on identical data.

A Subtler Trap: Leaking Information Through Repeated Peeking

A train/test split alone doesn't fully solve the problem. If a model is trained, checked against the test set, adjusted based on what went wrong, checked against the test set again, adjusted again — the test set is no longer serving its original purpose. Even without formally training on it, repeatedly using test-set performance to guide decisions lets information about the test set quietly leak into the model through every one of those adjustment cycles. The final "test accuracy" ends up flattered by the same underlying problem this chapter opened with, just one step more indirect.

The Third Piece: A Validation Set

The fix is a third split: a validation set, used for exactly this kind of iterative tuning, while the test set stays completely untouched until one single, final check.

A First Preview of Cross-Validation

With a small dataset, a single validation split can just be unlucky — by chance, an unusually easy or unusually hard subset. Cross-validation gets more reliable mileage out of limited data by rotating which portion serves as the validation set across several rounds, then averaging the results. This chapter only needs the concept to exist; ml1-8 delivers the full technique, once ml1-3–ml1-7 have given it real models to actually validate.

Why the Split Itself Has to Be Random

Splitting a dataset without shuffling it first is a real, easy-to-miss trap. If ds1-9's own used-car listings were sorted by year and split straight down the middle with no shuffling, the test set could end up containing only the newest cars while training saw only older ones — the model would never see a genuinely representative mix of ages during training, and the test set wouldn't represent the real population of cars it's meant to evaluate against either.

Hands-On Exercises

ds1-9's own used-car dataset returns. Its Step 7 hypothesis — "does mileage predict price more strongly than the car's year does?" — was raised from a heatmap and explicitly left untested. This chapter tests it for real.

What Linear Regression Actually Does

Linear regression fits a straight line (or, with more than one feature, a flat hyperplane) through data to predict a continuous numeric value — a price, not a category. With one feature: price = w × mileage + b. With several features at once (this chapter's own case): price = w₁ × mileage + w₂ × year + b. Each w is a learned coefficient; b is the intercept — what the model would predict if every feature were zero.

How "Fitting" Actually Works

ds1-6 already defined variance as the average squared distance between each value and the mean. Linear regression's own fitting process — least squares — minimizes a close cousin of that same idea: the total squared distance between each actual price and the price the line would have predicted for it. "Squared" matters for the same reason it did in ds1-6: it penalizes large errors disproportionately more than small ones, and keeps positive and negative errors from canceling out.

Coefficients Are Interpretable — With One Real Caveat

Each coefficient answers a direct question: how much does the predicted price change per one-unit change in that feature, holding the other features constant? This is the actual mechanism for testing this chapter's own hypothesis.

Fitting the Model — ds1-9's Own Dataset, For Real

from sklearn.linear_model import LinearRegression
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import train_test_split

X = df[["mileage", "year"]]
y = df["price"]

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)

model = LinearRegression()
model.fit(X_train_scaled, y_train)
model.coef_       # [coefficient for mileage, coefficient for year]
model.intercept_  # b

Why a Fitted Coefficient Is Stronger Evidence Than a Raw Correlation

ds1-9's own heatmap already showed mileage and price correlating. ds1-6's own confounding-variable warning applies at full strength to that pairwise number alone — a raw correlation between two variables says nothing about a third, unmeasured factor possibly driving both. Multiple linear regression is a genuine, practical (if partial) step beyond that: fitting mileage and year together means each coefficient already accounts for the other — "holding year constant" is quite literally controlling for one of the two candidate confounders directly in the model itself, rather than leaving it unaccounted for.

What This Chapter Doesn't Yet Tell You

Knowing mileage matters more than year is a real finding — but it says nothing about how good this model actually is at predicting price overall. A model could correctly rank mileage as the stronger factor while still being wildly inaccurate in its actual dollar predictions. ml1-4 covers exactly that next.

Hands-On Exercises

ml1-3 closed one question (which feature matters more) and left another open on purpose: is the model's own price prediction actually any good? Four metrics, applied directly to ml1-3's own fitted model, answer that.

MAE — The Plain-English Metric

from sklearn.metrics import mean_absolute_error
mae = mean_absolute_error(y_test, model.predict(X_test_scaled))

Directly readable: "on average, this model's price predictions are off by about $X." Every error contributes to the average in exact proportion to its own size — a $10,000 miss counts exactly ten times as much as a $1,000 miss, no more, no less.

MSE — The Training Objective, Reused as a Metric

ml1-3 already explained least squares as minimizing total squared error during fitting. MSE is that exact same squared-error idea, averaged, and computed on the test set instead of the training set — the fitting objective, repurposed as an honest, out-of-sample evaluation number. The squaring inherits ds1-6's own variance-style property: large errors are penalized disproportionately more than small ones. The real cost: the result is in dollars squared, a unit with no direct real-world meaning.

RMSE — MSE's Own Sensitivity, Back in Real Units

from sklearn.metrics import mean_squared_error
import numpy as np
rmse = np.sqrt(mean_squared_error(y_test, model.predict(X_test_scaled)))

RMSE is simply MSE's own square root — dollars-squared brought back to dollars, while keeping MSE's own large-error sensitivity intact. RMSE is mathematically guaranteed to be greater than or equal to MAE on the same data, and the gap between them is informative in its own right: a large gap means a few big misses are driving the error total; a small gap means errors are fairly uniform in size.

R² — A Different Kind of Question Entirely

R² doesn't measure error size at all — it measures what proportion of price's own total variance (ds1-6's own vocabulary, directly) the model actually accounts for. R² = 1.0 means the model explains all of it (a perfect fit); R² = 0 means the model does no better than simply always predicting the average price; a negative R² means the model is doing worse than that trivial baseline.

from sklearn.metrics import r2_score
r2 = r2_score(y_test, model.predict(X_test_scaled))

Applying All Four to ml1-3's Own Model

Choosing a Metric

Hands-On Exercises

ds1-10's own employee-attrition table returns. Two things it deliberately left open: "does lower salary predict a higher likelihood of leaving?", and a genuine, unresolved confounding-variable question — "does department matter, or is department's own apparent effect actually explained by department-level salary differences instead?" This chapter tests both, for real.

Why ml1-3's Own Tool Doesn't Fit Here

left_company is Yes/No — a category, not a continuous number. Fitting ml1-3's own plain linear regression directly against a 0/1-coded target would produce genuine nonsense: nothing stops the fitted line from predicting -0.3 or 1.4 for some employees — numbers with no sensible reading as "how likely is this person to leave."

Logistic Regression — Same Linear Core, a New Final Step

Logistic regression keeps ml1-3's own linear combination of weighted features underneath — this is genuinely still "regression" in that sense, which is exactly why the confusing name persists — but passes the result through the sigmoid function before treating it as an answer. Sigmoid takes any real number and squashes it into the range (0, 1): very negative inputs approach 0, very positive inputs approach 1, and an input of exactly 0 lands at 0.5. The output is now a genuine probability — "how likely is this employee to leave?" — never a nonsensical value like -0.3.

From Probability to a Decision — The Threshold

A probability alone isn't yet a Yes/No prediction. A threshold — commonly 0.5 — converts it: probability above the threshold predicts Yes, below predicts No. 0.5 is a genuine, adjustable choice, not a law of nature; ml1-6's own precision/recall material covers exactly why and when moving that threshold matters.

A New Wrinkle: Encoding department

ml1-3's own two features were already numeric. department is categorical text (Sales, Engineering, Support) — a model can't multiply a weight by a word. One-hot encoding converts one categorical column into several binary columns, one per category (is_sales, is_engineering, is_support, each 0 or 1), so each category gets its own learnable weight.

from sklearn.linear_model import LogisticRegression
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import train_test_split

X = pd.get_dummies(df[["department", "age", "years_at_company", "salary"]])
y = df["left_company"].map({"Yes": 1, "No": 0})

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)

model = LogisticRegression()
model.fit(X_train_scaled, y_train)
model.coef_

Testing ds1-10's Own Confounding Question — For Real

This is the payoff ds1-10 itself couldn't reach: because department's own one-hot columns and salary are now fit together, the model can show whether department carries any real, independent weight once salary is already accounted for.

What This Chapter Hasn't Checked Yet

Same structure as ml1-3 into ml1-4: interpreting coefficients answered which factors matter, not how good this model actually is at correctly predicting who leaves. ml1-6 covers that next — and, for this particular dataset, honestly: attrition is a genuinely imbalanced target (far more "No" than "Yes" in most real companies), which makes the choice of evaluation metric matter more here than it did for ml1-4's own regression case.

Hands-On Exercises

ml1-5 answered which factors matter for attrition, not how good the model actually is at predicting who leaves. ml1-5 also flagged something ml1-4 never had to deal with: attrition is genuinely imbalanced — far more No than Yes in most real companies. That fact changes which metric is trustworthy here.

The Confusion Matrix — Four Real Outcomes, Not One

Every prediction on a binary target falls into exactly one of four buckets, comparing the model's own prediction against the real outcome:

Accuracy — And Why It's Dangerous Here Specifically

Accuracy is simply (correct predictions) ÷ (total predictions) — the single most intuitive metric, and, on an imbalanced target, a genuinely misleading one.

Precision & Recall — Answering Two Different Questions

Precision = True Positives ÷ (True Positives + False Positives) — of everyone the model flagged as a flight risk, what fraction actually left? A precision-focused model minimizes false alarms. Recall = True Positives ÷ (True Positives + False Negatives) — of everyone who actually left, what fraction did the model catch? A recall-focused model minimizes missed cases.

For attrition specifically, a missed at-risk employee (a False Negative) is usually the more expensive mistake — a resignation nobody saw coming, with no chance to intervene — while a false alarm (a False Positive) costs, at worst, an unnecessary retention conversation. This is a genuine, business-specific judgment call about which error type matters more, not a fact the data itself decides.

F1 — One Number, When You Need One

F1 is the harmonic mean of precision and recall — a single combined number that only stays high when both precision and recall are reasonably good, punishing a model that's strong on one and collapses on the other far more than a plain average would. Useful specifically when precision and recall need to be compared or ranked with one number rather than two.

Applying This to ml1-5's Own Model

from sklearn.metrics import confusion_matrix, precision_score, recall_score, f1_score, accuracy_score

preds = model.predict(X_test_scaled)
confusion_matrix(y_test, preds)
accuracy_score(y_test, preds)
precision_score(y_test, preds)
recall_score(y_test, preds)
f1_score(y_test, preds)

Adjusting the Threshold — ml1-5's Own Preview, Delivered

ml1-5 flagged the 0.5 threshold as adjustable, not fixed. Lowering it flags more employees as at-risk — recall rises (fewer real leavers slip through), precision falls (more false alarms). Raising it does the reverse. Given this chapter's own reasoning that a missed at-risk employee is usually the costlier mistake for this specific problem, a real deployment might deliberately lower the threshold below 0.5, accepting more false alarms in exchange for catching more genuine flight risks — a business decision the metrics inform but don't make automatically.

Hands-On Exercises

ml1-3 and ml1-5 both fit a smooth mathematical formula — a weighted sum of features, squashed through a sigmoid for classification. This chapter introduces a structurally different family entirely: a model that looks, on paper, almost exactly like a flowchart of yes/no questions.

What a Decision Tree Actually Is

A decision tree is a series of nested yes/no questions about features — "is salary below $50,000?" — each one branching to a child node, until a leaf gives a final prediction. Unusually, the same structure works for both jobs this course has covered separately: predicting a category (a leaf holding a class label) or a continuous number (a leaf holding an average value) — one model family doing what took ml1-3 and ml1-5 two separate techniques to cover.

The Uncanny Resemblance to ml1-1's Own Expert Systems

How the Tree Actually Picks Its Own Questions

Entropy measures how mixed a group of labels is: a node where every single employee left has entropy 0 (perfectly "pure" — no uncertainty left at all); a node split evenly 50/50 between leaving and staying has maximum entropy (as uncertain as a coin flip). Information gain measures how much a candidate split would reduce that entropy. At every node, the tree-building algorithm greedily tries every possible feature and threshold, and picks whichever single split produces the largest information gain — the biggest reduction in mixed-up-ness.

A Worked Split — And an Independent Confirmation

from sklearn.tree import DecisionTreeClassifier

tree = DecisionTreeClassifier(max_depth=4, random_state=42)
tree.fit(X_train, y_train)

The Clearest Overfitting Example in This Course

Left unconstrained, a decision tree can keep splitting until individual leaves contain a single training example each — a tree that has, in effect, memorized the training set row by row. ml1-2's own warning about training-data evaluation becomes vividly concrete here: an unconstrained tree can reach close to 100% training accuracy while performing noticeably worse on the test set — the most visually obvious overfitting example this course covers, more intuitive than either regression model's own more abstract version of the same failure.

Random Forests — Averaging Away the Overfitting

A random forest builds many individual trees — each trained on a random, bootstrapped subset of the training rows, and restricted to a random subset of features at each split — then combines their predictions (majority vote for classification, average for regression).

from sklearn.ensemble import RandomForestClassifier

forest = RandomForestClassifier(n_estimators=200, random_state=42)
forest.fit(X_train, y_train)

Why this actually works: each individual tree, trained on its own random slice of data, tends to overfit in its own idiosyncratic way — memorizing quirks specific to whichever rows and features it happened to see. Averaged across many differently-trained trees, those individual idiosyncrasies tend to cancel each other out, while the genuine, real pattern every tree independently rediscovers — ds1-6's own variance vocabulary applies directly here — survives the averaging. The forest's own predictions end up with meaningfully lower variance than any single tree's own predictions, without needing to constrain any individual tree very much at all.

Hands-On Exercises

ml1-7 showed the clearest possible overfitting example in this course — a tree memorizing training rows one at a time. ml1-2 previewed cross-validation and promised this chapter would deliver it in full. Both threads close here.

Two Failure Modes, Formally

The diagnostic signature is the gap: overfitting shows low training error but high test error; underfitting shows high error on both — the model was never good, even on data it directly saw.

The Bias-Variance Tradeoff — Built on ds1-6's Own Vocabulary

Bias is systematic, consistent wrongness — error from a model too simple to capture the real pattern, present regardless of which training sample it happened to see (underfitting). Variance reuses ds1-6's own definition directly: how much would this model's own predictions change if trained on a different random sample from the same population? A single unconstrained tree (ml1-7) has high variance — a different training sample would grow a noticeably different tree. A random forest has lower variance — averaging many trees stabilizes the result regardless of which specific sample any one tree happened to see.

Reading a Learning Curve

Plotting training error and validation error against training-set size (or model complexity) gives a real diagnostic picture. High bias shows both curves converging to a similarly poor error — more data doesn't help, because the model itself is too simple to use it. High variance shows a persistent gap — low training error, meaningfully higher validation error — that more data typically narrows, since a larger, more representative sample gives a flexible model less room to fit pure noise.

Regularization — Deliberately Trading Bias for Variance

Regularization adds a penalty term to ml1-3's own least-squares fitting objective, discouraging large coefficient values. This is the bias-variance tradeoff, applied on purpose: a regularized model fits the training data very slightly worse (a bit more bias) in exchange for being noticeably less sensitive to the specific training sample it saw (less variance).

from sklearn.linear_model import Ridge, Lasso

ridge = Ridge(alpha=1.0)   # L2
lasso = Lasso(alpha=1.0)   # L1

Cross-Validation — ml1-2's Own Promise, Delivered

ml1-2 flagged the real risk: a single validation split can be unlucky on a small dataset. k-fold cross-validation fixes this directly: split the data into k equal folds, train on k−1 of them and validate on the one held out, rotate which fold is held out across k full rounds, then average the k resulting scores.

from sklearn.model_selection import cross_val_score

scores = cross_val_score(model, X, y, cv=5)
scores.mean(), scores.std()

Every row gets used for validation exactly once and for training k−1 times — a far more reliable estimate of real performance than any single split, and the standard deviation across the k scores is itself informative: a small spread means the model performs consistently regardless of which slice of data it sees; a large spread is itself a sign of high variance.

A Practical Checklist

Hands-On Exercises

ml1-1 named supervised and unsupervised learning as this course's own two branches — every chapter since has been supervised, working from a known, provided answer (price, left_company). This chapter is the other branch, made concrete rather than abstract: the exact same datasets, with every label deliberately removed.

What Changes When There's No Label

Every technique since ml1-3 learned by comparing its own predictions against a known, correct answer during training. Clustering has no such answer to learn from at all — it's asked only to group similar data points together, discovering whatever structure exists on its own, with nobody ever telling it what the "right" groups are.

K-Means — The Algorithm, Step by Step

A genuinely different fitting mechanism than anything earlier in this course: no least-squares fit against a known target (ml1-3), no entropy-driven split search against known labels (ml1-7) — k-means iteratively refines its own evolving guess about where the clusters are, with nothing external to check itself against.

Choosing k — The Elbow Method

Unlike supervised learning, where the number of real classes is simply given (ml1-5's own two — Yes/No), nothing hands k to this chapter directly. Plotting inertia (the total squared distance from each point to its own assigned centroid) against increasing values of k always trends downward — more clusters always let points sit closer to their own center — but with sharply diminishing returns past a certain point. The "elbow" — where adding another cluster stops meaningfully reducing inertia — is a reasonable, practical choice for k.

Running It for Real — ds1-9's Own Used-Car Data, Unlabeled

from sklearn.cluster import KMeans
from sklearn.preprocessing import StandardScaler

X = df[["mileage", "year", "price"]]
X_scaled = StandardScaler().fit_transform(X)

kmeans = KMeans(n_clusters=3, random_state=42)
clusters = kmeans.fit_predict(X_scaled)

Running It on ds1-10's Own Employees — A Genuinely New Finding

X = df[["age", "years_at_company", "salary"]]
X_scaled = StandardScaler().fit_transform(X)
clusters = KMeans(n_clusters=3, random_state=42).fit_predict(X_scaled)

Why Scaling Isn't Optional Here

ml1-3 standardized features so coefficient sizes could be compared fairly. K-means needs standardization for a stricter reason: the algorithm's own "nearest centroid" step is a literal distance calculation. Left unscaled, salary (tens of thousands) would swamp years_at_company (single digits) in every distance computation, and the resulting clusters would essentially just be salary bands wearing three other features' names. Scaling here isn't a nicety for readability — it determines whether the clustering result means anything at all.

Crisp Assignment — And a Preview of ch10

Every point in k-means belongs to exactly one cluster, fully — a crisp assignment, with no notion of "70% in this cluster, 30% in that one." ml1-10's own Fuzzy Logic chapter picks up exactly here, with a system built specifically around partial, graded membership instead.

Hands-On Exercises

ml1-9 closed on a deliberate setup: k-means gives every point crisp membership — fully in exactly one cluster, never partially in two. This chapter is that setup's own payoff: a system built specifically around membership that doesn't have to be all-or-nothing.

Crisp Logic Is Everywhere in This Course So Far, Once You Look

Classical Boolean logic says a statement is fully true or fully false — an element either is or isn't in a set, 1 or 0, nothing between. This isn't just ml1-9's own clustering; ml1-7's own decision tree splits are crisp too. "Is salary below $50,000?" draws a hard line: someone earning $49,999 and someone earning $50,001 — practically identical — land in completely different branches, with zero partial credit for how close the second person actually sits to the boundary.

Fuzzy Sets — Partial, Graded Membership

A fuzzy set allows membership to be any value between 0 and 1, not just the two endpoints. Take "long tenure": crisp logic needs a hard cutoff (years_at_company ≥ 5 = long, full stop). A fuzzy set instead assigns a graded degree of membership — four years might sit at 0.6 membership in "long tenure," eight years at 0.9 — genuinely closer to how people actually reason about vague, real-world categories, where "long" isn't a single sharp line anyone actually applies mentally.

Membership Functions

A membership function is what actually assigns each possible input a degree of membership in a fuzzy set — commonly triangular, trapezoidal, or S-shaped curves, plotted against the input value.

Fuzzy Inference Systems

Fuzzy sets combine using fuzzy versions of AND/OR/NOT (typically minimum, maximum, and complement), feeding into fuzzy IF...THEN rules — IF tenure is long AND salary is low THEN attrition_risk is high — where "long," "low," and "high" are all fuzzy sets with their own membership functions, and the rule's own output is itself a graded degree, not a hard yes/no.

The Real Historical Link to ml1-1's Own Expert Systems

Fuzzy logic and historyai2-6's own expert systems emerged in the same classical-AI era, and were often combined directly into "fuzzy expert systems." The rules in a fuzzy inference system are, like MYCIN's own, still typically hand-written by a human expert — ml1-1's own learned-vs-hand-coded distinction still applies exactly as before. What fuzzy logic actually changed was narrower and more specific: ml1-1's own tip-box already named brittleness as one of the real, documented reasons expert systems declined — a crisp rule system handling a value right at a boundary badly, with no graceful middle ground. Fuzzy logic was a genuine, historical attempt to fix exactly that brittleness, letting hand-coded rules degrade gracefully near a boundary instead of snapping sharply across it.

Back to ml1-9's Own Employee Clusters

Where Fuzzy Logic Actually Lives Today

Fuzzy logic isn't a mainstream competitor to ml1-3's regression, ml1-7's trees, or nn1's eventual neural networks for prediction tasks like this course's own two case studies. Its real, ongoing home is control systems — washing machines adjusting cycle length to load size, camera autofocus, some industrial process control — genuine, still-deployed applications where reasoning gracefully over graded inputs matters more than raw predictive accuracy on a labeled dataset.

Hands-On Exercises

Every chapter since ml1-3 has quietly used the same shape: fit() to train, predict() to apply, some scoring function to evaluate. This capstone makes that consistency explicit, and closes the loop this entire course opened — bringing both of its own resolved hypotheses together into one final pipeline.

The Consistent scikit-learn API — A Genuine Design Strength

Four genuinely different algorithm families — a smooth linear fit, a sigmoid-squashed linear fit, a greedy entropy-splitting tree, an iterative centroid-refinement process with no target at all — and switching between them required almost no change to the surrounding code. That consistency isn't an accident; it's a deliberate design choice this whole course has been quietly relying on since ml1-3.

Pipeline 1 — Closing ml1-3's Own Regression Story

from sklearn.model_selection import train_test_split, cross_val_score
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LinearRegression, Ridge
from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score
import numpy as np

X = df_cars[["mileage", "year"]]
y = df_cars["price"]

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)  # ml1-2

scaler = StandardScaler()
X_train_s = scaler.fit_transform(X_train)
X_test_s = scaler.transform(X_test)

model = Ridge(alpha=1.0)   # ml1-3's own LinearRegression, ml1-8's own L2 regularization applied
model.fit(X_train_s, y_train)

preds = model.predict(X_test_s)
mae = mean_absolute_error(y_test, preds)              # ml1-4
rmse = np.sqrt(mean_squared_error(y_test, preds))      # ml1-4
r2 = r2_score(y_test, preds)                           # ml1-4

cv_scores = cross_val_score(model, X_train_s, y_train, cv=5)   # ml1-8

Pipeline 2 — Closing ml1-5's Own Classification Story

from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import confusion_matrix, precision_score, recall_score, f1_score

X = pd.get_dummies(df_emp[["department", "age", "years_at_company", "salary"]])
y = df_emp["left_company"].map({"Yes": 1, "No": 0})

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)  # ml1-2
X_train_s = scaler.fit_transform(X_train)
X_test_s = scaler.transform(X_test)

logit = LogisticRegression()          # ml1-5
logit.fit(X_train_s, y_train)

forest = RandomForestClassifier(n_estimators=200, random_state=42)   # ml1-7
forest.fit(X_train, y_train)          # trees don't require scaling — a real, honest asymmetry worth noting

for name, preds in [("Logistic", logit.predict(X_test_s)), ("Forest", forest.predict(X_test))]:
    print(name, precision_score(y_test, preds), recall_score(y_test, preds), f1_score(y_test, preds))  # ml1-6

forest.feature_importances_   # does the forest agree with ml1-5's own salary finding?

A Final Coda — Revisiting ml1-9 and ml1-10

Rerunning ml1-9's own unsupervised clustering on both datasets one last time is worth doing here specifically because it asks a genuinely different question than either pipeline above — not "does this predict the target," but "what structure exists regardless of any target at all." And if either dataset had employees sitting persistently near a cluster boundary, unable to be assigned crisply with real confidence, ml1-10's own fuzzy membership approach — not implemented here, but conceptually available — is exactly where that specific problem would be addressed, rather than by forcing ml1-9's own crisp k-means to make an arbitrary, boundary-splitting call.

Chapter Attribution

Honest Scope Note

Hands-On Exercises