Exercise 2: Why NER Needs Softmax Over Multiple Tags, Not nlp1-6's Own Sigmoid — Possible Solution ==================================================================== WHAT nlp1-6's OWN SIGMOID OUTPUT ACTUALLY REPRESENTED ------------------------------ Per nlp1-6, the sentiment task was binary — a review is either positive or negative, nothing else. A single sigmoid-activated output neuron is exactly right for this: sigmoid squashes its input into a value between 0 and 1, which can be read directly as "the probability of the positive class," with the negative class implicitly being 1 minus that value. This is exactly nn1-1's own logistic regression neuron, reused unchanged. WHY NER IS A GENUINELY DIFFERENT KIND OF PROBLEM ------------------------------ Per this chapter, NER tags include PER, ORG, LOC, and O (at minimum) — four or more distinct, mutually exclusive categories, not two. A single sigmoid neuron has no way to represent "which one of four-plus categories" — it only ever produces one number between 0 and 1, which can only meaningfully encode a two-way choice. WHAT WOULD ACTUALLY GO WRONG IF SIGMOID WERE REUSED UNCHANGED ------------------------------ If the output layer were left as a single sigmoid neuron and simply pointed at an NER task, the model could only ever express something like "how confident am I in class 0 versus class 1" — there would be no way to distinguish PER from ORG from LOC from O at all, since the architecture itself only has room to represent two outcomes. Training would be attempting to teach a two-way discriminator to make a four-way distinction it structurally cannot represent, regardless of how much data or how long it trained. WHY SOFTMAX OVER num_tags OUTPUTS SOLVES THIS ------------------------------ This chapter's own classifier head produces `num_tags` separate output values — one score per possible tag — and softmax converts that set of scores into a proper probability distribution across all of them simultaneously (all values between 0 and 1, summing to exactly 1). This gives the model exactly as many "slots" as there are real tag categories, letting it express "probably ORG, unlikely PER, very unlikely LOC" in a way a single sigmoid value structurally cannot. WHY THIS IS THE SAME KIND OF DISTINCTION MADE ELSEWHERE ON THIS SITE ------------------------------ This mirrors the same binary-vs-multiclass distinction between logistic regression (nn1-1, ml1-5) and multiclass softmax classifiers covered elsewhere in the Data Science & ML subject — the underlying principle (match the number of output "slots" to the number of real possible categories) isn't NLP-specific, it's a general classifier-design rule this chapter is simply applying to a new task. WHY THIS WORKS AS AN ANSWER ------------------------------ It explains precisely what nlp1-6's own sigmoid neuron could represent (a single binary choice), shows why NER's multiple mutually exclusive tag categories exceed what that representation can express, and explains how softmax over num_tags outputs provides the correct number of "slots" the task actually requires.