Named Entity Extractor: Structured Information From Real Documents

What We're Building

An entity extractor identifying people, organizations, and locations across a real batch of documents — trained on CoNLL-2003, the canonical NER benchmark (comparable in stature to MNIST for images or IMDb for sentiment), then run over genuinely new text to produce structured, per-document extracted-entity output.

Step 1: A Real Labeling Scheme, More Detailed Than nlp1-7's Own

# CoNLL-2003 format: one token and tag per line
Sarah    B-PER
Chen     I-PER
works    O
at       O
New      B-ORG
York     I-ORG
Times    I-ORG

Step 2: Vocabulary — Reusing Chapter 1's Own Approach

from collections import Counter

all_tokens = [tok for sentence in sentences for tok, tag in sentence]
vocab = {word: i + 2 for i, (word, _) in enumerate(Counter(all_tokens).most_common(15000))}
vocab["<pad>"], vocab["<unk>"] = 0, 1

tag_set = sorted({tag for sentence in sentences for _, tag in sentence})
tag2idx = {tag: i for i, tag in enumerate(tag_set)}

Same frequency-capped vocabulary pattern dsproj2-1 established — plus a second small vocabulary, this time for the tag set itself.

Step 3: The Exact BiLSTM Tagger From nlp1-7

import torch.nn as nn

class SequenceLabeler(nn.Module):
    def __init__(self, vocab_size, embed_dim, hidden_dim, num_tags):
        super().__init__()
        self.embedding = nn.Embedding(vocab_size, embed_dim, padding_idx=0)
        self.lstm = nn.LSTM(embed_dim, hidden_dim, batch_first=True, bidirectional=True)
        self.output = nn.Linear(hidden_dim * 2, num_tags)

    def forward(self, x):
        embedded = self.embedding(x)
        outputs, _ = self.lstm(embedded)
        return self.output(outputs)   # one prediction per token, per nlp1-7

Character for character nlp1-7's own architecture — every hidden state kept, bidirectional for exactly the "Washington could be a place or a person" reason that chapter demonstrated.

Step 4: Training — A Real Batching Gotcha Toy Data Never Surfaced

import torch

loss_fn = nn.CrossEntropyLoss(ignore_index=tag2idx["<pad>"])

for epoch in range(5):
    for X_batch, y_batch in train_loader:
        optimizer.zero_grad()
        logits = model(X_batch)                     # (batch, seq_len, num_tags)
        loss = loss_fn(logits.view(-1, len(tag_set)), y_batch.view(-1))
        loss.backward()
        optimizer.step()

Step 5: Evaluating — The Accuracy Paradox, Again, in a New Form

from sklearn.metrics import classification_report

print(classification_report(all_true_tags, all_predicted_tags))

Step 6: Extracting Structured Output From New Documents

def extract_entities(text, model, vocab, idx2tag):
    tokens = text.split()
    ids = torch.tensor([[vocab.get(t, 1) for t in tokens]])
    predicted_tags = model(ids).argmax(dim=-1)[0]

    entities = []
    current = []
    for token, tag_idx in zip(tokens, predicted_tags):
        tag = idx2tag[tag_idx.item()]
        if tag.startswith("B-"):
            if current:
                entities.append(" ".join(current))
            current = [token]
        elif tag.startswith("I-") and current:
            current.append(token)
        else:
            if current:
                entities.append(" ".join(current))
            current = []
    return entities

This is the genuine payoff of the B-/I- scheme from Step 1 — reassembling consecutive B-/I- tokens back into whole multi-word entity strings, turning a stream of per-token tags into the structured, readable output the chapter's own title promised.

What's Next

Chapter 7: Using an LLM API as a Data Labeling Tool — a light, deliberate touch of llm1/claude-adv1, using an LLM as one stage inside a larger data pipeline rather than training a model from scratch.