Trigram Language Model
A dependency-free statistical language model applied to essay-proficiency classification.
The Problem
Before neural language models, statistical n-gram models were the standard approach to scoring how fluent a piece of text is — and they're still a clean way to build a fully interpretable, explainable classifier.
The Approach
Built a trigram language model from scratch — pure Python, no libraries — with linearly-interpolated smoothing across unigram/bigram/trigram probabilities, then applied it to a real, practical task: classifying TOEFL essays by English proficiency level based on which of two differently-trained models finds the essay more "fluent" (lower perplexity).
Try It Live
A live trigram model trained right now, in your browser, on a small demo corpus — using the same fixed 1/3 linear-interpolation smoothing as the real project (real language models need far more text than is practical to ship to a browser, so expect it to be charmingly repetitive rather than eloquent).
- ▹Counts unigrams, bigrams, and trigrams in separate hash maps; raw MLE probability = count(w₁,w₂,w₃) / count(w₁,w₂), with a 1/N fallback to avoid divide-by-zero on unseen contexts
- ▹Smoothing: linear interpolation with fixed, equal weights (λ₁=λ₂=λ₃=1/3) across all three n-gram orders — a deliberately simple, fixed-weight scheme rather than a tuned or Kneser-Ney backoff
Highlights
- ▹Fully dependency-free — no NumPy, no NLTK, just the Python standard library
- ▹Applies the language model to a genuinely interpretable classification task (essay proficiency via perplexity comparison) rather than stopping at a perplexity number
- ▹Evaluated on the Brown Corpus with proper START/STOP padding and closed-vocabulary UNK handling
- ▹Correct perplexity computation (2^(−average log₂ probability)) using consistent base-2 logs throughout