Neural Network Dependency Parser
A from-scratch neural reimplementation of the classic Chen & Manning transition-based parser.
PythonPyTorchNLPDependency Parsing
The Problem
Dependency parsing — figuring out which words in a sentence grammatically depend on which — is a foundational NLP task that classical rule-based parsers generalize poorly on.
The Approach
Reimplemented Chen & Manning's (2014) neural transition-based parsing architecture in PyTorch: an arc-standard shift-reduce parser guided by a small neural network that greedily predicts the next parsing action at each step.
Try It Live
The real parser is a neural network — this walks through the same arc-standard transition system (shift / left-arc / right-arc) it's trained to imitate, step by step, on an oracle-derived action sequence for one sentence.
thedogchasedthecat
Stack
ROOT
Buffer
thedogchasedthecat
Next: SHIFT
- ▹Model: word embeddings (128-dim) for the top-3 stack and top-3 buffer words, flattened to 768 dims → Linear+ReLU (128) → Linear (91 outputs: shift + 45 labels × 2 arc directions)
- ▹Trained with Adagrad (lr=0.01), batch size 16, 5 epochs, cross-entropy loss
Highlights
- ▹Oracle-based training data generation — walks each gold-standard tree to derive the exact shift/left-arc/right-arc sequence that reproduces it, then trains on those derived action sequences
- ▹POS-aware unknown-word handling: unknown proper nouns map to a distinct ⟨NNP⟩ token and unknown numbers to ⟨CD⟩, rather than collapsing all unknowns into one generic ⟨UNK⟩
- ▹Greedy decoder validates structural legality (e.g. buffer must be non-empty to shift) before executing the highest-scored action, rather than blindly trusting the model
- ▹Full evaluation harness computing both UAS and LAS, each micro- and macro-averaged