Papers I find interesting, read closely, and rebuild: a plain-language summary, the math reimplemented from scratch in TypeScript, live charts, and — where the idea maps onto a 2D toy — a link to poke at it on the model pages. Same rules as the rest of the site: no ML libraries, every number computed in your browser.
Speculative decoding verifies cheap guesses in parallel; we rebuild its exact sampler and find speed depends on agreement and spare compute.
ViT turns image patches into ordinary Transformer tokens; we rebuild the input pipeline and find scale—not patching alone—made it beat convolutions.
FlashAttention streams softmax tiles instead of storing an attention matrix; we rebuild the exact update and find the arithmetic stays quadratic.
DPO turns preference learning into logistic regression; we rebuild the loss and find every judgment is measured relative to a reference model.
LoRA replaces a dense weight update with two skinny matrices; we rebuild the merge and find the tiny adapter never replaces the frozen base model.
TIGER recommends products by generating semantic tokens; we rebuild its quantizer and find a three-order-of-magnitude arithmetic slip.
Exact match can turn smooth language-model improvement into an apparent breakthrough; we rebuild the metric and expose its hidden resolution limit.
DeepSeek-R1 used group-relative RL to reward better answers; we rebuild GRPO and find that all wins or all losses teach it nothing.
KANs replace weights with learnable splines; we rebuild the edge math and find the accuracy grid also taxes every connection.
Keeping old data stops recursive training error from exploding; we rebuild the proof and find “avoids collapse” does not mean nothing is forgotten.
Mamba replaces attention with a selective recurrence; we rebuild its memory gate and find why the fast algorithm is part of the idea.
Deep Hedging learns option strategies under real trading costs; a seeded reconstruction shows why its most useful action can be doing nothing.
Muon orthogonalizes matrix gradients and reportedly halves LLM training compute; we rebuild it and find raw update size silently depends on matrix shape.
BitNet packs 2B weights into 0.4 GB without losing much benchmark score; we rebuild its ternary math and find the savings begin after training.
A 2023 paper forecasts the future with a single linear layer and beats a stack of specialised Transformers. We rebuild it from scratch — and find the clever-looking decomposition barely matters, while the whole edge rests on one assumption nobody states.
After enough backtests, a Sharpe of 2 is what pure luck looks like. A 2014 paper computes exactly how much luck — and deflates the number back down. We rebuild it from scratch and find the verdict hinges on one figure nobody reports: how many strategies you tried.
A 2015 paper shows a tiny, invisible nudge flips a confident classifier — and argues the cause is that models are too LINEAR, not too complex. We rebuild the fast gradient sign method from scratch and watch the attack strengthen as the input grows.
NGBoost turns gradient boosting into a probability distribution with error bars. We rebuild it from scratch and find the whole thing hinges on one correction the title names and the demos gloss over: the natural gradient.
Adam trains almost every deep model — and its original convergence proof was wrong. A 2018 best paper builds a one-line convex problem where Adam walks to the worst point. We rebuild it from scratch and watch a one-word fix pull it back.
Sparse Mixture-of-Experts got the headline — a trillion parameters, a sliver of compute. We rebuild it from scratch and find the real trick is a one-line auxiliary loss: without it the gate collapses onto a few experts and the rest die.
A trading paper bolts a second model onto the first: keep its direction, learn which of its calls to trust. We rebuild the filter from scratch — and find it only adds value when it sees something the first model never did.
A 2019 paper throws away the hand-tuned momentum rule and trains a network to maximise the Sharpe ratio directly. We rebuild the Sharpe loss from scratch — and find the number it optimises is the one it never has to pay for.
A 2017 paper shows modern nets are confidently wrong — and that dividing the logits by one number fixes it. We rebuild ECE and temperature scaling from scratch, then find the one knob can make calibration worse.
Isolation Forest scores anomalies by how fast random cuts fence a point off — but its axis-parallel cuts smear a coordinate-frame bias across the score map. We rebuild it from scratch, expose the phantom corridors, and watch one tilted-cut change erase them.
A microstructure paper shows the mid-price move over a few seconds is a near-linear function of order-flow imbalance — and integrating the whole book explains ~84% of it. We rebuild OFI from scratch, then watch it forecast almost none of the next move.
A 2016 paper builds a diversified portfolio without ever inverting the covariance matrix — and beats the textbook optimiser out-of-sample. We rebuild it from scratch and find its edge is all estimation error, not a better objective.
A 2024 paper turns return forecasts into intervals with a distribution-free coverage guarantee, then picks portfolios from them. We rebuild it from scratch — and watch the guarantee quietly break the moment the market changes regime.
A NeurIPS 2022 paper explains why gradient-boosted trees still beat neural nets on tabular data. We reproduce its sharpest test from scratch: a random rotation flips the ranking, because a tree lives in its coordinate frame.
A Journal of Finance paper times the market with more parameters than data points — and wins. We rebuild its random-feature ridge from scratch, watch double descent appear, and find where the money actually comes from.
A gradient-boosting paper predicts next-day S&P 500 returns and builds a portfolio. We reproduce its indicators from scratch, chart its results — and read the Kelly number it buries.