Some things I've built
Drag this tetrahedron. Try to count the number of faces.
Efficient Decode Attention
Flash-Decoding and paged KV-cache memory management, written in CUDA.
A from-scratch CUDA build of the two pieces that dominate LLM decode: a split-K flash-decode attention kernel and a paged KV-cache manager, profiled on an NVIDIA A40.
Parallelizing over the token dimension instead of the softmax boundary took the kernel from 17 to 477 Gi/s, a 28× speedup at long context. Paged allocation fit 4 to 8× more concurrent sequences by killing fragmentation, at a flat 1.8× throughput cost that page size never changes.
MurmurMatch
College social platform scaled to 56,000+ students.
Built with my friend Maxwell and grown to 56,000+ users with $0 spent on marketing. Used by ½ of Dartmouth, ⅓ of Brown, ⅓ of Stanford, and ¼ of Princeton students, with 500+ on a $7.99/week subscription.
Murmur is a digital-postcard app for campuses. Students send anonymous superlatives to their peers, and both identities are revealed once the recipient sends one back. A second format, Chat Sparks, turns hot takes into conversation openers, and because every recipient is nudged to send their own, the whole thing spreads through a viral loop.
Running on load-balanced GCP infrastructure with autoscaling managed instance groups to keep up with demand.
Links: Website
FPGA Neural Inference Accelerator
Low-latency neural inference in fixed-point arithmetic.
Fixed-point arithmetic, activation functions baked into lookup tables, and quantization-aware training so the network survives the move off floating point.
Links: Writeup
Representational Regularization for Diverse LLM Ensembles
Improved LLM ensemble diversity and performance at negligible additional inference cost.
Ensembles only help when their members actually disagree. This regularizes the models' internal representations apart during training, so the diversity is already there at inference time instead of being bought with extra compute.
Links: Paper
ShuffleSpeak (JonesSpeak v4.2)
Data compression for physical, private communication.
A transformer as the probability model, arithmetic coding as the entropy coder — squeezing a message down far enough to hand it to someone.
Links: Try it
BlitzVision v2 for BlitzGo
What superhuman play looks like in a game we invented.
Maxwell and I created BlitzGo, a game whose simple rules give rise to profound strategic depth. BlitzVision explores what superhuman play looks like by training AI with self-play.
Links: GitHub
