Some things I've built
Paged Flash-Decoding Attention Kernel
Paged Flash-Decoding with GQA on tensor cores and an FP8 KV cache, written in CUDA.
A from-scratch CUDA decode attention kernel for a paged KV cache, profiled on an NVIDIA H100. It runs grouped-query attention on tensor cores, looks up each page inside its own 16-byte async copy, and schedules the work as one persistent wave.
An earlier FP32 version on an A40 found that paging cost a flat 1.8× in throughput. This one removes that cost: 16-token pages run within a point of an unpaged layout, and the kernel sustains 86% of peak DRAM bandwidth, within 4% of FlashInfer. An FP8 cache halves KV memory, fits twice the sequences, and decodes 1.86× faster, 7% ahead of FlashInfer.
Edge ML FPGA Accelerator for Speech Recognition
Always-on keyword spotting from a live microphone, running on an FPGA.
This listens to the microphone on a Nexys A7 and shows the word it hears on the 7-segment display. The training data comes from a synthetic pipeline built on open-weight text-to-speech models, so it can be trained on any word, including "Emaan" and "Heidari".
Everything after the mic is SystemVerilog: decimation filters, an FFT and mel filter bank, and an int8 CNN that matches the PyTorch model bit for bit. Instead of rerunning the full one-second window, the engine computes one new row of each layer every 40 ms, which cuts the work per result by 25×. At a 10 MHz clock it sits idle 93% of the time.
I trained it on Speech Commands plus those synthetic clips, then quantized it to int8. It scores 91.9% on the Speech Commands test set and uses about 9% of the FPGA's logic.
HotLoop
A benchmark for agents doing GPU performance engineering.
I wanted to know whether an agent could actually do GPU performance work, not just write code that runs. HotLoop hands it a PyTorch reference and a real A100, then checks whether what comes back is correct and faster. Tasks get generated and certified on the hardware, since the right optimization changes with the shapes.
The measurement turned out to be the hard part. It's easy to fool yourself with GPU timings, so correctness is checked against fp64, speedups use paired-block timing, and every submission gets rescored in a clean container on seeds the agent never saw. Some tasks have no headroom at all, to catch a benchmark that hands out free wins.
Across 12 tasks, a frontier model beat the best strategy I'd written myself on every one with room in it, 17.3× where I'd certified 3.48×. The controls stayed at 1.00×.
Links: GitHub
Markov
Private, paper-money prediction markets for small groups.
As treasurer of Theta Tau at USC I was looking for ways to keep members active longer and keep them coming to events as they got older, new member classes especially. Markov is what came of it. Anyone in the group can open a market on anything, and an LMSR automated market maker quotes both sides, so a market is live the moment it exists instead of waiting for someone to take the other end.
The incentives did something I didn't plan for. A market on whether more guys or more girls would turn out to an event had each side betting on themselves and then dragging people along to make the bet pay. The market moved the turnout it was trying to predict. People started showing up to events just to watch their positions resolve.
Good fun to watch prediction markets pull a group closer together instead of being pointed at strangers.
Links: markovmarkets.com
MurmurMatch
College social platform scaled to 56,000+ students.
Built with my friend Maxwell and grown to 56,000+ users with $0 spent on marketing. Used by ½ of Dartmouth, ⅓ of Brown, ⅓ of Stanford, and ¼ of Princeton students, with 500+ on a $7.99/week subscription.
Murmur is a digital-postcard app for campuses. Students send anonymous superlatives to their peers, and both identities are revealed once the recipient sends one back. A second format, Chat Sparks, turns hot takes into conversation openers, and because every recipient is nudged to send their own, the whole thing spreads through a viral loop.
Running on load-balanced GCP infrastructure with autoscaling managed instance groups to keep up with demand.
Links: murmurmatch.com
FPGA Inference Accelerator for MNIST
Low-latency neural inference in fixed-point arithmetic.
Fixed-point arithmetic, activation functions baked into lookup tables, and quantization-aware training so the network survives the move off floating point.
Links: Writeup
Training Diverse LLM Ensembles
Improved LLM ensemble diversity and performance at negligible additional inference cost.
Ensembles only help when their members actually disagree. This regularizes the models' internal representations apart during training, so the diversity is already there at inference time instead of being bought with extra compute.
Links: Paper
ShuffleSpeak
Data compression for physical, private communication.
A transformer as the probability model, arithmetic coding as the entropy coder, squeezing a message down far enough to hand it to someone.
Links: Try it
BlitzVision v2 for BlitzGo
What superhuman play looks like in a game we invented.
Maxwell and I created BlitzGo, a game whose simple rules give rise to profound strategic depth. BlitzVision explores what superhuman play looks like by training AI with self-play.
Links: GitHub
