Systems, GPUs, and other machinery.
Research and notes from the lower levels of the stack.
thesis
The gap
Hardware is years ahead of the software that runs on it. The speed is already in the chip. The work is getting it out.
The lower layers
The interesting limits sit at the bottom of the stack. A number format or a memory bus decides more than the code above it.
The method
Every measurement answers some question. The craft is making it answer yours.
results
fp4-speech
Hardware FP4 for speech on Blackwell. Weights are safe across three architectures, activations are the open problem, 2x Orpheus decode with CUDA graphs.
Orpheus-3B decode, NVFP4 + CUDA graphs vs BF16 baseline ยท RTX PRO 6000 Blackwell
research program
- NVFP4 for speech and audio in progress
- kernel-level profiling studies in progress
- benchmark methodology planned