In collaboration with Fireworks and fal, we are releasing a technical analysis of EAGLE-4 alongside a public dataset for benchmarking speculative decoding on creative tasks.
Why Speculative Decoding?
LLMs generate text auto-regressively, predicting one token at a time. At small batch sizes, decoding relies on "thin"