LiquidAI released DSpark, a speculative-decoding technique that speeds up vision-language model inference without shrinking the model.
LiquidAI published "LFM2.5-VL-DSpark," a technique for speeding up vision-language model (VLM) inference using speculative decoding: a 280M-parameter "drafter," an 8.9% overhead on the 3B target, proposes candidate tokens the target model verifies. It trades a small memory increase for faster generation without changing output quality. Reported gains: 2.30x-3.13x decode speedup on an M5 Max (MLX), 2.04x-2.66x on an H100 GPU.
Speculative decoding only speeds up the decode, token-generation, stage. In VLMs, encoding the image and the prefill step before the first token both eat a substantial share of wall-clock time, especially on edge devices. So a large decode speedup doesn't translate one-to-one into overall speed; by Amdahl's law, the gain caps out once another stage becomes the bottleneck. The headline decode multiplier shouldn't be read as the model's overall speedup.
Whether encode and prefill get similar speedups next, and whether the same draft-then-verify design ports to other VLMs or larger models, are the things to watch.