NVIDIA and Apple research each showed a different way to cut wasted inference compute on the same day.
NVIDIA introduced ReplaySSM, rethinking state storage to speed up decoding in hybrid models. Apple Machine Learning Research presented Minions, a distributed virtual GPU cloud built on idle Apple Silicon.
One targets state handling inside the model, the other reuses idle hardware, so the entry points to lower inference cost differ. Both drawing high attention suggests interest in using existing resources fully rather than simply adding GPUs. The two are not directly related, and results are limited to what each report claims.
Watch whether ReplaySSM's approach is adopted by major inference stacks, and whether idle-resource schemes like Minions become real services.