NVIDIA × Inference & Efficiency

NVIDIA: efficient, resilient inference

NVIDIA: efficient, resilient inference

✎ Story body

NVIDIA published three technical explainers on the same day. They sit at different layers - model, execution, interconnect - and none of them is about making the computation faster.

What happened

The first is about choosing a model. Nemotron 3.5 Lightning, its example, activates only 3B of its 30B total parameters to process a single token. The piece compares dense models, which run everything every time, against Mixture-of-Experts models, which switch on only the parts they need, framed by active parameter count and throughput. The second covers the deterministic execution of Groq 3 LPX on Vera Rubin, which fixes timing in advance to remove wasted waiting. The third is NVLink 6. In large-scale training every GPU in the cluster moves in lockstep, so one failed node halts the entire run; NVLink 6 stacks redundancy and recovery from the link layer up to the system layer so that training survives a failure.

Why it matters

Three separate technologies that, lined up, point the same way. Not faster arithmetic, but how to spend a fixed budget of resources and how not to stop. The caveat is that only the latter two actually name power as the constraint; the model-choice piece stays in the language of throughput. The measure of a system is shifting from how fast it goes toward whether it can keep going.

What to watch

What was published is design reasoning, not measurements. The test comes when operators can produce their own numbers - work done per watt, and how far a training run actually got after a failure - from their own environments.

▲ Official & Press
Official

Dense vs. MoE Models: Active Parameters, Throughput, and When to Choose Each

NVIDIA Developer Blog ・ 2026-09-15 ・ 📌

NVIDIA compares dense and MoE models on active parameters and throughput

Official

Bypassing inference bottlenecks: Accelerating complex AI search with Retrieve-for-Train

Google Research Blog ・ 2026-09-15

Google unveils Retrieve-for-Train, skipping inference-time reasoning

Official

How NVIDIA Groq 3 LPX Deterministic Execution Drives Power-Efficient High-Interactivity Inference on NVIDIA Vera Rubin

NVIDIA Developer Blog ・ 2026-09-15

NVIDIA on Groq 3 LPX deterministic execution on Vera Rubin

Official

How NVIDIA NVLink 6 Delivers Multi-Layer Resiliency for AI Factories

NVIDIA Developer Blog ・ 2026-09-15

NVIDIA details NVLink 6's multi-layer resiliency for AI factories

Press

How OpenAI Used Its Own LLMs to Design Its Jalapeño Chip

IEEE Spectrum (AI section) ・ 2026-09-14

OpenAI used its own LLMs to design the Jalapeño chip, RTL to tapeout in 9 months

Academic (arxiv etc.) 22 ▾
Academic

Stellar Colosseum: A Many-Agent Harness for Long-Horizon Research in Mathematics and Theoretical Computer Science

arXiv cs.AI (Artificial Intelligence) ・ 2026-09-14

Academic

Bridging Control, Inference, Transport, and Thermodynamics: From Theory to Applications in Learning

arXiv cs.LG (Machine Learning) ・ 2026-09-14

Academic

Anatomical Grounding and Leakage-Aware Multimodal Contrastive Learning for Alzheimer's Disease Classification from Structural MRI

arXiv cs.AI (Artificial Intelligence) ・ 2026-09-14

Academic

Learning to Coach for Experiential Learning

arXiv cs.CL (Computation and Language) ・ 2026-09-14

Academic

Proportional-Fair Resource Allocation and Dual-Threshold Early-Exit Inference for Secure Cooperative Multi-Layer Edge Intelligence

arXiv cs.LG (Machine Learning) ・ 2026-09-14

Academic

Enabling Streaming User Transcription in Full-Duplex Speech-to-Speech Models

arXiv cs.CL (Computation and Language) ・ 2026-09-14

Academic

Look Before You Leap: Factual Decoding with Internal Attribution Signals

arXiv cs.AI (Artificial Intelligence) ・ 2026-09-14

Academic

Merging the Knowledge of LLMs for Automatic Speech Recognition

arXiv cs.CL (Computation and Language) ・ 2026-09-14

Academic

Backward SDEs-based Diffusion for Physics-Constrained Generation

arXiv cs.LG (Machine Learning) ・ 2026-09-14

Academic

More Than Just Access: Generative AI as Communication Intermediary for Blind and Low-Vision Users

arXiv cs.AI (Artificial Intelligence) ・ 2026-09-14

Academic

Don't Send What You Don't Need: Question-Guided Token Pruning as a Privacy Defense for Vision-Language Models

arXiv cs.AI (Artificial Intelligence) ・ 2026-09-14

Academic

Circuit-MLLM: Topological Logic-Guided Latent-Space Visual Reasoning for Circuit Schematic Understanding

arXiv cs.AI (Artificial Intelligence) ・ 2026-09-14

Academic

Through the Eyes of the Beholder: Biometric and Demographic Conditioning for Multimodal Sexism Detection

arXiv cs.CL (Computation and Language) ・ 2026-09-14

Academic

Option-Aware Retrieval and Task-Specific VLM Adaptation for Medical VQA

arXiv cs.CL (Computation and Language) ・ 2026-09-14

Academic

To Each Language Its Tokenizer: Modular Tokenizers for Efficient Multilingual LLMs

arXiv cs.CL (Computation and Language) ・ 2026-09-14

Academic

Beyond Noise: Understanding and Overcoming Temperature Effects in Analog DNN Inference

arXiv cs.LG (Machine Learning) ・ 2026-09-14

Academic

How Lossless Is Lossless Speculative Decoding? The Role of Numerical Precision in Orthrus

arXiv cs.CL (Computation and Language) ・ 2026-09-14

Academic

MarKey: Marginal Utility Guided Greedy Keyframe Selection for Long Video Understanding

arXiv cs.CL (Computation and Language) ・ 2026-09-14

Academic

Dynamic Semantic Compression for Efficient Latent-Space Inference in Large Language Models

arXiv cs.CL (Computation and Language) ・ 2026-09-14

Academic

Reason What Matters: Retrieval-Grounded Reasoning for Universal Multimodal Embeddings

arXiv cs.CL (Computation and Language) ・ 2026-09-14

Academic

Semiotic Relations and Proof Methods: A Cross-Genre Study of Argument Structure with Large Language Models

arXiv cs.CL (Computation and Language) ・ 2026-09-14

Academic

EMR: Self-Evolving Medical Multi-Agent System via Experience Mining and Reuse

arXiv cs.CL (Computation and Language) ・ 2026-09-14

← Story Archive