Inference & Efficiency
A
Showing 1–30 of 93
-
DeepSeek-v4.1 Flash: Pushing the Limits of KV Cache CompressionDeepSeek-V4.1 Flash pushes KV cache compression 4x for agent workloadsA close read of the DeepSeek-V4.1 Flash technical report, whose stated aim is pushing KV cache compression to its limit for long-horizon agent workflows. Borrowing from YOCO, only 20 of the 40 layers run during prefill, cutting prefill active parameters to 8B against 16B at decode. GQA-style head reduction, cross-layer CSA2 block compression and an FP4 KV cache together shrink the KV cache roughly 4x while keeping task quality.
-
「中国AIがClaudeやGPTから数十億トークン抽出」 米当局が暴いた「知識蒸留」の実態US NSA, CISA and FBI warn of Chinese distillation of US frontier modelsThe NSA, CISA and FBI issued a joint advisory alleging Chinese AI companies are systematically distilling US frontier models, extracting reasoning capabilities and other behaviour through proxy access. The agencies recommend countermeasures.
-
TensorRT Edge-LLM Completes the MLPerf Edge Agentic Benchmark 6.4x Faster on Jetson AGX ThorNVIDIA runs MLPerf Edge Agentic 6.4x faster on one Jetson AGX ThorIn MLPerf Inference v6.1's Edge Agentic benchmark, NVIDIA's TensorRT Edge-LLM ran Qwen3.6-27B on one Jetson AGX Thor, finishing 1,007 turns in 24m36s — 6.4x faster than the llama.cpp reference (2h37m). NVFP4 quantization, tree-based MTP and KV cache reuse drive the gain.
-
rMuscle: Robotic Muscle Memory for Efficient Vision-Language-Action Model Inference
-
Long-Lived Characters, Local Inference: Incremental Memory Maintenance for Game NPCs
-
Structured Claim-Level Discourse Representations for Dense Health Narratives
-
Decodable but Misrouted: Sparse Features Uncover a Readout Gap in Vision-Language Models for Harmful Meme Detection
-
Taming the Agentic RAN: Stability-Guaranteed Arbitration of Autonomous AI Agents in O-RAN
-
Ask the Tool, Don't Guess: Agent Tool Calls Hold Their Progress, and the Serving System Should Read It
-
FRAUDSkill: Structured Frozen-Weight Skill Optimization for Audio Anti-Fraud Detection
-
A Scalable Framework for Automated NER Annotation Correction in Low-Resource Languages
-
Clueing up LLMs with Tool-Augmented Deductive Reasoning
-
Beyond Truncation: Rethinking LLM Decoding as Ensemble Pruning
-
Toward Composable Network Digital Twins: A Subgraph-Based Latency Prediction Study
-
RankGround: Efficient High-Resolution GUI Grounding via Lightweight Reranker-Guided Crop Selection
-
Generalist-Specialist Mixture-of-Experts for Rare Pathology Detection in Multimodal Imaging
-
Voice of Reason: Reinforcement Learning for Spoken Math
-
Beyond EER: Multi-Dimensional Evaluation of Information Leakage in Speaker De-Identification
-
Selection Is Retrieval, Abstention Is Not: On-Device Tool Routing over 70 Korean-English Actions
-
VLA-ULAP: Interleaving Cloud VLA Calls with Ultra-Lightweight Local Action Prediction at the Edge
-
Peak-Aware Short-Term Load Forecasting Across Distribution Grid Aggregation Levels
-
TTM-Bench: A Framework for Text-to-Music System Performance Benchmarking
-
On-the-Fly Homographies Calibration for Multi-Camera Tracking
-
Align, Integrate, and Fire: Efficient Token-Level Alignment for Zero-Shot SpeechLLMs
-
Size Matters: Foundation Model for Czech HTML documents
-
ActionPiece: Rethinking Action Tokenization for Autoregressive Vision-Language-Action Models
-
Dependency-Aware Trajectory Refinement for Efficient Multi-Turn Agent Fine-Tuning
-
Bypassing inference bottlenecks: Accelerating complex AI search with Retrieve-for-TrainGoogle unveils Retrieve-for-Train, skipping inference-time reasoningGoogle Research introduced Retrieve-for-Train, which returns a coherent slate of results rather than one best match. Instead of inference-time reasoning, it trains a lightweight diffusion model once via reinforcement learning to generate the slate instantly.
-
LACE: Layer-Wise Compression for Dynamic Frame Rate Codecs
-
JustFit: 200K-Token LLM Serving on a 24 GiB Laptop with Just-in-Time State Management