Safety & Evaluation A

Founded in 2013 by the creators of the open-source big-data engine Apache Spark, Databricks has grown into a data and AI platform valued around ¥20 trillion and used by roughly 70% of the Fortune 500. The article traces its rise and latest developments.

DiffusionGemma performs much of its computation in a continuous latent space, raising the question of whether this reduces reasoning transparency. The authors decompose transparency into variable transparency (understanding intermediate computational states) and algorithmic transparency (reconstructing the process behind a model's answer).

A minimax-optimal multicalibration algorithm that outputs a deterministic predictor, resolving the open question of whether randomization is needed for optimal sample complexity. The result is extended to deterministic predictors satisfying outcome indistinguishability and omniprediction.

The paper studies multi-task Bayesian in-context learning, using in-context learning to perform Bayesian predictive inference across tasks. It targets the intractability of exact inference and the cost or restrictiveness of scalable approximations, aiming for uncertainty quantification and data efficiency.

StylisticBias investigates the visual cues that shape how multimodal large language models judge people. The study finds that a small set of human visual cues drives most of the social biases exhibited by MLLMs, which are increasingly deployed in consequential settings.

Neurosymbolic systems such as DeepProbLog combine neural perception with probabilistic logic, but standard inference has limits. DeepSWIP introduces quotient-WMC counterfactuals to enable counterfactual reasoning in neural probabilistic logic programs.

Multimodal foundation models have advanced rapidly thanks to large optical benchmarks, but comparable SAR resources are scarce. SARLO-80 provides a worldwide slant-range SAR and optical dataset at 80cm resolution to fill this gap.

LiveCodeBench has become a widely adopted benchmark for evaluating large language models on code. Multi-LCB extends it to multiple programming languages to assess multilingual code generation.

In-context demonstrations can jailbreak language models, but it has been unclear what safety-aligned models learn when demonstrations mix compliant and non-compliant behavior. This work analyzes that learning behavior.

Style-content dual-reference generation aims to synthesize an image that preserves structure while adopting a reference style. FreeStyle leverages community LoRA mining to give free control over style and content.

The paper presents a systematic study of von Neumann entropy estimation in multi-qutrit quantum systems, comparing variational quantum algorithms with classical convolutional neural networks on an ideal noise-free simulator for systems of up to three qutrits.

It is unclear whether LLMs that score well on vulnerability benchmarks truly reason about security or merely pattern-match. This work diagnoses the limits of fine-tuning LLMs for vulnerability detection in systems software.

When large language models act as evaluators in multi-agent systems, their systematic evaluation biases can spread through the system. This work analyzes how such evaluator bias propagates across agents.

The paper proposes a hierarchical recovery mechanism for cross-device agent systems, moving beyond coarse-grained global replanning. It targets real-world computer-use tasks that span multiple applications and devices and must coordinate heterogeneous environments under dynamic runtime failures.

The paper proposes aligning large language models using implicit user signals—such as mouse and eye movements—instead of explicit human feedback. It addresses the limitation that users rarely provide explicit ratings, which makes high-quality preference data scarce for reward modeling.

The paper proposes marginal advantage accumulation, a cross-batch, operation-level mechanism for memory-driven agent self-evolution. It aims to distinguish stably effective memory operations from accidental hits, addressing contradictory feedback that the same operation can receive across different batches in trace distillation.

Agentic AI systems increasingly rely on language-model components to interpret instructions, exposing them to attacks. This paper analyzes defensive misdirection as a countermeasure against model-guided automated attacks.

The paper introduces a Fisher-geometric notion of sharpness to study the implicit bias of SGD toward flat minima. It addresses the fact that standard Euclidean flatness measures, such as the trace or maximum eigenvalue of the loss Hessian, are not invariant under reparametrizations that preserve the network function.

The paper proposes agentic symbolic search, an approach to characterize partial differential equation solutions through mathematical structures rather than tables of computed values. It targets the structural understanding that neither numerical simulation nor neural networks produce directly, traditionally derived by hand.

The paper studies data bias mitigation under coverage constraints and the resulting price of fairness. It addresses discriminatory outcomes for individuals at the intersection of multiple sensitive attributes, including the lack of principled measures for quantifying intersectional bias.

Malware analysts often inspect compiled binaries through decompiled pseudo-C when source code is unavailable. This work uses multi-view decompilation to improve LLM-based malware classification.

LLM agents are increasingly proposed as supervisory components for safety-critical systems. This work evaluates their safety via multi-turn red-teaming, jailbreak benchmarks, and adversarial robustness tests.

Data videos combine dynamic charts, voice narration, and synchronized animation to convey insights. DataMagic automatically transforms tabular data into such data-insight videos.

The paper addresses modality imbalance in multimodal federated graph learning with a data-synthesis-based approach. It targets two granularities of imbalance—client-level, where some clients lack entire modalities, and node-level, where individual nodes have missing modalities.

Safety is a core concern when deploying reinforcement learning agents in real-world domains. CRAX provides a framework for fast benchmarking of safe reinforcement learning methods.

Large language models show promise for code compilation tasks but struggle with runtime performance tuning. AutoPass uses evidence-guided LLM agents to perform compiler performance tuning.

The paper introduces CATCH-ME, a dataset of contextually annotated multi-turn counterspeech against overlapping hate speech and misinformation. It addresses NLP's tendency to treat the two threats in isolation and the tendency of zero-shot LLMs to produce repetitive, vague counterspeech.

The paper presents a de-biased VLM-as-3D-judge protocol for single-image 3D generation. Building on a cross-model judge that ranks single-image-to-3D mesh quality where geometry and CLIP proxies fall short, it asks whether the judge's preferences can cheaply specialize a strong open generator, TRELLIS, on one asset class such as furniture without human labels.

Explicit skill libraries make computer-using agents easier to inspect, but building them is costly. This work automates SKILL.md generation by mining agents' interaction trajectories.

The paper runs a four-arm head-to-head comparison of fine-tuning, retrieval, and their combination for producing correct statutory citations on the Ontario Residential Tenancies Act and its core regulation. It targets the practical need of tenants, landlords, and help-desk staff to be pointed at the governing provision.