Safety & Evaluation A

The paper presents REDACT, a systematically controlled multilingual benchmark for personal information (PII) detection. It addresses limitations of existing corpora—few entity types, ad hoc generation, and little insight into which surface conditions cause detector failures.

The paper discusses options for scaling up democratic deliberation and empowering people with AI as large language models become prominent in public discourse. It weighs opportunities against persistent concerns such as linguistic constraints, biases, and the sycophantic tendencies of LLMs, beyond what red teaming addresses.

The paper investigates whether semantic information can be encoded in compact, non-standard text that sacrifices human readability while remaining usable by models. It argues large language models do not always need human-readable language, especially when the intended reader is another model.

The paper presents Prompt, Plan, Extract, a zero-shot agentic LLM workflow for extracting lung pathology information from clinical narrative reports. It targets the labor-intensive, error-prone manual extraction needed for cancer staging and tumor registries, avoiding fully supervised NLP pipelines.

The paper proposes AtomMem, a simple and effective memory system for LLM agents built around atomic facts. It addresses the limits of fixed context windows for accumulating and reusing information across sessions, and the coarse, unstable memory of existing memory-augmented systems.

The paper develops a budget-normalized control-window framework for single-neuron steering in language models. It seeks to predict when intervening on one neuron coherently controls a behavior—such as refusal or language routing gated by sparse feed-forward neurons—rather than collapsing the output.

The paper introduces JAMER, a project-level code framework dataset and benchmark for professional game engines. It addresses the lack of large-scale datasets and deterministic evaluation for project-level code engineering, which has remained underexplored despite progress in AI-driven game asset and gameplay generation.

The paper presents CREDENCE, an approach to decomposing compound sentences into atomic, verifiable claims for automated fact-checking. It introduces semantic metrics that avoid token-overlap measures, which underestimate quality for paraphrastic claims, and adds convergence and termination analysis.

The paper presents CombEval, a dynamic benchmark for evaluating combinatorial counting in large language models. Each problem is expressed as a typed Cofola specification over entities, combinatorial objects, dependencies, and constraints, enabling controlled generation of natural-language counting problems.

The paper presents AgentFinVQA, a deployable multi-agent pipeline for auditable financial chart question answering. It targets regulated settings where practitioners must know which answers to trust and cannot send client data to external model providers, unlike existing accuracy-focused, opaque chart-QA agents.

The paper proposes Manifold Bandits, a Bayesian curriculum-learning method that samples training problems over the latent geometry of large language models. It targets reinforcement learning for LLM reasoning, where training efficiency depends heavily on how prompts are selected during optimization.

The paper benchmarks agentic review systems, which are emerging to relieve the pressure AI-assisted research places on peer review. It evaluates two open-source systems, one proprietary system, and a zero-shot baseline, addressing the open question of how such systems should be assessed.

The paper studies sequential Direct Preference Optimization (DPO) across different preference settings, examining how applying multiple alignment objectives one after another affects earlier ones. It looks beyond uniform forgetting to understand how later training stages interfere with previously learned preferences.

The paper presents NRITYAM, a benchmark for evaluating how well language models comprehend culture in the context of global dance traditions. It addresses the gap that the global effectiveness of language models depends on a nuanced understanding of local socio-cultural contexts.

Hugging Face explores how to judge whether open models are 'agentic enough' by benchmarking them on your own tooling rather than generic suites. The approach evaluates models under realistic, user-specific tool setups to better gauge practical agent capability.

Anthropic opened a Seoul office and announced new partnerships across Korea's AI ecosystem, including enterprises, startups, and researchers building on Claude. It frames Korea as treating innovation and safety as two sides of the same coin. Specifics are per the announcement and unverified independently.

Simulating human users in interactive settings could advance training of agent assistants, evaluation of personalization systems, and social-science research. This work learns user simulators using Turing rewards, aiming to reproduce more realistic user behavior.

Preference-based RL learns reward models from pairwise behavior comparisons, bypassing explicit reward design, but existing methods often rely on passive data collection. UBP2 introduces uncertainty-balanced preference planning to actively select comparisons and learn efficiently from fewer preferences.

Post-training of reasoning models often combines supervised distillation with reinforcement learning from verifiable rewards, but distillation relies on costly chain-of-thought annotations. This work proposes rubric-conditioned self-distillation to rethink reward supervision while cutting annotation cost.

Existing multi-speaker dialogue systems bind speakers to utterances through structured supervision such as per-turn tags, multi-stream transcriptions, or learnable speaker embeddings. This work generates multi-speaker audio scenes by drawing on in-the-wild reference priors for more natural synthesis.

Production data integration is bottlenecked by repeated, lossy handoffs among data owners, engineers, and analysts who must jointly discover, structure, and query enterprise data. The authors present Data Intelligence Agents (DIA), autonomous coding agents that interpret, model, and query that data.

A longstanding goal of interpretable deep learning is to replace opaque neural computations with human-meaningful symbolic descriptions. This paper approximates the behavior of attention components with synthesized programs, offering a route to explain attention and improve interpretability.

Glioma segmentation in multiparametric MRI is critical for treatment planning, and a model that fails silently on treatment-critical sub-regions is a patient-safety risk that overlap metrics miss. This work shows MC dropout confidence does not equal reliability, rethinking uncertainty estimation.

Embodied Vision-Language-Action (VLA) models are typically obtained by fine-tuning powerful pretrained VLMs on robotics data, yet how much commonsense and factual knowledge they retain is unclear. This work measures that retention, revealing how much fine-tuning erodes prior world knowledge.

Neurosymbolic semantics is fragmented: classical, fuzzy, probabilistic, and neural systems each define truth by their own rules. Extending ULLER, NeSyCat subsumes them under a single inductive definition of truth, delivered as a differentiable tensor implementation for neurosymbolic learning.

AI has driven rapid progress in medical imaging, yielding ever more sophisticated algorithms and steady benchmark gains. Yet this algorithm-centric trajectory reveals limits. This work argues for conceptual innovation beyond algorithms to achieve clinically meaningful advances in medical imaging AI.

As LLMs are adapted to specialized domains and languages, the effectiveness of adaptation strategies remains unclear. This empirical study on French medical question answering analyzes the trade-offs of various domain-adaptation methods, clarifying gains and losses in performance and generality.

Knowledge encoded in LLMs can serve as a substrate for structured reasoning over variables describing a complex world, but accessing it probabilistically is hard. This work performs structured inference over LLMs using Gibbs sampling, enabling probabilistic reasoning across interrelated variables.

Text-rich images often hold privacy-sensitive, transactional, or decision-relevant information. As multimodal generators synthesize realistic text and layouts, this work builds a multi-domain benchmark for detecting AI-generated text-rich images from GPT-Image-2, assessing detector reliability.

Block diffusion language models speed decoding via parallel block-wise denoising, but reliably scaling them for long chain-of-thought reasoning is unresolved. The authors develop DreamReasoner-8B, using block-size curriculum learning to strengthen long-CoT reasoning in diffusion reasoning models.