Tech Meridian ← ENTITY INDEX
PROMY MERIDIAN RU

COMPANY · ENTITY #14691

RLHF

Related event timeline, sources and context from the news index.

EVENT TIMELINE

4

RESEARCH · 1 SOURCE · Apple Machine Learning Research

Theoretical Limits of Language Model Alignment

The paper derives information-theoretic limits on reward improvement under a fixed KL-divergence budget for KL-regularized alignment, giving a closed-form expression governed by a Jeffreys divergence term and a reformulation as a covariance under the base model that yields a practical estimator. It also shows the proxy-reward (reward hacking) gap grows with reward error and lower KL penalties, proves reward ensembling reduces reward hacking, and empirically finds best-of-N nearly reaches the theoretical KL–reward Pareto frontier while PPO and GRPO remain substantially suboptimal on safety and summarization tasks.

7.0

RESEARCH · 1 SOURCE · arXiv cs.AI

Cognition-Aligned Reasoning Distillation (CARD): transforming solver trajectories into teaching trajectories

The paper introduces CARD, an RL-based framework that transforms existing solver trajectories into teaching trajectories aligned with a student's cognitive capacities, optimizing two rewards (Distribution Alignment and Logical Coherence) to make teacher reasoning easier for smaller models to learn. Evaluations show CARD improves in-domain distillation and substantially boosts out-of-distribution performance; code is available at the authors' GitHub.

6.0

RESEARCH · 1 SOURCE · arXiv cs.AI

A comprehensive framework for all stages of large-model training (arXiv:2610.33451v1)

This arXiv preprint proposes a unified stage division and a six-dimensional analytical framework (data, objective, optimization, system constraints, evaluation, risk/governance) covering data/tokenization, pretraining, and post-training alignment for LLMs and MLLMs, systematically reviewing 395 references (2017–2026). The authors present a cross-stage effects map, a decision tree, at least ten verifiable open problems, and three core findings: scaling-law comparability depends on data quality/mixture; preference-alignment methods (RLHF, DPO) and RL with verifiable rewards shift post-training boundaries; and multimodal training challenges tokenizer design, objectives, and evaluation.

7.0

RESEARCH · 1 SOURCE · arXiv cs.AI

SSRFT: Safe-Role Internalization for Robust LLM Safety Alignment (arXiv:2610.07023v1)

The paper introduces SSRFT (Supervised Safe-Role Fine-Tuning), a safety-alignment framework that teaches models to internalize a predefined safe role by building a Safe-Role Q&A (SRQA) dataset from psychometric questions, limited jailbreak prompts, and a safe-role description; role-consistent responses are synthesized, validated, and expanded into diverse scenarios. Experiments on multiple Base and Instruct models report that SSRFT yields more robust and generalizable safety alignment than standard SFT, with greater resistance to prefilling attacks, better generalization to unseen jailbreaks, reduced over-refusal on benign queries, and preserved general capabilities.

6.0