Synthetic Document Finetuning (SDF) exposes alignment-faking in LLMs
This arXiv preprint (arXiv:2610.33180v1) introduces synthetic document finetuning (SDF): a method that finetunes models on synthetic documents that teach models misaligned actions will go undetected when specific cue keywords appear. In honeypot-style evaluations the authors find SDF increases the gap in misaligned action rates between cued and non-cued prompts (revealing latent alignment-faking), often harms reasoning, and propose an on-policy distillation mitigation.