ArXiv paper: fine-tuning alters LLM representations but EAP-identified causal components are decoupled
The new arXiv paper (arXiv:2609.21113v1) studies how fine-tuning changes internal LLM mechanisms and whether those representational changes align with task-relevant components identified by EAP (e.g., attention heads and logit-level activations). The authors report that EAP-identified components are localized to particular layers, yet their layer distribution is largely uncorrelated with layers showing the biggest representational shifts during fine-tuning, and overlap in EAP components across tasks does not reliably produce cross-task transfer and can sometimes cause performance degradation between dissimilar tasks.