NEWS · RESEARCH · #371
CRN v2: logit-level correction module fixes many errors for frozen Gemma 4 E2B without capability loss
Researchers propose CRN v2, a ~34M-parameter logit-level correction module trained on top of a fully frozen Gemma 4 E2B (4.65B text module). Trained with supervised fine-tuning and reference-free DPO on 83,400 error-correction pairs, CRN v2 corrects 53.3% of base-model errors on a 60-question CEHRI exam (43.3% on a reworded variant) while showing no measured degradation on capability benchmarks (MMLU/BoolQ); code, main-result weights, and eval scripts are released (deep-code variant released without deep checkpoints).
KEY POINTS
- Researchers propose CRN v2, a ~34M-parameter logit-level correction module trained on top of a fully frozen Gemma 4 E2B (4.65B text module).
- Trained with supervised fine-tuning and reference-free DPO on 83,400 error-correction pairs, CRN v2 corrects 53.3% of base-model errors on a 60-question CEHRI exam (43.3% on a reworded variant) while showing no measured degradation on capability benchmarks (MMLU/BoolQ); code, main-result weights, and eval scripts are released (deep-code variant released without deep checkpoints).
- Demonstrates a practical design (frozen base + logit correction + KL anchoring) that can reduce model errors without fine-tuning the base, relevant for safe maintenance and targeted fixes of deployed LMs.
WHY IT MATTERS
Demonstrates a practical design (frozen base + logit correction + KL anchoring) that can reduce model errors without fine-tuning the base, relevant for safe maintenance and targeted fixes of deployed LMs.