RESEARCH · RESEARCH · #2081
Fine-tuning Qwen2.5-3B (LoRA) to withdraw actions in synthetic NAQD-Env
This arXiv v1 paper evaluates supervised fine-tuning (LoRA) of Qwen2.5-3B-Instruct in NAQD-Env to teach selective withdrawal (suspending actions when support is lost). SFT checkpoints improve aggregate decision agreement (0.77–0.92) and withdrawal recall (0.46–0.86) compared with instruction-tuned baselines (agreement 0.45–0.63, withdrawal recall ≤0.06), but exhibit rule-specific failures (e.g., an original-curriculum checkpoint wrongly withdrew 26/34 actions with alternative support) and other limitations; the authors present a method to detect such regressions and note experimental constraints that limit causal claims.
KEY POINTS
- This arXiv v1 paper evaluates supervised fine-tuning (LoRA) of Qwen2.5-3B-Instruct in NAQD-Env to teach selective withdrawal (suspending actions when support is lost).
- SFT checkpoints improve aggregate decision agreement (0.77–0.92) and withdrawal recall (0.46–0.86) compared with instruction-tuned baselines (agreement 0.45–0.63, withdrawal recall ≤0.06), but exhibit rule-specific failures (e.g., an original-curriculum checkpoint wrongly withdrew 26/34 actions with alternative support) and other limitations; the authors present a method to detect such regressions and note experimental constraints that limit causal claims.
- The paper presents a diagnostic method and empirical evidence that fine-tuning can improve withdrawal behavior while hiding rule-specific regressions that aggregate metrics miss, informing safer agent design and evaluation.
WHY IT MATTERS
The paper presents a diagnostic method and empirical evidence that fine-tuning can improve withdrawal behavior while hiding rule-specific regressions that aggregate metrics miss, informing safer agent design and evaluation.