DASA: synthetic continuous embeddings enable effective LLM fine-tuning (arXiv:2609.35868v1)
The paper introduces Desired-Update-Aligned Synthetic Data (DASA), which optimizes continuous synthetic input embeddings via activation-gradient feedback from a frozen reference model and uses them directly for downstream fine-tuning. Experiments on six Llama and Qwen models (1B–32B) across six benchmarks show DASA matches or exceeds natural-language fine-tuning in multiple settings, outperforms GRADMM in most comparisons, and yields a 3.6–4.9× speedup over GRADMM with comparable peak GPU memory.