RESEARCH · RESEARCH · #2697
LoRA on a Budget — Functional Rank Allocation (FRA) for adapter-fleet compression
This arXiv preprint introduces Functional Rank Allocation (FRA), a post-hoc method that builds per-adapter functional rank–distortion frontiers and solves a fleet-level minimax allocation to compress LoRA adapters under a shared rank budget. FRA combines isolated-module measurements, a dynamic-programming search surrogate, end-to-end verification and a complementary spectral proposal; evaluated on 78 adapters from three public fleets it improves worst-case and tail utility versus uniform spectral compression in constrained-budget regimes.
KEY POINTS
- This arXiv preprint introduces Functional Rank Allocation (FRA), a post-hoc method that builds per-adapter functional rank–distortion frontiers and solves a fleet-level minimax allocation to compress LoRA adapters under a shared rank budget.
- FRA combines isolated-module measurements, a dynamic-programming search surrogate, end-to-end verification and a complementary spectral proposal; evaluated on 78 adapters from three public fleets it improves worst-case and tail utility versus uniform spectral compression in constrained-budget regimes.
- Helps operators allocate limited LoRA rank across layers and many adapters to preserve worst-case and tail model utility, improving deployment efficiency for large adapter fleets.
WHY IT MATTERS
Helps operators allocate limited LoRA rank across layers and many adapters to preserve worst-case and tail model utility, improving deployment efficiency for large adapter fleets.