JIVEAdapter: an additive multi-task low-rank adapter via Joint and Individual Variation Explained
The paper introduces JIVEAdapter, an additive low-rank adapter that decomposes each weight update into a shared Joint component and per-task Individual components, enforces near-orthogonality between them, and adaptively allocates rank across shared and task-specific pools. Evaluated with DeBERTaV3-base on GLUE and SuperGLUE, JIVEAdapter matches strong single- and multi-task low-rank baselines at the same per-task effective rank and supports freezing a jointly learned Joint for reuse with cheap per-task updates.