NEWS · RESEARCH · #331
Cappy: 360M-parameter RoBERTa scorer that authors say boosts and adapts large multi-task LLMs without finetuning
Google Research introduces Cappy, a lightweight (≈360M params) scorer based on continual pretraining of RoBERTa that assigns a 0–1 correctness score to an instruction–response pair. The authors report Cappy can operate standalone on classification tasks or be used as an auxiliary component to boost large multi-task LLMs and enable downstream supervision and adaptation without backpropagating through the LLM or needing access to its parameters (e.g., closed-source WebAPI models).
KEY POINTS
- Google Research introduces Cappy, a lightweight (≈360M params) scorer based on continual pretraining of RoBERTa that assigns a 0–1 correctness score to an instruction–response pair.
- The authors report Cappy can operate standalone on classification tasks or be used as an auxiliary component to boost large multi-task LLMs and enable downstream supervision and adaptation without backpropagating through the LLM or needing access to its parameters (e.g., closed-source WebAPI models).
- If the reported results hold, a small external scorer that enables supervision and adaptation of large or closed-source multi-task LLMs without finetuning could reduce compute/memory costs and broaden practical use of such models.
WHY IT MATTERS
If the reported results hold, a small external scorer that enables supervision and adaptation of large or closed-source multi-task LLMs without finetuning could reduce compute/memory costs and broaden practical use of such models.