RESEARCH · RESEARCH · #2066
Study: Plan-then-Retrieve outperforms Retrieve-then-Plan for multi-tool queries
The paper compares two tool-discovery architectures for tool-augmented LLM agents — Retrieve-then-Plan (RtP) and Plan-then-Retrieve (PtR) — using ToolBench (≈47k APIs). RtP is better on single-tool queries, while PtR outperforms RtP on intra- and cross-category multi-tool queries across retrieval budgets, registry sizes, and evaluated LLMs (Gemini 2.5 Flash, gpt-oss-120b); PtR can hallucinate fictitious API stubs and roughly doubles inference cost, motivating complexity-adaptive routing using RtP’s top-1 score as a cheap signal.
KEY POINTS
- The paper compares two tool-discovery architectures for tool-augmented LLM agents — Retrieve-then-Plan (RtP) and Plan-then-Retrieve (PtR) — using ToolBench (≈47k APIs).
- RtP is better on single-tool queries, while PtR outperforms RtP on intra- and cross-category multi-tool queries across retrieval budgets, registry sizes, and evaluated LLMs (Gemini 2.5 Flash, gpt-oss-120b); PtR can hallucinate fictitious API stubs and roughly doubles inference cost, motivating complexity-adaptive routing using RtP’s top-1 score as a cheap signal.
- Finds a clear, model- and scale-robust crossover that guides when to use RtP vs PtR and proposes a practical zero-cost signal for adaptive routing, affecting design of tool-augmented LLM systems.
WHY IT MATTERS
Finds a clear, model- and scale-robust crossover that guides when to use RtP vs PtR and proposes a practical zero-cost signal for adaptive routing, affecting design of tool-augmented LLM systems.