Study: Plan-then-Retrieve outperforms Retrieve-then-Plan for multi-tool queries
The paper compares two tool-discovery architectures for tool-augmented LLM agents — Retrieve-then-Plan (RtP) and Plan-then-Retrieve (PtR) — using ToolBench (≈47k APIs). RtP is better on single-tool queries, while PtR outperforms RtP on intra- and cross-category multi-tool queries across retrieval budgets, registry sizes, and evaluated LLMs (Gemini 2.5 Flash, gpt-oss-120b); PtR can hallucinate fictitious API stubs and roughly doubles inference cost, motivating complexity-adaptive routing using RtP’s top-1 score as a cheap signal.