Not established from the available sources.
GOOGLE · MODEL RELEASE TRACKER
Gemini 2.5 Flash
Gemini 2.5 Flash is a variant of Google’s Gemini family that was used in the StreamVibe video-assistant benchmark. In the evaluation it suggested when to offer help and what to say; examples required unanimous approval by four human evaluators.CURRENT SNAPSHOT1/5 DIMENSIONS WITH DATA
The dimensions that change the decision.
Not established from the available sources.
Not established from the available sources.
Retrieve-then-Plan vs Plan-then-Retrieve performance on ToolBenchOn ToolBench (46,984 unique API endpoints; G1 single-tool, G2 intra-category multi-tool, G3 cross-category multi-tool), Retrieve-then-Plan (RtP) outperforms Plan-then-Retrieve (PtR) on single-tool queries while PtR outperforms RtP on multi-tool queries. This crossover holds at every retrieval budget (k=1..20) and is reported as consistent across model families including Gemini 2.5 Flash.
Used in StreamVibe benchmarkGemini 2.5 Flash was used in evaluations on the StreamVibe benchmark (about 4,100 examples from roughly 183 hours of recordings) that measure timing and content of proactive assistance.
Not established from the available sources.
VERIFIABLE FACTS
Every value stays attached to a source and date.
BENCHMARKS · Retrieve-then-Plan vs Plan-then-Retrieve performance on ToolBenchINDEPENDENTLY SUPPORTED
On ToolBench (46,984 unique API endpoints; G1 single-tool, G2 intra-category multi-tool, G3 cross-category multi-tool), Retrieve-then-Plan (RtP) outperforms Plan-then-Retrieve (PtR) on single-tool queries while PtR outperforms RtP on multi-tool queries. This crossover holds at every retrieval budget (k=1..20) and is reported as consistent across model families including Gemini 2.5 Flash.
SAFETY · Stub-hallucination failure mode observedINDEPENDENTLY SUPPORTED
The study characterizes a 'stub-hallucination' failure mode in which LLMs invent plausible but non-existent API names; this behavior affects PtR's generated stubs and is reported across model families including Gemini 2.5 Flash.
LIMITATIONS · Plan-then-Retrieve inference cost (token overhead)INDEPENDENTLY SUPPORTED
The analysis shows PtR's extra discovery call roughly doubles inference cost (token overhead). This higher cost tends to pay off on multi-tool queries but harms performance-cost tradeoffs on single-tool queries; the result is reported as consistent across model families including Gemini 2.5 Flash.
CAPABILITIES · Suggested when to offer helpDEVELOPER CLAIM
In the StreamVibe benchmark, Gemini 2.5 Flash suggested when to offer help and what to say; four human evaluators had to unanimously approve each example.
BENCHMARKS · Used in StreamVibe benchmarkDEVELOPER CLAIM
Gemini 2.5 Flash was used in evaluations on the StreamVibe benchmark (about 4,100 examples from roughly 183 hours of recordings) that measure timing and content of proactive assistance.
WHAT CHANGED
Stored passport versions, without reconstructed history.
CAPABILITIES · Suggested when to offer help+ In the StreamVibe benchmark, Gemini 2.5 Flash suggested when to offer help and what to say; four human evaluators had to unanimously approve each example.
BENCHMARKS · Used in StreamVibe benchmark+ Gemini 2.5 Flash was used in evaluations on the StreamVibe benchmark (about 4,100 examples from roughly 183 hours of recordings) that measure timing and content of proactive assistance.