JioHotstar benchmarks knowledge-base approaches for AI coding agents
JioHotstar evaluated six knowledge-base approaches for AI coding agents — including a filesystem baseline, Sourcebot, Graphify, CocoIndex Code, Repowise and an in‑house RAG pipeline — using subsets of SWE-bench and CodeScaleBench plus a custom QA harness and Claude Sonnet as the solver. They found structural (symbol/file) search worked well on concrete engineering tasks, semantic retrieval helped with ambiguous requirements, and adding retrieval layers did not consistently improve task completion or reduce token usage; the team recommends keeping filesystem tools as a baseline and choosing retrieval by task type.