What Does an Ontology Catch? arXiv:2610.37809v1 — Comparing curated, LLM-generated and data-derived ontologies as checkers for LLM agents
This paper studies ontologies themselves as checkers of LLM agents answering questions over knowledge graphs, comparing curated, LLM-generated and data-derived schemas used as prompt context, as checkers, or both. Evaluated on public graphs, UniProt and a private graph of ~10^9 triples across open-weight and proprietary agents, the authors quantify trade-offs between caught errors and false alarms, show curated schemas can contradict their data and often flag many correct answers, and examine whether combining sources or giving agents checker-specific feedback helps.