RESEARCH · RESEARCH · #1244
GeoOutageBench: benchmark for ambiguity-aware multimodal geospatiotemporal KGQA
GeoOutageBench (arXiv:2609.36082v1) is a new benchmark and dataset that evaluates LLM-based geospatiotemporal KGQA over a multimodal knowledge graph combining outage records, remote sensing, weather, storm and power events, geographic entities, and domain ontologies. It provides a competency query taxonomy (containment/proximity, co-occurrence, multimodal evidence, hypothetical evaluation) and user-configurable evaluations of NL→SPARQL interpretation, ontology utility, and multimodal KGQA retrieval; code, data, and results are available on GitHub.
KEY POINTS
- GeoOutageBench (arXiv:2609.36082v1) is a new benchmark and dataset that evaluates LLM-based geospatiotemporal KGQA over a multimodal knowledge graph combining outage records, remote sensing, weather, storm and power events, geographic entities, and domain ontologies.
- It provides a competency query taxonomy (containment/proximity, co-occurrence, multimodal evidence, hypothetical evaluation) and user-configurable evaluations of NL→SPARQL interpretation, ontology utility, and multimodal KGQA retrieval; code, data, and results are available on GitHub.
- This benchmark creates a focused framework for testing LLM–KG systems on ambiguous, spatiotemporal, multimodal queries relevant to infrastructure outage and resilience analysis—an underexplored but practical domain.
WHY IT MATTERS
This benchmark creates a focused framework for testing LLM–KG systems on ambiguous, spatiotemporal, multimodal queries relevant to infrastructure outage and resilience analysis—an underexplored but practical domain.