RESEARCH · RESEARCH · #2627
GEMS-RAG: graph-grounded multimodal RAG for engineering standards
GEMS-RAG is a framework that combines hierarchical parsing, fine-grained visual entity extraction, multimodal knowledge-graph construction, and graph-grounded hybrid retrieval to enable grounded reasoning over complex, multimodal engineering standards. Evaluated on the Manual on Uniform Traffic Control Devices (MUTCD), GEMS-RAG improves retrieval effectiveness and question-answering performance compared with text-only, multimodal, and graph-based RAG baselines.
KEY POINTS
- GEMS-RAG is a framework that combines hierarchical parsing, fine-grained visual entity extraction, multimodal knowledge-graph construction, and graph-grounded hybrid retrieval to enable grounded reasoning over complex, multimodal engineering standards.
- Evaluated on the Manual on Uniform Traffic Control Devices (MUTCD), GEMS-RAG improves retrieval effectiveness and question-answering performance compared with text-only, multimodal, and graph-based RAG baselines.
- Decomposing figures into linked visual entities and using graph-grounded retrieval helps LLMs reason over the hierarchical, multimodal, and cross-referenced structure of standards, which is important for accessing safety-critical technical rules.
WHY IT MATTERS
Decomposing figures into linked visual entities and using graph-grounded retrieval helps LLMs reason over the hierarchical, multimodal, and cross-referenced structure of standards, which is important for accessing safety-critical technical rules.
SOURCES & TIMELINE
1arXiv:2610.33277v1 Announce Type: new Abstract: Engineering standards are challenging knowledge sources for large language models (LLMs) because they are hierarchical, multimodal, and densely connected through figures, tables, cross-references, and normative rules. Existing retrieval-augmented generation (RAG) methods often rely on unstructured text retrieval and struggle with questions requiring joint reasoning over…
arXiv:2610.36433v1 Announce Type: new Abstract: Construction safety management still struggles to produce daily safety bulletins (DSBs), structured daily communications that identify site-specific hazards, convey safety protocols, and set preventive measures for workers. Manual DSB creation takes 2-3 hours daily and yields inconsistent quality, with 25% of crews never conducting pretask planning (Olson et al., 2016),…
arXiv:2610.36335v1 Announce Type: new Abstract: Large language models (LLMs) perform well across many real-world tasks but remain prone to hallucination, limiting their reliability in domains such as biomedicine and healthcare, where answers must be grounded in established knowledge. Retrieval-augmented generation (RAG) mitigates this problem by incorporating external information, but often overlooks logical relation…