RESEARCH · RESEARCH · #2707
MARQ: modular, LLM-agnostic prototype that infuses INCOSE requirement criteria for requirement-quality assessment
The arXiv preprint introduces MARQ, a modular, LLM-agnostic prototype that supplements LLMs with a criterion-specific knowledge stack implementing INCOSE requirement-quality criteria. Evaluated on an expert-labeled NASA requirement set with 100 executions each using Claude Opus 5, GPT-5.6 Sol, and Gemma 3 12B, MARQ raises median F1 from off-the-shelf baselines (Claude: 0.55 → 0.80; GPT-5.6 Sol: 0.52 → 0.62; Gemma 3 12B: 0.18 → 0.53), with gains concentrated in Correctness and Conformance and model-dependent trade-offs between misses and false alarms.
KEY POINTS
- The arXiv preprint introduces MARQ, a modular, LLM-agnostic prototype that supplements LLMs with a criterion-specific knowledge stack implementing INCOSE requirement-quality criteria.
- Evaluated on an expert-labeled NASA requirement set with 100 executions each using Claude Opus 5, GPT-5.6 Sol, and Gemma 3 12B, MARQ raises median F1 from off-the-shelf baselines (Claude: 0.55 → 0.80; GPT-5.6 Sol: 0.52 → 0.62; Gemma 3 12B: 0.18 → 0.53), with gains concentrated in Correctness and Conformance and model-dependent trade-offs between misses and false alarms.
- Infusing explicit systems-engineering (INCOSE) knowledge into LLM workflows can materially improve automated requirement-quality detection and model trustworthiness, while showing that gains are model- and criterion-specific and still require human-in-the-loop review.
WHY IT MATTERS
Infusing explicit systems-engineering (INCOSE) knowledge into LLM workflows can materially improve automated requirement-quality detection and model trustworthiness, while showing that gains are model- and criterion-specific and still require human-in-the-loop review.