NewsBench: expert-grounded benchmark for epistemic quality in AI news reporting (arXiv:2610.33240v1)
NewsBench is an expert-grounded benchmark and two-phase evaluation pipeline for AI-generated news and current-events reporting that operationalizes source quality, factuality, and neutrality using guidelines set by senior journalists, policy, and intelligence practitioners. The release comprises ≈3,000 evaluation prompts (≈500 refreshed monthly), ~2,500 human expert-annotated items for calibration and validation, and a suite of automated AI judges validated against expert-adjudicated gold labels; applied to four frontier models, it reveals distinct, non-overlapping failure profiles across sourcing, factuality, and neutrality.