NEWS · MODELS · #1925
GitHub deploys ModernBERT-based context-aware classifier to reduce secret scanning noise
GitHub replaced an LLM-heavy secret-scanning pipeline with a two-stage system: fast regex candidate generation followed by a fine-tuned ModernBERT classifier that evaluates each candidate plus surrounding context. The new pipeline yields an 86% faster median post-push scan execution time, over 98% lower serving cost versus the previous LLM-based approach, and an 11.5% improvement in user acceptance by reducing false positives.
KEY POINTS
- GitHub replaced an LLM-heavy secret-scanning pipeline with a two-stage system: fast regex candidate generation followed by a fine-tuned ModernBERT classifier that evaluates each candidate plus surrounding context.
- The new pipeline yields an 86% faster median post-push scan execution time, over 98% lower serving cost versus the previous LLM-based approach, and an 11.5% improvement in user acceptance by reducing false positives.
- Reducing false positives and latency makes secret scanning practical for push‑time protection and scales secret detection more cheaply and accurately across repositories.
WHY IT MATTERS
Reducing false positives and latency makes secret scanning practical for push‑time protection and scales secret detection more cheaply and accurately across repositories.