RESEARCH · RESEARCH · #2105
Counterfactual Neighborhood Risk Fields (CNRF) for selective safety in vision‑language models
The paper introduces Counterfactual Neighborhood Risk Fields (CNRF), a representation-based, evidence-grounded framework that builds context-matched benign–harmful counterfactual pairs to derive capability-related risk directions and separately estimate neighborhood support. CNRF enables selective intervention (routing requests for same-victim review or producing constrained alternatives) and reports 86.06–89.30% macro defense success across three VLMs and five multimodal jailbreak families while retaining substantial benign-task utility.
KEY POINTS
- The paper introduces Counterfactual Neighborhood Risk Fields (CNRF), a representation-based, evidence-grounded framework that builds context-matched benign–harmful counterfactual pairs to derive capability-related risk directions and separately estimate neighborhood support.
- CNRF enables selective intervention (routing requests for same-victim review or producing constrained alternatives) and reports 86.06–89.30% macro defense success across three VLMs and five multimodal jailbreak families while retaining substantial benign-task utility.
- This reframes VLM safety from boundary classification to constructing evidence-backed, context-matched risk assessments that reduce false positives and allow targeted mitigation, with strong reported empirical results.
WHY IT MATTERS
This reframes VLM safety from boundary classification to constructing evidence-backed, context-matched risk assessments that reduce false positives and allow targeted mitigation, with strong reported empirical results.