Tech Meridian ← LIVE FEED
PROMY MERIDIAN RU

RESEARCH · RESEARCH · #3188

CST-Bench: benchmark and annotations for counterfactual spatial reasoning, plus CT-OPSD method

The authors introduce CST-Bench, a dataset of 7,082 examples for counterfactual spatial reasoning covering 6 intervention types and 12 transformation operators with dense causal annotations (intervention sources, affected/invariant relations, expected effects, metric changes). They benchmark general-purpose multimodal LLMs on CST-Bench, compare post-training strategies, and propose CT-OPSD (Causal Transition On-Policy Self-Distillation) to teach counterfactual state transitions via annotated teacher chains distilled into a student model, reporting CST-Bench as a challenging evaluation and reusable supervision resource.

KEY POINTS

  1. The authors introduce CST-Bench, a dataset of 7,082 examples for counterfactual spatial reasoning covering 6 intervention types and 12 transformation operators with dense causal annotations (intervention sources, affected/invariant relations, expected effects, metric changes).
  2. They benchmark general-purpose multimodal LLMs on CST-Bench, compare post-training strategies, and propose CT-OPSD (Causal Transition On-Policy Self-Distillation) to teach counterfactual state transitions via annotated teacher chains distilled into a student model, reporting CST-Bench as a challenging evaluation and reusable supervision resource.
  3. CST-Bench supplies dense causal annotations and focused tasks that reveal failures of standard spatial reasoning methods under interventions and provides a benchmarking and supervision resource to improve MLLM causal spatial reasoning.

WHY IT MATTERS

CST-Bench supplies dense causal annotations and focused tasks that reveal failures of standard spatial reasoning methods under interventions and provides a benchmarking and supervision resource to improve MLLM causal spatial reasoning.

SOURCES & TIMELINE

1