RESEARCH · RESEARCH · #2031
LLM-referenced C-test using Levin's Kt to certify high-difficulty (supra-intelligence) items (arXiv:2610.14005v1)
The paper introduces a Levin's Kt–based benchmark that treats large language models as reference Universal Turing Machines, formalises a difficulty measure in bits, and describes a lightweight generation and certification pipeline. The authors produce a first battery of certified items, administer them to twelve LLMs, and report that the procedure can certify fresh, high-difficulty items reaching a 'supra-intelligence' region beyond current human and frontier-model performance.
KEY POINTS
- The paper introduces a Levin's Kt–based benchmark that treats large language models as reference Universal Turing Machines, formalises a difficulty measure in bits, and describes a lightweight generation and certification pipeline.
- The authors produce a first battery of certified items, administer them to twelve LLMs, and report that the procedure can certify fresh, high-difficulty items reaching a 'supra-intelligence' region beyond current human and frontier-model performance.
- If robust, this offers a principled, non-circular way to generate and certify problems whose difficulty scales with computational mechanism complexity, enabling evaluation of systems beyond current human-calibrated benchmarks.
WHY IT MATTERS
If robust, this offers a principled, non-circular way to generate and certify problems whose difficulty scales with computational mechanism complexity, enabling evaluation of systems beyond current human-calibrated benchmarks.