Neural networks are more modular at the subspace level than single neurons suggest
The paper (arXiv:2610.12842v1) defines a hierarchy of modularity—unit-level, orthogonal, and linear—and presents a unified framework to learn and causally evaluate decompositions at each level via selective ablation. Across vision models, language models, and multi-task RNNs, the authors find that distinct computations often occupy separable subspaces despite overlapping neuron populations; these subspaces also differentially align with brain activity in visual and language cortex.