NEWS · MODELS · #372
arXiv:2609.16129v1 proposes Fisher‑geodesic optimal parameter pruning for neural networks
The paper introduces a parameter‑pruning scheme derived from the geodesic distance in model space defined by the Fisher information metric. The authors derive a hierarchy of pruning approximations (recovering magnitude pruning as a first level) and report that the Fisher‑geodesic method outperforms magnitude and local‑Fisher pruning on fully‑connected networks and vision transformers evaluated on MNIST and CIFAR‑10 across the full 0–100% pruning range and five seeds, using accuracy and the Matthews correlation coefficient.
KEY POINTS
- The paper introduces a parameter‑pruning scheme derived from the geodesic distance in model space defined by the Fisher information metric.
- The authors derive a hierarchy of pruning approximations (recovering magnitude pruning as a first level) and report that the Fisher‑geodesic method outperforms magnitude and local‑Fisher pruning on fully‑connected networks and vision transformers evaluated on MNIST and CIFAR‑10 across the full 0–100% pruning range and five seeds, using accuracy and the Matthews correlation coefficient.
- Provides a mathematically grounded pruning criterion (Fisher‑geodesic distance) that empirically yields better compression–accuracy tradeoffs than common heuristics, potentially improving model compression practices.
WHY IT MATTERS
Provides a mathematically grounded pruning criterion (Fisher‑geodesic distance) that empirically yields better compression–accuracy tradeoffs than common heuristics, potentially improving model compression practices.