Exphormer: using expander graphs to scale graph transformers
Exphormer is a sparse-attention framework for graph transformers proposed by Google Research (presented at ICML 2023) that replaces the dense full-attention interaction graph with a sparse d-regular expander graph (using a randomized Friedman near-Ramanujan construction). The approach combines expander edges with the input graph and virtual nodes to achieve linear edges and reports strong empirical results across diverse graph datasets; an implementation is available on GitHub.