RESEARCH · RESEARCH · #1108
T-RoPE: time-aware rotary position embedding improves sequential recommendation
arXiv:2609.30576v1 introduces T-RoPE, a time-aware variant of Rotary Position Embedding (RoPE) for Transformer-based sequential recommendation that replaces index-only rotations with timestamp-based angles, learnable temporal coefficients, multiscale frequency banks, shifted query alignment, and non-stationary key rotation. The paper proves standard RoPE is time-translation invariant, reports state-of-the-art gains across five public benchmarks (e.g., +78–130% HR@10 on sparse PixelRec; +8–12% on Amazon Books), large improvements on a 6B-interaction industrial e-commerce dataset (+13–82%), and small but positive online A/B lifts in the Shop app (+0.33% conversion, +0.63% order count); it also provides forward/backward algorithms with linear added cost in sequence length and head dimension.
KEY POINTS
- arXiv:2609.30576v1 introduces T-RoPE, a time-aware variant of Rotary Position Embedding (RoPE) for Transformer-based sequential recommendation that replaces index-only rotations with timestamp-based angles, learnable temporal coefficients, multiscale frequency banks, shifted query alignment, and non-stationary key rotation.
- The paper proves standard RoPE is time-translation invariant, reports state-of-the-art gains across five public benchmarks (e.g., +78–130% HR@10 on sparse PixelRec; +8–12% on Amazon Books), large improvements on a 6B-interaction industrial e-commerce dataset (+13–82%), and small but positive online A/B lifts in the Shop app (+0.33% conversion, +0.63% order count); it also provides forward/backward algorithms with linear added cost in sequence length and head dimension.
- Making positional embeddings time-aware addresses a key limitation in sequential recommenders, yielding large offline and measurable online gains while remaining computationally practical.
WHY IT MATTERS
Making positional embeddings time-aware addresses a key limitation in sequential recommenders, yielding large offline and measurable online gains while remaining computationally practical.