T-RoPE: Advanced Positioning for Enhanced Recommendation Systems

Paper: T-RoPE: Time-Aware Rotary Position Embedding for Sequential Recommendation

Authors: Yang Liu, Noel Loo, Ali Khanafer, Shuying Sun, Akshay Soni, Zhong Wu, Linjun Yang

Executive Summary

Recent research has explored advancements in recommendation systems, particularly those leveraging large-scale models similar to those used in natural language processing. The study introduces a novel approach called T-RoPE, which enhances the traditional Rotary Position Embedding (RoPE) method used in recommendation algorithms.

In typical RoPE implementations, position indices are recorded to understand the order of user interactions, but this method does not account for crucial elements such as the timing of these interactions, behavioral cycles, or seasonal variations. T-RoPE addresses these gaps by integrating time-awareness into the model, using timestamp-based angles and learnable coefficients that reflect varying user behaviors over different periods. This innovation allows the model to distinguish between different contexts, such as seasonal trends, which the standard RoPE could not achieve.

The research demonstrates T-RoPE's effectiveness across five public benchmarks, where it outperformed all existing models significantly. In particular, it showed improvements of 78% to 130% in the HR@10 metric on the PixelRec dataset, and an 8% to 12% improvement across various metrics on the Amazon Books dataset. More impressively, when applied to a large-scale e-commerce dataset that includes over 6 billion user interactions, T-RoPE improved every performance metric compared to the previously established HSTU + Time RAB model, with enhancements ranging from 13% to 82%. The study found that the most substantial performance gains were attributed to its multiscale frequency handling and the use of non-stationary keys, which led to a notable 56% improvement in NDCG@50.

Further real-world applicability was confirmed through an online A/B test conducted in the Shop app, which resulted in positive outcomes, showing an increase in conversion rates by 0.33% and a 0.63% rise in the number of orders.

It is important to note that while the research presents strong results based on benchmarks and simulations, the model's practical implementation for large-scale recommendation systems remains feasible due to the linear computational cost associated with the added complexity of time-aware features. This positions T-RoPE as a promising enhancement for businesses looking to improve their recommendation systems, particularly those that rely on understanding user interactions over time.

Academic Abstract

Large-scale recommenders increasingly adopt the sequential generative recipe behind large language models, bringing the Transformer into recommendation along with design choices made for text, including Rotary Position Embedding (RoPE). In language models, RoPE encodes token indices for relative position reasoning, but in recommendation, an interaction index records only event order, saying nothing about elapsed time, behavioral cycles across scales, or calendar phase. We revisit this choice and propose T-RoPE, a time-aware RoPE for sequential generative recommendation that replaces index-only rotation with timestamp-based angles, learnable temporal coefficients, multiscale frequency banks, shifted query alignment, and non-stationary key rotation. We prove that standard RoPE, even on timestamps, remains time-translation invariant and cannot distinguish seasonal contexts, and that T-RoPE breaks this invariance while preserving the RoPE interface. Across five public benchmarks, T-RoPE achieves the best result on every metric on every dataset, improving over the strongest baseline by 78--130% in HR@10 on the sparse PixelRec data and 8--12% across metrics on Amazon Books. On an industrial-scale e-commerce dataset with more than 6B interactions, it improves every metric over the HSTU + Time RAB backbone by 13--82%, with ablations attributing the largest gains to multiscale frequencies ($+56%$ NDCG@50) and non-stationary keys ($+4%$). An online A/B test in the Shop app yields positive lifts in conversion rate ($+0.33%$) and order count ($+0.63%$). We also provide forward and backward algorithms whose added cost is linear in sequence length and head dimension, keeping time-aware RoPE practical for large generative recommenders.