A Dynamic MLP Architecture for Sequential Recommendation

Abstract

Sequential recommendation aims to predict users' next interested item based on their historical interaction sequence. In the past few years, the transformer-based model has achieved the most advanced performance in sequential recommendation. Recently, in order to improve the quadratic memory and time complexity in the attention mechanism of transformer, especially when processing extremely long sequences, Multi-Layer Perceptions (MLP)-based models have been proposed to replace transformer under the premise of linear computational complexity. However, due to the simplicity and static nature of the MLP's structure, it cannot effectively mine multiple semantics of user interests and dynamically adjust model weights according to different input data like Transformer. To this end, we propose a simple yet efficient model called Dynamic MLP architecture for sequential Recommendation(DM4Rec), which can capture users' preference by stacking a series of MLP layers whose weights are generated dynamically according to different input sequences. Specifically, we first utilize a dynamic sequence mixer to model the sequential dependency of a user interaction sequence because the item transition pattern may be completely distinct for various input sequences. Then we use a channel mixer to explore and exploit channel correlation within each item. Moreover, the same as multi-head attention, we introduce a novel multi-segment mixing mechanism, which effectively incorporates multiple semantics into user modeling. Extensive experiments on four real-world datasets show our dynamic MLP model outperforms Transformer-based models while maintaining linear computational complexity.