HRL-based Multi-Graph Fusion Framework for Sequential Recommendation

Abstract

 In sequential recommendation, both user historical behaviors and product attribute relationships represent crucial heterogeneous information sources. Fully leveraging this diverse information is vital for enhancing recommendation performance. While recent studies demonstrate that integrating these sources can effectively boost performance, existing methods often fail to adequately address their inherent disparities. This oversight leads to semantic conflicts during fusion, diminishing recommendation accuracy and interpretability. To address this, we propose a Hierarchical reinforcement learning-based MUlti-Graph Fusion framework(HUF for short) for adaptive heterogeneous information fusion. Specifically, we first model temporal properties and product attribute relationships as graphs, and transform the fusion task into an interactive task for intelligent agents. Next, we devise a three-level agent hierarchy: low-level agents first explore paths within their respective graphs; when path explorations intertwine, the next level (middle-level agent) determines the fusion approach and evaluates information enhancement needs; subsequently, the high-level agent finally selects the most appropriate low-level agent’s decision contextually. To tackle sparse coupling during learning, we introduce a rapid strategy: each low-level agent first generates paths; then, only paths exhibiting coupling are retained for subsequent highlevel fusion decisions to ensure learning efficiency. We compare our model with 8 methods on three real datasets, demonstrating its effectiveness. The relevant code can be found at https://anonymous.4open.science/r/HUF/.