Refining Graph Learning Dynamics with the Riemann–Liouville Fractional Derivative

Yufeng Peng; Kang Qiyu; Kai Zhao; Xuhao Li; Qinxu Ding

doi:10.64509/jicn.22.93

Authors

Yufeng Peng School of Information Science and Technology, University of Science and Technology of China, Hefei 230026, China Author
Kang Qiyu School of Information Science and Technology, University of Science and Technology of China, Hefei 230026, China Author
Kai Zhao School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore 639798, Singapore Author
Xuhao Li School of Mathematical Sciences, Anhui University, Hefei 230601, China Author
Qinxu Ding }School of Business, Singapore University of Social Sciences, Singapore 599494, Singapore Author

DOI:

https://doi.org/10.64509/jicn.22.93

Keywords:

Graph Neural Networks, Fractional Calculus, Riemann–Liouville Derivative

Abstract

We introduce the Riemann–Liouville Graph neural Fractional-order Differential Equation (RL-GFDE), a novel continuous Graph Neural Network (GNN) framework that incorporates the Riemann–Liouville (RL) fractional derivative. This framework effectively captures feature updating dynamics with pronounced memory effects, offering an advantage over standard graph neural ordinary differential equation (ODE) models that rely solely on integer-order derivatives. Our contributions are twofold: First, we establish a generalized continuous GNN framework grounded in fractional calculus. We also provide a comprehensive convergence analysis for the solvers applied to RL-GFDE, ensuring precision and efficiency in our method. Second, through theoretical analysis and extensive empirical experiments, we demonstrate the enhanced performance of RL-GFDE by comparing fractional adaptations of various well-known contemporary graph neural ODE models against their original integer-order counterparts. Our results validate the efficacy of RL-GFDE, showcasing its versatility as an extension to boost the performance of graph neural ODE models in various applications.

Downloads

Download data is not yet available.

References

[1] Kipf, T.N., Welling, M.: Semi-Supervised Classification with Graph Convolutional Networks. In 5th International Conference on Learning Representations, pp. 1-14 (2017)

[2] Velickovic, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y.: Graph attention networks. In 6th International Conference on Learning Representations, pp. 1-12 (2018)

[3] Hamilton, W., Ying, Z., Leskovec, J.: Inductive Representation Learning on Large Graphs. In Proceedings of the 31st International Conference on Neural Information Processing Systems, pp. 1025-1035 (2017)

[4] Shi, W., Rajkumar, R.: Point-GNN: Graph Neural Network for 3D Object Detection in a Point Cloud. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1711-1719 (2020). https://doi.org/10.1109/CVPR42600.2020.00178

[5] Reiser, P., Neubert, M., Eberhard, A., Torresi, L., Zhou, C., Shao, C., Metni, H., Hoesel, C., Schopmans, H., Sommer, T., et al.: Graph neural networks for materials science and chemistry. Communications Materials 3(1), 93 (2022). https://doi.org/10.1038/s43246-022-00315-6

[6] Fan, W., Ma, Y., Li, Q., He, Y., Zhao, E., Tang, J., Yin, D.: Graph neural networks for social recommendation. In The World Wide Web Conference, pp. 417-426 (2019). https://doi.org/10.1145/3308558.3313488

[7] Han, A., Shi, D., Lin, L., Gao, J.: From Continuous Dynamics to Graph Neural Networks: Neural Diffusion and Beyond. arXiv preprint arXiv:2310.10121 (2023). https://doi.org/10.48550/arXiv.2310.10121

[8] Poli, M., Massaroli, S., Park, J., Yamashita, A., Asama, H., Park, J.: Graph neural ordinary differential equations. arXiv preprint arXiv:1911.07532 (2019). https://doi.org/10.48550/arXiv.1911.07532

[9] Xhonneux, L.-P., Qu, M., Tang, J.: Continuous graph neural networks. In Proceedings of the 37th International Conference on Machine Learning, pp. 10432-10441 (2020)

[10] Chamberlain, B.P., Rowbottom, J., Goronova, M., Webb, S., Rossi, E., Bronstein, M.M.: GRAND: Graph Neural Diffusion. In Proceedings of the 38th International Conference on Machine Learning, pp. 1401-1418 (2021)

[11] Thorpe, M., Xia, H., Nguyen, T., Strohmer, T., Bertozzi, A., Osher, S., Wang, B.: GRAND++: Graph neural diffusion with a source term. In The Tenth International Conference on Learning Representations, pp. 1-21 (2022)

[12] Rusch, T.K., Chamberlain, B., Rowbottom, J., Mishra, S., Bronstein, M.: Graph-coupled oscillator networks. In Proceedings of the 39th International Conference on Machine Learning, pp. 18888-18909 (2022)

[13] Song, Y., Kang, Q., Wang, S., Zhao, K., Tay, W.P.: On the Robustness of Graph Neural Diffusion to Topology Perturbations. In Proceedings of the 36th International Conference on Neural Information Processing Systems, pp. 6384-6396 (2022)

[14] Choi, J., Hong, S., Park, N., Cho, S.-B.: GREAD: Graph Neural Reaction-Diffusion Equations. In Proceedings of the 40th International Conference on Machine Learning, pp. 5722-5747 (2023)

[15] Zhao, K., Kang, Q., Song, Y., She, R., Wang, S., Tay, W.P.: Graph Neural Convection-Diffusion with Heterophily. In Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, pp. 4656-4664 (2023). https://doi.org/10.24963/ijcai.2023/518

[16] Zhao, K., Kang, Q., Song, Y., She, R., Wang, S., Tay, W.P.: Adversarial Robustness in Graph Neural Networks: A Hamiltonian Approach. In 37th Conference on Neural Information Processing Systems (NeurIPS 2023), pp. 3338-3361 (2023)

[17] Baleanu, D., Diethelm, K., Scalas, E., Trujillo, J.J.: Fractional Calculus: Models and Numerical Methods in Series on Complexity, Nonlinearity and Chaos. World Scientific, Singapore (2012). https://doi.org/10.1142/8180

[18] Metzler, R., Klafter, J.: The random walk's guide to anomalous diffusion: a fractional dynamics approach. Physics reports 339(1), 1-77 (2000). https://doi.org/10.1016/S0370-1573(00)00070-3

[19] Yang, Q., Chen, D., Zhao, T., Chen, Y.: Fractional calculus in image processing: a review. Fractional Calculus and Applied Analysis 19(5), 1222-1249 (2016). https://doi.org/10.1515/fca-2016-0063

[20] Tarasov, V.E., Zaslavsky, G.M.: Fractional Ginzburg-Landau equation for fractal media. Physica A: Statistical Mechanics and its Applications 354, 249-261 (2005). https://doi.org/10.1016/j.physa.2005.02.047

[21] Tarasov, V.E.: Fractional dynamics: applications of fractional calculus to dynamics of particles, fields and media. Springer, Berlin, Heidelberg (2011). https://doi.org/10.1007/978-3-642-14003-7

[22] Cai, C., Wang, Y.: A Note on Over-Smoothing for Graph Neural Networks. arXiv preprint arXiv:2006.13318 (2020). https://doi.org/10.48550/arXiv.2006.13318

[23] Diethelm, K.: The analysis of fractional differential equations: an application-oriented exposition using differential operators of Caputo type. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-14574-2

[24] Chen, R.T., Rubanova, Y., Bettencourt, J., Duvenaud, D.: Neural ordinary differential equations. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, pp. 6572-6583 (2018)

[25] Elliott, D.: An asymptotic analysis of two algorithms for certain Hadamard finite-part integrals. IMA journal of numerical analysis 13(3), 445-462 (1993). https://doi.org/10.1093/imanum/13.3.445

[26] Cormen, T.H., Leiserson, C.E., Rivest, R.L., Stein, C.: Introduction to algorithms. MIT Press, Cambridge, Massachusetts, USA (2022)

[27] Chami, I., Ying, Z., Re, C., Leskovec, J.: Hyperbolic graph convolutional neural networks. In Proceedings of the 33rd International Conference on Neural Information Processing Systems, pp. 4868-4879 (2019)

[28] Kim, J., Goh, K.-I., Salvi, G., Oh, E., Kahng, B., Kim, D.: Fractality in complex networks: Critical and supercritical skeletons. Physical Review E 75(1), 016110 (2007). https://doi.org/10.1103/PhysRevE.75.016110

[29] Nigmatullin, R.: Fractional integral and its physical interpretation. Theoretical and Mathematical Physics 90(3), 242-251 (1992). https://doi.org/10.1007/BF01036529

[30] Zhu, S., Pan, S., Zhou, C., Wu, J., Cao, Y., Wang, B.: Graph Geometry Interaction Learning. In Proceedings of the 34th International Conference on Neural Information Processing Systems, pp. 7548-7558 (2020)

[31] Dwivedi, V.P., Rampasek, L., Galkin, M., Parviz, A., Wolf, G., Luu, A.T., Beaini, D.: Long Range Graph Benchmark. In Proceedings of the 36th International Conference on Neural Information Processing System, pp. 22326-22340 (2023)

[32] Dwivedi, V.P., Joshi, C.K., Luu, A.T., Laurent, T., Bengio, Y., Bresson, X.: Benchmarking Graph Neural Networks. The Journal of Machine Learning Research 24(1), 1730-1777 (2022)

[33] Dwivedi, V.P., Luu, A.T., Laurent, T., Bengio, Y., Bresson, X.: Graph Neural Networks with Learnable Structural and Positional Representations. In The Tenth International Conference on Learning Representations (ICLR 2022), pp. 1-25 (2022)

[34] Pei, H., Wei, B., Chang, K.C.-C., Lei, Y., Yang, B.: Geom-GCN: Geometric Graph Convolutional Networks. In International Conference on Learning Representations (ICLR 2020), pp. 1-12 (2019)

[35] Chen, M., Wei, Z., Huang, Z., Ding, B., Li, Y.: Simple and deep graph convolutional networks. In Proceedings of the 37th International Conference on Machine Learning, pp. 1725-1735 (2020)

[36] Hu, W., Liu, B., Gomes, J., Zitnik, M., Liang, P., Pande, V., Leskovec, J.: Strategies for Pre-training Graph Neural Networks. In International Conference on Learning Representations (ICLR 2020), pp. 1-22 (2020)

[37] Bresson, X., Laurent, T.: Residual Gated Graph ConvNets. In International Conference on Learning Representations (ICLR 2018), pp. 1-11 (2018)

[38] Vaswani, A., Shazeer, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Systems, pp. 6000-6010 (2017)

[39] Kreuzer, D., Beaini, D., Hamilton, W., Letourneau, V., Tossou, P.: Rethinking graph transformers with spectral attention. In Proceedings of the 35th International Conference on Neural Information Processing Systems, pp. 21618-21629 (2021)

[40] Gutteridge, B., Dong, X., Bronstein, M.M., Di Giovanni, F.: Drew: Dynamically rewired message passing with delay. In Proceedings of the 40th International Conference on Machine Learning, pp. 12252-12267 (2023)

[41] Michel, G., Nikolentzos, G., Lutzeyer, J.F., Vazirgiannis, M.: Path neural networks: Expressive and accurate graph neural networks. In Proceedings of the 40th International Conference on Machine Learning, pp. 24737-24755 (2023)

[42] Hu, W., Fey, M., Zitnik, M., Dong, Y., Ren, H., Liu, B., Catasta, M., Leskovec, J.: Open graph benchmark: Datasets for machine learning on graphs (2020)

[43] Zeng, H., Zhou, H., Srivastava, A., Kannan, R., Prasanna, V.: GraphSAINT: Graph Sampling Based Inductive Learning Method. In 8th International Conference on Learning Representations (ICLR 2020), pp. 1-19 (2020)

[44] Fey, M., Sunil, J., Nitta, A., Puri, R., Shah, M., Stojanovic, B., Bendias, R., Barghi, A., Kocijan, V., Zhang, Z., He, X., Lenssen, J.E., Leskovec, J.: PyG 2.0: Scalable Learning on Real World Graphs. In Temporal Graph Learning Workshop @ KDD 2025, pp. 1-9 (2025)

[45] Feng, J., Chen, Y., Li, F., Sarkar, A., Zhang, M.: How Powerful are K-hop Message Passing Graph Neural Networks. In Thirty-Sixth Conference on Neural Information Processing Systems (NeurIPS 2022), pp. 1-15 (2022)

[46] Oono, K., Suzuki, T.: Graph Neural Networks Exponentially Lose Expressive Power for Node Classification. In 8th International Conference on Learning Representations (ICLR 2020), pp. 1-37 (2020)

[47] Chung, F.R.: Spectral graph theory. American Mathematical Society, Providence, Rhode Island, USA (1997)

[48] Li, G., Muller, M., Thabet, A., Ghanem, B.: DeepGCNs: Can GCNs Go As Deep As CNNs? In 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 9267-9276 (2019). https://doi.org/10.1109/ICCV.2019.00936

[49] Xu, K., Li, C., Tian, Y., Sonobe, T., Kawarabayashi, K.-i., Jegelka, S.: Representation learning on graphs with jumping knowledge networks. In Proceedings of the 35th International Conference on Machine Learning, pp. 5453-5462 (2018)

[50] Li, G., Xiong, C., Thabet, A., Ghanem, B.: DeeperGCN: All You Need to Train Deeper GCNs. In The Tenth International Conference on Learning Representations (ICLR 2022), pp. 1-18 (2022)

[51] Jin, B., Li, B., Zhou, Z.: Numerical analysis of nonlinear subdiffusion equations. SIAM Journal on Numerical Analysis 56(1), 1-23 (2018). https://doi.org/10.1137/16M1089320

[52] Diethelm, K.: An algorithm for the numerical solution of differential equations of fractional order. Electronic Transactions on Numerical Analysis 5, 1-6 (1997)

[53] Podlubny, I.: Fractional Differential Equations. Academic Press, San Diego, CA, USA (1999)

[54] Horn, R.A., Johnson, C.R.: Matrix analysis. Cambridge university press, New York, USA (2012)