Edge-Aware Hybrid-Action Reinforcement Learning for Latency-Sensitive Cooperative Bus Signal Priority in Vehicular Edge Networks

Hui Deng; Xiaofei Wang; Zipeng Wang; Zesong Tian; Wen Du

doi:10.64509/jicn.23.133

Authors

Hui Deng College of Intelligence and Computing, Tianjin University, Tianjin 300350, China; China Telecom Digital City Technology Co., Ltd., Xiong'an New Area, Hebei 071000, China Author https://orcid.org/0009-0001-2341-4252
Xiaofei Wang College of Intelligence and Computing, Tianjin University, Tianjin 300350, China Author
Zipeng Wang College of Electrical and Control Engineering, North China University of Technology, Beijing 100144, China Author
Zesong Tian China Telecom Digital City Technology Co., Ltd., Xiong'an New Area, Hebei 071000, China Author
Wen Du Xiong'an National Innovation Center Technology Co., Ltd., Xiong'an New Area, Hebei 071000, China Author

DOI:

https://doi.org/10.64509/jicn.23.133

Keywords:

Vehicular Edge Computing, Edge Intelligence, Deadline-Aware Control, Bus Signal Priority, Hybrid-Action Reinforcement Learning, Multi-Agent Traffic Signal Control

Abstract

Dense short-block road networks require bus signal priority (BSP) decisions to be generated and delivered within short and reliable control windows. This paper presents Edge-HyAR-BSP, a deadline-aware cooperative BSP framework that supports edge-side execution in vehicular edge networks. Roadside edge nodes perform decentralized low-latency inference, while the cloud supports centralized training and model updates. The framework represents each priority decision as a coupled phase-duration action and checks its executability under bus ETA, signal-safety, compensation, and edge-side deadline constraints. Green extension, red truncation, and cross-cycle compensation are integrated to improve bus passage while limiting disturbance to general traffic. The project platform covers 102 signalized intersections in the Rongdong District of Xiong'an New Area. Detailed operational evaluation is conducted on a 15-intersection corridor served by Route 302, while robustness and ablation analyses are performed in simulation using a topology derived from the 102-intersection network. In the selected pre-/post-deployment periods, bus speed increased by 14.64%-30.09%, aggregate bus delay decreased by 38.28%-65.13%, and bus stops decreased by 42.01%-58.62%. Edge deployment reduced mean end-to-end decision latency from 89.7 ms to 24.6 ms. Simulation results show gradual degradation under increased delay, packet loss, and workload. The findings provide operational case-study evidence for the feasibility of edge-aware hybrid-action BSP, while broader multi-route and cross-city validation remains necessary.

Downloads

Download data is not yet available.

References

1. Lin, Y., Yang, X., Zou, N., Franz, M.: Transit signal priority control at signalized intersections: A comprehensive review. Transportation Letters, 7(3), 168-180 (2015). https://doi.org/10.1179/1942787514.0000000044 DOI: https://doi.org/10.1179/1942787514Y.0000000044

2. Skabardonis, A., Geroliminis, N.: Real-time monitoring and control on signalized arterials. Journal of Intelligent Transportation Systems, 12(2), 64-74 (2008). https://doi.org/10.1080/15472450802023337 DOI: https://doi.org/10.1080/15472450802023337

3. Wang, Z., Gupta, R., Han, K., Wang, H., Ganlath, A., Ammar, N.: Mobility digital twin: Concept, architecture, case study, and future challenges. IEEE Internet of Things Journal, 9(18), 17452-17467 (2022). https://doi.org/10.1109/JIOT.2022.3156028 DOI: https://doi.org/10.1109/JIOT.2022.3156028

4. Sun, Y., Lei, B., Liu, J., Huang, H., Zhang, X., Peng, J.: Computing power network: A survey. China Communications, 21(9), 109-145 (2024). https://doi.org/10.23919/JCC.ja.2021-0776 DOI: https://doi.org/10.23919/JCC.ja.2021-0776

5. Tang, X., Cao, C., Wang, Y., Zhang, S., Liu, Y., Li, M.: Computing power network: The architecture of convergence of computing and networking towards 6G requirement. China Communications, 18(2), 175-185 (2021). https://doi.org/10.23919/JCC.2021.02.011

6. Zhou, Z., Chen, X., Li, E., Zeng, L., Luo, K., Zhang, J.: Edge intelligence: Paving the last mile of artificial intelligence with edge computing. Proceedings of the IEEE, 107(8), 1738-1762 (2019). https://doi.org/10.1109/JPROC.2019.2918951 DOI: https://doi.org/10.1109/JPROC.2019.2918951

7. Deng, S., Zhao, H., Fang, W., Yin, J., Dustdar, S., Zomaya, A.Y.: Edge intelligence: The confluence of edge computing and artificial intelligence. IEEE Internet of Things Journal, 7(8), 7457-7469 (2020). https://doi.org/10.1109/JIOT.2020.2984887 DOI: https://doi.org/10.1109/JIOT.2020.2984887

8. Liu, L., Chen, C., Pei, Q., Maharjan, S., Zhang, Y.: Vehicular edge computing and networking: A survey. Mobile Networks and Applications, 26(3), 1145-1168 (2021). https://doi.org/10.1007/s11036-020-01624-1 DOI: https://doi.org/10.1007/s11036-020-01624-1

9. Wei, H., Zheng, G., Yao, H., Li, Z.: IntelliLight: A reinforcement learning approach for intelligent traffic light control. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 2496-2505 (2018). https://doi.org/10.1145/3219819.3220096 DOI: https://doi.org/10.1145/3219819.3220096

10. Wei, H., Chen, C., Zheng, G., Wu, K., Gayah, V., Xu, K.: PressLight: Learning Max Pressure control to coordinate traffic signals in arterial network. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1290-1298 (2019). https://doi.org/10.1145/3292500.3330949 DOI: https://doi.org/10.1145/3292500.3330949

11. Wei, H., Xu, N., Zhang, H., Zheng, G., Zang, X., Chen, C., Zhang, W., Zhu, Y., Xu, K., Li, Z.: CoLight: Learning network-level cooperation for traffic signal control. In: Proceedings of the 28th ACM International Conference on Information and Knowledge Management (CIKM), pp. 1913-1922 (2019). https://doi.org/10.1145/3357384.3357902 DOI: https://doi.org/10.1145/3357384.3357902

12. Bouktif, S., Cheniki, A., Ouni, A.: Traffic Signal Control Using Hybrid Action Space Deep Reinforcement Learning. Sensors, 21(7), 2302 (2021). https://doi.org/10.3390/s21072302 DOI: https://doi.org/10.3390/s21072302

13. Hu, W.X., Ishihara, H., Chen, C.: Deep reinforcement learning two-way transit signal priority algorithm for optimizing headway adherence and speed. IEEE Transactions on Intelligent Transportation Systems, 24(8), 7920-7931 (2023). https://doi.org/10.1109/TITS.2023.3266461 DOI: https://doi.org/10.1109/TITS.2023.3266461

14. Dong, Y., Huang, H., Zhang, G., Jin, J.: Adaptive transit signal priority control for traffic safety and efficiency optimization: A multi-objective deep reinforcement learning framework. Mathematics, 12(24), 3994 (2024). https://doi.org/10.3390/math12243994 DOI: https://doi.org/10.3390/math12243994

15. Luo, Q., Hu, S., Li, C., Li, G., Shi, W.: Resource scheduling in edge computing: A survey. IEEE Communications Surveys & Tutorials, 23(4), 2131-2165 (2021). https://doi.org/10.1109/COMST.2021.3106401 DOI: https://doi.org/10.1109/COMST.2021.3106401

16. Taleb, T., Samdanis, K., Mada, B., Flinck, H., Dutta, S., Sabella, D.: On multi-access edge computing: A survey of the emerging 5G network edge cloud architecture and orchestration. IEEE Communications Surveys & Tutorials, 19(3), 1657-1681 (2017). https://doi.org/10.1109/COMST.2017.2705720 DOI: https://doi.org/10.1109/COMST.2017.2705720

17. Wang, X., Deng, H., Qiu, C., Chen, Z., Luo, T., Ming, Z.: Computing power networks for unmanned aerial vehicles: A hierarchical resources trading market. Digital Communications and Networks, 12(4), 584-593 (2026). https://doi.org/10.1016/j.dcan.2024.12.002 DOI: https://doi.org/10.1016/j.dcan.2024.12.002

18. ETSI: Multi-access edge computing (MEC): Framework and reference architecture. ETSI GS MEC 003, V4.1.1 (2025). https://www.etsi.org/deliver/etsi_gs/MEC/001_099/003/04.01.01_60/gs_MEC003v040101p.pdf Accessed 30 June 2026.

19. Gu, H., Zhao, L., Han, Z., Zheng, G., Song, S.: AI-enhanced cloud-edge-terminal collaborative network: Survey, applications, and future directions. IEEE Communications Surveys & Tutorials, 26(2), 1322-1385 (2024). https://doi.org/10.1109/COMST.2023.3338153 DOI: https://doi.org/10.1109/COMST.2023.3338153

20. Ge, C., Qin, S.: Digital twin intelligent transportation system (DT-ITS): A systematic review. IET Intelligent Transport Systems, 18(12), 2325-2358 (2024). https://doi.org/10.1049/itr2.12539 DOI: https://doi.org/10.1049/itr2.12539

21. Thodi, B.T., Chilukuri, B.R., Vanajakshi, L.: An analytical approach to real-time bus signal priority system for isolated intersections. Journal of Intelligent Transportation Systems, 25(1), 145-167 (2021). https://doi.org/10.1080/15472450.2020.1797504 DOI: https://doi.org/10.1080/15472450.2020.1797504

22. Girijan, A., Vanajakshi, L., Chilukuri, B.R.: Dynamic Thresholds Identification for Green Extension and Red Truncation Strategies for Bus Priority. IEEE Access, 9, 64291-64305 (2021). https://doi.org/10.1109/ACCESS.2021.3074361 DOI: https://doi.org/10.1109/ACCESS.2021.3074361

23. Liu, H., Teng, K., Rai, L., Xing, J.: Transit Signal Priority Controlling Method Considering Non-Transit Traffic Benefits and Coordinated Phase States for Multi-Rings Timing Plan at Isolated Intersections. IEEE Transactions on Intelligent Transportation Systems, 22(2), 913-936 (2020). https://doi.org/10.1109/TITS.2019.2961420 DOI: https://doi.org/10.1109/TITS.2019.2961420

24. Xu, M., Chai, J., Yan, Y., Qu, X.: Multi-agent fuzzy-based transit signal priority control for traffic network considering conflicting priority requests. IEEE Transactions on Intelligent Transportation Systems, 23(2), 1554-1564 (2021). https://doi.org/10.1109/TITS.2020.3045122 DOI: https://doi.org/10.1109/TITS.2020.3045122

25. Zeng, X., Zhang, Y., Jiao, J., Yin, K.: Route-based transit signal priority using connected vehicle technology to promote bus schedule adherence. IEEE Transactions on Intelligent Transportation Systems, 22(2), 1174-1184 (2021). https://doi.org/10.1109/TITS.2020.2963839 DOI: https://doi.org/10.1109/TITS.2020.2963839

26. Abdelhalim, A., Abbas, M.: A value proposition of cooperative bus-holding transit signal priority strategy in connected and automated vehicles environment. IEEE Transactions on Intelligent Transportation Systems, 23(7), 9301-9306 (2021). https://doi.org/10.1109/TITS.2021.3086776 DOI: https://doi.org/10.1109/TITS.2021.3086776

27. Cvijovic, Z., Zlatkovic, M., Stevanovic, A., Song, Y.: Conditional transit signal priority for connected transit vehicles. Transportation Research Record: Journal of the Transportation Research Board, 2676(2), 490-503 (2021). https://doi.org/10.1177/03611981211044459 DOI: https://doi.org/10.1177/03611981211044459

28. Zheng, G., Xiong, Y., Zang, X., Feng, J., Wei, H., Zhang, H., Li, Y., Xu, K., Li, Z.: Learning phase competition for traffic signal control. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pp. 1936-1972 (2019). https://doi.org/10.1145/3357384.3357900 DOI: https://doi.org/10.1145/3357384.3357900

29. Noaeen, M., Naik, A., Goodman, L., Crebo, J., Abrar, T., Abad, Z.S.H., Bazzan, A.L.C., Far, B.: Reinforcement learning in urban network traffic signal control: A systematic literature review. Expert Systems with Applications, 199, 116830 (2022). https://doi.org/10.1016/j.eswa.2022.116830 DOI: https://doi.org/10.1016/j.eswa.2022.116830

30. Chu, T., Wang, J., Codeca, L., Li, Z.: Multi-agent deep reinforcement learning for large-scale traffic signal control. IEEE Transactions on Intelligent Transportation Systems, 21(3), 1086-1096 (2020). https://doi.org/10.1109/TITS.2019.2901791 DOI: https://doi.org/10.1109/TITS.2019.2901791

31. Wang, P., Deng, H., Zhang, J., Wang, L., Zhang, M., Li, Y.: Model predictive control for connected vehicle platoon under switching communication topology. IEEE Transactions on Intelligent Transportation Systems, 23(7), 7817-7830 (2022). https://doi.org/10.1109/TITS.2021.3073012 DOI: https://doi.org/10.1109/TITS.2021.3073012

32. Li, B., Tang, H., Zheng, Y., Hao, J., Li, P., Meng, Z., Wang, L.: HyAR: Addressing discrete-continuous action reinforcement learning via hybrid action representation. In International Conference on Learning Representations (ICLR), pp. 1-19 (2022). https://doi.org/10.48550/arXiv.2109.05490

33. Haarnoja, T., Zhou, A., Abbeel, P., Levine, S.: Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In Proceedings of the 35th International Conference on Machine Learning (ICML), pp. 1861-1870 (2018). https://doi.org/10.48550/arXiv.1801.01290

34. Tang, X., Cao, C., Wang, Y., Zhang, S., Liu, Y., Li, M., He, T.: Computing power network: The architecture of convergence of computing and networking towards 6G requirement. China Communications, 18(2), 175-185 (2021). https://doi.org/10.23919/JCC.2021.02.011 DOI: https://doi.org/10.23919/JCC.2021.02.011

35. Yu, W., Ngoudy, D., Hua, X., Wang, W.: On the stability of a heterogeneous platoon-based traffic system with multiple anticipations in the presence of connected and automated vehicles. Transportation Research Part C: Emerging Technologies, 157, 104389 (2023). https://doi.org/10.1016/j.trc.2023.104389 DOI: https://doi.org/10.1016/j.trc.2023.104389

36. Liu, M., Zhu, M., Yao, M., Li, P., Tang, R., Deng, H.: Cooperative lane-changing control method for autonomous vehicles based on dynamic trajectory planning. Journal of Transportation Systems Engineering and Information Technology, 24(5), 65-78 (2024). https://doi.org/10.16097/j.cnki.1009-6744.2024.05.007