Network-Wide Traffic Signal Control Based on MARL With Hierarchical Nash-Stackelberg Game Model

doi:10.1109/ACCESS.2023.3345448

CORC > 自动化研究所 > 中国科学院自动化研究所 > 多模态人工智能系统全国重点实验室

	Network-Wide Traffic Signal Control Based on MARL With Hierarchical Nash-Stackelberg Game Model
	Shen, Hui 1,2; Zhao, Hongxia 3; Zhang, Zundong 4; Yang, Xun 4; Song, Yutong 4; Liu, Xiaoming 4
刊名	IEEE ACCESS
	2023
卷号	11 页码:145085-145100
关键词	Games Roads Approximation algorithms Q-learning Multi-agent systems Process control Optimization Reinforcement learning Traffic control Network-wide traffic signal control hierarchical game model multi-agent reinforcement learning
ISSN号	2169-3536
DOI	10.1109/ACCESS.2023.3345448
通讯作者	Zhang, Zundong(zdzhang@ncut.edu.cn)
英文摘要	Network-wide traffic signal control is an important means of relieving urban congestion, reducing traffic accidents, and improving traffic efficiency. However, solving the problem of computational complexity caused by multi-intersection games is challenging. To address this issue, we propose a Nash-Stackelberg hierarchical game model that considers the importance of different intersections in the road network and the game relationships between intersections. The model takes into account traffic control strategies between and within sub-areas of the road network, with important intersections in the two sub-areas as the game subject at the upper layer and secondary intersections as the game subject at the lower layer. Furthermore, we propose two reinforcement learning algorithms (NSHG-QL and NSHG-DQN) based on the Nash-Stackelberg hierarchical game model to realize coordinated control of traffic signals in urban areas. Experimental results show that, compared to basic game model solving algorithms, NSHG-QL and NSHG-DQN algorithms can reduce the average travel time and time loss of vehicles at intersections, increase average speed and road occupancy, and coordinate secondary intersections to make optimal strategy selections based on satisfying the upper-layer game between important intersections. Moreover, the multi-agent reinforcement learning algorithms based on this hierarchical game model can significantly improve learning performance and convergence.
资助项目	National Natural Science Foundation Project
WOS关键词	MULTIAGENT SYSTEMS ; REINFORCEMENT
WOS研究方向	Computer Science ; Engineering ; Telecommunications
语种	英语
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
WOS记录号	WOS:001132233900001
资助机构	National Natural Science Foundation Project
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/54910]
专题	多模态人工智能系统全国重点实验室
通讯作者	Zhang, Zundong
作者单位	1.North China Univ Technol, Sch Elect & Control Engn, Beijing 100037, Peoples R China 2.Beijing Municipal Traff Management Bur, Beijing 100037, Peoples R China 3.Chinese Acad Sci, Inst Automat, State Key Lab Multimodal Artificial Intelligence S, Beijing 100190, Peoples R China 4.North China Univ Technol, Beijing Key Lab Urban Rd Traff Intelligent Technol, Beijing 100144, Peoples R China
推荐引用方式 GB/T 7714	Shen, Hui,Zhao, Hongxia,Zhang, Zundong,et al. Network-Wide Traffic Signal Control Based on MARL With Hierarchical Nash-Stackelberg Game Model[J]. IEEE ACCESS,2023,11:145085-145100.
APA	Shen, Hui,Zhao, Hongxia,Zhang, Zundong,Yang, Xun,Song, Yutong,&Liu, Xiaoming.(2023).Network-Wide Traffic Signal Control Based on MARL With Hierarchical Nash-Stackelberg Game Model.IEEE ACCESS,11,145085-145100.
MLA	Shen, Hui,et al."Network-Wide Traffic Signal Control Based on MARL With Hierarchical Nash-Stackelberg Game Model".IEEE ACCESS 11(2023):145085-145100.