Target Tracking Control of a Biomimetic Underwater Vehicle Through Deep Reinforcement Learning

doi:10.1109/TNNLS.2021.3054402

CORC > 自动化研究所 > 中国科学院自动化研究所 > 复杂系统管理与控制国家重点实验室 > 先进机器人控制团队

	Target Tracking Control of a Biomimetic Underwater Vehicle Through Deep Reinforcement Learning
	Wang, Yu 1; Tang, Chong 4,5; Wang, Shuo 1,2,3; Cheng, Long 1; Wang, Rui 1; Tan, Min 1,2; Hou, Zengguang 1
刊名	IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
	2021-02-08
页码	12
关键词	Reinforcement learning Target tracking Robots Sports Aerospace electronics Mobile robots Underwater vehicles Biomimetic underwater vehicle (BUV) reinforcement learning target tracking control
ISSN号	2162-237X
DOI	10.1109/TNNLS.2021.3054402
通讯作者	Cheng, Long(long.cheng@ia.ac.cn)
英文摘要	In this article, the underwater target tracking control problem of a biomimetic underwater vehicle (BUV) is addressed. Since it is difficult to build an effective mathematic model of a BUV due to the uncertainty of hydrodynamics, target tracking control is converted into the Markov decision process and is further achieved via deep reinforcement learning. The system state and reward function of underwater target tracking control are described. Based on the actor-critic reinforcement learning framework, the deep deterministic policy gradient actor-critic algorithm with supervision controller is proposed. The training tricks, including prioritized experience replay, actor network indirect supervision training, target network updating with different periods, and expansion of exploration space by applying random noise, are presented. Indirect supervision training is designed to address the issues of low stability and slow convergence of reinforcement learning in the continuous state and action space. Comparative simulations are performed to show the effectiveness of the training tricks. Finally, the proposed actor-critic reinforcement learning algorithm with supervision controller is applied to the physical BUV. Swimming pool experiments of underwater object tracking of the BUV are conducted in multiple scenarios to verify the effectiveness and robustness of the proposed method.
资助项目	Youth Innovation Promotion Association CAS[2018162] ; National Natural Science Foundation of China[U1713222] ; National Natural Science Foundation of China[62025307] ; National Natural Science Foundation of China[62073316] ; National Natural Science Foundation of China[U1806204] ; National Natural Science Foundation of China[62033013] ; Beijing Municipal Natural Science Foundation[JQ19020]
WOS关键词	MOVING-TARGET ; MOBILE ROBOT
WOS研究方向	Computer Science ; Engineering
语种	英语
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
WOS记录号	WOS:000732356900001
资助机构	Youth Innovation Promotion Association CAS ; National Natural Science Foundation of China ; Beijing Municipal Natural Science Foundation
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/46848]
专题	自动化研究所_复杂系统管理与控制国家重点实验室_先进机器人控制团队
通讯作者	Cheng, Long
作者单位	1.Chinese Acad Sci, Inst Automat, State Key Lab Management & Control Complex Syst, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Beijing 100049, Peoples R China 3.CAS Ctr Excellence Brain Sci & Intelligence Techn, Shanghai 200031, Peoples R China 4.Tsinghua Univ, Dept Comp Sci & Technol, Beijing 100084, Peoples R China 5.NUCTECH Co Ltd, Beijing 100084, Peoples R China
推荐引用方式 GB/T 7714	Wang, Yu,Tang, Chong,Wang, Shuo,et al. Target Tracking Control of a Biomimetic Underwater Vehicle Through Deep Reinforcement Learning[J]. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,2021:12.
APA	Wang, Yu.,Tang, Chong.,Wang, Shuo.,Cheng, Long.,Wang, Rui.,...&Hou, Zengguang.(2021).Target Tracking Control of a Biomimetic Underwater Vehicle Through Deep Reinforcement Learning.IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,12.
MLA	Wang, Yu,et al."Target Tracking Control of a Biomimetic Underwater Vehicle Through Deep Reinforcement Learning".IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS (2021):12.