From Rough to Precise: Human-Inspired Phased Target Learning Framework for Redundant Musculoskeletal Systems

doi:10.3389/fnbot.2019.00061

	From Rough to Precise: Human-Inspired Phased Target Learning Framework for Redundant Musculoskeletal Systems
	Zhou, Junjie1,2,3 ; Chen, Jiahao 2,3,4; Deng, Hu 1,3; Qiao, Hong1,2,5
刊名	FRONTIERS IN NEUROROBOTICS
	2019-07-31
卷号	13 页码:14
关键词	musculoskeletal system human-inspired motion learning noise in nervous system reinforcement learning phased target learning
ISSN号	1662-5218
DOI	10.3389/fnbot.2019.00061
通讯作者	Qiao, Hong(hong.qiao@ia.ac.cn)
英文摘要	Redundant muscles in human-like musculoskeletal robots provide additional dimensions to the solution space. Consequently, the computation of muscle excitations remains an open question. Conventional methods like dynamic optimization and reinforcement learning usually have high computational costs or unstable learning processes when applied to a complex musculoskeletal system. Inspired by human learning, we propose a phased target learning framework that provides different targets to learners at varying levels, to guide their training process and to avoid local optima. By introducing an extra layer of neurons reflecting a preference, we improve the Q-network method to generate continuous excitations. In addition, based on information transmission in the human nervous system, two kinds of biological noise sources are introduced into our framework to enhance exploration over the solution space. Tracking experiments based on a simplified musculoskeletal arm model indicate that under guidance of phased targets, the proposed framework prevents divergence of excitations, thus stabilizing training. Moreover, the enhanced exploration of solutions results in smaller motion errors. The phased target learning framework can be expanded for general-purpose reinforcement learning, and it provides a preliminary interpretation for modeling the mechanisms of human motion learning.
资助项目	National Key Research and Development Program of China[2017YFB1300200] ; National Key Research and Development Program of China[2017YFB1300203] ; National Natural Science Foundation of China[91648205] ; National Natural Science Foundation of China[61627808] ; Strategic Priority Research Program of Chinese Academy of Science[XDB32000000] ; development of science and technology of Guangdong Province special fund project[2016B090910001]
WOS关键词	PHYSICAL LIMITS ; MUSCLE ; MODEL ; MOVEMENT ; CONTRACTION ; PREDICTION ; CRITERION ; TENDON ; NOISE
WOS研究方向	Computer Science ; Robotics ; Neurosciences & Neurology
语种	英语
出版者	FRONTIERS MEDIA SA
WOS记录号	WOS:000478024100002
资助机构	National Key Research and Development Program of China ; National Natural Science Foundation of China ; Strategic Priority Research Program of Chinese Academy of Science ; development of science and technology of Guangdong Province special fund project
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/27763]
专题	中国科学院自动化研究所
通讯作者	Qiao, Hong
作者单位	1.Chinese Acad Sci, Inst Automat, State Key Lab Management & Control Complex Syst, Beijing, Peoples R China 2.Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing, Peoples R China 3.Beijing Key Lab Res & Applicat Robot Intelligence, Beijing, Peoples R China 4.Chinese Acad Sci, Inst Automat, Res Ctr Brain Inspired Intelligence, Beijing, Peoples R China 5.CAS Ctr Excellence Brain Sci & Intelligence Techn, Shanghai, Peoples R China
推荐引用方式 GB/T 7714	Zhou, Junjie,Chen, Jiahao,Deng, Hu,et al. From Rough to Precise: Human-Inspired Phased Target Learning Framework for Redundant Musculoskeletal Systems[J]. FRONTIERS IN NEUROROBOTICS,2019,13:14.
APA	Zhou, Junjie,Chen, Jiahao,Deng, Hu,&Qiao, Hong.(2019).From Rough to Precise: Human-Inspired Phased Target Learning Framework for Redundant Musculoskeletal Systems.FRONTIERS IN NEUROROBOTICS,13,14.
MLA	Zhou, Junjie,et al."From Rough to Precise: Human-Inspired Phased Target Learning Framework for Redundant Musculoskeletal Systems".FRONTIERS IN NEUROROBOTICS 13(2019):14.