A Basal Ganglia Network Centric Reinforcement Learning Model and Its Application in Unmanned Aerial Vehicle

doi:10.1109/TCDS.2017.2649564

	A Basal Ganglia Network Centric Reinforcement Learning Model and Its Application in Unmanned Aerial Vehicle
	Zeng, Yi1,2 ; Wang, Guixiang1 ; Xu, Bo 1,2
刊名	IEEE TRANSACTIONS ON COGNITIVE AND DEVELOPMENTAL SYSTEMS
	2018-06-01
卷号	10 期号:2 页码:290-303
关键词	Basal ganglia (BG) network brain-inspired intelligence precise encoding reinforcement learning model unmanned aerial vehicle (UAV) autonomous learning
ISSN号	2379-8920
DOI	10.1109/TCDS.2017.2649564
通讯作者	Zeng, Yi(yi.zeng@ia.ac.cn)
英文摘要	Reinforcement learning brings flexibility and generality for machine learning, while most of them are mathematical optimization driven approaches, and lack of cognitive and neural evidence. In order to provide a more cognitive and neural mechanisms driven foundation and validate its applicability in complex task, we develop a basal ganglia (BG) network centric reinforcement learning model. Compared to existing work on modeling BG, this paper is unique from the following perspectives: 1) the orbitofrontal cortex (OFC) is taken into consideration. OFC is critical in decision making because of its responsibility for reward representation and is critical in controlling the learning process, while most of the BG centric models do not include OFC; 2) to compensate the inaccurate memory of numeric values, precise encoding is proposed to enable working memory system remember important values during the learning process. The method combines vector convolution and the idea of storage by digit bit and is efficient for accurate value storage; and 3) for information coding, the Hodgkin-Huxley model is used to obtain a more biological plausible description of action potential with plenty of ionic activities. To validate the effectiveness of the proposed model, we apply the model to the unmanned aerial vehicle (UAV) autonomous learning process in a 3-D environment. Experimental results show that our model is able to give the UAV the ability of free exploration in the environment and has comparable learning speed as the Q learning algorithm, while the major advances for our model is that it is with solid cognitive and neural basis.
资助项目	Chinese Academy of Sciences[XDB02060007] ; Beijing Municipal Commission of Science and Technology[Z151100000915070] ; Beijing Municipal Commission of Science and Technology[Z161100000216124]
WOS关键词	ORBITOFRONTAL CORTEX ; FUNCTIONAL-ANATOMY ; DECISION-MAKING ; BRAIN ; CIRCUITS
WOS研究方向	Computer Science ; Robotics ; Neurosciences & Neurology
语种	英语
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
WOS记录号	WOS:000435198600015
资助机构	Chinese Academy of Sciences ; Beijing Municipal Commission of Science and Technology
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/27937]
专题	中国科学院自动化研究所
通讯作者	Zeng, Yi
作者单位	1.Chinese Acad Sci, Inst Automat, Beijing 100190, Peoples R China 2.Chinese Acad Sci, Ctr Excellence Brain Sci & Intelligence Technol, Shanghai 200031, Peoples R China
推荐引用方式 GB/T 7714	Zeng, Yi,Wang, Guixiang,Xu, Bo. A Basal Ganglia Network Centric Reinforcement Learning Model and Its Application in Unmanned Aerial Vehicle[J]. IEEE TRANSACTIONS ON COGNITIVE AND DEVELOPMENTAL SYSTEMS,2018,10(2):290-303.
APA	Zeng, Yi,Wang, Guixiang,&Xu, Bo.(2018).A Basal Ganglia Network Centric Reinforcement Learning Model and Its Application in Unmanned Aerial Vehicle.IEEE TRANSACTIONS ON COGNITIVE AND DEVELOPMENTAL SYSTEMS,10(2),290-303.
MLA	Zeng, Yi,et al."A Basal Ganglia Network Centric Reinforcement Learning Model and Its Application in Unmanned Aerial Vehicle".IEEE TRANSACTIONS ON COGNITIVE AND DEVELOPMENTAL SYSTEMS 10.2(2018):290-303.