Incremental Learning Framework for Autonomous Robots based on Q-learning and the Adaptive Kernel Linear Model | |
Hu YM(胡艳明)2,3,4; Li DC(李德才)2,3; He YQ(何玉庆)2,3; Han JD(韩建达)1,2,3 | |
刊名 | IEEE Transactions on Cognitive and Developmental Systems |
2022 | |
卷号 | 14期号:1页码:64-74 |
关键词 | incremental learning path planning Q-learning autonomous robots recursive least squares algorithm L2-norm |
ISSN号 | 2379-8920 |
产权排序 | 1 |
英文摘要 | The performance of autonomous robots in varying environments needs to be improved. For such incremental improvement, here we propose an incremental learning framework based on Q-learning and the adaptive kernel linear (AKL) model. The AKL model is used for storing behavioral policies that are learned by Q-learning. Both the structure and parameters of the AKL model can be trained using a novel L2-norm kernel recursive least squares (L2-KRLS) algorithm. AKL model initially without nodes and gradually accumulates content. The proposed framework allows to learn new behaviors without forgetting the previous ones. A novel local -greedy policy is proposed to speed the convergence rate of Q-learning. It calculates the exploration probability of each state for generating and selecting more important training samples. The performance of our incremental learning framework was validated in two experiments. A curve fitting example shows that the L2-KRLS based AKL model is suitable for incremental learning. The second experiment is based on robot learning tasks. The results show that our framework can incrementally learn behaviors in varying environments. Local -greedy policy-based Q-learning is faster than existing Q-learning algorithms. |
资助项目 | Nature Sciences Foundation of China[U1608253] ; Nature Sciences Foundation of China[61473282] ; Chinese Academy of Sciences[6141A01061601] ; State Key Laboratory of Robotics[2017-Z07] |
WOS研究方向 | Computer Science ; Robotics ; Neurosciences & Neurology |
语种 | 英语 |
WOS记录号 | WOS:000767846600010 |
资助机构 | Nature Sciences Foundation of China (Grant Nos.U1608253, 61473282) ; Chinese Academy of Sciences (Grant No. 6141A01061601) |
内容类型 | 期刊论文 |
源URL | [http://ir.sia.cn/handle/173321/26185] |
专题 | 沈阳自动化研究所_机器人学研究室 |
通讯作者 | He YQ(何玉庆) |
作者单位 | 1.College of Artificial Intelligence, Nankai University, 300071, Tianjing,China. 2.State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, 110016, Shenyang, Liaoning Province, China 3.Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, 110016, Shenyang, Liaoning Province, China 4.University of Chinese Academy of Sciences, 100049, Beijing, China |
推荐引用方式 GB/T 7714 | Hu YM,Li DC,He YQ,et al. Incremental Learning Framework for Autonomous Robots based on Q-learning and the Adaptive Kernel Linear Model[J]. IEEE Transactions on Cognitive and Developmental Systems,2022,14(1):64-74. |
APA | Hu YM,Li DC,He YQ,&Han JD.(2022).Incremental Learning Framework for Autonomous Robots based on Q-learning and the Adaptive Kernel Linear Model.IEEE Transactions on Cognitive and Developmental Systems,14(1),64-74. |
MLA | Hu YM,et al."Incremental Learning Framework for Autonomous Robots based on Q-learning and the Adaptive Kernel Linear Model".IEEE Transactions on Cognitive and Developmental Systems 14.1(2022):64-74. |
个性服务 |
查看访问统计 |
相关权益政策 |
暂无数据 |
收藏/分享 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论