Robot Subgoal-guided Navigation in Dynamic Crowded Environments with Hierarchical Deep Reinforcement Learning | |
Zhang TL(张天乐)2,3; Liu Z(刘振)2,3; Pu ZQ(蒲志强)2,3; Yi JQ(易建强)2,3; Liang YY(梁延研)1; Zhang D(张渡)1 | |
刊名 | International Journal of Control, Automation and Systems |
2023 | |
页码 | 1-13 |
文献子类 | 研究性论文 |
英文摘要 | Although deep reinforcement learning has recently achieved some successes in robot navigation, there are still unsolved problems. Particularly, a robot guided by a distant ultimate goal is easy to get stuck in danger or encounter collisions in dynamic crowded environments due to the lack of long-term perspectives. In this paper, a novel subgoal-guided approach based on two-level hierarchical deep reinforcement learning with spatial-temporal graph attention networks (ST-GANets), called SG-HDRL, is proposed for a robot navigating in a dynamic crowded environment with autonomous obstacles, e.g., crowd. Specifically, the high-level policy, that models the spatial-temporal relation between the robot and the obstacles using the obstacles’ trajectories by the designed high-level ST-GANet, generates intermediate subgoals from a longer-term perspective over higher temporal scales. The sub-goals give a favorable and collision-free direction to avoid encountering danger or collisions while approaching the ultimate goal. The low-level policy, that similarly implements the designed low-level ST-GANet to implicitly predict the obstacles’ motions, takes the subgoals as short-term guidance through an intrinsic reward incentive to generate primitive actions for the robot. Simulation results demonstrate that SG-HDRL using ST-GANets has better performances compared with state-of-the-art baselines. Furthermore, the proposed SG-HDRL is deployed to an experimental platform based on omnidirectional cars, and experiment results validate the effectiveness and practicability of the proposed SG-HDRL. |
语种 | 英语 |
内容类型 | 期刊论文 |
源URL | [http://ir.ia.ac.cn/handle/173211/51958] |
专题 | 综合信息系统研究中心_飞行器智能技术 |
通讯作者 | Liu Z(刘振) |
作者单位 | 1.澳门科技大学 2.中国科学院大学人工智能学院 3.中国科学院自动化研究所 |
推荐引用方式 GB/T 7714 | Zhang TL,Liu Z,Pu ZQ,et al. Robot Subgoal-guided Navigation in Dynamic Crowded Environments with Hierarchical Deep Reinforcement Learning[J]. International Journal of Control, Automation and Systems,2023:1-13. |
APA | Zhang TL,Liu Z,Pu ZQ,Yi JQ,Liang YY,&Zhang D.(2023).Robot Subgoal-guided Navigation in Dynamic Crowded Environments with Hierarchical Deep Reinforcement Learning.International Journal of Control, Automation and Systems,1-13. |
MLA | Zhang TL,et al."Robot Subgoal-guided Navigation in Dynamic Crowded Environments with Hierarchical Deep Reinforcement Learning".International Journal of Control, Automation and Systems (2023):1-13. |
个性服务 |
查看访问统计 |
相关权益政策 |
暂无数据 |
收藏/分享 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论