Hand gesture recognition using multimodal data fusion and multiscale parallel convolutional neural network for human-robot interaction | |
Gao Q(高庆)1,3; Liu JG(刘金国)1 | |
刊名 | EXPERT SYSTEMS |
2021 | |
卷号 | 38期号:5页码:1-12 |
关键词 | hand gesture recognition multimodal data fusion parallel CNN sEMG signal |
ISSN号 | 0266-4720 |
产权排序 | 1 |
英文摘要 | Hand gesture recognition plays an important role in human-robot interaction. The accuracy and reliability of hand gesture recognition are the keys to gesture-based human-robot interaction tasks. To solve this problem, a method based on multimodal data fusion and multiscale parallel convolutional neural network (CNN) is proposed in this paper to improve the accuracy and reliability of hand gesture recognition. First of all, data fusion is conducted on the sEMG signal, the RGB image, and the depth image of hand gestures. Then, the fused images are generated to two different scale images by downsampling, which are respectively input into two subnetworks of the parallel CNN to obtain two hand gesture recognition results. After that, hand gesture recognition results of the parallel CNN are combined to obtain the final hand gesture recognition result. Finally, experiments are carried out on a self-made database containing 10 common hand gestures, which verify the effectiveness and superiority of the proposed method for hand gesture recognition. In addition, the proposed method is applied to a seven-degree-of-freedom bionic manipulator to achieve robotic manipulation with hand gestures. |
资助项目 | CAS Interdisciplinary Innovation Team[JCTD-2018-11] ; National Key R&D Program of China[2018YFB1304600] ; Natural Science Foundation of China[51575412] ; Natural Science Foundation of China[51775541] ; EU Seventh Framework Programme (FP7)-ICT[611391] |
WOS研究方向 | Computer Science |
语种 | 英语 |
WOS记录号 | WOS:000505898400001 |
资助机构 | CAS Interdisciplinary Innovation Team [JCTD-2018-11] ; National Key R&D Program of China [2018YFB1304600] ; Natural Science Foundation of ChinaNational Natural Science Foundation of China [51575412, 51775541] ; EU Seventh Framework Programme (FP7)-ICT [611391] |
内容类型 | 期刊论文 |
源URL | [http://ir.sia.cn/handle/173321/26217] |
专题 | 沈阳自动化研究所_空间自动化技术研究室 |
通讯作者 | Liu JG(刘金国) |
作者单位 | 1.State Key Laboratory of Robotics, Shenyang Institute of Automation, Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, Shenyang, China 2.School of Computing, University of Portsmouth, Portsmouth, United Kingdom 3.University of Chinese Academy of Sciences, Beijing, China |
推荐引用方式 GB/T 7714 | Gao Q,Liu JG. Hand gesture recognition using multimodal data fusion and multiscale parallel convolutional neural network for human-robot interaction[J]. EXPERT SYSTEMS,2021,38(5):1-12. |
APA | Gao Q,&Liu JG.(2021).Hand gesture recognition using multimodal data fusion and multiscale parallel convolutional neural network for human-robot interaction.EXPERT SYSTEMS,38(5),1-12. |
MLA | Gao Q,et al."Hand gesture recognition using multimodal data fusion and multiscale parallel convolutional neural network for human-robot interaction".EXPERT SYSTEMS 38.5(2021):1-12. |
个性服务 |
查看访问统计 |
相关权益政策 |
暂无数据 |
收藏/分享 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论