Dynamic Hand Gesture Recognition Based on Short-Term Sampling Neural Networks

doi:10.1109/JAS.2020.1003465

CORC > 自动化研究所 > 中国科学院自动化研究所 > 学术期刊 > IEEE/CAA Journal of Automatica Sinica

	Dynamic Hand Gesture Recognition Based on Short-Term Sampling Neural Networks
	Wenjin Zhang; Jiacun Wang; Fangping Lan
刊名	IEEE/CAA Journal of Automatica Sinica
	2021
卷号	8 期号:1 页码:110-120
关键词	Convolutional neural network (ConvNet) hand gesture recognition long short-term memory (LSTM) network short-term sampling transfer learning
ISSN号	2329-9266
DOI	10.1109/JAS.2020.1003465
英文摘要	Hand gestures are a natural way for human-robot interaction. Vision based dynamic hand gesture recognition has become a hot research topic due to its various applications. This paper presents a novel deep learning network for hand gesture recognition. The network integrates several well-proved modules together to learn both short-term and long-term features from video inputs and meanwhile avoid intensive computation. To learn short-term features, each video input is segmented into a fixed number of frame groups. A frame is randomly selected from each group and represented as an RGB image as well as an optical flow snapshot. These two entities are fused and fed into a convolutional neural network (ConvNet) for feature extraction. The ConvNets for all groups share parameters. To learn long-term features, outputs from all ConvNets are fed into a long short-term memory (LSTM) network, by which a final classification result is predicted. The new model has been tested with two popular hand gesture datasets, namely the Jester dataset and Nvidia dataset. Comparing with other models, our model produced very competitive results. The robustness of the new model has also been proved with an augmented dataset with enhanced diversity of hand gestures.
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/43900]
专题	自动化研究所_学术期刊_IEEE/CAA Journal of Automatica Sinica
推荐引用方式 GB/T 7714	Wenjin Zhang,Jiacun Wang,Fangping Lan. Dynamic Hand Gesture Recognition Based on Short-Term Sampling Neural Networks[J]. IEEE/CAA Journal of Automatica Sinica,2021,8(1):110-120.
APA	Wenjin Zhang,Jiacun Wang,&Fangping Lan.(2021).Dynamic Hand Gesture Recognition Based on Short-Term Sampling Neural Networks.IEEE/CAA Journal of Automatica Sinica,8(1),110-120.
MLA	Wenjin Zhang,et al."Dynamic Hand Gesture Recognition Based on Short-Term Sampling Neural Networks".IEEE/CAA Journal of Automatica Sinica 8.1(2021):110-120.