Temporal Modeling on Multi-Temporal-Scale Spatiotemporal Atoms for Action Recognition

doi:10.3390/app8101835

CORC > 光电技术研究所 > 中国科学院光电技术研究所 > 光电探测技术研究室（三室）

	Temporal Modeling on Multi-Temporal-Scale Spatiotemporal Atoms for Action Recognition
	Guangle Yao 1,2,3; Tao Lei 1; Xianyuan Liu 1,3; Ping Jiang 1
刊名	APPLIED SCIENCES-BASEL
	2018-10-01
卷号	8 期号:10 页码:1835
关键词	action recognition action atom convolutional neural network long short-term memory
ISSN号	2076-3417
DOI	10.3390/app8101835
文献子类	J
英文摘要	As an important branch of video analysis, human action recognition has attracted extensive research attention in computer vision and artificial intelligence communities. In this paper, we propose to model the temporal evolution of multi-temporal-scale atoms for action recognition. An action can be considered as a temporal sequence of action units. These action units which we referred to as action atoms, can capture the key semantic and characteristic spatiotemporal features of actions in different temporal scales. We first investigate Res3D, a powerful 3D CNN architecture and create the variants of Res3D for different temporal scale. In each temporal scale, we design some practices to transfer the knowledge learned from RGB to optical flow (OF) and build RGB and OF streams to extract deep spatiotemporal information using Res3D. Then we propose an unsupervised method to mine action atoms in the deep spatiotemporal space. Finally, we use long short-term memory (LSTM) to model the temporal evolution of atoms for action recognition. The experimental results show that our proposed multi-temporal-scale spatiotemporal atoms modeling method achieves recognition performance comparable to that of state-of-the-art methods on two challenging action recognition datasets: UCF101 and HMDB51.
语种	英语
WOS记录号	WOS:000448653700130
内容类型	期刊论文
源URL	[http://ir.ioe.ac.cn/handle/181551/9265]
专题	光电技术研究所_光电探测技术研究室（三室）
作者单位	1.Institute of Optics and Electronics, Chinese Academy of Sciences, P.O. Box 350, Shuangliu, Chengdu 610209, China; 2.School of Optoelectronic Information, University of Electronic Science and Technology of China, No. 4, Section 2, North Jianshe Road, Chengdu 610054, China; 3.University of Chinese Academy of Sciences, 19 A Yuquan Rd, Shijingshan District, Beijing 100039, China
推荐引用方式 GB/T 7714	Guangle Yao,Tao Lei,Xianyuan Liu,et al. Temporal Modeling on Multi-Temporal-Scale Spatiotemporal Atoms for Action Recognition[J]. APPLIED SCIENCES-BASEL,2018,8(10):1835.
APA	Guangle Yao,Tao Lei,Xianyuan Liu,&Ping Jiang.(2018).Temporal Modeling on Multi-Temporal-Scale Spatiotemporal Atoms for Action Recognition.APPLIED SCIENCES-BASEL,8(10),1835.
MLA	Guangle Yao,et al."Temporal Modeling on Multi-Temporal-Scale Spatiotemporal Atoms for Action Recognition".APPLIED SCIENCES-BASEL 8.10(2018):1835.