RhythmNet: End-to-End Heart Rate Estimation From Face via Spatial-Temporal Representation

doi:10.1109/TIP.2019.2947204

CORC > 计算技术研究所 > 中国科学院计算技术研究所 > 中国科学院计算技术研究所期刊论文 > 英文

	RhythmNet: End-to-End Heart Rate Estimation From Face via Spatial-Temporal Representation
	Niu, Xuesong 2,3; Shan, Shiguang 1,3,4; Han, Hu 1; Chen, Xilin 2,3
刊名	IEEE TRANSACTIONS ON IMAGE PROCESSING
	2020
卷号	29 页码:2409-2423
关键词	Heart rate Estimation Webcams Databases Skin Image color analysis Head Remote heart rate estimation rPPG spatial-temporal representation end-to-end learning
ISSN号	1057-7149
DOI	10.1109/TIP.2019.2947204
英文摘要	Heart rate (HR) is an important physiological signal that reflects the physical and emotional status of a person. Traditional HR measurements usually rely on contact monitors, which may cause inconvenience and discomfort. Recently, some methods have been proposed for remote HR estimation from face videos; however, most of them focus on well-controlled scenarios, their generalization ability into less-constrained scenarios (e.g., with head movement, and bad illumination) are not known. At the same time, lacking large-scale HR databases has limited the use of deep models for remote HR estimation. In this paper, we propose an end-to-end RhythmNet for remote HR estimation from the face. In RyhthmNet, we use a spatial-temporal representation encoding the HR signals from multiple ROI volumes as its input. Then the spatial-temporal representations are fed into a convolutional network for HR estimation. We also take into account the relationship of adjacent HR measurements from a video sequence via Gated Recurrent Unit (GRU) and achieves efficient HR measurement. In addition, we build a large-scale multi-modal HR database (named as VIPL-HR (1) ), which contains 2,378 visible light videos (VIS) and 752 near-infrared (NIR) videos of 107 subjects. Our VIPL-HR database contains various variations such as head movements, illumination variations, and acquisition device changes, replicating a less-constrained scenario for HR estimation. The proposed approach outperforms the state-of-the-art methods on both the public-domain and our VIPL-HR databases. (1) VIPL-HR is available at: http://vipl.ict.ac.cn/view_database.php?id=15
资助项目	National Key R&D Program of China[2017YFA0700800] ; Natural Science Foundation of China[61672496] ; Natural Science Foundation of China[61702486] ; External Cooperation Program of Chinese Academy of Sciences (CAS)[GJHZ1843]
WOS研究方向	Computer Science ; Engineering
语种	英语
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
WOS记录号	WOS:000507869900016
内容类型	期刊论文
源URL	[http://119.78.100.204/handle/2XEOYT63/15024]
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Shan, Shiguang
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, Beijing 100190, Peoples R China 2.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, Beijing 100190, Peoples R China 3.Univ Chinese Acad Sci, Sch Comp Sci & Technol, Beijing 100049, Peoples R China 4.CAS Ctr Excellence Brain Sci & Intelligence Techn, Shanghai 200031, Peoples R China
推荐引用方式 GB/T 7714	Niu, Xuesong,Shan, Shiguang,Han, Hu,et al. RhythmNet: End-to-End Heart Rate Estimation From Face via Spatial-Temporal Representation[J]. IEEE TRANSACTIONS ON IMAGE PROCESSING,2020,29:2409-2423.
APA	Niu, Xuesong,Shan, Shiguang,Han, Hu,&Chen, Xilin.(2020).RhythmNet: End-to-End Heart Rate Estimation From Face via Spatial-Temporal Representation.IEEE TRANSACTIONS ON IMAGE PROCESSING,29,2409-2423.
MLA	Niu, Xuesong,et al."RhythmNet: End-to-End Heart Rate Estimation From Face via Spatial-Temporal Representation".IEEE TRANSACTIONS ON IMAGE PROCESSING 29(2020):2409-2423.