High-Fidelity Face Manipulation With Extreme Poses and Expressions

doi:10.1109/TIFS.2021.3050065

CORC > 自动化研究所 > 中国科学院自动化研究所 > 智能感知与计算研究中心

	High-Fidelity Face Manipulation With Extreme Poses and Expressions
	Fu, Chaoyou 3,4,5,6; Hu, Yibo 3,4,5,6; Wu, Xiang 3,4,5,6; Wang, Guoli 2; Zhang, Qian 1; He, Ran 3,4,5,6
刊名	IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY
	2021
卷号	16 页码:2218-2231
关键词	Face manipulation extreme pose and expression high-resolution MVF-HQ
ISSN号	1556-6013
DOI	10.1109/TIFS.2021.3050065
通讯作者	He, Ran(rhe@nlpr.ia.ac.cn)
英文摘要	Face manipulation has shown remarkable advances with the flourish of Generative Adversarial Networks. However, due to the difficulties of controlling structures and textures, it is challenging to model poses and expressions simultaneously, especially for the extreme manipulation at high-resolution. In this article, we propose a novel framework that simplifies face manipulation into two correlated stages: a boundary prediction stage and a disentangled face synthesis stage. The first stage models poses and expressions jointly via boundary images. Specifically, a conditional encoder-decoder network is employed to predict the boundary image of the target face in a semi-supervised way. Pose and expression estimators are introduced to improve the prediction performance. In the second stage, the predicted boundary image and the input face image are encoded into the structure and the texture latent space by two encoder networks, respectively. A proxy network and a feature threshold loss are further imposed to disentangle the latent space. Furthermore, due to the lack of high-resolution face manipulation databases to verify the effectiveness of our method, we collect a new high-quality Multi-View Face (MVF-HQ) database. It contains 120,283 images at 6000 x 4000 resolution from 479 identities with diverse poses, expressions, and illuminations. MVF-HQ is much larger in scale and much higher in resolution than publicly available high-resolution face manipulation databases. We will release MVF-HQ soon to push forward the advance of face manipulation. Qualitative and quantitative experiments on four databases show that our method dramatically improves the synthesis quality.
资助项目	Beijing Natural Science Foundation[JQ18017] ; National Natural Science Foundation of China[61721004] ; National Natural Science Foundation of China[U20A20223]
WOS研究方向	Computer Science ; Engineering
语种	英语
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
WOS记录号	WOS:000617315800004
资助机构	Beijing Natural Science Foundation ; National Natural Science Foundation of China
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/43249]
专题	自动化研究所_智能感知与计算研究中心
通讯作者	He, Ran
作者单位	1.Horizon Robot, Beijing 100190, Peoples R China 2.Tsinghua Univ, Dept Automat, Beijing 100084, Peoples R China 3.Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing 100190, Peoples R China 4.Chinese Acad Sci, Ctr Excellence Brain Sci & Intelligence Technol, Beijing 100864, Peoples R China 5.Chinese Acad Sci CASIA, Inst Automat, Ctr Res Intelligent Percept & Comp, Beijing 100190, Peoples R China 6.Chinese Acad Sci CASIA, Natl Lab Pattern Recognit, Inst Automat, Beijing 100190, Peoples R China
推荐引用方式 GB/T 7714	Fu, Chaoyou,Hu, Yibo,Wu, Xiang,et al. High-Fidelity Face Manipulation With Extreme Poses and Expressions[J]. IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY,2021,16:2218-2231.
APA	Fu, Chaoyou,Hu, Yibo,Wu, Xiang,Wang, Guoli,Zhang, Qian,&He, Ran.(2021).High-Fidelity Face Manipulation With Extreme Poses and Expressions.IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY,16,2218-2231.
MLA	Fu, Chaoyou,et al."High-Fidelity Face Manipulation With Extreme Poses and Expressions".IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY 16(2021):2218-2231.