Face Forgery Detection by 3D Decomposition and Composition Search

doi:10.1109/TPAMI.2022.3233586

CORC > 自动化研究所 > 中国科学院自动化研究所 > 多模态人工智能系统全国重点实验室

	Face Forgery Detection by 3D Decomposition and Composition Search
	Zhu, Xiangyu 1,5,6; Fei, Hongyan 1,5,6; Zhang, Bin 4; Zhang, Tianshuo 1,5,6; Zhang, Xiaoyu 4; Li, Stan Z.3; Lei, Zhen 1,2,5,6
刊名	IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
	2023-07-01
卷号	45 期号:7 页码:8342-8357
关键词	Faces Forgery Three-dimensional displays Face recognition Feature extraction Lighting Computer architecture Composition search differentiable search fake face forgery detection 3D decomposition 3D face model
ISSN号	0162-8828
DOI	10.1109/TPAMI.2022.3233586
通讯作者	Lei, Zhen(zlei@nlpr.ia.ac.cn)
英文摘要	Detecting digital face manipulation has attracted extensive attention due to fake media's potential risks to the public. However, recent advances have been able to reduce the forgery signals to a low magnitude. Decomposition, which reversibly decomposes an image into several constituent elements, is a promising way to highlight the hidden forgery details. In this paper, we investigate a novel 3D decomposition based method that considers a face image as the production of the interaction between 3D geometry and lighting environment. Specifically, we disentangle a face image into four graphics components including 3D shape, lighting, common texture, and identity texture, which are respectively constrained by 3D morphable model, harmonic reflectance illumination, and PCA texture model. Meanwhile, we build a fine-grained morphing network to predict 3D shapes with pixel-level accuracy to reduce the noise in the decomposed elements. Moreover, we propose a composition search strategy that enables an automatic construction of an architecture to mine forgery clues from forgery-relevant components. Extensive experiments validate that the decomposed components highlight forgery artifacts, and the searched architecture extracts discriminative forgery features. Thus, our method achieves the state-of-the-art performance.
资助项目	National Key Research & Development Program[2020AAA0140000] ; Chinese National Natural Science Foundation[62176256] ; Chinese National Natural Science Foundation[62276254] ; Chinese National Natural Science Foundation[62106264] ; Youth Innovation Promotion Association CAS[Y2021131] ; InnoHK program
WOS关键词	IMAGE
WOS研究方向	Computer Science ; Engineering
语种	英语
出版者	IEEE COMPUTER SOC
WOS记录号	WOS:001004665900026
资助机构	National Key Research & Development Program ; Chinese National Natural Science Foundation ; Youth Innovation Promotion Association CAS ; InnoHK program
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/53660]
专题	多模态人工智能系统全国重点实验室
通讯作者	Lei, Zhen
作者单位	1.Chinese Acad Sci, Ctr Biometr & Secur Res, Inst Automat, Beijing 100190, Peoples R China 2.Chinese Acad Sci, Ctr Artificial Intelligence & Robot, Hong Kong Inst Sci & Innovat, Hong Kong, Peoples R China 3.Westlake Univ, Sch Engn, Hangzhou 310024, Peoples R China 4.Chinese Acad Sci, Inst Informat Engn, Beijing 100045, Peoples R China 5.UCAS, Sch Artificial Intelligence, Beijing 100049, Peoples R China 6.Chinese Acad Sci, Natl Lab Pattern Recognit, Inst Automat, Beijing 100190, Peoples R China
推荐引用方式 GB/T 7714	Zhu, Xiangyu,Fei, Hongyan,Zhang, Bin,et al. Face Forgery Detection by 3D Decomposition and Composition Search[J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,2023,45(7):8342-8357.
APA	Zhu, Xiangyu.,Fei, Hongyan.,Zhang, Bin.,Zhang, Tianshuo.,Zhang, Xiaoyu.,...&Lei, Zhen.(2023).Face Forgery Detection by 3D Decomposition and Composition Search.IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,45(7),8342-8357.
MLA	Zhu, Xiangyu,et al."Face Forgery Detection by 3D Decomposition and Composition Search".IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 45.7(2023):8342-8357.