DeepFaceDrawing: Deep Generation of Face Images from Sketches

doi:10.1145/3386569.3392386

	DeepFaceDrawing: Deep Generation of Face Images from Sketches
	Chen, Shu-Yu 1,2,4; Su, Wanchao 3; Gao, Lin 1,2,4; Xia, Shihong 1,2,4; Fu, Hongbo 3
刊名	ACM TRANSACTIONS ON GRAPHICS
	2020-07-01
卷号	39 期号:4 页码:16
关键词	image-to-image translation feature embedding sketch-based generation face synthesis
ISSN号	0730-0301
DOI	10.1145/3386569.3392386
英文摘要	Recent deep image-to-image translation techniques allow fast generation of face images from freehand sketches. However, existing solutions tend to overfit to sketches, thus requiring professional sketches or even edge maps as input. To address this issue, our key idea is to implicitly model the shape space of plausible face images and synthesize a face image in this space to approximate an input sketch. We take a local-to-global approach. We first learn feature embeddings of key face components, and push corresponding parts of input sketches towards underlying component manifolds defined by the feature vectors of face component samples. We also propose another deep neural network to learn the mapping from the embedded component features to realistic images with multi-channel feature maps as intermediate results to improve the information flow. Our method essentially uses input sketches as soft constraints and is thus able to produce high-quality face images even from rough and/or incomplete sketches. Our tool is easy to use even for non-artists, while still supporting fine-grained control of shape details. Both qualitative and quantitative evaluations show the superior generation ability of our system to existing and alternative solutions. The usability and expressiveness of our system are confirmed by a user study.
资助项目	Beijing Program for International S&T Cooperation Project[Z191100001619003] ; Royal Society Newton Advanced Fellowship[NAFnR2n192151] ; Youth Innovation Promotion Association CAS ; CCF-Tencent Open Fund ; Open Project Program of the National Laboratory of Pattern Recognition (NLPR)[201900055] ; Research Grants Council of the Hong Kong Special Administrative Region, China[CityU 11212119] ; Research Grants Council of the Hong Kong Special Administrative Region, China[11237116] ; City University of Hong Kong[SRG 7005176] ; Centre for Applied Computing and Interactive Media (ACIM) of School of Creative Media, CityU
WOS研究方向	Computer Science
语种	英语
出版者	ASSOC COMPUTING MACHINERY
WOS记录号	WOS:000583700300045
内容类型	期刊论文
源URL	[http://119.78.100.204/handle/2XEOYT63/15978]
专题	中国科学院计算技术研究所
通讯作者	Chen, Shu-Yu
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Beijing, Peoples R China 2.Univ Chinese Acad Sci, Beijing, Peoples R China 3.City Univ Hong Kong, Sch Creat Media, Hong Kong, Peoples R China 4.Chinese Acad Sci, Inst Comp Technol, Beijing Key Lab Mobile Comp & Pervas Device, Beijing, Peoples R China
推荐引用方式 GB/T 7714	Chen, Shu-Yu,Su, Wanchao,Gao, Lin,et al. DeepFaceDrawing: Deep Generation of Face Images from Sketches[J]. ACM TRANSACTIONS ON GRAPHICS,2020,39(4):16.
APA	Chen, Shu-Yu,Su, Wanchao,Gao, Lin,Xia, Shihong,&Fu, Hongbo.(2020).DeepFaceDrawing: Deep Generation of Face Images from Sketches.ACM TRANSACTIONS ON GRAPHICS,39(4),16.
MLA	Chen, Shu-Yu,et al."DeepFaceDrawing: Deep Generation of Face Images from Sketches".ACM TRANSACTIONS ON GRAPHICS 39.4(2020):16.