Lifelong CycleGAN for continual multi-task image restoration

doi:10.1016/j.patrec.2021.12.010

CORC > 自动化研究所 > 中国科学院自动化研究所 > 复杂系统管理与控制国家重点实验室 > 机器人应用与理论组

	Lifelong CycleGAN for continual multi-task image restoration
	Li, Yuping 1,5; Nie, Xiangli 1,5; Diao, Wenhui 2,3; Zheng, Suiwu 1,4,5
刊名	PATTERN RECOGNITION LETTERS
	2022
卷号	153 页码:183-189
关键词	Image restoration Lifelong continual learning CycleGAN Knowledge distillation
ISSN号	0167-8655
DOI	10.1016/j.patrec.2021.12.010
通讯作者	Nie, Xiangli(xiangli.nie@ia.ac.cn)
英文摘要	Recent years have witnessed the great success of deep learning in the applications of image restoration. However, there are still some challenging problems. First, most deep learning methods rely heavily on paired training images which are difficult to capture in the real world. Second, most existing methods are generally designed for a specific task of image restoration and they suffer from catastrophic forgetting when learn multiple tasks continually. Third, for most multi-task image restoration methods, they learn an individual network for each task and require to know the type of distortion for both training and test, which costs a lot of computational time and memory. To address the above issues, we propose a new lifelong learning framework based on CycleGAN for continual multi-task image restoration, called Lifelong CycleGAN (LCGAN), which can enhance low-light images, deblur and denoise simultaneously. The model utilizes knowledge distillation and memory replay to transfer knowledge and replay information that learned previously to alleviate forgetting. In addition, to regularize the unpaired training, we introduce the local discriminator and feature consistency constraint to preserve the color, edge and texture of input images. The proposed method can continually learn the three tasks using one network model and doesn't need to prejudge the type of distortion, which has low time and memory requirements. Experimental results demonstrate that LCGAN can achieve better visual and numerical results across the three tasks. (c) 2021 Elsevier B.V. All rights reserved.
资助项目	National Natural Science Foundation of China[62076241] ; National Natural Science Foundation of China[91948303] ; National Natural Science Foundation of China[61933001] ; CETC Key Laboratory of Data Link Technology[CLDL-20202208_1] ; National Key Research and Development Program of China[2020AAA0105900] ; Huizhou Science and Technology Project[2020SB0106006]
WOS研究方向	Computer Science
语种	英语
出版者	ELSEVIER
WOS记录号	WOS:000740766100007
资助机构	National Natural Science Foundation of China ; CETC Key Laboratory of Data Link Technology ; National Key Research and Development Program of China ; Huizhou Science and Technology Project
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/47193]
专题	自动化研究所_复杂系统管理与控制国家重点实验室_机器人应用与理论组
通讯作者	Nie, Xiangli
作者单位	1.Chinese Acad Sci, Inst Automat, State Key Lab Management & Control Complex Syst, Beijing 100190, Peoples R China 2.Chinese Acad Sci, Aerosp Informat Res Inst, Key Lab Network Informat Syst Technol NIST, Beijing 100094, Peoples R China 3.Chinese Acad Sci, Aerosp Informat Res Inst, Beijing 100190, Peoples R China 4.Huizhou Zhongke Adv Mfg Res Ctr Co Ltd, Huizhou 516000, Peoples R China 5.Univ Chinese Acad Sci, Beijing 100049, Peoples R China
推荐引用方式 GB/T 7714	Li, Yuping,Nie, Xiangli,Diao, Wenhui,et al. Lifelong CycleGAN for continual multi-task image restoration[J]. PATTERN RECOGNITION LETTERS,2022,153:183-189.
APA	Li, Yuping,Nie, Xiangli,Diao, Wenhui,&Zheng, Suiwu.(2022).Lifelong CycleGAN for continual multi-task image restoration.PATTERN RECOGNITION LETTERS,153,183-189.
MLA	Li, Yuping,et al."Lifelong CycleGAN for continual multi-task image restoration".PATTERN RECOGNITION LETTERS 153(2022):183-189.