Parallel multiscale context-based edge-preserving optical flow estimation with occlusion detection

doi:10.1016/j.image.2021.116560

CORC > 自动化研究所 > 中国科学院自动化研究所 > 模式识别国家重点实验室 > 视频内容安全团队

	Parallel multiscale context-based edge-preserving optical flow estimation with occlusion detection
	Zhang, Congxuan 1,2; Feng, Cheng 2; Chen, Zhen 2; Hu, Weiming 1; Li, Ming 2
刊名	SIGNAL PROCESSING-IMAGE COMMUNICATION
	2022-02-01
卷号	101 页码:14
关键词	Optical flow Occlusion detection Edge-preserving Parallel multiscale context Convolutional neural network
ISSN号	0923-5965
DOI	10.1016/j.image.2021.116560
通讯作者	Chen, Zhen(dr_chenzhen@163.com)
英文摘要	Although convolutional neural network (CNN)-based optical flow approaches have exhibited good performance in terms of computational accuracy and efficiency in recent years, the issue of edge-blurring caused by motion occlusions remains. In this paper, we propose a parallel multiscale context-based edge-preserving optical flow estimation method with occlusion detection, named PMC-PWC. First, we exploit a parallel multiscale context (PMC) network for occlusion detection, in which the proposed PMC model is able to aggregate the multiscale context information to develop the performance of occlusion detection near motion boundaries. Second, we combine the PMC model with a context network to plan an occlusion estimation module and incorporate it into a pyramid, warping, and cost volume model to construct an edge-preserving optical flow computation network. Third, we design a novel loss function including an endpoint error (EPE)-based loss, a binary cross-entropy loss and an edge loss to supervise the proposed PMC-PWC network to produce optical flow and occlusion simultaneously. Finally, we run the proposed PMC-PWC method on the MPI-Sintel and KITTI datasets to conduct a comprehensive comparison with several state-of-the-art approaches. The experimental results indicate that the proposed PMC-PWC method performed well in terms of both accuracy and robustness, especially due to the significant benefits of edge preservation and occlusion handling.
资助项目	National Key Research and Development Program of China[2020YFC2003800] ; National Natural Science Foundation of China[61772255] ; National Natural Science Foundation of China[61866026] ; National Natural Science Foundation of China[61866025] ; Advantage Subject Team Project of Jiangxi Province, China[20165BCB19007] ; Outstanding Young Talents Program of Jiangxi Province, China[20192BCB23011] ; National Natural Science Foundation of Jiangxi Province, China[20202ACB214007] ; Aeronautical Science Foundation of China[2018ZC56008] ; China Postdoctoral Science Foundation[2019M650894]
WOS研究方向	Engineering
语种	英语
出版者	ELSEVIER
WOS记录号	WOS:000724345500006
资助机构	National Key Research and Development Program of China ; National Natural Science Foundation of China ; Advantage Subject Team Project of Jiangxi Province, China ; Outstanding Young Talents Program of Jiangxi Province, China ; National Natural Science Foundation of Jiangxi Province, China ; Aeronautical Science Foundation of China ; China Postdoctoral Science Foundation
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/46575]
专题	自动化研究所_模式识别国家重点实验室_视频内容安全团队
通讯作者	Chen, Zhen
作者单位	1.Chinese Acad Sci, Inst Automat, Beijing 100190, Peoples R China 2.Nanchang Hangkong Univ, Key Lab Nondestruct Testing, Minist Educ, Nanchang 330063, Jiangxi, Peoples R China
推荐引用方式 GB/T 7714	Zhang, Congxuan,Feng, Cheng,Chen, Zhen,et al. Parallel multiscale context-based edge-preserving optical flow estimation with occlusion detection[J]. SIGNAL PROCESSING-IMAGE COMMUNICATION,2022,101:14.
APA	Zhang, Congxuan,Feng, Cheng,Chen, Zhen,Hu, Weiming,&Li, Ming.(2022).Parallel multiscale context-based edge-preserving optical flow estimation with occlusion detection.SIGNAL PROCESSING-IMAGE COMMUNICATION,101,14.
MLA	Zhang, Congxuan,et al."Parallel multiscale context-based edge-preserving optical flow estimation with occlusion detection".SIGNAL PROCESSING-IMAGE COMMUNICATION 101(2022):14.