Robust Local Light Field Synthesis via Occlusion-aware Sampling and Deep Visual Feature Fusion

doi:10.1007/s11633-022-1381-9

CORC > 自动化研究所 > 中国科学院自动化研究所 > 学术期刊 > International Journal of Automation and Computing

	Robust Local Light Field Synthesis via Occlusion-aware Sampling and Deep Visual Feature Fusion
	Wenpeng Xing 2
刊名	Machine Intelligence Research
	2023
卷号	20 期号:3 页码:408-420
关键词	Novel view synthesis, light field (LF) imaging, multi-view stereo, occlusion sampling, deep visual feature (DVF) fusion
ISSN号	2731-538X
DOI	10.1007/s11633-022-1381-9
英文摘要	Novel view synthesis has attracted tremendous research attention recently for its applications in virtual reality and immersive telepresence. Rendering a locally immersive light field (LF) based on arbitrary large baseline RGB references is a challenging problem that lacks efficient solutions with existing novel view synthesis techniques. In this work, we aim at truthfully rendering local immersive novel views/LF images based on large baseline LF captures and a single RGB image in the target view. To fully explore the precious information from source LF captures, we propose a novel occlusion-aware source sampler (OSS) module which efficiently transfers the pixels of source views to the target view's frustum in an occlusion-aware manner. An attention-based deep visual fusion module is proposed to fuse the revealed occluded background content with a preliminary LF into a final refined LF. The proposed source sampling and fusion mechanism not only helps to provide information for occluded regions from varying observation angles, but also proves to be able to effectively enhance the visual rendering quality. Experimental results show that our proposed method is able to render high-quality LF images/novel views with sparse RGB references and outperforms state-of-the-art LF rendering and novel view synthesis methods.
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/51709]
专题	自动化研究所_学术期刊_International Journal of Automation and Computing
作者单位	1.Data Science Institute, Imperial College London, London SW7 2AZ, UK 2.Department of Computer Science, Hong Kong Baptist University, Hong Kong 999077, China
推荐引用方式 GB/T 7714	Wenpeng Xing. Robust Local Light Field Synthesis via Occlusion-aware Sampling and Deep Visual Feature Fusion[J]. Machine Intelligence Research,2023,20(3):408-420.
APA	Wenpeng Xing.(2023).Robust Local Light Field Synthesis via Occlusion-aware Sampling and Deep Visual Feature Fusion.Machine Intelligence Research,20(3),408-420.
MLA	Wenpeng Xing."Robust Local Light Field Synthesis via Occlusion-aware Sampling and Deep Visual Feature Fusion".Machine Intelligence Research 20.3(2023):408-420.