CANet: Co-attention network for RGB-D semantic segmentation

doi:10.1016/j.patcog.2021.108468

CORC > 自动化研究所 > 中国科学院自动化研究所 > 复杂系统管理与控制国家重点实验室 > 机器人应用与理论组

	CANet: Co-attention network for RGB-D semantic segmentation
	Zhou, Hao 2,3,5; Qi, Lu 4; Huang, Hai 5; Yang, Xu 2,3; Wan, Zhaoliang 5; Wen, Xianglong 1
刊名	PATTERN RECOGNITION
	2022-04-01
卷号	124 页码:11
关键词	RGB-D Multi -modal fusion Co-attention Semantic segmentation
ISSN号	0031-3203
DOI	10.1016/j.patcog.2021.108468
通讯作者	Huang, Hai(haihus@163.com)
英文摘要	Incorporating the depth (D) information to RGB images has proven the effectiveness and robustness in semantic segmentation. However, the fusion between them is not trivial due to their inherent physical meaning discrepancy, in which RGB represents RGB information but D depth information. In this paper, we propose a co-attention network (CANet) to build sound interaction between RGB and depth features. The key part in the CANet is the co-attention fusion part. It includes three modules. Specifically, the po-sition and channel co-attention fusion modules adaptively fuse RGB and depth features in spatial and channel dimensions. An additional fusion co-attention module further integrates the outputs of the posi-tion and channel co-attention fusion modules to obtain a more representative feature which is used for the final semantic segmentation. Extensive experiments witness the effectiveness of the CANet in fus-ing RGB and depth features, achieving state-of-the-art performance on two challenging RGB-D semantic segmentation datasets, i.e., NYUDv2 and SUN-RGBD. (c) 2021 Elsevier Ltd. All rights reserved.
资助项目	National Natural Science Foundation (NSFC) of China[61633009] ; National Natural Science Foundation (NSFC) of China[61973301] ; National Natural Science Foundation (NSFC) of China[61972020] ; National Natural Science Foundation (NSFC) of China[51579053] ; National Natural Science Foundation (NSFC) of China[51779058] ; Beijing Science and Technology Plan Project[Z18110 0 0 08918018] ; National Key R&D Program of China[2016YFC0300801] ; National Key R&D Program of China[2017YFB1300202] ; National Key R&D Program of China[2020AAA0108902]
WOS关键词	FEATURES
WOS研究方向	Computer Science ; Engineering
语种	英语
出版者	ELSEVIER SCI LTD
WOS记录号	WOS:000736972200013
资助机构	National Natural Science Foundation (NSFC) of China ; Beijing Science and Technology Plan Project ; National Key R&D Program of China
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/47130]
专题	自动化研究所_复杂系统管理与控制国家重点实验室_机器人应用与理论组
通讯作者	Huang, Hai
作者单位	1.Jihua Lab, Foshan, Peoples R China 2.Univ Chinese Acad Sci, Beijing, Peoples R China 3.Chinese Acad Sci, Inst Automat, State Key Lab Management & Control Complex Syst, Beijing, Peoples R China 4.Chinese Univ Hong Kong, Hong Kong, Peoples R China 5.Harbin Engn Univ, Natl Key Lab Sci & Technol Underwater Vehicle, Harbin, Peoples R China
推荐引用方式 GB/T 7714	Zhou, Hao,Qi, Lu,Huang, Hai,et al. CANet: Co-attention network for RGB-D semantic segmentation[J]. PATTERN RECOGNITION,2022,124:11.
APA	Zhou, Hao,Qi, Lu,Huang, Hai,Yang, Xu,Wan, Zhaoliang,&Wen, Xianglong.(2022).CANet: Co-attention network for RGB-D semantic segmentation.PATTERN RECOGNITION,124,11.
MLA	Zhou, Hao,et al."CANet: Co-attention network for RGB-D semantic segmentation".PATTERN RECOGNITION 124(2022):11.