A Two-Stream CNN With Simultaneous Detection and Segmentation for Robotic Grasping

doi:10.1109/TSMC.2020.3018757

	A Two-Stream CNN With Simultaneous Detection and Segmentation for Robotic Grasping
	Yu, Yingying 1,2; Cao, Zhiqiang 1,2; Liu, Zhicheng 1,2; Geng, Wenjie 1,2; Yu, Junzhi 2,4; Zhang, Weimin 3
刊名	IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS
	2022-02-01
卷号	52 期号:2 页码:1167-1181
关键词	Grasping Robot kinematics Manipulators Image segmentation Deconvolution Machine learning Global deconvolution network (GDN) robotic grasping simultaneous detection and segmentation two-stream grasping convolutional neural network (CNN)
ISSN号	2168-2216
DOI	10.1109/TSMC.2020.3018757
通讯作者	Cao, Zhiqiang(zhiqiang.cao@ia.ac.cn)
英文摘要	The manipulating robots receive much attention by offering better services, where object grasping is still challenging especially under background interferences. In this article, a novel two-stream grasping convolutional neural network (CNN) with simultaneous detection and segmentation is proposed. The proposed method is cascaded by an improved simultaneous detection and segmentation network BlitzNet and a two-stream grasping CNN TsGNet. The improved BlitzNet introduces the channel-based attention mechanism, and achieves an improvement of detection accuracy and segmentation accuracy with the combination of the learning of multitask loss weightings and background suppression. Based on the obtained bounding box and the segmentation mask of the target object, the target object is separated from the background, and the corresponding depth map and grayscale map are sent to TsGNet. By adopting depthwise separable convolution and designed global deconvolution network, TsGNet achieves the best grasp detection with only a small amount of network parameters. This best grasp in the pixel coordinate system is converted to a desired 6-D pose for the robot, which drives the manipulator to execute grasping. The proposed method combines a grasping CNN with simultaneous detection and segmentation to achieve the best grasp with a good adaptability to background. With the Cornell grasping dataset, the image-wise accuracy and object-wise accuracy of the proposed TsGNet are 93.13% and 92.99%, respectively. The effectiveness of the proposed method is verified by the experiments.
资助项目	National Natural Science Foundation of China[61633017] ; National Natural Science Foundation of China[61633020] ; National Natural Science Foundation of China[61836015] ; Beijing Advanced Innovation Center for Intelligent Robots and Systems[2018IRS21]
WOS研究方向	Automation & Control Systems ; Computer Science
语种	英语
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
WOS记录号	WOS:000742732900053
资助机构	National Natural Science Foundation of China ; Beijing Advanced Innovation Center for Intelligent Robots and Systems
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/47915]
专题	中国科学院自动化研究所
通讯作者	Cao, Zhiqiang
作者单位	1.Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing 100049, Peoples R China 2.Chinese Acad Sci, Inst Automat, State Key Lab Management & Control Complex Syst, Beijing 100190, Peoples R China 3.Beijing Inst Technol, Beijing Adv Innovat Ctr Intelligent Robots & Syst, Beijing 100081, Peoples R China 4.Peking Univ, State Key Lab Turbulence & Complex Syst, Dept Mech & Engn Sci, BIC ESAT Coll Engn, Beijing 100871, Peoples R China
推荐引用方式 GB/T 7714	Yu, Yingying,Cao, Zhiqiang,Liu, Zhicheng,et al. A Two-Stream CNN With Simultaneous Detection and Segmentation for Robotic Grasping[J]. IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS,2022,52(2):1167-1181.
APA	Yu, Yingying,Cao, Zhiqiang,Liu, Zhicheng,Geng, Wenjie,Yu, Junzhi,&Zhang, Weimin.(2022).A Two-Stream CNN With Simultaneous Detection and Segmentation for Robotic Grasping.IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS,52(2),1167-1181.
MLA	Yu, Yingying,et al."A Two-Stream CNN With Simultaneous Detection and Segmentation for Robotic Grasping".IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS 52.2(2022):1167-1181.