Real-Time Digital Video Stabilization of Bioinspired Robotic Fish Using Estimation-and-Prediction Framework

doi:10.1109/TMECH.2022.3155696

CORC > 自动化研究所 > 中国科学院自动化研究所 > 复杂系统管理与控制国家重点实验室 > 先进机器人控制团队

	Real-Time Digital Video Stabilization of Bioinspired Robotic Fish Using Estimation-and-Prediction Framework
	Meng, Yan 1,3; Wu, Zhengxing 1,3; Zhang, Pengfei 1,3; Wang, Jian 1,3; Yu, Junzhi 2,3
刊名	IEEE-ASME TRANSACTIONS ON MECHATRONICS
	2022-03-11
页码	12
关键词	Robots Cameras Robot vision systems Streaming media Bio-inspired robotics Real-time systems Estimation Bioinspired robot digital video stabilization estimation-and-prediction framework robotic fish vision system
ISSN号	1083-4435
DOI	10.1109/TMECH.2022.3155696
通讯作者	Yu, Junzhi(junzhi.yu@ia.ac.cn)
英文摘要	The rhythmic movement of bioinspired robotic fish brings about undesirable visual jitter. Note that the unstable camera path of this kind of robot is characterized by obvious regularity and predictability, of which traditional stabilization methods have not made full advantage. This article proposes a novel estimation-and-prediction framework for real-time digital video stabilization of bioinspired robotic fish. First, based on the attitude information of an inertial measurement unit (IMU), a camera-IMU model is established, where the homography transformation with eight degrees of freedom (DOFs) is reduced to translation transformation with two DOFs. Second, traditional optical flow and gray projection methods as well as a novel translation estimation network are employed to estimate the translations between consecutive frames. Third, a lightweight long short-term memory (LSTM) network is constructed, allowing remarkable prediction and smoothing of the camera path. Finally, aquatic experiments under various scenarios are conducted on a manta-inspired robot, demonstrating the effectiveness of the proposed method. Specifically, compared with the state-of-the-art commercial offline stabilization software, our online stabilization algorithm achieves approximate visual stability and remarkably faster stabilization speed. The obtained results shed light on visual sensing and control applications of bioinspired underwater vehicles.
资助项目	National Natural Science Foundation of China[61973303] ; National Natural Science Foundation of China[62033013] ; National Natural Science Foundation of China[61725305] ; National Natural Science Foundation of China[U1909206] ; National Natural Science Foundation of China[62022090] ; Beijing Natural Science Foundation[4192060] ; Beijing Nova Program[Z201100006820078] ; Youth Innovation Promotion Association, Chinese Academy of Sciences[2019138]
WOS研究方向	Automation & Control Systems ; Engineering
语种	英语
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
WOS记录号	WOS:000770572900001
资助机构	National Natural Science Foundation of China ; Beijing Natural Science Foundation ; Beijing Nova Program ; Youth Innovation Promotion Association, Chinese Academy of Sciences
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/48111]
专题	自动化研究所_复杂系统管理与控制国家重点实验室_先进机器人控制团队
通讯作者	Yu, Junzhi
作者单位	1.Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing 100049, Peoples R China 2.Peking Univ, Coll Engn, Beijing Innovat Ctr Engn Sci & Adv Technol, Dept Adv Mfg & Robot,State Key Lab Turbulence & C, Beijing 100871, Peoples R China 3.Chinese Acad Sci, Inst Automat, State Key Lab Management & Control Complex Syst, Beijing 100190, Peoples R China
推荐引用方式 GB/T 7714	Meng, Yan,Wu, Zhengxing,Zhang, Pengfei,et al. Real-Time Digital Video Stabilization of Bioinspired Robotic Fish Using Estimation-and-Prediction Framework[J]. IEEE-ASME TRANSACTIONS ON MECHATRONICS,2022:12.
APA	Meng, Yan,Wu, Zhengxing,Zhang, Pengfei,Wang, Jian,&Yu, Junzhi.(2022).Real-Time Digital Video Stabilization of Bioinspired Robotic Fish Using Estimation-and-Prediction Framework.IEEE-ASME TRANSACTIONS ON MECHATRONICS,12.
MLA	Meng, Yan,et al."Real-Time Digital Video Stabilization of Bioinspired Robotic Fish Using Estimation-and-Prediction Framework".IEEE-ASME TRANSACTIONS ON MECHATRONICS (2022):12.