Multi-modal tag localization for mobile video search

doi:10.1007/s00530-016-0506-9

CORC > 计算技术研究所 > 中国科学院计算技术研究所 > 中国科学院计算技术研究所期刊论文 > 英文

	Multi-modal tag localization for mobile video search
	Zhang, Rui 2,3; Tang, Sheng 2; Liu, Wu 1; Zhang, Yongdong 2; Li, Jintao 2
刊名	MULTIMEDIA SYSTEMS
	2017-11-01
卷号	23 期号:6 页码:713-724
关键词	Tag localization Mobile video search Multi-modality fusion
ISSN号	0942-4962
DOI	10.1007/s00530-016-0506-9
英文摘要	Given the tremendous growth of mobile videos, video tag localization, which localizes the relevant video clips for an associated semantic tag, is becoming increasingly important to influence users browsing and searching experience. However, most existing approaches adopt and depend to large degree on carefully selected visual features, which are manually designed by experts and do not take multi-modality into consideration. Aiming to take into account complementarity of different modalities, in this paper, we propose a multi-modal tag localization framework by exploiting deep learning to learn visual, auditory, and semantic features of videos for tag localization. Furthermore, we showcase that the framework can be applied to two novel mobile video search applications: (1) automatic time-code-level tags generation and (2) query-dependent video thumbnail selection. Extensive experiments on the public dataset show that the proposed approach achieves promising results, which obtains improvement beyond the state-of-the-arts. Finally, the subjective evaluation of usability demonstrates the proposed applications can significantly improve the user's mobile video search experience.
资助项目	863 Project[2014AA015202] ; National Nature Science Foundation of China[61525206] ; National Nature Science Foundation of China[61173054] ; National Nature Science Foundation of China[61572472] ; Beijing Natural Science Foundation[4152050] ; Funds for Creative Research Groups of China[61421061] ; Beijing Committee of Education ; Beijing Advanced Innovation Center for Imaging Technology
WOS研究方向	Computer Science
语种	英语
出版者	SPRINGER
WOS记录号	WOS:000415313700007
内容类型	期刊论文
源URL	[http://119.78.100.204/handle/2XEOYT63/6520]
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Liu, Wu
作者单位	1.Beijing Univ Posts & Telecommun, Beijing Key Lab Intelligent Telecommun Software &, Beijing, Peoples R China 2.Chinese Acad Sci, Inst Comp Technol, Beijing, Peoples R China 3.Univ Chinese Acad Sci, Beijing, Peoples R China
推荐引用方式 GB/T 7714	Zhang, Rui,Tang, Sheng,Liu, Wu,et al. Multi-modal tag localization for mobile video search[J]. MULTIMEDIA SYSTEMS,2017,23(6):713-724.
APA	Zhang, Rui,Tang, Sheng,Liu, Wu,Zhang, Yongdong,&Li, Jintao.(2017).Multi-modal tag localization for mobile video search.MULTIMEDIA SYSTEMS,23(6),713-724.
MLA	Zhang, Rui,et al."Multi-modal tag localization for mobile video search".MULTIMEDIA SYSTEMS 23.6(2017):713-724.