Cross-domain personalized image captioning

doi:10.1007/s11042-019-7441-7

CORC > 自动化研究所 > 中国科学院自动化研究所 > 模式识别国家重点实验室 > 多媒体计算与图形学团队

	Cross-domain personalized image captioning
	Long, Cuirong 2; Yang, Xiaoshan 1,3; Xu, Changsheng 1,2,3
刊名	MULTIMEDIA TOOLS AND APPLICATIONS
	2020-12-01
卷号	79 期号:45-46 页码:33333-33348
关键词	Personalization Image captioning Domain adaptation
ISSN号	1380-7501
DOI	10.1007/s11042-019-7441-7
通讯作者	Yang, Xiaoshan(xiaoshan.yang@nlpr.ia.ac.cn)
英文摘要	Image captioning aims to translate an image to a complete and natural sentence. It involves both computer vision and natural language processing. Though image captioning has achieved good results under the rapid development of deep neural networks, excessively pursuing the evaluation results of the captioning models makes the generated text description too conservative in practical applications. It is necessary to increase the diversity of the text description and account for prior knowledge such as the user's favorite vocabularies and writing styles. In this paper, we study the personalized image captioning which can generate sentences to describe the user's own story and feelings of life with the most preferred word expression. Moreover, we propose cross-domain personalized image captioning (CDPIC) to learn domain-invariant captioning models which can be applied on different social media platforms. The proposed method can flexibly model user interest by embedding the user ID as an interest vector. To the best of our knowledge, we propose the first cross-domain personalized image captioning approach by combining the user interest modeling and a simple and effective domain-invariant constraint. The effectiveness of the proposed method is verified on datasets from the Instagram and Lookbook platforms.
WOS研究方向	Computer Science ; Engineering
语种	英语
出版者	SPRINGER
WOS记录号	WOS:000594855000001
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/42698]
专题	自动化研究所_模式识别国家重点实验室_多媒体计算与图形学团队
通讯作者	Yang, Xiaoshan
作者单位	1.Chinese Acad Sci, Inst Automat, Beijing, Peoples R China 2.HeFei Univ Technol, Hefei, Peoples R China 3.Univ Chinese Acad Sci, Beijing, Peoples R China
推荐引用方式 GB/T 7714	Long, Cuirong,Yang, Xiaoshan,Xu, Changsheng. Cross-domain personalized image captioning[J]. MULTIMEDIA TOOLS AND APPLICATIONS,2020,79(45-46):33333-33348.
APA	Long, Cuirong,Yang, Xiaoshan,&Xu, Changsheng.(2020).Cross-domain personalized image captioning.MULTIMEDIA TOOLS AND APPLICATIONS,79(45-46),33333-33348.
MLA	Long, Cuirong,et al."Cross-domain personalized image captioning".MULTIMEDIA TOOLS AND APPLICATIONS 79.45-46(2020):33333-33348.