Prior-knowledge and attention based meta-learning for few-shot learning

doi:10.1016/j.knosys.2020.106609

CORC > 自动化研究所 > 中国科学院自动化研究所 > 模式识别国家重点实验室 > 生物识别与安全技术研究中心

	Prior-knowledge and attention based meta-learning for few-shot learning
	Qin, Yunxiao 3; Zhang, Weiguo 3; Zhao, Chenxu 5; Wang, Zezheng 6; Zhu, Xiangyu 1; Shi, Jingping 3; Qi, Guojun 4; Lei, Zhen 1,2
刊名	KNOWLEDGE-BASED SYSTEMS
	2021-02-15
卷号	213 页码:12
关键词	Meta-learning Few-shot learning Prior-knowledge Representation Attention mechanism
ISSN号	0950-7051
DOI	10.1016/j.knosys.2020.106609
通讯作者	Qin, Yunxiao(qyxqyx@mail.nwpu.edu.cn)
英文摘要	Recently, meta-learning has been shown to be a promising way to solve few-shot learning. In this paper, inspired by the human cognition process, which utilizes both prior-knowledge and visual attention when learning new knowledge, we present a novel paradigm of meta-learning approach that capitalizes on three developments to introduce attention mechanism and prior-knowledge to meta-learning. In our approach, prior-knowledge is responsible for helping the meta-learner express the input data in a high-level representation space, and the attention mechanism enables the meta-learner to focus on key data features in the representation space. Compared with the existing meta-learning approaches that pay little attention to prior-knowledge and visual attention, our approach alleviates the meta-learner's few-shot cognition burden. Furthermore, we discover a Task-Over-Fitting (TOF) problem,(1) which indicates that the meta-learner has poor generalization across different K-shot learning tasks. To model the TOF problem, we propose a novel Cross-Entropy across Tasks (CET) metric.(2) Extensive experiments demonstrate that our techniques improve the meta-learner to state-of-the-art performance on several few-shot learning benchmarks while also substantially alleviating the TOF problem. (C) 2020 Elsevier B.V. All rights reserved.
资助项目	National Key Research and Development Program of China[2020YFC2003901] ; National Natural Science Foundation of China[61573286] ; National Natural Science Foundation of China[61876178] ; National Natural Science Foundation of China[61976229]
WOS研究方向	Computer Science
语种	英语
出版者	ELSEVIER
WOS记录号	WOS:000614644100011
资助机构	National Key Research and Development Program of China ; National Natural Science Foundation of China
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/43350]
专题	自动化研究所_模式识别国家重点实验室_生物识别与安全技术研究中心
通讯作者	Qin, Yunxiao
作者单位	1.Chinese Acad Sci, Inst Automat, Natl Lab Pattern Recognit NLPR, Beijing 100000, Peoples R China 2.Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing 100049, Peoples R China 3.Northwestern Polytech Univ, Xian 710129, Peoples R China 4.Huawei Cloud, Seattle, WA 90876 USA 5.MiningLamp Technol, Beijing 100094, Peoples R China 6.Beijing Kwai Technol, Beijing 102600, Peoples R China
推荐引用方式 GB/T 7714	Qin, Yunxiao,Zhang, Weiguo,Zhao, Chenxu,et al. Prior-knowledge and attention based meta-learning for few-shot learning[J]. KNOWLEDGE-BASED SYSTEMS,2021,213:12.
APA	Qin, Yunxiao.,Zhang, Weiguo.,Zhao, Chenxu.,Wang, Zezheng.,Zhu, Xiangyu.,...&Lei, Zhen.(2021).Prior-knowledge and attention based meta-learning for few-shot learning.KNOWLEDGE-BASED SYSTEMS,213,12.
MLA	Qin, Yunxiao,et al."Prior-knowledge and attention based meta-learning for few-shot learning".KNOWLEDGE-BASED SYSTEMS 213(2021):12.