Multi-Modal Domain Adaptation Variational Auto-encoder for EEG-Based Emotion Recognition

doi:10.1109/JAS.2022.105515

CORC > 自动化研究所 > 中国科学院自动化研究所 > 类脑智能研究中心 > 神经计算及脑机交互

	Multi-Modal Domain Adaptation Variational Auto-encoder for EEG-Based Emotion Recognition
	Wang, Yixin 3,5,6; Qiu, Shuang 3,6; Li, Dan 3,4,6; Du, Changde 3,6; Lu, Bao-Liang 1; He, Huiguang 2,3,6
刊名	IEEE-CAA JOURNAL OF AUTOMATICA SINICA
	2022-09-01
卷号	9 期号:9 页码:1612-1626
关键词	Cycle-consistency domain adaptation electroencephalograph (EEG) multi modality variational autoencoder
ISSN号	2329-9266
DOI	10.1109/JAS.2022.105515
通讯作者	He, Huiguang(huiguang.he@ia.ac.cn)
英文摘要	Traditional electroencephalograph (EEG)-based emotion recognition requires a large number of calibration samples to build a model for a specific subject, which restricts the application of the affective brain computer interface (BCI) in practice. We attempt to use the multi-modal data from the past session to realize emotion recognition in the case of a small amount of calibration samples. To solve this problem, we propose a multi-modal domain adaptive variational autoencoder (MMDA-VAE) method, which learns shared cross-domain latent representations of the multi-modal data. Our method builds a multi-modal variational autoencoder (MVAE) to project the data of multiple modalities into a common space. Through adversarial learning and cycle-consistency regularization, our method can reduce the distribution difference of each domain on the shared latent representation layer and realize the transfer of knowledge. Extensive experiments are conducted on two public datasets, SEED and SEED-IV, and the results show the superiority of our proposed method. Our work can effectively improve the performance of emotion recognition with a small amount of labelled multi-modal data.
资助项目	National Natural Science Foundation of China[61976209] ; National Natural Science Foundation of China[62020106015] ; National Natural Science Foundation of China[U21A20388] ; CAS International Collaboration Key Project[173211KYSB20190024] ; Strategic Priority Research Program of CAS[XDB32040000]
WOS研究方向	Automation & Control Systems
语种	英语
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
WOS记录号	WOS:000844142700010
资助机构	National Natural Science Foundation of China ; CAS International Collaboration Key Project ; Strategic Priority Research Program of CAS
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/50053]
专题	类脑智能研究中心_神经计算及脑机交互
通讯作者	He, Huiguang
作者单位	1.Shanghai Jiao Tong Univ, Dept Comp Sci & Engn, Shanghai 200240, Peoples R China 2.Chinese Acad Sci, Ctr Excellence Brain Sci & Intelligence Technol, Beijing, Peoples R China 3.Chinese Acad Sci, Res Ctr Brain Inspired Intelligence, Inst Automat, Natl Lab Pattern Recognit, Beijing 100190, Peoples R China 4.Yantai Univ, Sch Math & Informat Sci, Yantai 264003, Peoples R China 5.Beijing Inst Control & Elect Technol, Beijing 100038, Peoples R China 6.Univ Chinese Acad Sci, Beijing 100049, Peoples R China
推荐引用方式 GB/T 7714	Wang, Yixin,Qiu, Shuang,Li, Dan,et al. Multi-Modal Domain Adaptation Variational Auto-encoder for EEG-Based Emotion Recognition[J]. IEEE-CAA JOURNAL OF AUTOMATICA SINICA,2022,9(9):1612-1626.
APA	Wang, Yixin,Qiu, Shuang,Li, Dan,Du, Changde,Lu, Bao-Liang,&He, Huiguang.(2022).Multi-Modal Domain Adaptation Variational Auto-encoder for EEG-Based Emotion Recognition.IEEE-CAA JOURNAL OF AUTOMATICA SINICA,9(9),1612-1626.
MLA	Wang, Yixin,et al."Multi-Modal Domain Adaptation Variational Auto-encoder for EEG-Based Emotion Recognition".IEEE-CAA JOURNAL OF AUTOMATICA SINICA 9.9(2022):1612-1626.