Investigating Gated Recurrent Neural Networks for Acoustic Modeling

CORC > 自动化研究所 > 中国科学院自动化研究所 > 数字内容技术与服务研究中心 > 听觉模型与认知计算

	Investigating Gated Recurrent Neural Networks for Acoustic Modeling
	Zhao, Yuanyuan; Li, Jie; Xu, Shuang; Xu, Bo; Yuanyuan Zhao
	2016-10
会议日期	October 17-20
会议地点	Tianjin, China
关键词	Gated Recurrent Neural Networks Long Short-term Memory Unit Gated Recurrent Neural Networks Long Short-term Memory Projected Unit
英文摘要	Recurrent neural networks (RNNs) with a gating mechanism have been shown to give state-of-the-art performance in acoustic modeling, such as gated recurrent unit (GRU), long short-term memory (LSTM), long short-term memory projected (LSTMP), etc. But little is known about why these gated RNNs work and what the differences are among these networks. Based on a series of experimental comparison and analysis, we find that: a) GRU usually performs better than LSTM, for possibly GRU is able to modulate the previous memory content through the learned reset gates, helping to model the long-span dependence more efficiently for speech sequence; b) LSTMP shows comparable performance with GRU, since LSTMP has the similar ability of information selection and combination by an automatic learned linear transformation in a weight-sharing way. In experiments, a visual analysis method is adopted to understand the historical information selection mechanism in RNNs in contrast to DNN. Experimental results on three different speech recognition tasks demonstrate the above conclusions and 5%-13% relative PER or CER reduction is observed.
语种	英语
内容类型	会议论文
源URL	[http://ir.ia.ac.cn/handle/173211/19652]
专题	数字内容技术与服务研究中心_听觉模型与认知计算
通讯作者	Yuanyuan Zhao
作者单位	Institute of Automation, Chinese Academy of Sciences
推荐引用方式 GB/T 7714	Zhao, Yuanyuan,Li, Jie,Xu, Shuang,et al. Investigating Gated Recurrent Neural Networks for Acoustic Modeling[C]. 见:. Tianjin, China. October 17-20.