Model Aggregation Method for Data Parallelism in Distributed Real-Time Machine Learning of Smart Sensing Equipment

doi:10.1109/ACCESS.2019.2955547

CORC > 计算技术研究所 > 中国科学院计算技术研究所 > 中国科学院计算技术研究所期刊论文 > 英文

	Model Aggregation Method for Data Parallelism in Distributed Real-Time Machine Learning of Smart Sensing Equipment
	Fan, Yuchen 1,2; Zhang, Jilin 1,2,3; Zhao, Nailiang 1,2; Ren, Yongjian 1,2; Wan, Jian 1,2,4; Zhou, Li 1,2; Shen, Zhongyu 1,2; Wang, Jue 5; Zhang, Juncong 6; Wei, Zhenguo 6
刊名	IEEE ACCESS
	2019
卷号	7 页码:172065-172073
关键词	Distributed machine learning stochastic gradient descent model aggregation method smart sensing equipment
ISSN号	2169-3536
DOI	10.1109/ACCESS.2019.2955547
英文摘要	In distributed real-time machine learning of smart sensing equipment, training speed and training accuracy are two hard-to-choose trade-off performance measures directly influenced by the design of distributed machine learning algorithms. And it will influence effort of smart sensing equipment directly. We take the model aggregation method of distributed machine learning as a starting point. Due to the loss of accuracy caused by the direct averaging of the parameter average method, we developed the loss function weight reorder stochastic gradient descent method (LR-SGD). LR-SGD uses the loss function value to determine the weight of the work nodes when aggregating the model parameters, and it improves the performance of the parameter average method for nonconvex problems. As shown in the experiment results, our algorithm can improve the training accuracy by a maximum of approximately 0.57% for the Bulk Synchronous Parallel (BSP) model and approximately 6.30% for the Stale Synchronous Parallel (SSP) model.
资助项目	National Key Technology Research and Development Program[2018YFB0204001] ; National Natural Science Foundation of China[61672200] ; National Natural Science Foundation of China[61572163] ; Key Technology Research and Development Program of the Zhejiang Province[2019C01059] ; Zhejiang Natural Science Funds[LY17F020029] ; Zhejiang Natural Science Funds[LY16F020018] ; State Key Laboratory of Computer Architecture Project[CARCH201712] ; Hangzhou Dianzi University Postgraduate Research Innovation Fund Program[CXJJ2018052]
WOS研究方向	Computer Science ; Engineering ; Telecommunications
语种	英语
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
WOS记录号	WOS:000509374200057
内容类型	期刊论文
源URL	[http://119.78.100.204/handle/2XEOYT63/14735]
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Zhao, Nailiang; Ren, Yongjian
作者单位	1.Hangzhou Dianzi Univ, Sch Comp Sci & Technol, Hangzhou 310018, Peoples R China 2.Minist Educ, Key Lab Complex Syst Modeling & Simulat, Hangzhou 310018, Peoples R China 3.Chinese Acad Sci, Inst Comp Technol, State Key Lab Comp Architecture, Beijing 100190, Peoples R China 4.Zhejiang Univ Sci & Technol, Hangzhou 310023, Peoples R China 5.Chinese Acad Sci, Comp Network Informat Ctr, Beijing 100190, Peoples R China 6.Zhejiang Dawning Informat Technol Co Ltd, Hangzhou 310051, Zhejiang, Peoples R China
推荐引用方式 GB/T 7714	Fan, Yuchen,Zhang, Jilin,Zhao, Nailiang,et al. Model Aggregation Method for Data Parallelism in Distributed Real-Time Machine Learning of Smart Sensing Equipment[J]. IEEE ACCESS,2019,7:172065-172073.
APA	Fan, Yuchen.,Zhang, Jilin.,Zhao, Nailiang.,Ren, Yongjian.,Wan, Jian.,...&Wei, Zhenguo.(2019).Model Aggregation Method for Data Parallelism in Distributed Real-Time Machine Learning of Smart Sensing Equipment.IEEE ACCESS,7,172065-172073.
MLA	Fan, Yuchen,et al."Model Aggregation Method for Data Parallelism in Distributed Real-Time Machine Learning of Smart Sensing Equipment".IEEE ACCESS 7(2019):172065-172073.