Stochastic proximal quasi-Newton methods for non-convex composite optimization

doi:10.1080/10556788.2018.1471141

	Stochastic proximal quasi-Newton methods for non-convex composite optimization
	Wang, Xiaoyu 1,2; Wang, Xiao 1; Yuan, Ya-xiang 2
刊名	OPTIMIZATION METHODS & SOFTWARE
	2019-09-03
卷号	34 期号:5 页码:922-948
关键词	Non-convex composite optimization Polyak-Lojasiewicz (PL) inequality stochastic gradient stochastic variance reduction gradient symmetric rank one method rank one proximity operator complexity bound
ISSN号	1055-6788
DOI	10.1080/10556788.2018.1471141
英文摘要	In this paper, we propose a generic algorithmic framework for stochastic proximal quasi-Newton (SPQN) methods to solve non-convex composite optimization problems. Stochastic second-order information is explored to construct proximal subproblem. Under mild conditions we show the non-asympotic convergence of the proposed algorithm to stationary point of original problems and analyse its computational complexity. Besides, we extend the proximal form of Polyak-Lojasiewicz (PL) inequality to constrained settings and obtain the constrained proximal PL (CP-PL) inequality. Under CP-PL inequality linear convergence rate of the proposed algorithm is achieved. Moreover, we propose a modified self-scaling symmetric rank one incorporated in the framework for SPQN method, which is called stochastic symmetric rank one method. Finally, we report some numerical experiments to reveal the effectiveness of the proposed algorithm.
资助项目	National Natural Science Foundation of China[11331012] ; National Natural Science Foundation of China[11301505] ; National Natural Science Foundation of China[11731013] ; National Natural Science Foundation of China[11688101]
WOS研究方向	Computer Science ; Operations Research & Management Science ; Mathematics
语种	英语
出版者	TAYLOR & FRANCIS LTD
WOS记录号	WOS:000486079100002
内容类型	期刊论文
源URL	[http://ir.amss.ac.cn/handle/2S8OKBNM/35631]
专题	中国科学院数学与系统科学研究院
通讯作者	Wang, Xiao
作者单位	1.Univ Chinese Acad Sci, Sch Math Sci, Beijing, Peoples R China 2.Chinese Acad Sci, LSEC, Inst Computat Math & Sci Engn Comp, AMSS, Beijing, Peoples R China
推荐引用方式 GB/T 7714	Wang, Xiaoyu,Wang, Xiao,Yuan, Ya-xiang. Stochastic proximal quasi-Newton methods for non-convex composite optimization[J]. OPTIMIZATION METHODS & SOFTWARE,2019,34(5):922-948.
APA	Wang, Xiaoyu,Wang, Xiao,&Yuan, Ya-xiang.(2019).Stochastic proximal quasi-Newton methods for non-convex composite optimization.OPTIMIZATION METHODS & SOFTWARE,34(5),922-948.
MLA	Wang, Xiaoyu,et al."Stochastic proximal quasi-Newton methods for non-convex composite optimization".OPTIMIZATION METHODS & SOFTWARE 34.5(2019):922-948.