MEST: A Model-Driven Efficient Searching Approach for MapReduce Self-Tuning

	MEST: A Model-Driven Efficient Searching Approach for MapReduce Self-Tuning
	Zhendong Bei; Zhibin Yu; Qixiao Liu; Chengzhong Xu; Shengzhong Feng; Shuang Song
刊名	IEEE Access
	2017
文献子类	期刊论文
英文摘要	Hadoop is the most popular implementation framework of the MapReduce programming model, and it has a number of performance-critical configuration parameters. However, manually setting these parameters to their optimal values not only needs in-depth knowledge on Hadoop as well as the job itself, but also requires a large amount of time and efforts. Automatic approaches have therefore been proposed. Their usage, however, is still quite limited due to the intolerably long searching time. In this paper, we introduce MapreducE Self-Tuning (MEST), a framework that accelerates the searching process for the optimal configuration of a given Hadoop application. We have devised a novel mechanism by integrating the model trees algorithm with the genetic algorithm. As such, MEST significantly reduces the searching time by removing unnecessary profiling, modeling, and searching steps, which are mandatory for existing approaches. Our experiments using five benchmarks, each with two input data sets (DS1 and 2× DS1 ) show that MEST improves the searching efficiency (SE) by factors of 1.37× and 2.18× on average respectively over the state-of-the-art approach.
URL标识	查看原文
语种	英语
内容类型	期刊论文
源URL	[http://ir.siat.ac.cn:8080/handle/172644/12524]
专题	深圳先进技术研究院_数字所
作者单位	IEEE Access
推荐引用方式 GB/T 7714	Zhendong Bei,Zhibin Yu,Qixiao Liu,et al. MEST: A Model-Driven Efficient Searching Approach for MapReduce Self-Tuning[J]. IEEE Access,2017.
APA	Zhendong Bei,Zhibin Yu,Qixiao Liu,Chengzhong Xu,Shengzhong Feng,&Shuang Song.(2017).MEST: A Model-Driven Efficient Searching Approach for MapReduce Self-Tuning.IEEE Access.
MLA	Zhendong Bei,et al."MEST: A Model-Driven Efficient Searching Approach for MapReduce Self-Tuning".IEEE Access (2017).