Minimizing resource waste in heterogeneous resource allocation for data stream processing on clouds

Wu Chun Chung, Tsung Lin Wu, Yi Hsuan Lee, Kuo Chan Huang, Hung Chang Hsiao, Kuan Chou Lai

研究成果: Article同行評審

摘要

Resource allocation is vital for improving system performance in big data processing. The resource demand for various applications can be heterogeneous in cloud computing. Therefore, a resource gap occurs while some resource capacities are exhausted and other resource capacities on the same server are still available. This phenomenon is more apparent when the computing resources are more heterogeneous. Previous resource-allocation algorithms paid limited attention to this situation. When such an algorithm is applied to a server with heterogeneous resources, resource allocation may result in considerable resource wastage for the available but unused resources. To reduce resource wastage, a resource-allocation algorithm, called the minimizing resource gap (MRG) algorithm, for heterogeneous resources is proposed in this study. In MRG, the gap between resource usages for each server in cloud computing and the resource demands among various applications are considered. When an application is launched, MRG calculates resource usage and allocates resources to the server with the minimized usage gap to reduce the amount of available but unused resources. To demonstrate MRG performance, the MRG algorithm was implemented in Apache Spark. CPU-and memory-intensive applications were applied as benchmarks with different resource demands. Experimental results proved the superiority of the proposed MRG approach for improving the system utilization to reduce the overall completion time by up to 24.7% for heterogeneous servers in cloud computing.

原文English
文章編號149
頁(從 - 到)1-17
頁數17
期刊Applied Sciences (Switzerland)
11
發行號1
DOIs
出版狀態Published - 2021 一月 1

All Science Journal Classification (ASJC) codes

  • Materials Science(all)
  • Instrumentation
  • Engineering(all)
  • Process Chemistry and Technology
  • Computer Science Applications
  • Fluid Flow and Transfer Processes

指紋 深入研究「Minimizing resource waste in heterogeneous resource allocation for data stream processing on clouds」主題。共同形成了獨特的指紋。

引用此