Minimizing resource waste in heterogeneous resource allocation for data stream processing on clouds

Wu Chun Chung, Tsung Lin Wu, Yi Hsuan Lee, Kuo Chan Huang, Hung Chang Hsiao, Kuan Chou Lai

Research output: Contribution to journalArticlepeer-review

Abstract

Resource allocation is vital for improving system performance in big data processing. The resource demand for various applications can be heterogeneous in cloud computing. Therefore, a resource gap occurs while some resource capacities are exhausted and other resource capacities on the same server are still available. This phenomenon is more apparent when the computing resources are more heterogeneous. Previous resource-allocation algorithms paid limited attention to this situation. When such an algorithm is applied to a server with heterogeneous resources, resource allocation may result in considerable resource wastage for the available but unused resources. To reduce resource wastage, a resource-allocation algorithm, called the minimizing resource gap (MRG) algorithm, for heterogeneous resources is proposed in this study. In MRG, the gap between resource usages for each server in cloud computing and the resource demands among various applications are considered. When an application is launched, MRG calculates resource usage and allocates resources to the server with the minimized usage gap to reduce the amount of available but unused resources. To demonstrate MRG performance, the MRG algorithm was implemented in Apache Spark. CPU-and memory-intensive applications were applied as benchmarks with different resource demands. Experimental results proved the superiority of the proposed MRG approach for improving the system utilization to reduce the overall completion time by up to 24.7% for heterogeneous servers in cloud computing.

Original languageEnglish
Article number149
Pages (from-to)1-17
Number of pages17
JournalApplied Sciences (Switzerland)
Volume11
Issue number1
DOIs
Publication statusPublished - 2021 Jan 1

All Science Journal Classification (ASJC) codes

  • Materials Science(all)
  • Instrumentation
  • Engineering(all)
  • Process Chemistry and Technology
  • Computer Science Applications
  • Fluid Flow and Transfer Processes

Fingerprint Dive into the research topics of 'Minimizing resource waste in heterogeneous resource allocation for data stream processing on clouds'. Together they form a unique fingerprint.

Cite this