A Feedback Control Mechanism for Balancing I/O- and Memory-Intensive Applications on Clusters
Xiao Qin1 Hong Jiang2 Yifeng Zhu2 David R. Swanson21 Department of Computer Science New Mexico Institute of Mining and Technology 801 Leroy Place, Socorro, New Mexico 87801 2 Department of Computer Science and Engineering
University of Nebraska-Lincoln
Lincoln, NE 68588-0115
One common assumption of existing models of load balancing is that the weights of resources and I/O buffer size are statically configured and cannot be adjusted based on a dynamic workload. Though the static configuration of these parameters performs well in a cluster where the workload can be modeled and predicted, its performance is poor in dynamic systems in which the workload is unknown. In this paper, a new feedback control mechanism is proposed to improve the overall performance of a cluster with a general and practical workload including I/O-intensive and memory-intensive load. This mechanism is also shown to be effective in complementing and enhancing the performance of a number of existing dynamic load-balancing schemes. To capture the current and past workload characteristics, the primary objectives of the feedback mechanism are: (1) dynamically adjusting the resource weights, which indicate the significance of the resources, and (2) minimizing the number of page faults for memory-intensive jobs while increasing the utilization of the I/O buffers for I/O-intensive jobs by manipulating the I/O buffer size. Results from extensive trace-driven simulation experiments show that, compared with the schemes with fixed resource weights and buffer sizes, the feedback control mechanism delivers a performance improvement in terms of the mean slowdown by up to 282% (with an average of 125%).
Appeared in Parallel and Distributed Computing Practices Journal, 2005 .