[Back] [PDF

A Feedback Control Mechanism for Balancing I/O- and Memory-Intensive Applications on Clusters

Xiao Qin1    Hong Jiang2    Yifeng Zhu2    David R. Swanson2

1 Department of Computer Science
   New Mexico Institute of Mining and Technology
   801 Leroy Place, Socorro, New Mexico 87801
2 Department of Computer Science and Engineering
   University of Nebraska-Lincoln
   Lincoln, NE 68588-0115 

One common assumption of existing models of load balancing is that the weights of resources and I/O buffer size are statically configured and cannot be adjusted based on a dynamic workload. Though the static configuration of these parameters performs well in a cluster where the workload can be modeled and predicted, its performance is poor in dynamic systems in which the workload is unknown. In this paper, a new feedback control mechanism is proposed to improve the overall performance of a cluster with a general and practical workload including I/O-intensive and memory-intensive load. This mechanism is also shown to be effective in complementing and enhancing the performance of a number of existing dynamic load-balancing schemes. To capture the current and past workload characteristics, the primary objectives of the feedback mechanism are: (1) dynamically adjusting the resource weights, which indicate the significance of the resources, and (2) minimizing the number of page faults for memory-intensive jobs while increasing the utilization of the I/O buffers for I/O-intensive jobs by manipulating the I/O buffer size. Results from extensive trace-driven simulation experiments show that, compared with the schemes with fixed resource weights and buffer sizes, the feedback control mechanism delivers a performance improvement in terms of the mean slowdown by up to 282% (with an average of 125%).

Appeared in Parallel and Distributed Computing Practices Journal, 2005 .