Improving the Performance of I/O-Intensive Applications on Clusters of Workstations
Xiao Qin1 Hong Jiang2 Yifeng Zhu2 David R. Swanson21 Department of Computer Science New Mexico Institute of Mining and Technology 801 Leroy Place, Socorro, New Mexico 87801 2 Department of Computer Science and Engineering
University of Nebraska-Lincoln
Lincoln, NE 68588-0115
Load balancing in a workstation-based cluster system has been investigated extensively, mainly focusing on the effective usage of global CPU and memory resources. However, if a significant portion of applications running in the system is I/O-intensive, traditional load balancing policies that focus on CPU and memory usage may ignore I/O load imbalance, thereby causing system performance to decrease substantially. In this paper, two I/O-aware load-balancing schemes, referred to as IOCM and WAL-PM, are presented to improve the overall performance of a cluster system with a general and practical workload including I/O activities. The proposed schemes dynamically detect I/O load imbalance on nodes of a cluster, and determine whether to migrate some I/O load or running jobs from overloaded nodes to other less- or under-loaded nodes. The current running jobs are eligible to be migrated in WAL-PM only if it overall performance improves. Besides balancing I/O load, the scheme judiciously takes into account both CPU and memory load sharing in the system, thereby maintaining the same level of performance as existing schemes when I/O load is low or well balanced. Extensive trace-driven simulations for both synthetic and real I/O-intensive applications show that: (1) Compared with existing schemes that only consider CPU and memory, the proposed schemes improve performance with respect to mean slowdown by up to a factor of 20; (2) When compared to the existing approaches that only consider I/O with non-preemptive job migrations, the proposed schemes achieve improvement in mean slowdown by up to a factor of 10; (3) Under CPU-memory intensive workload, our scheme improves the performance over the existing approaches that only consider I/O by up to 47.5%.
Appeared in Cluster Computing: The Journal of Networks, Software Tools and Applications, Special Issue on Cluster Computing in Science and Engineering, vol. 9, no. 3, pp. 297-311, July 2006.