Parallelizing a level 3 BLAS library for LAN-connected workstations

shared by more than one user at a time and the user number for a workstation might change as time goes on. Therefore, each workstation would provide unequal and time-varying computing power to a parallel application. Computing models are different in LAN-connected workstations and conventional parallel computers, where a user may allocate a group of processors for dedicated use in a time period. Dynamic load balancing mechanisms are necessary for parallel applications to run efficiently on LAN-connected workstations. However, most existing parallel implementations of BLAS were designed based on conventional parallel computers, without concern for the need for dynamic load balancing capabilities on LAN-connected workstations. This paper explores the parallelization of Level 3 BLAS for LAN-connected workstations taking dynamic load balancing into consideration. Dynamic load balancing has received a great deal of attention in the literature [5, 6, 18, 19]. The basic idea of dynamic load balancing is to balance the workloads on all processors to increase the system throughput or reduce the wall clock execution time of an application. One approach is to allow the load to be migrated from heavy nodes to light ones. Our approach divides a task into subtasks whose number is larger than the processors’ and then assigns each processor one subtask at a time. Only when a processor completes its subtask is it assigned a new one. The approach achieves dynamic load balancing through proper data partition and task assignment. We have conducted several experiments to investigate various data partition methods. According to the results of experiments which will be described in detail in later sections, we propose the dynamic column-blocking method as the best data partition method to run parallel Level 3 BLAS efficiently on LAN-connected workstations. We have implemented a parallel LU factorization routine using our parallel Level 3 BLAS to show the effectiveness of our parallelizing method and discuss the issue of parallel library interface. In the next section we give high level descriptions of parallel implementations of Level 3 BLAS. Section 3 discusses the importance, advantages, and issues of LANconnected workstations. Section 4 presents various data distribution methods, our experiments and comparative analysis. Section 5 discusses the implementation of LU JOURNAL OF PARALLEL AND DISTRIBUTED COMPUTING 38, 28–36 (1996) ARTICLE NO. 0126