Hybrid Programming with OpenMP and MPI

The basic aims of parallel programming are to decrease the runtime for the solution to a problem and increase the size of the problem that can be solved. The conventional parallel programming practices involve a a pure OpenMP implementation on a shared memory architecture (Fig. 1) or a pure MPI implementation on distributed memory computer architectures (Fig. 2). The largest and fastest computers today employ both shared and distributed memory architecture (Fig. 3). This gives a exibility in tuning the parallelism in the programs to generate maximum e ciency and balance the computational and communication loads in the program. A wise implementation of hybrid parallel programs utilizing the modern hybrid computer hardware can generate massive speedups in the otherwise pure MPI and pure OpenMP implementations. A brief introduction to OpenMP, MPI and Hybrid programs is given in the following subsections.