Using Moldability to Improve the Performance of Supercomputer Jobs PhD Thesis

Distributed-memory parallel supercomputers are an important platform for the execution of high-performance parallel jobs. In order to submit a job for execution in most supercomputers, one has to specify the number of processors to be allocated to the job. However, most parallel jobs in production today are moldable. A job is moldable when the number of processors it needs to execute can vary, although such a number has to be fixed before the job starts executing. Consequently, users have to decide how many processors to request whenever they submit a moldable job. In this thesis, we show that the request that submits a moldable job can be automatically selected in a way that often reduces the job’s turn-around time. The turn-around time of a job is the time elapsed between the job’s submission and its completion. More precisely, we will introduce and evaluate SA, an application scheduler that chooses which request to use to submit a moldable job on behalf of the user. The user provides SA with a set of possible requests that can be used to submit a given moldable job. SA estimates the turn-around time of each request based on the current state of the supercomputer, and then forwards to the supercomputer the request with the smallest expected turn-around time. Users are thus relieved by SA of a task unrelated with their final goals, namely that of selecting which request to use. Moreover and more importantly, SA often improves the turn-around time of the job under a variety of conditions. The conditions under which SA was studied cover variations on the characteristics of the job, the state of the supercomputer, and the information available to SA. The emergent behavior generated by having most jobs using SA to craft their requests was also investigated.