OpenDist: Incremental Software Distribution

OpenDist provides efficient procedures and tools to synchronize our software file servers. This simple goal becomes challenging because of the size and complexity of supported software, the diversity of platforms, and because of network constraints. Our current solution is based on rdist (1) [1]. However, it is not possible anymore to synchronize file servers nightly, because it takes several days just to compare distant servers. We have analyzed the update process to find bottlenecks in the current solution. We measured the effects of network bandwidth and latency on rdist. We created statistics on the number of files and file sizes within all software packages. We found that not only the line speed, but also the line delay contributes substantially to the overall update time. Our measurements revealed that adding a compression mode to rdist would not have solved our problem, so we decided to look for a new solution. We have compiled a list of requirements for evaluating software distribution solutions. Based on these requirements, we evaluated both commercial and freely available tools. None of the tools fulfilled our most important requirements, so we implemented our own solution. In the following we will describe the overall architecture of the toolset and present performance figures for the distribution engine that replaces rdist. The results of the prototype implementation are promising. We conclude with a description of the next steps for enhancing the OpenDist toolset.