Homomorphic Self-repairing Codes for Agile Maintenance of Distributed Storage Systems

Distributed data storage systems are essential to deal with the need to store massive volumes of data. In order to make such a system fault-tolerant, some form of redundancy becomes crucial. There are various overheads that are incurred due to such redundancy most prominent ones being overheads in terms of storage space and maintenance bandwidth requirements. Erasure codes provide a storage efficient alternative to replication based redundancy in storage systems. They however entail high communication overhead for maintenance in a networked setting, when some of the encoded fragments are lost due to failure of storage devices and need to be replenished in new ones. Such overheads arise from the fundamental need in storage systems to recreate (or keep separately) first a copy of the whole object before any individual encoded fragment can be generated and replenished. Traditional erasure codes, originally designed for communication over lossy channels, are optimized for recreation of the original message (object), but not for regeneration of individual lost encoded parts. We propose as an alternative a new family of erasure codes called self-repairing codes (SRC) taking into account the peculiarities of distributed storage systems, specifically to improve the maintenance process. SRC has the following salient features: (a) encoded fragments can be repaired directly from other subsets of encoded fragments by downloading less data than the size of the complete object, ensuring that (b) a fragment is repaired from a fixed number of encoded fragments, the number depending only on how many encoded blocks are missing and independent of which specific blocks are missing. This paper lays the foundations by defining the novel self-repairing codes, elaborating why the defined characteristics are desirable for distributed storage systems. Then a concrete family of such code, namely, homomorphic self-repairing codes (HSRC) are proposed and various aspects and properties of the same are studied in detail and compared quantitatively or qualitatively (as may be suitable) with respect to other codes including traditional erasure codes as well as other recent codes designed specifically for storage applications.