Auto RAID hierarchical storage system

once the total number of disks gets large, it becomes more cost-effective to employ an array controller that uses some form of partial redundancy (such as parity) to protect the data it stores. Such RAIDs (for Redundant Arrays of Independent Disks) were first described in the early 1980s [Lawlor81, Park86], and popularized by the work of a group at UC Berkeley [Patterson88, Patterson89]. By storing only partial redundancy for the data, the incremental cost of the desired high availability is reduced to as little as 1/N of the total storage-capacity cost (where N is the number of disks in the array), plus the cost of the array controller itself. The UC Berkeley RAID terminology has a number of different RAID levels, each one representing a different amount of redundancy and a placement rule for the redundant data. Most disk array products implement RAID level 3 or 5. In RAID level 3, host data blocks are bit-or byte-interleaved across a set of data disks, and parity is stored on a dedicated data disk (see Figure 1). In RAID level 5, host data blocks are block-interleaved across the disks, and the disk on which the parity block is stored rotates in round-robin fashion for different stripes. Both hardware and software RAID products are available from many vendors. Unfortunately, current RAID arrays are often difficult to use [Chen93]: the different RAID levels have different performance characteristics, and perform well only for a relatively narrow range of workloads. To accommodate this, RAID systems typically offer a great many configuration parameters: data-and parity-layout choice, stripe depth, stripe width, cache sizes and write-back policies, etc. Setting these correctly is difficult, and requires knowledge of workload characteristics that most people are unable (and unwilling) to acquire. As a result, setting up a RAID array is often a daunting task, that requires skilled, expensive people and— in too many cases—a painful process of trial and error. Making the wrong choice has two costs: the resulting system may perform poorly; and changing from one layout to another almost inevitably requires copying data off to a second device, Figure 1. Data and parity layout for two different RAID levels. data parity data parity a. RAID 3. Abstract Configuring redundant disk arrays is a black art. To properly configure an array, a system administrator must understand the details of both the array and the workload it will support; incorrect understanding of …