Piecewise Linear Histograms for Selectivity Estimation

Selectivity estimation of queries is of critical importance to query optimization. In order to get accurate estimations, database management systems must maintain statistics to capture the underlying data distribution. Histograms are extensively used in commercial database systems for this purpose. Most current histogram techniques make the assumption that all values in a single bucket appear with the same frequency, which rarely holds true in practice. Though wavelet-based histograms have recently appear as a strong alternative, several problems inherent with the wavelet techniques prevent it from practical use in database systems. In this paper, we propose a new type of histograms called piecewise linear histograms. Frequencies of attribute values in a bucket of the piecewise linear histogram is t by a line using linear least squares regression, and the coeecients are stored as synopsis of the underlying data distribution. Moreover, since nding the best partition of the domain into buckets is a NP-hard problem, we propose a heuris-tic to eeciently determine the boundaries of buckets. Experimental results show that the piecewise linear histograms signiicantly outperform wavelet-based histograms.