Weighted entropy profiles and a new bound, the weighted coordinates bound, on the state complexity profile of block codes are presented. These profiles and bound generalize the notion of dimension/length profile (DLP) and entropy/length profile (ELP) to block codes whose symbols are not drawn from a common alphabet set, and in particular, group codes. Likewise, the new bound may improve upon th...

We compared entropy for texts written in natural languages (English, Spanish) and artificial languages (computer software) based on a simple expression for the entropy as a function of message length and specific word diversity. Code text written in artificial languages showed higher entropy than text of similar length expressed in natural languages. Spanish texts exhibit more symbolic diversit...

A universal variable-to-fixed length algorithm for binary memoryless sources which converges to the entropy of the source at the optimal rate is known. We study the problem of universal variable-to-fixed length coding for the class of Markov sources with finite alphabets. We give an upper bound on the performance of the code for large dictionary sizes and show that the code is optimal in the se...

For an entropy-coded Joint Photographic Experts Group (JPEG) image, a transmission error in a codeword will not only affect the underlying codeword but also may affect subsequent codewords, resulting in a great degradation of the received image. In this study, an error resilient coding scheme for JPEG image transmission based on data embedding and side-match vector quantization (VQ) is proposed...

We will show that • the entropy for a random variable gives a lower bound on the number of bits needed per character for a binary coding • Huffman codes are optimal in the average number of bits used per character among binary codes • the average bits per character used by Huffman codes is close to the entropy of the underlying random variable • one can get arbitrarily close to the entropy of a...

In this paper, we consider the problem of variable-length source coding allowing errors. The exponential moment of the codeword length is analyzed in the non-asymptotic regime and in the asymptotic regime. Our results show that the smooth Rényi entropy characterizes the optimal exponential moment of the codeword length. Index Terms ε source coding, exponential moment, the smooth Rényi entropy, ...

We present a new class of MDS array codes of size n n (n a prime number) called X-Code. The X-Codes are of minimum column distance 3, namely, they can correct either one column error or two column erasures. The key novelty in X-code is that it has a simple geometrical construction which achieves encoding/update optimal complexity, namely, a change of any single information bit a ects exactly tw...

Second order lower bounds for the entropy function expressed in terms of the index of coincidence are derived. Equivalently, these bounds involve entropy and Rényi entropy of order 2. The constants found either explicitly or implicitly are best possible in a natural sense. The inequalities developed originated with certain problems in universal prediction and coding which are briefly discussed.

Approaching a multi-codebook system as a codebook of codebooks, weighted universal vector quantization (WUVQ) uses traditional codeword design techniques to design locally optimal multi-codebook systems. Application of this technique to a sequence of medical images produces a 10.3 dB improvement over standard full search vector quantization followed by entropy coding at the cost of increased co...

We present a feature extraction and tracking method which is based on pure rich texture analysis and has robustness under irregular conditions, such as illumination change, deformation of objects and so on. The method is composed mainly of two algorithms: Entropy Filtering and Orientation Code Matching (OCM). Entropy filter points up areas of images being messy distribution of orientation codes...