Properties of scalar quantization with rth power distortion and constrained Rényi entropy of order α ∈ (0, 1) are investigated. For an asymptotically (high-rate) optimal sequence of quantizers, the contribution to the Rényi entropy due to source values in a fixed interval is identified in terms of the “entropy density” of the quantizer sequence. This extends results related to the well-known po...

In the paper, a theoretical work is done for investigating effects of splitting data sequence into packs of data set. We proved that a partitioning of data sequence is possible to find such that the entropy rate at each subsequence is lower than entropy rate of the source. Effects of sequence partitioning on overall compression rate are argued on the bases of partitioning statistics, and then, ...

A new interpretation of transform coding is developed that downplays quantization and emphasizes entropy coding, allowing a comparison of entropy coding methods with different memory requirements. With conventional transform coding, based on computing Karhunen–Loève transform coefficients and then quantizing them, vector entropy coding can be replaced by scalar entropy coding without an increas...

Transfer entropy is a measure of the magnitude and the direction of information flow between jointly distributed stochastic processes. In recent years, its permutation analogues are considered in the literature to estimate the transfer entropy by counting the number of occurrences of orderings of values, not the values themselves. It has been suggested that the method of permutation is easy to ...

We introduce the concept of entropy rate to characterize a diffusion process on a complex network. The entropy rate represents the minimal amount of information necessary to describe the diffusion on the network, and is a quantity extremely sensitive to the network topology and dynamics. By opportunely tuning the kind of diffusion, the entropy rate allows one to extract different properties of ...