For a large class of stationary probability measures on AN, where A is a finite alphabet, we compute the specific Rényi entropy of order α and the specific guesswork moments of order β > −1. We show that the specific guesswork moment of order β equals the specific Rényi entropy of order α = 1/(1+ β) multiplied by β. The method is based on energy–entropy estimates suggested by statistical physic...

A hidden Markov process (HMP) is a discrete-time finite-state homogeneous Markov chain observed through a discrete-time memoryless invariant channel. The non-asymptotic equipartition property (NEP) is a bound on the probability of the sample entropy deviating from the entropy rate of a stochastic process, so it can be viewed as a refinement of Shannon-McMillan-Breiman theorem. In this report, w...

For all p > 1, we demonstrate the existence of quantum channels with non-multiplicative maximal output p-norms. Equivalently, for all p > 1, the minimum output Rényi entropy of order p of a quantum channel is not additive. The violations found are large; in all cases, the minimum output Rényi entropy of order p for a product channel need not be significantly greater than the minimum output entr...

Kolmogorov complexity and Shannon entropy are conceptually different measures. However, for any recursive probability distribution, the expected value of Kolmogorov complexity equals its Shannon entropy, up to a constant. We study if a similar relationship holds for Rényi and Tsallis entropies of order α, showing that it only holds for α = 1. Regarding a time-bounded analogue relationship, we s...

This paper introduces a new class of fractal dimension measures which we call relative multifractal measures. The relative multifractal measures developed are formed through a melding of the Rényi dimension spectrum, which is based on the Rényi generalized entropy, and relative entropy as given with the Kullback-Leibler distance. This new class of multifractal measures is then used to find the ...

The Shannon entropy is a widely used summary statistic, for example, network traffic measurement, anomaly detection, neural computations, spike trains, etc. This study focuses on estimating Shannon entropy of data streams. It is known that Shannon entropy can be approximated by Rényi entropy or Tsallis entropy, which are both functions of the αth frequency moments and approach Shannon entropy a...

Information theoretic cryptography is discussed based on conditional Rényi entropies. Our discussion focuses not only on cryptography but also on the definitions of conditional Rényi entropies and the related information theoretic inequalities. First, we revisit conditional Rényi entropies, and clarify what kind of properties are required and actually satisfied. Then, we propose security criter...

We consider optimal scalar quantization with rth power distortion and constrained Rényi entropy of order α. For sources with absolutely continuous distributions the high rate asymptotics of the quantizer distortion has long been known for α = 0 (fixed-rate quantization) and α = 1 (entropyconstrained quantization). These results have recently been extended to quantization with Rényi entropy cons...

In this paper, we explore the issue of network infrastructure and architecture in Deep Space Communication Networks (DSCNs). Firstly, we present a typical structural sample infrastructure of our vision of DSCNs by separating the whole network into three major parts. We then use hypergraph theory and Rényi entropy paradigm to formulate Relaying Networks for Deep Space Communications (RNDSC) as a...

We derive a new upper bound for Eve’s information in secret key generation from a common random number without communication. This bound improves on Bennett[7]’s bound based on the Rényi entropy of order 2 because the bound obtained here uses the Rényi entropy of order 1+s for s ∈ [0, 1]. This bound is applied to a wire-tap channel. Then, we derive an exponential upper bound for Eve’s informati...