The optimal design of data acquisition is not obvious in Bayesian Network models. The dependency structure may vary dramatically, which makes learning and information evaluation complicated and sometimes non-intuitive. Our application, and the motivation for working on this topic, is prospect selection for petroleum exploration in the North Sea. Here, the data gathering is often carried out dur...

The concept of Shannon entropy as a measure of disorder is introduced and the generalisations of the Rényi and Tsallis entropy are motivated and defined. A number of different estimators for Shannon, Rényi and Tsallis entropy are defined in the theoretical part and compared by simulation in the practical part. In this work the nearest neighbour estimator presented in Leonenko and Pronzato (2010...

Estimation of Shannon and Rényi entropies of unknown discrete distributions is a fundamental problem in statistical property testing and an active research topic in both theoretical computer science and information theory. Tight bounds on the number of samples to estimate these entropies have been established in the classical setting, while little is known about their quantum counterparts. In t...

We apply Shannon entropy as a measure of information content in survey data, and define information efficiency as the empirical entropy divided by the maximum attainable entropy. In a case study of the Norwegian Function Assessment Scale, entropy calculations show that the 5-point response version has higher information efficiency than the 4-point version.

We presents a refined multiscale Shannon entropy for analyzing electroencephalogram (EEG), which reflects the underlying dynamics of EEG over multiple scales. The rationale behind this method is that neurological signals such as EEG possess distinct dynamics over different spectral modes. To deal with the nonlinear and nonstationary nature of EEG, the recently developed empirical mode decomposi...

The physical concept of entropy as it is used in thermodynamics is related to the mathematical formulation of a Shannon entropy. Usually only the Shannon entropy of equilibrium distributions such as a canonical distribution is considered. Large deviations statistics goes beyond that framework. Entropies are considered for arbitrary distributions or physical states, and they describe, e.g., \how...

Data mining is an interdisciplinary field of computer science and is referred to extracting or mining knowledge from large amounts of data. Classification is one of the data mining techniques that maps the data into the predefined classes and groups. It is used to predict group membership for data instances. There are many areas that adapt Data mining techniques such as medical, marketing, tele...

The global maximum of an entropy function with different decision levels for a three-level scalar quantizer performed after a discrete wavelet transform was derived. Herein, we considered the case of entropy-constrained scalar quantization capable of avoiding many compression ratio reductions as the mean squared error was minimized. We also dealt with the problem of minimum entropy with an erro...

This work is focused on the numerical determination of Shannon probabilistic entropy for MEMS devices exhibiting some uncertainty in their structural response. a universal measure statistical or stochastic disorder static deformation dynamic vibrations engineering systems and available both continuous discrete distributions functions parameters. An interval algorithm using Monte Carlo simulatio...