In this report we analyze different model selection criteria that were used in statistics, reconstruction, recognition and other areas. We concentrate on criteria based on Bayesian theory, MDL theory, and predicting future residuals. Our analysis showed that these criteria often are not consistent when the number of points in the underlying data of the model is changed or if the scale of the da...

The mathematical principles of quantum theory provide a general foundation for assigning probabilities to events. This paper examines the application of these principles to the probabilistic inference problem in which hypotheses are evaluated on the basis of a sequence of evidence (observations). The probabilistic inference problem is usually addressed using Bayesian updating rules. Here we der...

One of the fundamental problems of epistemology is to say when the evidence in an agent’s possession justifies the beliefs she holds. In this paper and its prequel, we defend the Bayesian solution to this problem by appealing to the following fundamental norm: Accuracy An epistemic agent ought to minimize the inaccuracy of her partial beliefs. In the prequel, we made this norm mathematically pr...

There are different methods of statistical hypotheses testing. 1 – 4 Among them, a special place has Bayesian approach. A generalization of Bayesian rule of many hypotheses testing is given below. It consists in increasing of decision rule dimensionality with respect to the number of tested hypoteses, which allows to make decisions more differentiated than in the classical case and to state, in...

Adaptive control problems are notoriously difficult to solve even in the presence of plantspecific controllers. One way to by-pass the intractable computation of the optimal policy is to restate the adaptive control as the minimization of the relative entropy of a controller that ignores the true plant dynamics from an informed controller. The solution is given by the Bayesian control rule— a s...

We investigate the influence of different kinds of structure on the learning behaviour of a perceptron performing a classification task defined by a teacher rule. The underlying pattern distribution is permitted to have spatial correlations. The prior distribution for the teacher coupling vectors itself is assumed to be nonuniform. Thus classification tasks of quite different difficulty are inc...

This essay defends the view that inductive reasoning involves following inductive rules against objections that inductive rules are undesirable because they ignore background knowledge and unnecessary because Bayesianism is not an inductive rule. I propose that inductive rules be understood as sets of functions from data to hypotheses that are intended as solutions to inductive problems. Accord...

The certainty-factor (CF) model is a method for managing uncertainty in rule-based systems. Shortliffe and Buchanan (1975) developed the CF model in the mid-1970s for MYCIN, an expert system for the diagnosis and treatment of meningitis and infections of the blood. Since then, the CF model has become the standard approach to uncertainty management in rule-based systems. When the model was creat...

The main problem faced by all association rule/pattern mining algorithms is their production of a large number of rules which incurred a secondary mining problem; namely, mining interesting association rules/patterns. The problem is compounded by the fact that ‘common knowledge’ discovered rules are not interesting, but they are usually strong rules with high support and confidence levels – the...

We discuss a Bayesian discovery procedure for multiple comparison problems. We show that under a coherent decision theoretic framework, a loss function combining true positive and false positive counts leads to a decision rule based on a threshold of the posterior probability of the alternative. Under a semi-parametric model for the data, we show that the Bayes rule can be approximated by the o...