Searching for Dependencies in Bayesian Classifiers

Naive Bayesian classi ers which make independence assumptions perform remarkably well on some data sets but poorly on others. We explore ways to improve the Bayesian classi er by searching for dependencies among attributes. We propose and evaluate two algorithms for detecting dependencies among attributes and show that the backward sequential elimination and joining algorithm provides the most improvement over the naive Bayesian classi er. The domains on which the most improvement occurs are those domains on which the naive Bayesian classi er is signi cantly less accurate than a decision tree learner. This suggests that the attributes used in some common databases are not independent conditioned on the class and that the violations of the independence assumption that a ect the accuracy of the classi er can be detected from training data.