In this project, I describe how I address the ICML 2004 Physiological Data Modeling Contest. For the gender prediction task, I compressed the large entry-based dataset to small session-based dataset and manually devised 90 features using a histogram method. Weighted naive Bayes (WNB) which is an extension of naive Bayes was applied and Markov Chain Monte Carlo was combined to solve the weight u...

Bayesian network classifiers are used in many fields, and one common class of classifiers are naive Bayes classifiers. In this paper, we introduce an approach for reasoning about Bayesian network classifiers in which we explicitly convert them into Ordered Decision Diagrams (ODDs), which are then used to reason about the properties of these classifiers. Specifically, we present an algorithm for...

Despite the simplicity of the Naive Bayes classifier, it has continued to perform well against more sophisticated newcomers and has remained, therefore, of great interest to the machine learning community. Of numerous approaches to refining the naive Bayes classifier, attribute weighting has received less attention than it warrants. Most approaches, perhaps influenced by attribute weighting in ...

The Naive Bayes classifier exists in different versions. One version, called multi-variate Bernoulli or binary independence model, uses binary word occurrence vectors, while the multinomial model uses word frequency counts. Many publications cite this difference as the main reason for the superior performance of the multinomial Naive Bayes classifier. We argue that this is not true. We show tha...

A maximum entropy classi er can be used to extract sentences from documents. Experiments using technical documents show that such a classi er tends to treat features in a categorical manner. This results in performance that is worse than when extracting sentences using a naive Bayes classi er. Addition of an optimised prior to the maximum entropy classi er improves performance over and above th...

Naive Bayes classifier is the simplest among Bayesian Network classifiers. It has shown to be very efficient on a variety of data classification problems. However, the strong assumption that all features are conditionally independent given the class is often violated on many real world applications. Therefore, improvement of the Naive Bayes classifier by alleviating the feature independence ass...

In many machine learning applications that deal with sequences, there is a need for learning algorithms that can effectively utilize the hierarchical grouping of words. We introduce Word Taxonomy guided Naive Bayes Learner for the Multinomial Event Model (WTNBL-MN) that exploits word taxonomy to generate compact classifiers, and Word Taxonomy Learner (WTL) for automated construction of word tax...

Bayesian networks are powerful tools for decision and reasoning under uncertainty. A very simple form of these networks is called naive Bayes, which is particularly efficient for learning and inference tasks. This paper offers an experimental study of the use of naive Bayes in intrusion detection. We show that eventhough they have a simple structure, naive Bayes provide satisfactory results. We...

Naive Bayes is an eecient and eeective learning algorithm, but previous results show that its representation ability is severely limited since it can only represent certain linearly separable functions in the binary domain. We give necessary and suucient conditions on linearly separable functions in the binary domain to be learnable by Naive Bayes under uniform representation. We then show that...

For this project, I implement 3 popular text classification algorithms on spam detection, namely AdaBoost, Support Vector Machines and Naive Bayes. The performance are evaluated on some testing datasets. All experiments are done in Matlab. The experimental result is, all 3 algorithms have a satisfactory performance on spam detection. In term of accuracy, Adaboost has the best error bound. On th...