We present a novel alert correlation approach based on the factor analysis statistical technique for malware characterization. Our approach involves mechanically computing a set of abstract quantities, called factors, for expressing the intrusion detection system (IDS) alerts pertaining to malware instances. These factors correspond to patterns of alerts, and can be used to succinctly character...

Various grammar compression algorithms have been proposed in the last decade. A grammar compression is a restricted CFG deriving the string deterministically. An efficient grammar compression develops a smaller CFG by finding duplicated patterns and removing them. This process is just a frequent pattern discovery by grammatical inference. While we can get any frequent pattern in linear time usi...

The problem of multiple hypothesis testing arises when there are more than one hypothesis to be tested simultaneously for statistical significance. This is a very common situation in many data mining applications. For instance, assessing simultaneously the significance of all frequent itemsets of a single dataset entails a host of hypothesis, one for each itemset. A multiple hypothesis testing ...

Abstract. We develop a hierarchical approach for pattern discovery in many-body stochastic systems, motivated by challenges in guiding engineering tasks for nanopattern formation in heteroepitaxial processes. Patterns in such systems have rich morphologies at mesoscales that change dramatically as control parameters vary; typically they form as a result of microscopic particle dynamics in a com...

In this paper we investigate the general problem of discovering recurrent patterns that are embedded in categorical sequences. An important real-world problem of this nature is motif discovery in DNA sequences. There are a number of fundamental aspects of this data mining problem that can make discovery “easy” or “hard”—we characterize the difficulty of learning in this context using an analysi...

Given m groups of streams which consist of n1, . . . , nm coevolving streams in each group, we want to: (i) incrementally find local patterns within a single group, (ii) efficiently obtain global patterns across groups, and more importantly, (iii) efficiently do that in real time while limiting shared information across groups. In this paper, we present a distributed, hierarchical algorithm add...

Article history: Received 28 October 2011 Received in revised form 23 June 2013 Accepted 25 June 2013 Available online 13 July 2013 Frequent episode discovery is a popular framework for pattern discovery from sequential data. It has found many applications in domains like alarmmanagement in telecommunication networks, fault analysis in the manufacturing plants, predicting user behavior in web c...

Pattern discovery is at the heart of bioinformatics, and algorithms from computer science have been widely used for identifying biological patterns. The assumption behind pattern discovery approaches is that a pattern that occurs often enough in biological sequences/structures or is conserved across organisms is expected to play a role in defining the respective sequence’s or structure’s functi...

Traditional research in information retrieval (IR) focuses on retrieving documents. This paper introduces the idea that valuable information exists within a document collection as thematic patterns that can be found without retrieving any individual documents from the collection. This pattern information is valuable in its own right and as an aid to the IR search process, and is often not expli...