Content Defined Chunking (CDC) is an important component in data deduplication, which affects both the deduplication ratio as well as deduplication performance. The sliding-window-based CDC algorithm and its variants have been the most popular CDC algorithms for the last 15 years. However, their performance is limited in certain application scenarios since they have to slide byte by byte. The a...

The curtailment of disambiguation decisions is crucial for eecient and precise analysis of sentences in the view of parsing as making a sequence of disambiguation. In this paper we propose three types of chunking in Korean for purpose of the reduction of search space. We present the parsing method based on chunking and the association among chunks and words in a chunk. Test was conducted on 237...

This paper illustrates a technique of shallow parsing named “text chunking” whereby “parse incompleteness” is reinterpreted as “parse underspecification”. A text is chunked into structured units which can be identified with certainty on the basis of available knowledge. The chunking process stops at that level of granularity beyond which the analysis gets undecidable. We argue that a chunked sy...

Training a Support Vector Machine (SVM) requires the solution of a very large quadratic programming (QP) problem. This paper proposes an algorithm for training SVMs: Sequential Minimal Optimization, or SMO. SMO breaks the large QP problem into a series of smallest possible QP problems which are analytically solvable. Thus, SMO does not require a numerical QP library. SMO’s computation time is d...

One of the challenging problems in Thai NLP is to manage a problem on a syntactical analysis of a long sentence. This paper applies conditional random field and categorical grammar to develop a chunking method, which can group words into larger unit. Based on the experiment, we found the impressive results. We gain around 74.17% on sentence level chunking. Furthermore we got a more correct pars...

In the present study, we investigated possible influences on the unitization of responses. In Experiments 1, 2, 3, and 6, we found that when the same small fragment (i.e., a few consecutive responses in a sequence) was presented as part of two larger sequences, participants responded to it faster when it was part of the sequence that was presented more often. This indicates that chunking can be...

This paper proposes a boosting algorithm that uses a semi-Markov perceptron. The training algorithm repeats the training of a semi-Markov model and the update of the weights of training samples. In the boosting, training samples that are incorrectly segmented or labeled have large weights. Such training samples are aggressively learned in the training of the semi-Markov perceptron because the w...

Planning in a dynamic environment is a complex task that requires several issues to be investigated in order to manage the associated search complexity. In this paper, an adaptive behavior that integrates planning with learning is presented. The former is performed adopting a hierarchical approach, interleaved with execution. The latter, devised to identify new abstract operators, adopts a chun...

Parsing is often seen as a combinatorial problem. It is not due to the properties of the natural languages, but due to the parsing strategies. This paper investigates a Constrained Grammar extracted from a Treebank and applies it in a non-combinatorial partial parser. This parser is a simpler version of a chunking-and-raising parser. The chunking and raising actions can be done in linear time. ...

In this paper we present an integrated system for tagging and chunking texts from a certain language. The approach is based on stochastic nite-state models that are learnt automatically. This includes bigram models or nite-state automata learnt using grammatical inference techniques. As the models involved in our system are learnt automatically, this is a very exible and portable system. In ord...