Maximum Entropy (MaxEnt) model has been proven to be a very effective approach in the topic classification task, where a specific topic from a pre-defined topic set will be assigned to each sentence. Although it is originally developed based on the motivation of maximizing the conditional probability entropy under certain constraints, MaxEnt model is indeed an exponential distribution model tha...

The k-support norm has been recently introduced to perform correlated sparsity regularization [1]. Although Argyriou et al. only reported experiments using squared loss, here we apply it to several other commonly used settings resulting in novel machine learning algorithms with interesting and familiar limit cases. Source code for the algorithms described here is available from https://github.c...

Measuring independence and k-wise independence is a fundamental problem that has multiple applications and it has been the subject of intensive research during the last decade (see, among others, the recent work of Batu, Fortnow, Fischer, Kumar, Rubinfeld and White [11] and of Alon, Andoni, Kaufman, Matulef, Rubinfeld and Xie [2] ). In the streaming environment, this problem was first addressed...

The decision tree classifier is a well-known methodology for classification. It is widely accepted that a fully grown tree is usually over-fit to the training data and thus should be pruned back. In this paper, we analyze the overtraining issue theoretically using an the k-norm risk estimation approach with Lidstone’s Estimate. Our analysis allows the deeper understanding of decision tree class...

In one-bit compressed sensing (1-bit CS), one attempts to estimate a structured parameter (signal) only using the sign of suitable linear measurements. In this paper, we investigate 1-bit CS problems for sparse signals using the recently proposed k-support norm. We show that the new estimator has a closed-form solution, so no optimization is needed. We establish consistency and recovery guarant...

The k-means algorithm is one of the most often used method for data clustering. However, the standard k-means can only be applied in the original feature space. The kernel k-means, which extends k-means into the kernel space, can be used to capture the non-linear structure and identify arbitrarily shaped clusters. Since both the standard k-means and kernel k-means apply the squared error to mea...

The Radon transform constitutes a fundamental concept for x-rays in medical imaging, and more generally, in image reconstruction problems from diverse fields. The Radon transform in Euclidean spaces assigns to functions their integrals over affine hyperplanes. This can be extended so that the integration is performed on affine k-dimensional subspaces, the corresponding transform is called k-pla...

Positive definite matrix approximation with a condition number constraint is an optimization problem to find the nearest positive definite matrix whose condition number is smaller than a given constant. We demonstrate that this problem can be converted to a simpler one in this note when we use a unitary similarity invariant norm as a metric. We can especially convert it to a univariate piecewis...

Network inference is an important problem in a variety of domains. In computational biology, gene interaction networks can be learned using the mRNA expression levels of genes. These networks capture how genes influence each other and can be used to identify potential malfunctions. In the context of social network analysis, network inference refers to the problem of inferring underlying network...