We consider practical methods for adding constraints to the K-Means clustering algorithm in order to avoid local solutions with empty clusters or clusters having very few points. We often observe this phenomena when applying K-Means to datasets where the number of dimensions is n 10 and the number of desired clusters is k 20. We propose explicitly adding k constraints to the underlying clusteri...

To achieve an insightful clustering of multivariate data, we propose subspace K-means. Its central idea is to model the centroids and cluster residuals in reduced spaces, which allows for dealing with a wide range of cluster types and yields rich interpretations of the clusters. We review the existing related clustering methods, including deterministic, stochastic, and unsupervised learning app...

The sets Sj are the sets of points to which μj is the closest center. In each step of the algorithm the potential function is reduced. Let’s examine that. First, if the set of centers μj are fixed, the best assignment is clearly the one which assigns each data point to its closest center. Also, assume that μ is the center of a set of points S. Then, if we move μ to 1 |S| ∑ i∈S xi then we only r...

In this paper we provide a fully distributed implementation of the k-means clustering algorithm, intended for wireless sensor networks where each agent is endowed with a possibly high-dimensional observation (e.g., position, humidity, temperature, etc.). The proposed algorithm, by means of one-hop communication, partitions the agents into measure-dependent groups that have small ingroup and lar...

We study how to learn multiple dictionaries from a dataset, and approximate any data point by the sum of the codewords each chosen from the corresponding dictionary. Although theoretically low approximation errors can be achieved by the global solution, an effective solution has not been well studied in practice. To solve the problem, we propose a simple yet effective algorithm Group K-Means. S...