ویراساینس.
سند علمی

Unifying Low-Rank Models for Visual Learning

Ricardo da Silveira Cabral

چکیده

Many problems in signal processing, machine learning and computer vision can be solved by learning low rank models from data. In computer vision, problems such as rigid structure from motion have been formulated as an optimization over subspaces with fixed rank. These hard -rank constraints have traditionally been imposed by a factorization that parameterizes subspaces as a product of two matrices of fixed rank. Whilst factorization approaches lead to efficient and kernelizable optimization algorithms, they have been shown to be NP-Hard in presence of missing data. Inspired by recent work in compressed sensing, hard-rank constraints have been replaced by soft-rank constraints, such as the nuclear norm regularizer. Vis-à-vis hard-rank approaches, soft-rank models are convex even in presence of missing data: but how is convex optimization solving a NP-Hard problem? This thesis addresses this question by analyzing the relationship between hard and soft rank constraints in the unsupervised factorization with missing data problem. Moreover, we extend soft rank models to weakly supervised and fully supervised learning problems in computer vision. There are four main contributions of our work: (1) The analysis of a new unified low-rank model for matrix factorization with missing data. Our model subsumes soft and hard-rank approaches and merges advantages from previous formulations, such as efficient algorithms and kernelization. It also provides justifications on the choice of algorithms and regions that guarantee convergence to global minima. (2) A deterministic “rank continuation” strategy for the NP-hard unsupervised factorization with missing data problem, that is highly competitive with the state-of-the-art and often achieves globally optimal solutions. In preliminary work, we show that this optimization strategy is applicable to other NP-hard problems which are typically relaxed to convex semidefinite programs (e.g., MAX-CUT, quadratic assignment problem). (3) A new soft-rank fully supervised robust regression model. This convex model is able to deal with noise, outliers and missing data in the input variables. (4) A new soft-rank model for weakly supervised image classification and localization. Unlike existing multiple-instance approaches for this problem, our model is convex.

اسناد مشابه

  • Learning of Generalized Low-Rank Models: A Greedy Approach

    سند علمی۲۰۱۶CoRRQuanming Yao, James T. Kwok

    Learning of low-rank matrices is fundamental to many machine learning applications. A state-of-the-art algorithm is the rank-one matrix pursuit (R1MP). However, it can only be used in matrix completion problems with the square loss. In this paper, we develop a more flexible greed

  • Greedy Learning of Generalized Low-Rank Models

    سند علمی۲۰۱۶IJCAIQuanming Yao, James T. Kwok

    Learning of low-rank matrices is fundamental to many machine learning applications. A state-ofthe-art algorithm is the rank-one matrix pursuit (R1MP). However, it can only be used in matrix completion problems with the square loss. In this paper, we develop a more flexible greedy

  • Topics in learning sparse and low-rank models of non-negative data

    سند علمی۲۰۱۵Martin Slawski

    Advances in information and measurement technology have led to a surge in prevalence of high-dimensional data. Sparse and low-rank modeling can both be seen as techniques of dimensionality reduction, which is essential for obtaining compact and interpretable representations of su

  • Low-Rank Similarity Metric Learning in High Dimensions

    سند علمی۲۰۱۵AAAIWei Liu, Cun Mu, Rongrong Ji, Shiqian Ma

    Metric learning has become a widespreadly used tool in machine learning. To reduce expensive costs brought in by increasing dimensionality, low-rank metric learning arises as it can be more economical in storage and computation. However, existing low-rank metric learning algorith

  • Time Series Forecasting via Learning Convolutionally Low-Rank Models

    مقاله‌ی دسترسی آزاد۲۰۲۲IEEE Transactions on Information Theory['Guangcan Liu', '']

    Recently, Liu and Zhang studied the rather challenging problem of time series forecasting from perspective compressed sensing. They proposed a no-learning method, named Convolution Nuclear Norm Minimization (CNNM), proved that CNNM can exactly recover future part its observed par

  • Riemannian Low-Rank Model Compression for Federated Learning with Over-the-Air Aggregation

    مقاله‌ی دسترسی آزاد۲۰۲۳IEEE Transactions on Signal Processing['Ye Xue', 'V.K.N. Lau', '']

    Low-rank model compression is a widely used technique for reducing the computational load when training machine learning models. However, existing methods often rely on relaxing low-rank constraint of weights using regularized nuclear norm penalty, which requires an appropriate h