Block-wise ADMM with a Relaxation Factor for Multiple-block Convex Programming

It has been shown that the alternating direction method of multipliers (ADMM) is not necessarily convergent when it is directly extended to a multiple-block linearly constrained convex minimization model with an objective function in the sum of more than two functions without coupled variables. Most recently, we proposed the block-wise ADMM, which was obtained by regrouping the variables and functions of such a model as two blocks and then applying the original ADMM in block-wise. This note is a further study on this topic, with the purpose of showing that a well-known relaxation factor for iteratively updating the Lagrangian multiplier of the original ADMM can also be used in the block-wise ADMM. We thus propose the block-wise ADMM with a relaxation factor for the multiple-block convex minimization model. Like the block-wise ADMM, we also suggest further decomposing the resulting subproblems and regularizing them by proximal terms to ensure the convergence. For the block-wise ADMM with a relaxation factor, its convergence and worst-case convergence rate measured by the iteration complexity in the ergodic sense are derived.