On the Behavior of the Conjugate-gradient Method on Ill-conditioned Problems

We study the behavior of the conjugate-gradient method for solving a set of linear equations, where the matrix is symmetric and positive definite with one set of eigenvalues that are large and the remaining are small. We characterize the behavior of the residuals associated with the large eigenvalues throughout the iterations, and also characterize the behavior of the residuals associated with the small eigenvalues for the early iterations. Our results show that the residuals associated with the large eigenvalues are made small first, without changing very much the residuals associated with the small eigenvalues. A conclusion is that the ill-conditioning of the matrix is not reflected in the conjugate-gradient iterations until the residuals associated with the large eigenvalues have been made small.