Analysis of Krylov subspace methods: Moments, error estimators, numerical stability and unexpected consequences

Krylov subspace methods play an important role in many areas of scientific computing, including numerical solution of linear algebraic systems arising from discretization of partial differential and/or integral equations. By their nature they represent model reductions based on matching moments. Such view naturally complements, to our opinion, the standard description using the projection processes framework, and it reveals in an instructive way the nonlinear character of Krylov subspace methods. In order to explain the deep link between matching moments and Krylov subspace methods, we present a modification of the classical Stieltjes moment problem, and show that its solution is given: • in the language of orthogonal polynomials by Gauss-Christoffel quadrature; • in the algebraic matrix form by the conjugate gradient method (CG). In order to allow straightforward generalizations, we use the Vorobyev vector moment problem, and present some basic Krylov subspace iterations as moment matching model reduction. Numerical stability analysis for iterative methods is considered inherently more difficult than for direct methods, and the results are harder to interpret. Within the last decades, an interest in rounding errors analysis for Krylov subspace methods has, however, led to many important results. It has been shown that rounding errors can delay convergence and significantly increase the cost of computation even when stopping at a modest accuracy level. The delay phenomenon in CG has been linked to the finite precision behaviour of the Lanczos method, and it has been understood. Estimates of the energy norm of the error in CG have been justified by a mathematically rigorous rounding error analysis. Results on limiting accuracy of various methods have offered a guidance for their implementations. On the other hand, rounding error analysis for seemingly attractive implementations used in practice has revealed their hidden potentially dangerous numerical instabilities. We illustrate on two recent developments that rounding error analysis can also have a surprising impact in other areas of research. First, we show that GaussChristoffel quadrature for a small number of quadrature nodes can be highly sensitive to small changes in the distribution function, and we describe the relationship of the sensitivity of Gauss-Christoffel quadrature to the convergence properties of the CG and Lanczos methods in finite precision arithmetic. Second, we explain how rounding error analysis of the generalized minimal residual method (GMRES) has resulted in the core problem decomposition of orthogonally invariant linear algebraic approximation problems. In both cases, rounding error analysis for an iterative method has inspired questions in the areas far from the original problem. Their solution has offered a new insight to mathematical foundations of the classical problems.