Upper bounds for convergence rates of vector extrapolation methods on linear systems with initial iterations

The application of the minimal polynomial extrapolation (MPE) and the reduced rank extrapolation (RRE) to a vector sequence obtained by the linear iterative technique x(sub j) + 1 = Ax(sub j) = b,j = 1,2,..., is considered. Both methods produce a two dimensional array of approximations s(sub n,k) to the solution of the system (I - A)x = b. Here, s(sub n,k) is obtained from the vectors x(sub j), n is less than or equal to j is less than or equal to n + k + 1. It was observed in an earlier publication by the first author that the sequence s(sub n,k), k = 1,2,..., for n greater than 0, but fixed, possesses better convergence properties than the sequence s(sub 0,k), k = 1,2,.... A detailed theoretical explanation for this phenomenon is provided in the present work. This explanation is heavily based on approximations by incomplete polynomials. It is demonstrated by numerical examples when the matrix A is sparse that cycling with s(sub n,k) for n greater than 0, but fixed, produces better convergence rates and costs less computationally than cycling with s(sub 0,k). It is also illustrated numerically with a convection-diffusion problem that the former may produce excellent results where the latter may fail completely. As has been shown in an earlier publication, the results produced by s(sub 0,k) are identical to the corresponding results obtained by applying the Arnoldi method or generalized minimal residual scheme (GMRES) to the system (I - A)x = b.