Randomized Iterative Solver as Iterative Refinement: A Simple Fix Towards Backward Stability

Iterative sketching and sketch-and-precondition are well-established randomized algorithms for solving large-scale over-determined linear least-squares problems. In this paper, we introduce a new perspective that interprets Iterative Sketching and Sketching-and-Precondition as forms of Iterative Refinement. We also examine the numerical stability of two distinct refinement strategies: iterative refinement and recursive refinement, which progressively improve the accuracy of a sketched linear solver. Building on this insight, we propose a novel algorithm, Sketched Iterative and Recursive Refinement (SIRR), which combines both refinement methods. SIRR aims to develop a stable randomized algorithm for least-squares problems, ensuring that the computed solution exactly solves a modified least-squares system, where the coefficient matrix deviates only slightly from the original matrix. To the best of our knowledge, SIRR is the first asymptotically fast, single-stage randomized least-squares solver that achieves both forward and backward stability.