This book offers a detailed treatment of the mathematical theory of Krylov subspace methods with focus on solving systems of linear algebraic equations. Starting from the idea of projections, Krylov subspace methods are characterised by their orthogonality and minimisation properties. Projections onto highly nonlinear Krylov subspaces can be linked with the underlying problem of moments, and therefore Krylov subspace methods can be viewed as matching moments model reduction. This allows enlightening reformulations of questions from matrix computations into the language of orthogonal polynomials, Gauss–Christoffel quadrature, continued fractions, and, more generally, of Vorobyev method of moments. Using the concept of cyclic invariant subspaces conditions are studied that allow generation of orthogonal Krylov subspace bases via short recurrences. The results motivate the practically important distinction between Hermitian and non-Hermitian problems. Finally, the book thoroughly addresses the computational cost while using Krylov subspace methods. The investigation includes effects of finite precision arithmetic and focuses on the method of conjugate gradients (CG) and generalised minimal residuals (GMRES) as major examples. The book emphasises that algebraic computations must always be considered in the context of solving real-world problems, where the mathematical modelling, discretisation, and computation cannot be separated from each other. Moreover, the book underlines the importance of the historical context and it demonstrates that knowledge of early developments can play an important role in understanding and resolving very recent computational problems. Many extensive historical notes are therefore included as an inherent part of the text. The book ends with formulating some omitted issues and challenges which need to be addressed in future work. The book is intended as a research monograph which can be used in a wide scope of graduate courses on related subjects. It can be beneficial also for readers interested in the history of mathematics.