Inexact Newton Methods

Abstract

A classical algorithm for solving the system of nonlinear equations $F(x) = 0$ is Newton’s method \[ x_{k + 1} = x_k + s_k ,\quad {\text{where }}F'(x_k )s_k = - F(x_k ),\quad x_0 {\text{ given}}.\] The method is attractive because it converges rapidly from any sufficiently good initial guess $x_0 $. However, solving a system of linear equations (the Newton equations) at each stage can be expensive if the number of unknowns is large and may not be justified when $x_k $ is far from a solution. Therefore, we consider the class of inexact Newton methods: \[ x_{k + 1} = x_k + s_k ,\quad {\text{where }}F'(x_k )s_k = - F(x_k ) + r_k ,\quad {{\left\| {r_k } \right\|} / {\left\| {F(x_k )} \right\|}} \leqq \eta _k \] which solve the Newton equations only approximately and in some unspecified manner. Under the natural assumption that the forcing sequence $\{ n_k \} $ is uniformly less than one, we show that all such methods are locally convergent and characterize the order of convergence in terms of the rate of convergence of the relative residuals $\{ {{\|r_k \|} / {\|F(x_k )\|}}\} $.Finally, we indicate how these general results can be used to construct and analyze specific methods for solving systems of nonlinear equations.