Understanding algorithms and starting values in nonlinear regression

Both linear and nonlinear regression minimize the sum of squares of the residual error (SSE) to estimate the parameters. However, they use very different approaches. For linear regression, Minitab mathematically derives the minimum sum of squares of the residual error by solving equations. After you choose the model, there are no more choices. If you fit the same model to the same data you obtain the same results.

However, for nonlinear regression, there is no direct solution for minimizing the sum of squares of the residual error. Thus, an iterative algorithm estimates parameters by systematically adjusting the parameter estimates to reduce the sum of squares of the residual error. After you decide on the model, you choose the algorithm and supply the starting value for each parameter. The algorithm uses these starting values to calculate the initial sum of squares of the residual error.

For each iteration, the algorithm adjusts the parameter estimates in a way that it predicts should reduce the sum of squares of the residual error compared to the previous iteration. Different algorithms use different approaches to determine the adjustments at each iteration. The iterations continue until the algorithm converges on the minimum sum of squares of the residual error, a problem prevents the subsequent iteration, or Minitab obtains the maximum number of iterations. If the algorithm fails to converge, you can try different starting values and/or the other algorithm.

For some expectation functions and data sets, the starting values can significantly affect the results. Certain starting values can lead to failure to converge or convergence to a local, instead of global, sum of squares of the residual error minimum. Sometimes, it might take much effort to develop good starting values.