What does the condition number on the optimization report mean?

Notation: Let the observed data be given as:

hence (nh-nx) represents the degrees of freedom.

The n x n correlation matrix is given by

Correlation(P) = Cov(k,l)/sqrt(Cov(k,k)*Cov(l,l)).

** T **

**The condition number is calculated for the matrix [S * W * S]. **

**What is a large versus small condition number?**

Back when single precision was the standard for scientific programming

in FORTRAN, a real variable had approximately 6.5 digits of precision.

Matrices with condition numbers near 1 were said to be well-conditioned.

Matrices with condition numbers much greater than one (such as around

10^5 for a 5 x 5 Hilbert matrix) were said to be ill-conditioned.

If kappa(A), the condition number of A, equals 10^k, then you may lose

up to k digits of accuracy on top of what would be lost to the numerical

method due to loss of precision from arithmetic methods.

If a condition number equals 10^16, then you have lost 16 digits of

precision in your result.

**What causes large condition numbers?**

If two parameters are close to being highly dependent on each other

(defined as having a correlation value whose absolute value is greater

than 0.98 can cause a large condition number.

Having parameters that the fit is insensitive to can also result in a

large condition number. This may be manifested as confidence intervals

that are an order of magnitude or larger then the parameter value. In

this case the model fit tends to be insensitive to changes in that parameter.

**What is the relationship between condition numbers and the probable **

**reliability of estimates of parameter confidence intervals or ranges quantitatively?**

This question makes an unwarranted assumption, that there is an estimate

of confidence intervals which is independent of the numerical process used

to calculate them. The confidence intervals given by the JSim optimizers

are those derived by equations 18 and 19 in Chan et al. (1993).

The question is not "Are the confidence intervals

correct?", the question is "Are the confidence intervals

useful?" A large condition number implies that the

optimization problem you are doing is poorly formulated.

For example, if you optimize on a parameter which does

not affect the fit, you will probably get a NaN for

a condition number.

From the SENSOP paper, "Near the solution for small!

residuals, the right hand side of Eq. 18 below is an

approximation! to the covariance matrix.

Cov(theta) =SSR(theta)/(m-p)* (S(theta)^T W S(theta))^-1.

The matrix (S(theta) transpose *W* S*(theta))

is the matrix whose condition number is calculated.

## Reference

Chan IS, Goldstein AA, Bassingthwaighte JB. SENSOP: a derivative-free solver for non-linear least squares with sensitivity scaling. Ann. Biomed. Eng. 21: 621-631, 1993.

Comments or Questions?

**Model development and archiving support at https://www.imagwiki.nibib.nih.gov/physiome provided by the following grants:** NIH U01HL122199 Analyzing the Cardiac Power Grid, 09/15/2015 - 05/31/2020, NIH/NIBIB BE08407 Software Integration, JSim and SBW 6/1/09-5/31/13; NIH/NHLBI T15 HL88516-01 Modeling for Heart, Lung and Blood: From Cell to Organ, 4/1/07-3/31/11; NSF BES-0506477 Adaptive Multi-Scale Model Simulation, 8/15/05-7/31/08; NIH/NHLBI R01 HL073598 Core 3: 3D Imaging and Computer Modeling of the Respiratory Tract, 9/1/04-8/31/09; as well as prior support from NIH/NCRR P41 RR01243 Simulation Resource in Circulatory Mass Transport and Exchange, 12/1/1980-11/30/01 and NIH/NIBIB R01 EB001973 JSim: A Simulation Analysis Platform, 3/1/02-2/28/07.