This page was created by the IDL library routine
mk_html_help. For more information on
this routine, refer to the IDL Online Help Navigator
or type:
? mk_html_help
at the IDL command line prompt.
Last modified: Thu Sep 11 14:53:59 2003.
NAME:
GAUSS1
AUTHOR:
Craig B. Markwardt, NASA/GSFC Code 662, Greenbelt, MD 20770
craigm@lheamail.gsfc.nasa.gov
PURPOSE:
Compute Gaussian curve given the mean, sigma and area.
MAJOR TOPICS:
Curve and Surface Fitting
CALLING SEQUENCE:
YVALS = GAUSS1(XVALS, [MEAN, SIGMA, AREA], SKEW=skew)
DESCRIPTION:
This routine computes the values of a Gaussian function whose
X-values, mean, sigma, and total area are given. It is meant to be
a demonstration for curve-fitting.
XVALS can be an array of X-values, in which case the returned
Y-values are an array as well. The second parameter to GAUSS1
should be an array containing the MEAN, SIGMA, and total AREA, in
that order.
INPUTS:
X - Array of X-values.
[MEAN, SIGMA, AREA] - the mean, sigma and total area of the
desired Gaussian curve.
INPUT KEYWORD PARAMETERS:
SKEW - You may specify a skew value. Default is no skew.
PEAK - if set then AREA is interpreted as the peak value rather
than the area under the peak.
RETURNS:
Returns the array of Y-values.
EXAMPLE:
p = [2.2D, 1.4D, 3000.D]
x = dindgen(200)*0.1 - 10.
y = gauss1(x, p)
Computes the values of the Gaussian at equispaced intervals
(spacing is 0.1). The gaussian has a mean of 2.2, standard
deviation of 1.4, and total area of 3000.
REFERENCES:
MODIFICATION HISTORY:
Written, Jul 1998, CM
Correct bug in normalization, CM, 01 Nov 1999
Optimized for speed, CM, 02 Nov 1999
Added copyright notice, 25 Mar 2001, CM
Added PEAK keyword, 30 Sep 2001, CM
$Id: gauss1.pro,v 1.4 2001/10/13 17:41:48 craigm Exp $
(See gauss1.pro)
NAME:
GAUSS1P
AUTHOR:
Craig B. Markwardt, NASA/GSFC Code 662, Greenbelt, MD 20770
craigm@lheamail.gsfc.nasa.gov
PURPOSE:
Compute Gaussian curve given the mean, sigma and area (procedure).
MAJOR TOPICS:
Curve and Surface Fitting
CALLING SEQUENCE:
GAUSS1, XVALS, [MEAN, SIGMA, AREA], YVALS, SKEW=skew
DESCRIPTION:
This routine computes the values of a Gaussian function whose
X-values, mean, sigma, and total area are given. It is meant to be
a demonstration for curve-fitting.
XVALS can be an array of X-values, in which case the returned
Y-values are an array as well. The second parameter to GAUSS1
should be an array containing the MEAN, SIGMA, and total AREA, in
that order.
INPUTS:
X - Array of X-values.
[MEAN, SIGMA, AREA] - the mean, sigma and total area of the
desired Gaussian curve.
YVALS - returns the array of Y-values.
KEYWORD PARAMETERS:
SKEW - You may specify a skew value. Default is no skew.
EXAMPLE:
p = [2.2D, 1.4D, 3000.D]
x = dindgen(200)*0.1 - 10.
gauss1p, x, p, y
Computes the values of the Gaussian at equispaced intervals
(spacing is 0.1). The gaussian has a mean of 2.2, standard
deviation of 1.4, and total area of 3000.
REFERENCES:
MODIFICATION HISTORY:
Transcribed from GAUSS1, 13 Dec 1999, CM
Added copyright notice, 25 Mar 2001, CM
$Id: gauss1p.pro,v 1.2 2001/03/25 18:55:12 craigm Exp $
(See gauss1p.pro)
NAME: GAUSS2 AUTHOR: Craig B. Markwardt, NASA/GSFC Code 662, Greenbelt, MD 20770 craigm@lheamail.gsfc.nasa.gov PURPOSE: Compute Gaussian curve given the mean, sigma and area. MAJOR TOPICS: Curve and Surface Fitting CALLING SEQUENCE: YVALS = GAUSS2(X, Y, [XCENT, YCENT, SIGMA, PEAK]) DESCRIPTION: This routine computes the values of a Gaussian function whose X-values, mean, sigma, and total area are given. It is meant to be a demonstration for curve-fitting. XVALS can be an array of X-values, in which case the returned Y-values are an array as well. The second parameter to GAUSS1 should be an array containing the MEAN, SIGMA, and total AREA, in that order. INPUTS: X - 2-dimensional array of "X"-values. Y - 2-dimensional array of "Y"-values. XCENT - X-position of gaussian centroid. YCENT - Y-position of gaussian centroid. SIGMA - sigma of the curve (X and Y widths are the same). PEAK - the peak value of the gaussian function. RETURNS: Returns the array of Y-values. EXAMPLE: p = [2.2D, -0.7D, 1.4D, 3000.D] x = (dindgen(200)*0.1 - 10.) # (dblarr(200) + 1) y = (dblarr(200) + 1) # (dindgen(200)*0.1 - 10.) z = gauss2(x, y, p) Computes the values of the Gaussian at equispaced intervals in X and Y (spacing is 0.1). The gaussian has a centroid position of (2.2, -0.7), standard deviation of 1.4, and peak value of 3000. REFERENCES: MODIFICATION HISTORY: Written, 02 Oct 1999, CM Added copyright notice, 25 Mar 2001, CM $Id: gauss2.pro,v 1.2 2001/03/25 18:55:13 craigm Exp $
(See gauss2.pro)
NAME:
MPCHILIM
AUTHOR:
Craig B. Markwardt, NASA/GSFC Code 662, Greenbelt, MD 20770
craigm@lheamail.gsfc.nasa.gov
UPDATED VERSIONs can be found on my WEB PAGE:
http://cow.physics.wisc.edu/~craigm/idl/idl.html
PURPOSE:
Compute confidence limits for chi-square statistic
MAJOR TOPICS:
Curve and Surface Fitting, Statistics
CALLING SEQUENCE:
DELCHI = MPCHILIM(PROB, DOF, [/SIGMA, /CLEVEL, /SLEVEL ])
DESCRIPTION:
The function MPCHILIM() computes confidence limits of the
chi-square statistic for a desired probability level. The returned
values, DELCHI, are the limiting chi-squared values: a chi-squared
value of greater than DELCHI will occur by chance with probability
PROB:
P_CHI(CHI > DELCHI; DOF) = PROB
In specifying the probability level the user has three choices:
* give the confidence level (default);
* give the significance level (i.e., 1 - confidence level) and
pass the /SLEVEL keyword; OR
* give the "sigma" of the probability (i.e., compute the
probability based on the normal distribution) and pass the
/SIGMA keyword.
Note that /SLEVEL, /CLEVEL and /SIGMA are mutually exclusive.
INPUTS:
PROB - scalar or vector number, giving the desired probability
level as described above.
DOF - scalar or vector number, giving the number of degrees of
freedom in the chi-square distribution.
RETURNS:
Returns a scalar or vector of chi-square confidence limits.
KEYWORD PARAMETERS:
SLEVEL - if set, then PROB describes the significance level.
CLEVEL - if set, then PROB describes the confidence level
(default).
SIGMA - if set, then PROB is the number of "sigma" away from the
mean in the normal distribution.
EXAMPLES:
print, mpchilim(0.99d, 2d, /clevel)
Print the 99% confidence limit for a chi-squared of 2 degrees of
freedom.
print, mpchilim(5d, 2d, /sigma)
Print the "5 sigma" confidence limit for a chi-squared of 2
degrees of freedom. Here "5 sigma" indicates the gaussian
probability of a 5 sigma event or greater.
P_GAUSS(5D) = 1D - 5.7330314e-07
REFERENCES:
Algorithms taken from CEPHES special function library, by Stephen
Moshier. (http://www.netlib.org/cephes/)
MODIFICATION HISTORY:
Completed, 1999, CM
Documented, 16 Nov 2001, CM
Reduced obtrusiveness of common block and math error handling, 18
Nov 2001, CM
$Id: mpchilim.pro,v 1.4 2001/11/18 12:59:16 craigm Exp $
(See mpchilim.pro)
NAME:
MPCHITEST
AUTHOR:
Craig B. Markwardt, NASA/GSFC Code 662, Greenbelt, MD 20770
craigm@lheamail.gsfc.nasa.gov
UPDATED VERSIONs can be found on my WEB PAGE:
http://cow.physics.wisc.edu/~craigm/idl/idl.html
PURPOSE:
Compute the probability of a given chi-squared value
MAJOR TOPICS:
Curve and Surface Fitting, Statistics
CALLING SEQUENCE:
PROB = MPCHITEST(CHI, DOF, [/SIGMA, /CLEVEL, /SLEVEL ])
DESCRIPTION:
The function MPCHITEST() computes the probability for a value drawn
from the chi-square distribution to equal or exceed the given value
CHI. This can be used for confidence testing of a measured value
obeying the chi-squared distribution.
P_CHI(X > CHI; DOF) = PROB
In specifying the returned probability level the user has three
choices:
* return the confidence level when the /CLEVEL keyword is passed;
OR
* return the significance level (i.e., 1 - confidence level) when
the /SLEVEL keyword is passed (default); OR
* return the "sigma" of the probability (i.e., compute the
probability based on the normal distribution) when the /SIGMA
keyword is passed.
Note that /SLEVEL, /CLEVEL and /SIGMA are mutually exclusive.
INPUTS:
CHI - chi-squared value to be tested.
DOF - scalar or vector number, giving the number of degrees of
freedom in the chi-square distribution.
RETURNS:
Returns a scalar or vector of probabilities, as described above,
and according to the /SLEVEL, /CLEVEL and /SIGMA keywords.
KEYWORD PARAMETERS:
SLEVEL - if set, then PROB describes the significance level
(default).
CLEVEL - if set, then PROB describes the confidence level.
SIGMA - if set, then PROB is the number of "sigma" away from the
mean in the normal distribution.
EXAMPLES:
print, mpchitest(1300d,1252d)
Print the probability for a chi-squared value with 1252 degrees of
freedom to exceed a value of 1300, as a confidence level.
REFERENCES:
Algorithms taken from CEPHES special function library, by Stephen
Moshier. (http://www.netlib.org/cephes/)
MODIFICATION HISTORY:
Completed, 1999, CM
Documented, 16 Nov 2001, CM
Reduced obtrusiveness of common block and math error handling, 18
Nov 2001, CM
$Id: mpchitest.pro,v 1.5 2001/11/18 12:59:16 craigm Exp $
(See mpchitest.pro)
NAME:
MPCURVEFIT
AUTHOR:
Craig B. Markwardt, NASA/GSFC Code 662, Greenbelt, MD 20770
craigm@lheamail.gsfc.nasa.gov
UPDATED VERSIONs can be found on my WEB PAGE:
http://cow.physics.wisc.edu/~craigm/idl/idl.html
PURPOSE:
Perform Levenberg-Marquardt least-squares fit (replaces CURVEFIT)
MAJOR TOPICS:
Curve and Surface Fitting
CALLING SEQUENCE:
YFIT = MPCURVEFIT(X, Y, WEIGHTS, P, [SIGMA,] FUNCTION_NAME=FUNC,
FUNCTARGS=functargs, ITMAX=itmax, PARINFO=parinfo,
FTOL=ftol, XTOL=xtol, GTOL=gtol,
ITERPROC=iterproc, ITERARGS=iterargs,
NPRINT=nprint, QUIET=quiet, NOCOVAR=nocovar,
NFEV=nfev, ITER=iter, ERRMSG=errmsg,
CHISQ=chisq, COVAR=covar, STATUS=status)
DESCRIPTION:
MPCURVEFIT fits a user-supplied model -- in the form of an IDL
function -- to a set of user-supplied data. MPCURVEFIT calls
MPFIT, the MINPACK-1 least-squares minimizer, to do the main
work.
Given the data and their uncertainties, MPCURVEFIT finds the best
set of model parameters which match the data (in a least-squares
sense) and returns them in the parameter P.
MPCURVEFIT returns the best fit function.
The user must supply the following items:
- An array of independent variable values ("X").
- An array of "measured" *dependent* variable values ("Y").
- An array of weighting values ("WEIGHTS").
- The name of an IDL function which computes Y given X ("FUNC").
- Starting guesses for all of the parameters ("P").
There are very few restrictions placed on X, Y or FUNCT. Simply
put, FUNCT must map the "X" values into "Y" values given the
model parameters. The "X" values may represent any independent
variable (not just Cartesian X), and indeed may be multidimensional
themselves. For example, in the application of image fitting, X
may be a 2xN array of image positions.
MPCURVEFIT carefully avoids passing large arrays where possible to
improve performance.
See below for an example of usage.
USER FUNCTION
The user must define a function which returns the model value. For
applications which use finite-difference derivatives -- the default
-- the user function should be declared in the following way:
PRO MYFUNCT, X, P, YMOD
; The independent variable is X
; Parameter values are passed in "P"
YMOD = ... computed model values at X ...
END
The returned array YMOD must have the same dimensions and type as
the "measured" Y values.
User functions may also indicate a fatal error condition
using the ERROR_CODE common block variable, as described
below under the MPFIT_ERROR common block definition.
See the discussion under "ANALYTIC DERIVATIVES" and AUTODERIVATIVE
in MPFIT.PRO if you wish to compute the derivatives for yourself.
AUTODERIVATIVE is accepted and passed directly to MPFIT. The user
function must accept one additional parameter, DP, which contains
the derivative of the user function with respect to each parameter
at each data point, as described in MPFIT.PRO.
CONSTRAINING PARAMETER VALUES WITH THE PARINFO KEYWORD
The behavior of MPFIT can be modified with respect to each
parameter to be fitted. A parameter value can be fixed; simple
boundary constraints can be imposed; limitations on the parameter
changes can be imposed; properties of the automatic derivative can
be modified; and parameters can be tied to one another.
These properties are governed by the PARINFO structure, which is
passed as a keyword parameter to MPFIT.
PARINFO should be an array of structures, one for each parameter.
Each parameter is associated with one element of the array, in
numerical order. The structure can have the following entries
(none are required):
.VALUE - the starting parameter value (but see the START_PARAMS
parameter for more information).
.FIXED - a boolean value, whether the parameter is to be held
fixed or not. Fixed parameters are not varied by
MPFIT, but are passed on to MYFUNCT for evaluation.
.LIMITED - a two-element boolean array. If the first/second
element is set, then the parameter is bounded on the
lower/upper side. A parameter can be bounded on both
sides. Both LIMITED and LIMITS must be given
together.
.LIMITS - a two-element float or double array. Gives the
parameter limits on the lower and upper sides,
respectively. Zero, one or two of these values can be
set, depending on the values of LIMITED. Both LIMITED
and LIMITS must be given together.
.PARNAME - a string, giving the name of the parameter. The
fitting code of MPFIT does not use this tag in any
way. However, the default ITERPROC will print the
parameter name if available.
.STEP - the step size to be used in calculating the numerical
derivatives. If set to zero, then the step size is
computed automatically. Ignored when AUTODERIVATIVE=0.
This value is superceded by the RELSTEP value.
.RELSTEP - the *relative* step size to be used in calculating
the numerical derivatives. This number is the
fractional size of the step, compared to the
parameter value. This value supercedes the STEP
setting. If the parameter is zero, then a default
step size is chosen.
.MPSIDE - the sidedness of the finite difference when computing
numerical derivatives. This field can take four
values:
0 - one-sided derivative computed automatically
1 - one-sided derivative (f(x+h) - f(x) )/h
-1 - one-sided derivative (f(x) - f(x-h))/h
2 - two-sided derivative (f(x+h) - f(x-h))/(2*h)
Where H is the STEP parameter described above. The
"automatic" one-sided derivative method will chose a
direction for the finite difference which does not
violate any constraints. The other methods do not
perform this check. The two-sided method is in
principle more precise, but requires twice as many
function evaluations. Default: 0.
.MPMINSTEP - the minimum change to be made in the parameter
value. During the fitting process, the parameter
will be changed by multiples of this value. The
actual step is computed as:
DELTA1 = MPMINSTEP*ROUND(DELTA0/MPMINSTEP)
where DELTA0 and DELTA1 are the estimated parameter
changes before and after this constraint is
applied. Note that this constraint should be used
with care since it may cause non-converging,
oscillating solutions.
A value of 0 indicates no minimum. Default: 0.
.MPMAXSTEP - the maximum change to be made in the parameter
value. During the fitting process, the parameter
will never be changed by more than this value.
A value of 0 indicates no maximum. Default: 0.
.TIED - a string expression which "ties" the parameter to other
free or fixed parameters. Any expression involving
constants and the parameter array P are permitted.
Example: if parameter 2 is always to be twice parameter
1 then use the following: parinfo(2).tied = '2 * P(1)'.
Since they are totally constrained, tied parameters are
considered to be fixed; no errors are computed for them.
[ NOTE: the PARNAME can't be used in expressions. ]
Future modifications to the PARINFO structure, if any, will involve
adding structure tags beginning with the two letters "MP".
Therefore programmers are urged to avoid using tags starting with
the same letters; otherwise they are free to include their own
fields within the PARINFO structure, and they will be ignored.
PARINFO Example:
parinfo = replicate({value:0.D, fixed:0, limited:[0,0], $
limits:[0.D,0]}, 5)
parinfo(0).fixed = 1
parinfo(4).limited(0) = 1
parinfo(4).limits(0) = 50.D
parinfo(*).value = [5.7D, 2.2, 500., 1.5, 2000.]
A total of 5 parameters, with starting values of 5.7,
2.2, 500, 1.5, and 2000 are given. The first parameter
is fixed at a value of 5.7, and the last parameter is
constrained to be above 50.
INPUTS:
FUNCT - a string variable containing the name of an IDL function.
This function computes the "model" Y values given the
X values and model parameters, as described above.
X - Array of independent variable values.
Y - Array of "measured" dependent variable values. Y should have
the same data type as X. The function FUNCT should map
X->Y.
WEIGHTS - Array of weights to be used in calculating the
chi-squared value. If WEIGHTS is specified then the ERR
parameter is ignored. The chi-squared value is computed
as follows:
CHISQ = TOTAL( (Y-FUNCT(X,P))^2 * ABS(WEIGHTS) )
Here are common values of WEIGHTS:
1D/ERR^2 - Normal weighting (ERR is the measurement error)
1D/Y - Poisson weighting (counting statistics)
1D - Unweighted
P - An array of starting values for each of the parameters of the
model. The number of parameters should be fewer than the
number of measurements. Also, the parameters should have the
same data type as the measurements (double is preferred).
Upon successful completion the new parameter values are
returned in P.
If both START_PARAMS and PARINFO are passed, then the starting
*value* is taken from START_PARAMS, but the *constraints* are
taken from PARINFO.
SIGMA - The formal 1-sigma errors in each parameter, computed from
the covariance matrix. If a parameter is held fixed, or
if it touches a boundary, then the error is reported as
zero.
If the fit is unweighted (i.e. no errors were given, or
the weights were uniformly set to unity), then SIGMA will
probably not represent the true parameter uncertainties.
*If* you can assume that the true reduced chi-squared
value is unity -- meaning that the fit is implicitly
assumed to be of good quality -- then the estimated
parameter uncertainties can be computed by scaling SIGMA
by the measured chi-squared value.
DOF = N_ELEMENTS(X) - N_ELEMENTS(P) ; deg of freedom
CSIGMA = SIGMA * SQRT(CHISQ / DOF) ; scaled uncertainties
RETURNS:
Returns the array containing the best-fitting function.
KEYWORD PARAMETERS:
CHISQ - the value of the summed squared residuals for the
returned parameter values.
COVAR - the covariance matrix for the set of parameters returned
by MPFIT. The matrix is NxN where N is the number of
parameters. The square root of the diagonal elements
gives the formal 1-sigma statistical errors on the
parameters IF errors were treated "properly" in MYFUNC.
Parameter errors are also returned in PERROR.
To compute the correlation matrix, PCOR, use this:
IDL> PCOR = COV * 0
IDL> FOR i = 0, n-1 DO FOR j = 0, n-1 DO $
PCOR(i,j) = COV(i,j)/sqrt(COV(i,i)*COV(j,j))
If NOCOVAR is set or MPFIT terminated abnormally, then
COVAR is set to a scalar with value !VALUES.D_NAN.
ERRMSG - a string error or warning message is returned.
FTOL - a nonnegative input variable. Termination occurs when both
the actual and predicted relative reductions in the sum of
squares are at most FTOL (and STATUS is accordingly set to
1 or 3). Therefore, FTOL measures the relative error
desired in the sum of squares. Default: 1D-10
FUNCTARGS - A structure which contains the parameters to be passed
to the user-supplied function specified by FUNCT via
the _EXTRA mechanism. This is the way you can pass
additional data to your user-supplied function without
using common blocks.
By default, no extra parameters are passed to the
user-supplied function.
GTOL - a nonnegative input variable. Termination occurs when the
cosine of the angle between fvec and any column of the
jacobian is at most GTOL in absolute value (and STATUS is
accordingly set to 4). Therefore, GTOL measures the
orthogonality desired between the function vector and the
columns of the jacobian. Default: 1D-10
ITER - the number of iterations completed.
ITERARGS - The keyword arguments to be passed to ITERPROC via the
_EXTRA mechanism. This should be a structure, and is
similar in operation to FUNCTARGS.
Default: no arguments are passed.
ITERPROC - The name of a procedure to be called upon each NPRINT
iteration of the MPFIT routine. It should be declared
in the following way:
PRO ITERPROC, FUNCT, p, iter, fnorm, FUNCTARGS=fcnargs, $
PARINFO=parinfo, QUIET=quiet, ...
; perform custom iteration update
END
ITERPROC must either accept all three keyword
parameters (FUNCTARGS, PARINFO and QUIET), or at least
accept them via the _EXTRA keyword.
FUNCT is the user-supplied function to be minimized,
P is the current set of model parameters, ITER is the
iteration number, and FUNCTARGS are the arguments to be
passed to FUNCT. FNORM should be the
chi-squared value. QUIET is set when no textual output
should be printed. See below for documentation of
PARINFO.
In implementation, ITERPROC can perform updates to the
terminal or graphical user interface, to provide
feedback while the fit proceeds. If the fit is to be
stopped for any reason, then ITERPROC should set the
common block variable ERROR_CODE to negative value (see
MPFIT_ERROR common block below). In principle,
ITERPROC should probably not modify the parameter
values, because it may interfere with the algorithm's
stability. In practice it is allowed.
Default: an internal routine is used to print the
parameter values.
ITMAX - The maximum number of iterations to perform. If the
number is exceeded, then the STATUS value is set to 5
and MPFIT returns.
Default: 200 iterations
NFEV - the number of FUNCT function evaluations performed.
NOCOVAR - set this keyword to prevent the calculation of the
covariance matrix before returning (see COVAR)
NPRINT - The frequency with which ITERPROC is called. A value of
1 indicates that ITERPROC is called with every iteration,
while 2 indicates every other iteration, etc. Note that
several Levenberg-Marquardt attempts can be made in a
single iteration.
Default value: 1
PARINFO - Provides a mechanism for more sophisticated constraints
to be placed on parameter values. When PARINFO is not
passed, then it is assumed that all parameters are free
and unconstrained. Values in PARINFO are never
modified during a call to MPFIT.
See description above for the structure of PARINFO.
Default value: all parameters are free and unconstrained.
QUIET - set this keyword when no textual output should be printed
by MPFIT
STATUS - an integer status code is returned. All values other
than zero can represent success. It can have one of the
following values:
0 improper input parameters.
1 both actual and predicted relative reductions
in the sum of squares are at most FTOL.
2 relative error between two consecutive iterates
is at most XTOL
3 conditions for STATUS = 1 and STATUS = 2 both hold.
4 the cosine of the angle between fvec and any
column of the jacobian is at most GTOL in
absolute value.
5 the maximum number of iterations has been reached
6 FTOL is too small. no further reduction in
the sum of squares is possible.
7 XTOL is too small. no further improvement in
the approximate solution x is possible.
8 GTOL is too small. fvec is orthogonal to the
columns of the jacobian to machine precision.
XTOL - a nonnegative input variable. Termination occurs when the
relative error between two consecutive iterates is at most
XTOL (and STATUS is accordingly set to 2 or 3). Therefore,
XTOL measures the relative error desired in the approximate
solution. Default: 1D-10
EXAMPLE:
; First, generate some synthetic data
npts = 200
x = dindgen(npts) * 0.1 - 10. ; Independent variable
yi = gauss1(x, [2.2D, 1.4, 3000.]) ; "Ideal" Y variable
y = yi + randomn(seed, npts) * sqrt(1000. + yi); Measured, w/ noise
sy = sqrt(1000.D + y) ; Poisson errors
; Now fit a Gaussian to see how well we can recover
p0 = [1.D, 1., 1000.] ; Initial guess
yfit = mpcurvefit(x, y, 1/sy^2, p0, $
FUNCTION_NAME='GAUSS1P') ; Fit a function
print, p
Generates a synthetic data set with a Gaussian peak, and Poisson
statistical uncertainty. Then the same function is fitted to the
data to see how close we can get. GAUSS1 and GAUSS1P are
available from the same web page.
COMMON BLOCKS:
COMMON MPFIT_ERROR, ERROR_CODE
User routines may stop the fitting process at any time by
setting an error condition. This condition may be set in either
the user's model computation routine (MYFUNCT), or in the
iteration procedure (ITERPROC).
To stop the fitting, the above common block must be declared,
and ERROR_CODE must be set to a negative number. After the user
procedure or function returns, MPFIT checks the value of this
common block variable and exits immediately if the error
condition has been set. By default the value of ERROR_CODE is
zero, indicating a successful function/procedure call.
REFERENCES:
MINPACK-1, Jorge More', available from netlib (www.netlib.org).
"Optimization Software Guide," Jorge More' and Stephen Wright,
SIAM, *Frontiers in Applied Mathematics*, Number 14.
MODIFICATION HISTORY:
Translated from MPFITFUN, 25 Sep 1999, CM
Alphabetized documented keywords, 02 Oct 1999, CM
Added QUERY keyword and query checking of MPFIT, 29 Oct 1999, CM
Check to be sure that X and Y are present, 02 Nov 1999, CM
Documented SIGMA for unweighted fits, 03 Nov 1999, CM
Changed to ERROR_CODE for error condition, 28 Jan 2000, CM
Copying permission terms have been liberalized, 26 Mar 2000, CM
Propagated improvements from MPFIT, 17 Dec 2000, CM
Corrected behavior of NODERIVATIVE, 13 May 2002, CM
Documented RELSTEP field of PARINFO (!!), CM, 25 Oct 2002
$Id: mpcurvefit.pro,v 1.4 2002/11/07 00:12:54 craigm Exp $
(See mpcurvefit.pro)
NAME:
MPFIT
AUTHOR:
Craig B. Markwardt, NASA/GSFC Code 662, Greenbelt, MD 20770
craigm@lheamail.gsfc.nasa.gov
UPDATED VERSIONs can be found on my WEB PAGE:
http://cow.physics.wisc.edu/~craigm/idl/idl.html
PURPOSE:
Perform Levenberg-Marquardt least-squares minimization (MINPACK-1)
MAJOR TOPICS:
Curve and Surface Fitting
CALLING SEQUENCE:
parms = MPFIT(MYFUNCT, start_parms, FUNCTARGS=fcnargs, NFEV=nfev,
MAXITER=maxiter, ERRMSG=errmsg, NPRINT=nprint, QUIET=quiet,
FTOL=ftol, XTOL=xtol, GTOL=gtol, NITER=niter,
STATUS=status, ITERPROC=iterproc, ITERARGS=iterargs,
COVAR=covar, PERROR=perror, BESTNORM=bestnorm,
PARINFO=parinfo)
DESCRIPTION:
MPFIT uses the Levenberg-Marquardt technique to solve the
least-squares problem. In its typical use, MPFIT will be used to
fit a user-supplied function (the "model") to user-supplied data
points (the "data") by adjusting a set of parameters. MPFIT is
based upon MINPACK-1 (LMDIF.F) by More' and collaborators.
For example, a researcher may think that a set of observed data
points is best modelled with a Gaussian curve. A Gaussian curve is
parameterized by its mean, standard deviation and normalization.
MPFIT will, within certain constraints, find the set of parameters
which best fits the data. The fit is "best" in the least-squares
sense; that is, the sum of the weighted squared differences between
the model and data is minimized.
The Levenberg-Marquardt technique is a particular strategy for
iteratively searching for the best fit. This particular
implementation is drawn from MINPACK-1 (see NETLIB), and seems to
be more robust than routines provided with IDL. This version
allows upper and lower bounding constraints to be placed on each
parameter, or the parameter can be held fixed.
The IDL user-supplied function should return an array of weighted
deviations between model and data. In a typical scientific problem
the residuals should be weighted so that each deviate has a
gaussian sigma of 1.0. If X represents values of the independent
variable, Y represents a measurement for each value of X, and ERR
represents the error in the measurements, then the deviates could
be calculated as follows:
DEVIATES = (Y - F(X)) / ERR
where F is the analytical function representing the model. You are
recommended to use the convenience functions MPFITFUN and
MPFITEXPR, which are driver functions that calculate the deviates
for you. If ERR are the 1-sigma uncertainties in Y, then
TOTAL( DEVIATES^2 )
will be the total chi-squared value. MPFIT will minimize the
chi-square value. The values of X, Y and ERR are passed through
MPFIT to the user-supplied function via the FUNCTARGS keyword.
Simple constraints can be placed on parameter values by using the
PARINFO keyword to MPFIT. See below for a description of this
keyword.
MPFIT does not perform more general optimization tasks. See TNMIN
instead. MPFIT is customized, based on MINPACK-1, to the
least-squares minimization problem.
USER FUNCTION
The user must define a function which returns the appropriate
values as specified above. The function should return the weighted
deviations between the model and the data. For applications which
use finite-difference derivatives -- the default -- the user
function should be declared in the following way:
FUNCTION MYFUNCT, p, X=x, Y=y, ERR=err
; Parameter values are passed in "p"
model = F(x, p)
return, (y-model)/err
END
See below for applications with analytical derivatives.
The keyword parameters X, Y, and ERR in the example above are
suggestive but not required. Any parameters can be passed to
MYFUNCT by using the FUNCTARGS keyword to MPFIT. Use MPFITFUN and
MPFITEXPR if you need ideas on how to do that. The function *must*
accept a parameter list, P.
In general there are no restrictions on the number of dimensions in
X, Y or ERR. However the deviates *must* be returned in a
one-dimensional array, and must have the same type (float or
double) as the input arrays.
User functions may also indicate a fatal error condition using the
ERROR_CODE common block variable, as described below under the
MPFIT_ERROR common block definition (by setting ERROR_CODE to a
number between -15 and -1).
ANALYTIC DERIVATIVES
In the search for the best-fit solution, MPFIT by default
calculates derivatives numerically via a finite difference
approximation. The user-supplied function need not calculate the
derivatives explicitly. However, if you desire to compute them
analytically, then the AUTODERIVATIVE=0 keyword must be passed. As
a practical matter, it is often sufficient and even faster to allow
MPFIT to calculate the derivatives numerically, and so
AUTODERIVATIVE=0 is not necessary.
Also, the user function must be declared with one additional
parameter, as follows:
FUNCTION MYFUNCT, p, dp, X=x, Y=y, ERR=err
model = F(x, p)
if n_params() GT 1 then begin
; Compute derivatives
dp = make_array(n_elements(x), n_elements(p), value=x(0)*0)
for i = 0, n_elements(p)-1 do $
dp(*,i) = FGRAD(x, p, i)
endif
return, (y-model)/err
END
where FGRAD(x, p, i) is a user function which must compute the
derivative of the model with respect to parameter P(i) at X. When
finite differencing is used for computing derivatives (ie, when
AUTODERIVATIVE=1), the parameter DP is not passed. Therefore
functions can use N_PARAMS() to indicate whether they must compute
the derivatives or not.
Derivatives should be returned in the DP array. DP should be an m x
n array, where m is the number of data points and n is the number
of parameters. dp(i,j) is the derivative at the ith point with
respect to the jth parameter.
The derivatives with respect to fixed parameters are ignored; zero
is an appropriate value to insert for those derivatives. Upon
input to the user function, DP is set to a vector with the same
length as P, with a value of 1 for a parameter which is free, and a
value of zero for a parameter which is fixed (and hence no
derivative needs to be calculated). This input vector may be
overwritten as needed.
If the data is higher than one dimensional, then the *last*
dimension should be the parameter dimension. Example: fitting a
50x50 image, "dp" should be 50x50xNPAR.
CONSTRAINING PARAMETER VALUES WITH THE PARINFO KEYWORD
The behavior of MPFIT can be modified with respect to each
parameter to be fitted. A parameter value can be fixed; simple
boundary constraints can be imposed; limitations on the parameter
changes can be imposed; properties of the automatic derivative can
be modified; and parameters can be tied to one another.
These properties are governed by the PARINFO structure, which is
passed as a keyword parameter to MPFIT.
PARINFO should be an array of structures, one for each parameter.
Each parameter is associated with one element of the array, in
numerical order. The structure can have the following entries
(none are required):
.VALUE - the starting parameter value (but see the START_PARAMS
parameter for more information).
.FIXED - a boolean value, whether the parameter is to be held
fixed or not. Fixed parameters are not varied by
MPFIT, but are passed on to MYFUNCT for evaluation.
.LIMITED - a two-element boolean array. If the first/second
element is set, then the parameter is bounded on the
lower/upper side. A parameter can be bounded on both
sides. Both LIMITED and LIMITS must be given
together.
.LIMITS - a two-element float or double array. Gives the
parameter limits on the lower and upper sides,
respectively. Zero, one or two of these values can be
set, depending on the values of LIMITED. Both LIMITED
and LIMITS must be given together.
.PARNAME - a string, giving the name of the parameter. The
fitting code of MPFIT does not use this tag in any
way. However, the default ITERPROC will print the
parameter name if available.
.STEP - the step size to be used in calculating the numerical
derivatives. If set to zero, then the step size is
computed automatically. Ignored when AUTODERIVATIVE=0.
This value is superceded by the RELSTEP value.
.RELSTEP - the *relative* step size to be used in calculating
the numerical derivatives. This number is the
fractional size of the step, compared to the
parameter value. This value supercedes the STEP
setting. If the parameter is zero, then a default
step size is chosen.
.MPSIDE - the sidedness of the finite difference when computing
numerical derivatives. This field can take four
values:
0 - one-sided derivative computed automatically
1 - one-sided derivative (f(x+h) - f(x) )/h
-1 - one-sided derivative (f(x) - f(x-h))/h
2 - two-sided derivative (f(x+h) - f(x-h))/(2*h)
Where H is the STEP parameter described above. The
"automatic" one-sided derivative method will chose a
direction for the finite difference which does not
violate any constraints. The other methods do not
perform this check. The two-sided method is in
principle more precise, but requires twice as many
function evaluations. Default: 0.
.MPMAXSTEP - the maximum change to be made in the parameter
value. During the fitting process, the parameter
will never be changed by more than this value in
one iteration.
A value of 0 indicates no maximum. Default: 0.
.TIED - a string expression which "ties" the parameter to other
free or fixed parameters. Any expression involving
constants and the parameter array P are permitted.
Example: if parameter 2 is always to be twice parameter
1 then use the following: parinfo(2).tied = '2 * P(1)'.
Since they are totally constrained, tied parameters are
considered to be fixed; no errors are computed for them.
[ NOTE: the PARNAME can't be used in expressions. ]
.MPPRINT - if set to 1, then the default ITERPROC will print the
parameter value. If set to 0, the parameter value
will not be printed. This tag can be used to
selectively print only a few parameter values out of
many. Default: 1 (all parameters printed)
Future modifications to the PARINFO structure, if any, will involve
adding structure tags beginning with the two letters "MP".
Therefore programmers are urged to avoid using tags starting with
the same letters; otherwise they are free to include their own
fields within the PARINFO structure, and they will be ignored.
PARINFO Example:
parinfo = replicate({value:0.D, fixed:0, limited:[0,0], $
limits:[0.D,0]}, 5)
parinfo(0).fixed = 1
parinfo(4).limited(0) = 1
parinfo(4).limits(0) = 50.D
parinfo(*).value = [5.7D, 2.2, 500., 1.5, 2000.]
A total of 5 parameters, with starting values of 5.7,
2.2, 500, 1.5, and 2000 are given. The first parameter
is fixed at a value of 5.7, and the last parameter is
constrained to be above 50.
HARD-TO-COMPUTE FUNCTIONS: "EXTERNAL" EVALUATION
The normal mode of operation for MPFIT is for the user to pass a
function name, and MPFIT will call the user function multiple times
as it iterates toward a solution.
Some user functions are particularly hard to compute using the
standard model of MPFIT. Usually these are functions that depend
on a large amount of external data, and so it is not feasible, or
at least highly impractical, to have MPFIT call it. In those cases
it may be possible to use the "(EXTERNAL)" evaluation option.
In this case the user is responsible for making all function *and
derivative* evaluations. The function and Jacobian data are passed
in through the EXTERNAL_FVEC and EXTERNAL_FJAC keywords,
respectively. The user indicates the selection of this option by
specifying a function name (MYFUNCT) of "(EXTERNAL)". No
user-function calls are made when EXTERNAL evaluation is being
used.
At the end of each iteration, control returns to the user, who must
reevaluate the function at its new parameter values. Users should
check the return value of the STATUS keyword, where a value of 9
indicates the user should supply more data for the next iteration,
and re-call MPFIT. The user may refrain from calling MPFIT
further; as usual, STATUS will indicate when the solution has
converged and no more iterations are required.
Because MPFIT must maintain its own data structures between calls,
the user must also pass a named variable to the EXTERNAL_STATE
keyword. This variable must be maintained by the user, but not
changed, throughout the fitting process. When no more iterations
are desired, the named variable may be discarded.
INPUTS:
MYFUNCT - a string variable containing the name of the function to
be minimized. The function should return the weighted
deviations between the model and the data, as described
above.
For EXTERNAL evaluation of functions, this parameter
should be set to a value of "(EXTERNAL)".
START_PARAMS - An array of starting values for each of the
parameters of the model. The number of parameters
should be fewer than the number of measurements.
Also, the parameters should have the same data type
as the measurements (double is preferred).
This parameter is optional if the PARINFO keyword
is used (but see PARINFO). The PARINFO keyword
provides a mechanism to fix or constrain individual
parameters. If both START_PARAMS and PARINFO are
passed, then the starting *value* is taken from
START_PARAMS, but the *constraints* are taken from
PARINFO.
RETURNS:
Returns the array of best-fit parameters.
KEYWORD PARAMETERS:
AUTODERIVATIVE - If this is set, derivatives of the function will
be computed automatically via a finite
differencing procedure. If not set, then MYFUNCT
must provide the (analytical) derivatives.
Default: set (=1)
NOTE: to supply your own analytical derivatives,
explicitly pass AUTODERIVATIVE=0
BESTNORM - the value of the summed squared residuals for the
returned parameter values.
COVAR - the covariance matrix for the set of parameters returned
by MPFIT. The matrix is NxN where N is the number of
parameters. The square root of the diagonal elements
gives the formal 1-sigma statistical errors on the
parameters IF errors were treated "properly" in MYFUNC.
Parameter errors are also returned in PERROR.
To compute the correlation matrix, PCOR, use this example:
IDL> PCOR = COV * 0
IDL> FOR i = 0, n-1 DO FOR j = 0, n-1 DO $
PCOR(i,j) = COV(i,j)/sqrt(COV(i,i)*COV(j,j))
If NOCOVAR is set or MPFIT terminated abnormally, then
COVAR is set to a scalar with value !VALUES.D_NAN.
ERRMSG - a string error or warning message is returned.
EXTERNAL_FVEC - upon input, the function values, evaluated at
START_PARAMS. This should be an M-vector, where M
is the number of data points.
EXTERNAL_FJAC - upon input, the Jacobian array of partial
derivative values. This should be a M x N array,
where M is the number of data points and N is the
number of parameters. NOTE: that all FIXED or
TIED parameters must *not* be included in this
array.
EXTERNAL_STATE - a named variable to store MPFIT-related state
information between iterations (used in input and
output to MPFIT). The user must not manipulate
or discard this data until the final iteration is
performed.
FASTNORM - set this keyword to select a faster algorithm to
compute sum-of-square values internally. For systems
with large numbers of data points, the standard
algorithm can become prohibitively slow because it
cannot be vectorized well. By setting this keyword,
MPFIT will run faster, but it will be more prone to
floating point overflows and underflows. Thus, setting
this keyword may sacrifice some stability in the
fitting process.
FTOL - a nonnegative input variable. Termination occurs when both
the actual and predicted relative reductions in the sum of
squares are at most FTOL (and STATUS is accordingly set to
1 or 3). Therefore, FTOL measures the relative error
desired in the sum of squares. Default: 1D-10
FUNCTARGS - A structure which contains the parameters to be passed
to the user-supplied function specified by MYFUNCT via
the _EXTRA mechanism. This is the way you can pass
additional data to your user-supplied function without
using common blocks.
Consider the following example:
if FUNCTARGS = { XVAL:[1.D,2,3], YVAL:[1.D,4,9],
ERRVAL:[1.D,1,1] }
then the user supplied function should be declared
like this:
FUNCTION MYFUNCT, P, XVAL=x, YVAL=y, ERRVAL=err
By default, no extra parameters are passed to the
user-supplied function, but your function should
accept *at least* one keyword parameter. [ This is to
accomodate a limitation in IDL's _EXTRA
parameter-passing mechanism. ]
GTOL - a nonnegative input variable. Termination occurs when the
cosine of the angle between fvec and any column of the
jacobian is at most GTOL in absolute value (and STATUS is
accordingly set to 4). Therefore, GTOL measures the
orthogonality desired between the function vector and the
columns of the jacobian. Default: 1D-10
ITERARGS - The keyword arguments to be passed to ITERPROC via the
_EXTRA mechanism. This should be a structure, and is
similar in operation to FUNCTARGS.
Default: no arguments are passed.
ITERPROC - The name of a procedure to be called upon each NPRINT
iteration of the MPFIT routine. It should be declared
in the following way:
PRO ITERPROC, MYFUNCT, p, iter, fnorm, FUNCTARGS=fcnargs, $
PARINFO=parinfo, QUIET=quiet, DOF=dof, ...
; perform custom iteration update
END
ITERPROC must either accept all three keyword
parameters (FUNCTARGS, PARINFO and QUIET), or at least
accept them via the _EXTRA keyword.
MYFUNCT is the user-supplied function to be minimized,
P is the current set of model parameters, ITER is the
iteration number, and FUNCTARGS are the arguments to be
passed to MYFUNCT. FNORM should be the chi-squared
value. QUIET is set when no textual output should be
printed. DOF is the number of degrees of freedom,
normally the number of points less the number of free
parameters. See below for documentation of PARINFO.
In implementation, ITERPROC can perform updates to the
terminal or graphical user interface, to provide
feedback while the fit proceeds. If the fit is to be
stopped for any reason, then ITERPROC should set the
common block variable ERROR_CODE to negative value
between -15 and -1 (see MPFIT_ERROR common block
below). In principle, ITERPROC should probably not
modify the parameter values, because it may interfere
with the algorithm's stability. In practice it is
allowed.
Default: an internal routine is used to print the
parameter values.
ITERSTOP - Set this keyword if you wish to be able to stop the
fitting by hitting Control-G on the keyboard. This
only works if you use the default ITERPROC.
MAXITER - The maximum number of iterations to perform. If the
number is exceeded, then the STATUS value is set to 5
and MPFIT returns.
Default: 200 iterations
NFEV - the number of MYFUNCT function evaluations performed.
NFREE - the number of free parameters in the fit. This includes
parameters which are not FIXED and not TIED, but it does
include parameters which are pegged at LIMITS.
NITER - the number of iterations completed.
NOCOVAR - set this keyword to prevent the calculation of the
covariance matrix before returning (see COVAR)
NPEGGED - the number of free parameters which are pegged at a
LIMIT.
NPRINT - The frequency with which ITERPROC is called. A value of
1 indicates that ITERPROC is called with every iteration,
while 2 indicates every other iteration, etc. Note that
several Levenberg-Marquardt attempts can be made in a
single iteration.
Default value: 1
PARINFO - Provides a mechanism for more sophisticated constraints
to be placed on parameter values. When PARINFO is not
passed, then it is assumed that all parameters are free
and unconstrained. Values in PARINFO are never
modified during a call to MPFIT.
See description above for the structure of PARINFO.
Default value: all parameters are free and unconstrained.
PERROR - The formal 1-sigma errors in each parameter, computed
from the covariance matrix. If a parameter is held
fixed, or if it touches a boundary, then the error is
reported as zero.
If the fit is unweighted (i.e. no errors were given, or
the weights were uniformly set to unity), then PERROR
will probably not represent the true parameter
uncertainties.
*If* you can assume that the true reduced chi-squared
value is unity -- meaning that the fit is implicitly
assumed to be of good quality -- then the estimated
parameter uncertainties can be computed by scaling PERROR
by the measured chi-squared value.
DOF = N_ELEMENTS(X) - N_ELEMENTS(PARMS) ; deg of freedom
PCERROR = PERROR * SQRT(BESTNORM / DOF) ; scaled uncertainties
QUIET - set this keyword when no textual output should be printed
by MPFIT
RESDAMP - a scalar number, indicating the cut-off value of
residuals where "damping" will occur. Residuals with
magnitudes greater than this number will be replaced by
their logarithm. This partially mitigates the so-called
large residual problem inherent in least-squares solvers
(as for the test problem CURVI, http://www.maxthis.com/-
curviex.htm). A value of 0 indicates no damping.
Default: 0
Note: RESDAMP doesn't work with AUTODERIV=0
STATUS - an integer status code is returned. All values greater
than zero can represent success (however STATUS EQ 5 may
indicate failure to converge). It can have one of the
following values:
-16 a parameter or function value has become infinite or an
undefined number. This is usually a consequence of
numerical overflow in the user's model function, which
must be avoided.
-15 to -1
these are error codes that either MYFUNCT or ITERPROC
may return to terminate the fitting process (see
description of MPFIT_ERROR common below). If either
MYFUNCT or ITERPROC set ERROR_CODE to a negative number,
then that number is returned in STATUS. Values from -15
to -1 are reserved for the user functions and will not
clash with MPFIT.
0 improper input parameters.
1 both actual and predicted relative reductions
in the sum of squares are at most FTOL.
2 relative error between two consecutive iterates
is at most XTOL
3 conditions for STATUS = 1 and STATUS = 2 both hold.
4 the cosine of the angle between fvec and any
column of the jacobian is at most GTOL in
absolute value.
5 the maximum number of iterations has been reached
6 FTOL is too small. no further reduction in
the sum of squares is possible.
7 XTOL is too small. no further improvement in
the approximate solution x is possible.
8 GTOL is too small. fvec is orthogonal to the
columns of the jacobian to machine precision.
9 A successful single iteration has been completed, and
the user must supply another "EXTERNAL" evaluation of
the function and its derivatives. This status indicator
is neither an error nor a convergence indicator.
XTOL - a nonnegative input variable. Termination occurs when the
relative error between two consecutive iterates is at most
XTOL (and STATUS is accordingly set to 2 or 3). Therefore,
XTOL measures the relative error desired in the approximate
solution. Default: 1D-10
EXAMPLE:
p0 = [5.7D, 2.2, 500., 1.5, 2000.]
fa = {X:x, Y:y, ERR:err}
p = mpfit('MYFUNCT', p0, functargs=fa)
Minimizes sum of squares of MYFUNCT. MYFUNCT is called with the X,
Y, and ERR keyword parameters that are given by FUNCTARGS. The
resulting parameter values are returned in p.
COMMON BLOCKS:
COMMON MPFIT_ERROR, ERROR_CODE
User routines may stop the fitting process at any time by
setting an error condition. This condition may be set in either
the user's model computation routine (MYFUNCT), or in the
iteration procedure (ITERPROC).
To stop the fitting, the above common block must be declared,
and ERROR_CODE must be set to a negative number. After the user
procedure or function returns, MPFIT checks the value of this
common block variable and exits immediately if the error
condition has been set. This value is also returned in the
STATUS keyword: values of -1 through -15 are reserved error
codes for the user routines. By default the value of ERROR_CODE
is zero, indicating a successful function/procedure call.
COMMON MPFIT_PROFILE
COMMON MPFIT_MACHAR
COMMON MPFIT_CONFIG
These are undocumented common blocks are used internally by
MPFIT and may change in future implementations.
THEORY OF OPERATION:
There are many specific strategies for function minimization. One
very popular technique is to use function gradient information to
realize the local structure of the function. Near a local minimum
the function value can be taylor expanded about x0 as follows:
f(x) = f(x0) + f'(x0) . (x-x0) + (1/2) (x-x0) . f''(x0) . (x-x0)
----- --------------- ------------------------------- (1)
Order 0th 1st 2nd
Here f'(x) is the gradient vector of f at x, and f''(x) is the
Hessian matrix of second derivatives of f at x. The vector x is
the set of function parameters, not the measured data vector. One
can find the minimum of f, f(xm) using Newton's method, and
arrives at the following linear equation:
f''(x0) . (xm-x0) = - f'(x0) (2)
If an inverse can be found for f''(x0) then one can solve for
(xm-x0), the step vector from the current position x0 to the new
projected minimum. Here the problem has been linearized (ie, the
gradient information is known to first order). f''(x0) is
symmetric n x n matrix, and should be positive definite.
The Levenberg - Marquardt technique is a variation on this theme.
It adds an additional diagonal term to the equation which may aid the
convergence properties:
(f''(x0) + nu I) . (xm-x0) = -f'(x0) (2a)
where I is the identity matrix. When nu is large, the overall
matrix is diagonally dominant, and the iterations follow steepest
descent. When nu is small, the iterations are quadratically
convergent.
In principle, if f''(x0) and f'(x0) are known then xm-x0 can be
determined. However the Hessian matrix is often difficult or
impossible to compute. The gradient f'(x0) may be easier to
compute, if even by finite difference techniques. So-called
quasi-Newton techniques attempt to successively estimate f''(x0)
by building up gradient information as the iterations proceed.
In the least squares problem there are further simplifications
which assist in solving eqn (2). The function to be minimized is
a sum of squares:
f = Sum(hi^2) (3)
where hi is the ith residual out of m residuals as described
above. This can be substituted back into eqn (2) after computing
the derivatives:
f' = 2 Sum(hi hi')
f'' = 2 Sum(hi' hj') + 2 Sum(hi hi'') (4)
If one assumes that the parameters are already close enough to a
minimum, then one typically finds that the second term in f'' is
negligible [or, in any case, is too difficult to compute]. Thus,
equation (2) can be solved, at least approximately, using only
gradient information.
In matrix notation, the combination of eqns (2) and (4) becomes:
hT' . h' . dx = - hT' . h (5)
Where h is the residual vector (length m), hT is its transpose, h'
is the Jacobian matrix (dimensions n x m), and dx is (xm-x0). The
user function supplies the residual vector h, and in some cases h'
when it is not found by finite differences (see MPFIT_FDJAC2,
which finds h and hT'). Even if dx is not the best absolute step
to take, it does provide a good estimate of the best *direction*,
so often a line minimization will occur along the dx vector
direction.
The method of solution employed by MINPACK is to form the Q . R
factorization of h', where Q is an orthogonal matrix such that QT .
Q = I, and R is upper right triangular. Using h' = Q . R and the
ortogonality of Q, eqn (5) becomes
(RT . QT) . (Q . R) . dx = - (RT . QT) . h
RT . R . dx = - RT . QT . h (6)
R . dx = - QT . h
where the last statement follows because R is upper triangular.
Here, R, QT and h are known so this is a matter of solving for dx.
The routine MPFIT_QRFAC provides the QR factorization of h, with
pivoting, and MPFIT_QRSOLV provides the solution for dx.
REFERENCES:
MINPACK-1, Jorge More', available from netlib (www.netlib.org).
"Optimization Software Guide," Jorge More' and Stephen Wright,
SIAM, *Frontiers in Applied Mathematics*, Number 14.
More', Jorge J., "The Levenberg-Marquardt Algorithm:
Implementation and Theory," in *Numerical Analysis*, ed. Watson,
G. A., Lecture Notes in Mathematics 630, Springer-Verlag, 1977.
MODIFICATION HISTORY:
Translated from MINPACK-1 in FORTRAN, Apr-Jul 1998, CM
Fixed bug in parameter limits (x vs xnew), 04 Aug 1998, CM
Added PERROR keyword, 04 Aug 1998, CM
Added COVAR keyword, 20 Aug 1998, CM
Added NITER output keyword, 05 Oct 1998
D.L Windt, Bell Labs, windt@bell-labs.com;
Made each PARINFO component optional, 05 Oct 1998 CM
Analytical derivatives allowed via AUTODERIVATIVE keyword, 09 Nov 1998
Parameter values can be tied to others, 09 Nov 1998
Fixed small bugs (Wayne Landsman), 24 Nov 1998
Added better exception error reporting, 24 Nov 1998 CM
Cosmetic documentation changes, 02 Jan 1999 CM
Changed definition of ITERPROC to be consistent with TNMIN, 19 Jan 1999 CM
Fixed bug when AUTDERIVATIVE=0. Incorrect sign, 02 Feb 1999 CM
Added keyboard stop to MPFIT_DEFITER, 28 Feb 1999 CM
Cosmetic documentation changes, 14 May 1999 CM
IDL optimizations for speed & FASTNORM keyword, 15 May 1999 CM
Tried a faster version of mpfit_enorm, 30 May 1999 CM
Changed web address to cow.physics.wisc.edu, 14 Jun 1999 CM
Found malformation of FDJAC in MPFIT for 1 parm, 03 Aug 1999 CM
Factored out user-function call into MPFIT_CALL. It is possible,
but currently disabled, to call procedures. The calling format
is similar to CURVEFIT, 25 Sep 1999, CM
Slightly changed mpfit_tie to be less intrusive, 25 Sep 1999, CM
Fixed some bugs associated with tied parameters in mpfit_fdjac, 25
Sep 1999, CM
Reordered documentation; now alphabetical, 02 Oct 1999, CM
Added QUERY keyword for more robust error detection in drivers, 29
Oct 1999, CM
Documented PERROR for unweighted fits, 03 Nov 1999, CM
Split out MPFIT_RESETPROF to aid in profiling, 03 Nov 1999, CM
Some profiling and speed optimization, 03 Nov 1999, CM
Worst offenders, in order: fdjac2, qrfac, qrsolv, enorm.
fdjac2 depends on user function, qrfac and enorm seem to be
fully optimized. qrsolv probably could be tweaked a little, but
is still <10% of total compute time.
Made sure that !err was set to 0 in MPFIT_DEFITER, 10 Jan 2000, CM
Fixed small inconsistency in setting of QANYLIM, 28 Jan 2000, CM
Added PARINFO field RELSTEP, 28 Jan 2000, CM
Converted to MPFIT_ERROR common block for indicating error
conditions, 28 Jan 2000, CM
Corrected scope of MPFIT_ERROR common block, CM, 07 Mar 2000
Minor speed improvement in MPFIT_ENORM, CM 26 Mar 2000
Corrected case where ITERPROC changed parameter values and
parameter values were TIED, CM 26 Mar 2000
Changed MPFIT_CALL to modify NFEV automatically, and to support
user procedures more, CM 26 Mar 2000
Copying permission terms have been liberalized, 26 Mar 2000, CM
Catch zero value of zero a(j,lj) in MPFIT_QRFAC, 20 Jul 2000, CM
(thanks to David Schlegel )
MPFIT_SETMACHAR is called only once at init; only one common block
is created (MPFIT_MACHAR); it is now a structure; removed almost
all CHECK_MATH calls for compatibility with IDL5 and !EXCEPT;
profiling data is now in a structure too; noted some
mathematical discrepancies in Linux IDL5.0, 17 Nov 2000, CM
Some significant changes. New PARINFO fields: MPSIDE, MPMINSTEP,
MPMAXSTEP. Improved documentation. Now PTIED constraints are
maintained in the MPCONFIG common block. A new procedure to
parse PARINFO fields. FDJAC2 now computes a larger variety of
one-sided and two-sided finite difference derivatives. NFEV is
stored in the MPCONFIG common now. 17 Dec 2000, CM
Added check that PARINFO and XALL have same size, 29 Dec 2000 CM
Don't call function in TERMINATE when there is an error, 05 Jan
2000
Check for float vs. double discrepancies; corrected implementation
of MIN/MAXSTEP, which I still am not sure of, but now at least
the correct behavior occurs *without* it, CM 08 Jan 2001
Added SCALE_FCN keyword, to allow for scaling, as for the CASH
statistic; added documentation about the theory of operation,
and under the QR factorization; slowly I'm beginning to
understand the bowels of this algorithm, CM 10 Jan 2001
Remove MPMINSTEP field of PARINFO, for now at least, CM 11 Jan
2001
Added RESDAMP keyword, CM, 14 Jan 2001
Tried to improve the DAMP handling a little, CM, 13 Mar 2001
Corrected .PARNAME behavior in _DEFITER, CM, 19 Mar 2001
Added checks for parameter and function overflow; a new STATUS
value to reflect this; STATUS values of -15 to -1 are reserved
for user function errors, CM, 03 Apr 2001
DAMP keyword is now a TANH, CM, 03 Apr 2001
Added more error checking of float vs. double, CM, 07 Apr 2001
Fixed bug in handling of parameter lower limits; moved overflow
checking to end of loop, CM, 20 Apr 2001
Failure using GOTO, TERMINATE more graceful if FNORM1 not defined,
CM, 13 Aug 2001
Add MPPRINT tag to PARINFO, CM, 19 Nov 2001
Add DOF keyword to DEFITER procedure, and print degrees of
freedom, CM, 28 Nov 2001
Add check to be sure MYFUNCT is a scalar string, CM, 14 Jan 2002
Addition of EXTERNAL_FJAC, EXTERNAL_FVEC keywords; ability to save
fitter's state from one call to the next; allow '(EXTERNAL)'
function name, which implies that user will supply function and
Jacobian at each iteration, CM, 10 Mar 2002
Documented EXTERNAL evaluation code, CM, 10 Mar 2002
Corrected signficant bug in the way that the STEP parameter, and
FIXED parameters interacted (Thanks Andrew Steffl), CM, 02 Apr
2002
Allow COVAR and PERROR keywords to be computed, even in case of
'(EXTERNAL)' function, 26 May 2002
Add NFREE and NPEGGED keywords; compute NPEGGED; compute DOF using
NFREE instead of n_elements(X), thanks to Kristian Kjaer, CM 11
Sep 2002
Hopefully PERROR is all positive now, CM 13 Sep 2002
Documented RELSTEP field of PARINFO (!!), CM, 25 Oct 2002
$Id: mpfit.pro,v 1.24 2002/10/26 00:44:47 craigm Exp $
(See mpfit.pro)
NAME:
MPFIT2DFUN
AUTHOR:
Craig B. Markwardt, NASA/GSFC Code 662, Greenbelt, MD 20770
craigm@lheamail.gsfc.nasa.gov
UPDATED VERSIONs can be found on my WEB PAGE:
http://cow.physics.wisc.edu/~craigm/idl/idl.html
PURPOSE:
Perform Levenberg-Marquardt least-squares fit to a 2-D IDL function
MAJOR TOPICS:
Curve and Surface Fitting
CALLING SEQUENCE:
parms = MPFIT2DFUN(MYFUNCT, X, Y, Z, ERR, start_parms, ...)
DESCRIPTION:
MPFIT2DFUN fits a user-supplied model -- in the form of an IDL
function -- to a set of user-supplied data. MPFIT2DFUN calls
MPFIT, the MINPACK-1 least-squares minimizer, to do the main
work. MPFIT2DFUN is a specialized version for two-dimensional
data.
Given the data and their uncertainties, MPFIT2DFUN finds the best set
of model parameters which match the data (in a least-squares
sense) and returns them in an array.
The user must supply the following items:
- Two arrays of independent variable values ("X", "Y").
- An array of "measured" *dependent* variable values ("Z").
- An array of "measured" 1-sigma uncertainty values ("ERR").
- The name of an IDL function which computes Z given (X,Y) ("MYFUNCT").
- Starting guesses for all of the parameters ("START_PARAMS").
There are very few restrictions placed on X, Y, Z, or MYFUNCT.
Simply put, MYFUNCT must map the (X,Y) values into Z values given
the model parameters. The (X,Y) values are usually the independent
X and Y coordinate positions in the two dimensional plane, but need
not be.
MPFIT2DFUN carefully avoids passing large arrays where possible to
improve performance.
See below for an example of usage.
USER FUNCTION
The user must define a function which returns the model value. For
applications which use finite-difference derivatives -- the default
-- the user function should be declared in the following way:
FUNCTION MYFUNCT, X, Y, P
; The independent variables are X and Y
; Parameter values are passed in "P"
ZMOD = ... computed model values at (X,Y) ...
return, ZMOD
END
The returned array YMOD must have the same dimensions and type as
the "measured" Z values.
User functions may also indicate a fatal error condition
using the ERROR_CODE common block variable, as described
below under the MPFIT_ERROR common block definition.
See the discussion under "ANALYTIC DERIVATIVES" and AUTODERIVATIVE
in MPFIT.PRO if you wish to compute the derivatives for yourself.
AUTODERIVATIVE is accepted and passed directly to MPFIT. The user
function must accept one additional parameter, DP, which contains
the derivative of the user function with respect to each parameter
at each data point, as described in MPFIT.PRO.
CREATING APPROPRIATELY DIMENSIONED INDEPENDENT VARIABLES
The user must supply appropriate independent variables to
MPFIT2DFUN. For image fitting applications, this variable should
be two-dimensional *arrays* describing the X and Y positions of
every *pixel*. [ Thus any two dimensional sampling is permitted,
including irregular sampling. ]
If the sampling is regular, then the x coordinates are the same for
each row, and the y coordinates are the same for each column. Call
the x-row and y-column coordinates XR and YC respectively. You can
then compute X and Y as follows:
X = XR # (YC*0 + 1) eqn. 1
Y = (XR*0 + 1) # YC eqn. 2
For example, if XR and YC have the following values:
XR = [ 1, 2, 3, 4, 5,] ;; X positions of one row of pixels
YC = [ 15,16,17 ] ;; Y positions of one column of
pixels
Then using equations 1 and 2 above will give these values to X and
Y:
X : 1 2 3 4 5 ;; X positions of all pixels
1 2 3 4 5
1 2 3 4 5
Y : 15 15 15 15 15 ;; Y positions of all pixels
16 16 16 16 16
17 17 17 17 17
Using the above technique is suggested, but *not* required. You
can do anything you wish with the X and Y values. This technique
only makes it easier to compute your model function values.
CONSTRAINING PARAMETER VALUES WITH THE PARINFO KEYWORD
The behavior of MPFIT can be modified with respect to each
parameter to be fitted. A parameter value can be fixed; simple
boundary constraints can be imposed; limitations on the parameter
changes can be imposed; properties of the automatic derivative can
be modified; and parameters can be tied to one another.
These properties are governed by the PARINFO structure, which is
passed as a keyword parameter to MPFIT.
PARINFO should be an array of structures, one for each parameter.
Each parameter is associated with one element of the array, in
numerical order. The structure can have the following entries
(none are required):
.VALUE - the starting parameter value (but see the START_PARAMS
parameter for more information).
.FIXED - a boolean value, whether the parameter is to be held
fixed or not. Fixed parameters are not varied by
MPFIT, but are passed on to MYFUNCT for evaluation.
.LIMITED - a two-element boolean array. If the first/second
element is set, then the parameter is bounded on the
lower/upper side. A parameter can be bounded on both
sides. Both LIMITED and LIMITS must be given
together.
.LIMITS - a two-element float or double array. Gives the
parameter limits on the lower and upper sides,
respectively. Zero, one or two of these values can be
set, depending on the values of LIMITED. Both LIMITED
and LIMITS must be given together.
.PARNAME - a string, giving the name of the parameter. The
fitting code of MPFIT does not use this tag in any
way. However, the default ITERPROC will print the
parameter name if available.
.STEP - the step size to be used in calculating the numerical
derivatives. If set to zero, then the step size is
computed automatically. Ignored when AUTODERIVATIVE=0.
This value is superceded by the RELSTEP value.
.RELSTEP - the *relative* step size to be used in calculating
the numerical derivatives. This number is the
fractional size of the step, compared to the
parameter value. This value supercedes the STEP
setting. If the parameter is zero, then a default
step size is chosen.
.MPSIDE - the sidedness of the finite difference when computing
numerical derivatives. This field can take four
values:
0 - one-sided derivative computed automatically
1 - one-sided derivative (f(x+h) - f(x) )/h
-1 - one-sided derivative (f(x) - f(x-h))/h
2 - two-sided derivative (f(x+h) - f(x-h))/(2*h)
Where H is the STEP parameter described above. The
"automatic" one-sided derivative method will chose a
direction for the finite difference which does not
violate any constraints. The other methods do not
perform this check. The two-sided method is in
principle more precise, but requires twice as many
function evaluations. Default: 0.
.MPMINSTEP - the minimum change to be made in the parameter
value. During the fitting process, the parameter
will be changed by multiples of this value. The
actual step is computed as:
DELTA1 = MPMINSTEP*ROUND(DELTA0/MPMINSTEP)
where DELTA0 and DELTA1 are the estimated parameter
changes before and after this constraint is
applied. Note that this constraint should be used
with care since it may cause non-converging,
oscillating solutions.
A value of 0 indicates no minimum. Default: 0.
.MPMAXSTEP - the maximum change to be made in the parameter
value. During the fitting process, the parameter
will never be changed by more than this value.
A value of 0 indicates no maximum. Default: 0.
.TIED - a string expression which "ties" the parameter to other
free or fixed parameters. Any expression involving
constants and the parameter array P are permitted.
Example: if parameter 2 is always to be twice parameter
1 then use the following: parinfo(2).tied = '2 * P(1)'.
Since they are totally constrained, tied parameters are
considered to be fixed; no errors are computed for them.
[ NOTE: the PARNAME can't be used in expressions. ]
Future modifications to the PARINFO structure, if any, will involve
adding structure tags beginning with the two letters "MP".
Therefore programmers are urged to avoid using tags starting with
the same letters; otherwise they are free to include their own
fields within the PARINFO structure, and they will be ignored.
PARINFO Example:
parinfo = replicate({value:0.D, fixed:0, limited:[0,0], $
limits:[0.D,0]}, 5)
parinfo(0).fixed = 1
parinfo(4).limited(0) = 1
parinfo(4).limits(0) = 50.D
parinfo(*).value = [5.7D, 2.2, 500., 1.5, 2000.]
A total of 5 parameters, with starting values of 5.7,
2.2, 500, 1.5, and 2000 are given. The first parameter
is fixed at a value of 5.7, and the last parameter is
constrained to be above 50.
INPUTS:
MYFUNCT - a string variable containing the name of an IDL
function. This function computes the "model" Z values
given the X,Y values and model parameters, as described above.
X - Array of "X" independent variable values, as described above.
These values are passed directly to the fitting function
unmodified.
Y - Array of "Y" independent variable values, as described
above. X and Y should have the same data type.
Z - Array of "measured" dependent variable values. Z should have
the same data type as X and Y. The function MYFUNCT should
map (X,Y)->Z.
ERR - Array of "measured" 1-sigma uncertainties. ERR should have
the same data type as Z. ERR is ignored if the WEIGHTS
keyword is specified.
START_PARAMS - An array of starting values for each of the
parameters of the model. The number of parameters
should be fewer than the number of measurements.
Also, the parameters should have the same data type
as the measurements (double is preferred).
This parameter is optional if the PARINFO keyword
is used (see MPFIT). The PARINFO keyword provides
a mechanism to fix or constrain individual
parameters. If both START_PARAMS and PARINFO are
passed, then the starting *value* is taken from
START_PARAMS, but the *constraints* are taken from
PARINFO.
RETURNS:
Returns the array of best-fit parameters.
KEYWORD PARAMETERS:
BESTNORM - the value of the summed squared residuals for the
returned parameter values.
COVAR - the covariance matrix for the set of parameters returned
by MPFIT. The matrix is NxN where N is the number of
parameters. The square root of the diagonal elements
gives the formal 1-sigma statistical errors on the
parameters IF errors were treated "properly" in MYFUNC.
Parameter errors are also returned in PERROR.
To compute the correlation matrix, PCOR, use this:
IDL> PCOR = COV * 0
IDL> FOR i = 0, n-1 DO FOR j = 0, n-1 DO $
PCOR(i,j) = COV(i,j)/sqrt(COV(i,i)*COV(j,j))
If NOCOVAR is set or MPFIT terminated abnormally, then
COVAR is set to a scalar with value !VALUES.D_NAN.
ERRMSG - a string error or warning message is returned.
FTOL - a nonnegative input variable. Termination occurs when both
the actual and predicted relative reductions in the sum of
squares are at most FTOL (and STATUS is accordingly set to
1 or 3). Therefore, FTOL measures the relative error
desired in the sum of squares. Default: 1D-10
FUNCTARGS - A structure which contains the parameters to be passed
to the user-supplied function specified by MYFUNCT via
the _EXTRA mechanism. This is the way you can pass
additional data to your user-supplied function without
using common blocks.
By default, no extra parameters are passed to the
user-supplied function.
GTOL - a nonnegative input variable. Termination occurs when the
cosine of the angle between fvec and any column of the
jacobian is at most GTOL in absolute value (and STATUS is
accordingly set to 4). Therefore, GTOL measures the
orthogonality desired between the function vector and the
columns of the jacobian. Default: 1D-10
ITERARGS - The keyword arguments to be passed to ITERPROC via the
_EXTRA mechanism. This should be a structure, and is
similar in operation to FUNCTARGS.
Default: no arguments are passed.
ITERPROC - The name of a procedure to be called upon each NPRINT
iteration of the MPFIT routine. It should be declared
in the following way:
PRO ITERPROC, MYFUNCT, p, iter, fnorm, FUNCTARGS=fcnargs, $
PARINFO=parinfo, QUIET=quiet, ...
; perform custom iteration update
END
ITERPROC must either accept all three keyword
parameters (FUNCTARGS, PARINFO and QUIET), or at least
accept them via the _EXTRA keyword.
MYFUNCT is the user-supplied function to be minimized,
P is the current set of model parameters, ITER is the
iteration number, and FUNCTARGS are the arguments to be
passed to MYFUNCT. FNORM should be the
chi-squared value. QUIET is set when no textual output
should be printed. See below for documentation of
PARINFO.
In implementation, ITERPROC can perform updates to the
terminal or graphical user interface, to provide
feedback while the fit proceeds. If the fit is to be
stopped for any reason, then ITERPROC should set the
common block variable ERROR_CODE to negative value (see
MPFIT_ERROR common block below). In principle,
ITERPROC should probably not modify the parameter
values, because it may interfere with the algorithm's
stability. In practice it is allowed.
Default: an internal routine is used to print the
parameter values.
MAXITER - The maximum number of iterations to perform. If the
number is exceeded, then the STATUS value is set to 5
and MPFIT returns.
Default: 200 iterations
NFEV - the number of MYFUNCT function evaluations performed.
NITER - the number of iterations completed.
NOCOVAR - set this keyword to prevent the calculation of the
covariance matrix before returning (see COVAR)
NPRINT - The frequency with which ITERPROC is called. A value of
1 indicates that ITERPROC is called with every iteration,
while 2 indicates every other iteration, etc. Note that
several Levenberg-Marquardt attempts can be made in a
single iteration.
Default value: 1
PARINFO - Provides a mechanism for more sophisticated constraints
to be placed on parameter values. When PARINFO is not
passed, then it is assumed that all parameters are free
and unconstrained. Values in PARINFO are never
modified during a call to MPFIT.
See description above for the structure of PARINFO.
Default value: all parameters are free and unconstrained.
PERROR - The formal 1-sigma errors in each parameter, computed
from the covariance matrix. If a parameter is held
fixed, or if it touches a boundary, then the error is
reported as zero.
If the fit is unweighted (i.e. no errors were given, or
the weights were uniformly set to unity), then PERROR
will probably not represent the true parameter
uncertainties. *If* you can assume that the true reduced
chi-squared value is unity -- meaning that the fit is
implicitly assumed to be of good quality -- then the
estimated parameter uncertainties can be computed by
scaling PERROR by the measured chi-squared value.
DOF = N_ELEMENTS(Z) - N_ELEMENTS(PARMS) ; deg of freedom
PCERROR = PERROR * SQRT(BESTNORM / DOF) ; scaled uncertainties
QUIET - set this keyword when no textual output should be printed
by MPFIT
STATUS - an integer status code is returned. All values other
than zero can represent success. It can have one of the
following values:
0 improper input parameters.
1 both actual and predicted relative reductions
in the sum of squares are at most FTOL.
2 relative error between two consecutive iterates
is at most XTOL
3 conditions for STATUS = 1 and STATUS = 2 both hold.
4 the cosine of the angle between fvec and any
column of the jacobian is at most GTOL in
absolute value.
5 the maximum number of iterations has been reached
6 FTOL is too small. no further reduction in
the sum of squares is possible.
7 XTOL is too small. no further improvement in
the approximate solution x is possible.
8 GTOL is too small. fvec is orthogonal to the
columns of the jacobian to machine precision.
WEIGHTS - Array of weights to be used in calculating the
chi-squared value. If WEIGHTS is specified then the ERR
parameter is ignored. The chi-squared value is computed
as follows:
CHISQ = TOTAL( (Z-MYFUNCT(X,Y,P))^2 * ABS(WEIGHTS) )
Here are common values of WEIGHTS:
1D/ERR^2 - Normal weighting (ERR is the measurement error)
1D/Z - Poisson weighting (counting statistics)
1D - Unweighted
XTOL - a nonnegative input variable. Termination occurs when the
relative error between two consecutive iterates is at most
XTOL (and STATUS is accordingly set to 2 or 3). Therefore,
XTOL measures the relative error desired in the approximate
solution. Default: 1D-10
YFIT - the best-fit model function, as returned by MYFUNCT.
EXAMPLE:
p = [2.2D, -0.7D, 1.4D, 3000.D]
x = (dindgen(200)*0.1 - 10.) # (dblarr(200) + 1)
y = (dblarr(200) + 1) # (dindgen(200)*0.1 - 10.)
zi = gauss2(x, y, p)
sz = sqrt(zi)
z = zi + randomn(seed, 200, 200) * sz
p0 = [0D, 0D, 1D, 10D]
p = mpfit2dfun('GAUSS2', x, y, z, sz, p0)
Generates a synthetic data set with a Gaussian peak, and Poisson
statistical uncertainty. Then the same function (but different
starting parameters) is fitted to the data to see how close we can
get.
It is especially worthy to notice that the X and Y values are
created as full images, so that a coordinate is attached to each
pixel independently. This is the format that GAUSS2 accepts, and
the easiest for you to use in your own functions.
COMMON BLOCKS:
COMMON MPFIT_ERROR, ERROR_CODE
User routines may stop the fitting process at any time by
setting an error condition. This condition may be set in either
the user's model computation routine (MYFUNCT), or in the
iteration procedure (ITERPROC).
To stop the fitting, the above common block must be declared,
and ERROR_CODE must be set to a negative number. After the user
procedure or function returns, MPFIT checks the value of this
common block variable and exits immediately if the error
condition has been set. By default the value of ERROR_CODE is
zero, indicating a successful function/procedure call.
REFERENCES:
MINPACK-1, Jorge More', available from netlib (www.netlib.org).
"Optimization Software Guide," Jorge More' and Stephen Wright,
SIAM, *Frontiers in Applied Mathematics*, Number 14.
MODIFICATION HISTORY:
Written, transformed from MPFITFUN, 26 Sep 1999, CM
Alphabetized documented keywords, 02 Oct 1999, CM
Added example, 02 Oct 1999, CM
Tried to clarify definitions of X and Y, 29 Oct 1999, CM
Added QUERY keyword and query checking of MPFIT, 29 Oct 1999, CM
Check to be sure that X, Y and Z are present, 02 Nov 1999, CM
Documented PERROR for unweighted fits, 03 Nov 1999, CM
Changed to ERROR_CODE for error condition, 28 Jan 2000, CM
Copying permission terms have been liberalized, 26 Mar 2000, CM
Propagated improvements from MPFIT, 17 Dec 2000, CM
Documented RELSTEP field of PARINFO (!!), CM, 25 Oct 2002
$Id: mpfit2dfun.pro,v 1.3 2002/11/07 00:12:54 craigm Exp $
(See mpfit2dfun.pro)
NAME:
MPFIT2DPEAK
AUTHOR:
Craig B. Markwardt, NASA/GSFC Code 662, Greenbelt, MD 20770
craigm@lheamail.gsfc.nasa.gov
UPDATED VERSIONs can be found on my WEB PAGE:
http://cow.physics.wisc.edu/~craigm/idl/idl.html
PURPOSE:
Fit a gaussian, lorentzian or Moffat model to data
MAJOR TOPICS:
Curve and Surface Fitting
CALLING SEQUENCE:
yfit = MPFIT2DPEAK(Z, A [, X, Y, /TILT ...] )
DESCRIPTION:
MPFIT2DPEAK fits a gaussian, lorentzian or Moffat model using the
non-linear least squares fitter MPFIT. MPFIT2DPEAK is meant to be
a drop-in replacement for IDL's GAUSS2DFIT function (and requires
MPFIT and MPFIT2DFUN).
The choice of the fitting function is determined by the keywords
GAUSSIAN, LORENTZIAN and MOFFAT. By default the gaussian model
function is used. [ The Moffat function is a modified Lorentzian
with variable power law index. ] The two-dimensional peak has
independent semimajor and semiminor axes, with an optional
rotation term activated by setting the TILT keyword. The baseline
is assumed to be a constant.
GAUSSIAN A(0) + A(1)*exp(-0.5*u)
LORENTZIAN A(0) + A(1)/(u + 1)
MOFFAT A(0) + A(1)/(u + 1)^A(7)
u = ( (x-A(4))/A(2) )^2 + ( (y-A(5))/A(3) )^2
where x and y are cartesian coordinates in rotated
coordinate system if TILT keyword is set.
The returned parameter array elements have the following meanings:
A(0) Constant baseline level
A(1) Peak value
A(2) Peak half-width (x) -- gaussian sigma or half-width at half-max
A(3) Peak half-width (y) -- gaussian sigma or half-width at half-max
A(4) Peak centroid (x)
A(5) Peak centroid (y)
A(6) Rotation angle (radians) if TILT keyword set
A(7) Moffat power law index if MOFFAT keyword set
By default the initial starting values for the parameters A are
estimated from the data. However, explicit starting values can be
supplied using the ESTIMATES keyword. Also, error or weighting
values can optionally be provided; otherwise the fit is
unweighted.
RESTRICTIONS:
If no starting parameter ESTIMATES are provided, then MPFIT2DPEAK
attempts to estimate them from the data. This is not a perfect
science; however, the author believes that the technique
implemented here is more robust than the one used in IDL's
GAUSS2DFIT. The author has tested cases of strong peaks, noisy
peaks and broad peaks, all with success.
INPUTS:
Z - Two dimensional array of "measured" dependent variable values.
Z should be of the same type and dimension as (X # Y).
X - Optional vector of x positions for a single row of Z.
X(i) should provide the x position of Z(i,*)
Default: X values are integer increments from 0 to NX-1
Y - Optional vector of y positions for a single column of Z.
Y(j) should provide the j position of Z(*,j)
Default: Y values are integer increments from 0 to NY-1
OUTPUTS:
A - Upon return, an array of best fit parameter values. See the
table above for the meanings of each parameter element.
RETURNS:
Returns the best fitting model function as a 2D array.
KEYWORDS:
** NOTE ** Additional keywords such as PARINFO, BESTNORM, and
STATUS are accepted by MPFIT2DPEAK but not documented
here. Please see the documentation for MPFIT for the
description of these advanced options.
ERROR - upon input, the measured 1-sigma uncertainties in the "Z"
values. If no ERROR or WEIGHTS are given, then the fit is
unweighted.
ESTIMATES - Array of starting values for each parameter of the
model.
Default: parameter values are estimated from data.
GAUSSIAN - if set, fit a gaussian model function. The Default.
LORENTZIAN - if set, fit a lorentzian model function.
MOFFAT - if set, fit a Moffat model function.
NEGATIVE - if set, and ESTIMATES is not provided, then MPFIT2DPEAK
will assume that a negative peak is present -- a
valley. Specifying this keyword is not normally
required, since MPFIT2DPEAK can determine this
automatically.
PERROR - upon return, the 1-sigma uncertainties of the parameter
values A. These values are only meaningful if the ERRORS
or WEIGHTS keywords are specified properly.
If the fit is unweighted (i.e. no errors were given, or
the weights were uniformly set to unity), then PERROR
will probably not represent the true parameter
uncertainties.
*If* you can assume that the true reduced chi-squared
value is unity -- meaning that the fit is implicitly
assumed to be of good quality -- then the estimated
parameter uncertainties can be computed by scaling PERROR
by the measured chi-squared value.
DOF = N_ELEMENTS(Z) - N_ELEMENTS(A) ; deg of freedom
PCERROR = PERROR * SQRT(BESTNORM / DOF) ; scaled uncertainties
QUIET - if set then diagnostic fitting messages are suppressed.
Default: QUIET=1 (i.e., no diagnostics)
CIRCULAR - if set, then the peak profile is assumed to be
azimuthally symmetric. When set, the parameters A(2)
and A(3) will be identical and the TILT keyword will
have no effect.
TILT - if set, then the major and minor axes of the peak profile
are allowed to rotate with respect to the image axes.
Parameter A(6) will be set to the clockwise rotation angle
of the A(2) axis in radians.
WEIGHTS - Array of weights to be used in calculating the
chi-squared value. If WEIGHTS is specified then the ERR
parameter is ignored. The chi-squared value is computed
as follows:
CHISQ = TOTAL( (Z-MYFUNCT(X,Y,P))^2 * ABS(WEIGHTS) )
Here are common values of WEIGHTS:
1D/ERR^2 - Normal weighting (ERR is the measurement error)
1D/Y - Poisson weighting (counting statistics)
1D - Unweighted
The ERROR keyword takes precedence over any WEIGHTS
keyword values. If no ERROR or WEIGHTS are given, then
the fit is unweighted.
EXAMPLE:
; Construct a sample gaussian surface in range [-5,5] centered at [2,-3]
x = findgen(100)*0.1 - 5. & y = x
xx = x # (y*0 + 1)
yy = (x*0 + 1) # y
rr = sqrt((xx-2.)^2 + (yy+3.)^2)
; Gaussian surface with sigma=0.5, peak value of 3, noise with sigma=0.2
z = 3.*exp(-(rr/0.5)^2) + randomn(seed,100,100)*.2
; Fit gaussian parameters A
zfit = mpfit2dpeak(z, a, x, y)
REFERENCES:
MINPACK-1, Jorge More', available from netlib (www.netlib.org).
"Optimization Software Guide," Jorge More' and Stephen Wright,
SIAM, *Frontiers in Applied Mathematics*, Number 14.
MODIFICATION HISTORY:
New algorithm for estimating starting values, CM, 31 Oct 1999
Documented, 02 Nov 1999
Small documentation fixes, 02 Nov 1999
Documented PERROR for unweighted fits, 03 Nov 1999, CM
Copying permission terms have been liberalized, 26 Mar 2000, CM
Small cosmetic changes, 21 Sep 2000, CM
Corrected bug introduced by cosmetic changes, 11 Oct 2000, CM :-)
Added POSITIVE keyword, 17 Nov 2000, CM
Removed TILT in common, in favor of FUNCTARGS approach, 23 Nov
2000, CM
Added SYMMETRIC keyword, documentation for TILT, and an example,
24 Nov 2000, CM
Changed SYMMETRIC to CIRCULAR, 17 Dec 2000, CM
Really change SYMMETRIC to CIRCULAR!, 13 Sep 2002, CM
Add error messages for SYMMETRIC and CIRCLE, 08 Nov 2002, CM
$Id: mpfit2dpeak.pro,v 1.4 2002/11/08 15:44:24 craigm Exp $
(See mpfit2dpeak.pro)
NAME:
MPFITELLIPSE
AUTHOR:
Craig B. Markwardt, NASA/GSFC Code 662, Greenbelt, MD 20770
craigm@lheamail.gsfc.nasa.gov
UPDATED VERSIONs can be found on my WEB PAGE:
http://cow.physics.wisc.edu/~craigm/idl/idl.html
PURPOSE:
Approximate fit to points forming an ellipse
MAJOR TOPICS:
Curve and Surface Fitting
CALLING SEQUENCE:
parms = MPFITELLIPSE(X, Y, start_parms, [/TILT, WEIGHTS=wts, ...])
DESCRIPTION:
MPFITELLIPSE fits a closed elliptical or circular curve to a two
dimensional set of data points. The user specifies the X and Y
positions of the points, and an optional set of weights. The
ellipse may also be tilted at an arbitrary angle.
The best fitting ellipse parameters are returned from by
MPFITELLIPSE as a vector, whose values are:
P(0) Ellipse semi axis 1
P(1) Ellipse semi axis 2 ( = P(0) if CIRCLE keyword set)
P(2) Ellipse center - x value
P(3) Ellipse center - y value
P(4) Ellipse rotation angle (radians) if TILT keyword set
The user may specify an initial set of trial parameters, but by
default MPFITELLIPSE will estimate the parameters automatically.
Users should be aware that in the presence of large amounts of
noise, namely when the measurement error becomes significant
compared to the ellipse axis length, then the estimated parameters
become unreliable. Generally speaking the computed axes will
overestimate the true axes. For example when (SIGMA_R/R) becomes
0.5, the radius of the ellipse is overestimated by about 40%.
Users can weight their data as they see appropriate. However, the
following prescription for the weighting appears to be the correct
one, and produces values comparable to the typical chi-squared
value.
WEIGHTS = 0.75/(SIGMA_X^2 + SIGMA_Y^2)
where SIGMA_X and SIGMA_Y are the measurement error vectors in the
X and Y directions respectively. However, it should be pointed
out that this weighting is only appropriate for a set of points
whose measurement errors are comparable. If a more robust
estimation of the parameter values is needed, the so-called
orthogonal distance regression package should be used (ODRPACK,
available in FORTRAN at www.netlib.org).
INPUTS:
X - measured X positions of the points in the ellipse.
Y - measured Y positions of the points in the ellipse.
START_PARAMS - an array of starting values for the ellipse
parameters, as described above. This parameter is
optional; if not specified by the user, then the
ellipse parameters are estimated automatically from
the properties of the data.
RETURNS:
Returns the best fitting model ellipse parameters.
KEYWORDS:
** NOTE ** Additional keywords such as PARINFO, BESTNORM, and
STATUS are accepted by MPFITELLIPSE but not documented
here. Please see the documentation for MPFIT for the
description of these advanced options.
PERROR - upon return, the 1-sigma uncertainties of the returned
ellipse parameter values. These values are only
meaningful if the WEIGHTS keyword is specified properly.
If the fit is unweighted (i.e. no errors were given, or
the weights were uniformly set to unity), then PERROR
will probably not represent the true parameter
uncertainties.
QUIET - if set then diagnostic fitting messages are suppressed.
Default: QUIET=1 (i.e., no diagnostics)
CIRCULAR - if set, then the curve is assumed to be a circle
instead of ellipse. When set, the parameters P(0) and
P(1) will be identical and the TILT keyword will have
no effect.
TILT - if set, then the major and minor axes of the ellipse
are allowed to rotate with respect to the data axes.
Parameter P(4) will be set to the clockwise rotation angle
of the P(0) axis in radians.
WEIGHTS - Array of weights to be used in calculating the
chi-squared value. If WEIGHTS is specified then the ERR
parameter is ignored. The chi-squared value is computed
as follows:
CHISQ = TOTAL( (Z-MYFUNCT(X,Y,P))^2 * ABS(WEIGHTS)^2 )
Users may wish to follow the guidelines for WEIGHTS
described above.
EXAMPLE:
; Construct a set of points on an ellipse, with some noise
ph0 = 2*!pi*randomu(seed,50)
x = 50. + 32.*cos(ph0) + 4.0*randomn(seed, 50)
y = -75. + 65.*sin(ph0) + 0.1*randomn(seed, 50)
; Compute weights function
weights = 0.75/(4.0^2 + 0.1^2)
; Fit ellipse and plot result
p = mpfitellipse(x, y)
plot, x, y, psym=1
phi = dindgen(101)*2D*!dpi/100
oplot, p(2)+p(0)*cos(phi), p(3)+p(1)*sin(phi)
REFERENCES:
MINPACK-1, Jorge More', available from netlib (www.netlib.org).
"Optimization Software Guide," Jorge More' and Stephen Wright,
SIAM, *Frontiers in Applied Mathematics*, Number 14.
MODIFICATION HISTORY:
Ported from MPFIT2DPEAK, 17 Dec 2000, CM
More documentation, 11 Jan 2001, CM
Example corrected, 18 Nov 2001, CM
Change CIRCLE keyword to the correct CIRCULAR keyword, 13 Sep
2002, CM
Add error messages for SYMMETRIC and CIRCLE, 08 Nov 2002, CM
Found small error in computation of _EVAL (when CIRCULAR) was set;
sanity check when CIRCULAR is set, 21 Jan 2003, CM
$Id: mpfitellipse.pro,v 1.8 2003/01/24 04:04:30 craigm Exp $
(See mpfitellipse.pro)
NAME:
MPFITEXPR
AUTHOR:
Craig B. Markwardt, NASA/GSFC Code 662, Greenbelt, MD 20770
craigm@lheamail.gsfc.nasa.gov
UPDATED VERSIONs can be found on my WEB PAGE:
http://cow.physics.wisc.edu/~craigm/idl/idl.html
PURPOSE:
Perform Levenberg-Marquardt least-squares fit to arbitrary expression
MAJOR TOPICS:
Curve and Surface Fitting
CALLING SEQUENCE:
MYFUNCT = 'X*(1-X)+3'
parms = MPFITEXPR(MYFUNCT, XVAL, YVAL, ERR, start_parms, ...)
DESCRIPTION:
MPFITEXPR fits a user-supplied model -- in the form of an arbitrary IDL
expression -- to a set of user-supplied data. MPFITEXPR calls
MPFIT, the MINPACK-1 least-squares minimizer, to do the main
work.
Given the data and their uncertainties, MPFITEXPR finds the best set
of model parameters which match the data (in a least-squares
sense) and returns them in an array.
The user must supply the following items:
- An array of independent variable values ("X").
- An array of "measured" *dependent* variable values ("Y").
- An array of "measured" 1-sigma uncertainty values ("ERR").
- A text IDL expression which computes Y given X.
- Starting guesses for all of the parameters ("START_PARAMS").
There are very few restrictions placed on X, Y or the expression of
the model. Simply put, the expression must map the "X" values into
"Y" values given the model parameters. The "X" values may
represent any independent variable (not just Cartesian X), and
indeed may be multidimensional themselves. For example, in the
application of image fitting, X may be a 2xN array of image
positions.
Some rules must be obeyed in constructing the expression. First,
the independent variable name *MUST* be "X" in the expression,
regardless of the name of the variable being passed to MPFITEXPR.
This is demonstrated in the above calling sequence, where the X
variable passed in is called "XVAL" but the expression still refers
to "X". Second, parameter values must be referred to as an array
named "P".
If you do not pass in starting values for the model parameters,
MPFITEXPR will attempt to determine the number of parameters you
intend to have (it does this by looking for references to the array
variable named "P"). When no starting values are passed in, the
values are assumed to start at zero.
MPFITEXPR carefully avoids passing large arrays where possible to
improve performance.
See below for an example of usage.
EVALUATING EXPRESSIONS
This source module also provides a function called MPEVALEXPR. You
can use this function to evaluate your expression, given a list of
parameters. This is one of the easier ways to compute the model
once the best-fit parameters have been found. Here is an example:
YMOD = MPEVALEXPR(MYFUNCT, XVAL, PARMS)
where MYFUNCT is the expression (see MYFUNCT below), XVAL is the
list of "X" values, and PARMS is an array of parameters. The
returned array YMOD contains the expression MYFUNCT evaluated at
each point in XVAL.
PASSING PRIVATE DATA TO AN EXPRESSION
The most complicated optimization problems typically involve other
external parameters, in addition to the fitted parameters. While
it is extremely easy to rewrite an expression dynamically, in case
one of the external parameters changes, the other possibility is to
use the PRIVATE data structure.
The user merely passes a structure to the FUNCTARGS keyword. The
user expression receives this value as the variable PRIVATE.
MPFITEXPR nevers accesses this variable so it can contain any
desired values. Usually it would be an IDL structure so that any
named external parameters can be passed to the expression.
CONSTRAINING PARAMETER VALUES WITH THE PARINFO KEYWORD
The behavior of MPFIT can be modified with respect to each
parameter to be fitted. A parameter value can be fixed; simple
boundary constraints can be imposed; limitations on the parameter
changes can be imposed; properties of the automatic derivative can
be modified; and parameters can be tied to one another.
These properties are governed by the PARINFO structure, which is
passed as a keyword parameter to MPFIT.
PARINFO should be an array of structures, one for each parameter.
Each parameter is associated with one element of the array, in
numerical order. The structure can have the following entries
(none are required):
.VALUE - the starting parameter value (but see the START_PARAMS
parameter for more information).
.FIXED - a boolean value, whether the parameter is to be held
fixed or not. Fixed parameters are not varied by
MPFIT, but are passed on to MYFUNCT for evaluation.
.LIMITED - a two-element boolean array. If the first/second
element is set, then the parameter is bounded on the
lower/upper side. A parameter can be bounded on both
sides. Both LIMITED and LIMITS must be given
together.
.LIMITS - a two-element float or double array. Gives the
parameter limits on the lower and upper sides,
respectively. Zero, one or two of these values can be
set, depending on the values of LIMITED. Both LIMITED
and LIMITS must be given together.
.PARNAME - a string, giving the name of the parameter. The
fitting code of MPFIT does not use this tag in any
way. However, the default ITERPROC will print the
parameter name if available.
.STEP - the step size to be used in calculating the numerical
derivatives. If set to zero, then the step size is
computed automatically. Ignored when AUTODERIVATIVE=0.
This value is superceded by the RELSTEP value.
.RELSTEP - the *relative* step size to be used in calculating
the numerical derivatives. This number is the
fractional size of the step, compared to the
parameter value. This value supercedes the STEP
setting. If the parameter is zero, then a default
step size is chosen.
.MPSIDE - the sidedness of the finite difference when computing
numerical derivatives. This field can take four
values:
0 - one-sided derivative computed automatically
1 - one-sided derivative (f(x+h) - f(x) )/h
-1 - one-sided derivative (f(x) - f(x-h))/h
2 - two-sided derivative (f(x+h) - f(x-h))/(2*h)
Where H is the STEP parameter described above. The
"automatic" one-sided derivative method will chose a
direction for the finite difference which does not
violate any constraints. The other methods do not
perform this check. The two-sided method is in
principle more precise, but requires twice as many
function evaluations. Default: 0.
.MPMINSTEP - the minimum change to be made in the parameter
value. During the fitting process, the parameter
will be changed by multiples of this value. The
actual step is computed as:
DELTA1 = MPMINSTEP*ROUND(DELTA0/MPMINSTEP)
where DELTA0 and DELTA1 are the estimated parameter
changes before and after this constraint is
applied. Note that this constraint should be used
with care since it may cause non-converging,
oscillating solutions.
A value of 0 indicates no minimum. Default: 0.
.MPMAXSTEP - the maximum change to be made in the parameter
value. During the fitting process, the parameter
will never be changed by more than this value.
A value of 0 indicates no maximum. Default: 0.
.TIED - a string expression which "ties" the parameter to other
free or fixed parameters. Any expression involving
constants and the parameter array P are permitted.
Example: if parameter 2 is always to be twice parameter
1 then use the following: parinfo(2).tied = '2 * P(1)'.
Since they are totally constrained, tied parameters are
considered to be fixed; no errors are computed for them.
[ NOTE: the PARNAME can't be used in expressions. ]
Future modifications to the PARINFO structure, if any, will involve
adding structure tags beginning with the two letters "MP".
Therefore programmers are urged to avoid using tags starting with
the same letters; otherwise they are free to include their own
fields within the PARINFO structure, and they will be ignored.
PARINFO Example:
parinfo = replicate({value:0.D, fixed:0, limited:[0,0], $
limits:[0.D,0]}, 5)
parinfo(0).fixed = 1
parinfo(4).limited(0) = 1
parinfo(4).limits(0) = 50.D
parinfo(*).value = [5.7D, 2.2, 500., 1.5, 2000.]
A total of 5 parameters, with starting values of 5.7,
2.2, 500, 1.5, and 2000 are given. The first parameter
is fixed at a value of 5.7, and the last parameter is
constrained to be above 50.
INPUTS:
MYFUNCT - a string variable containing an IDL expression. The
only restriction is that the independent variable *must*
be referred to as "X" and model parameters *must* be
referred to as an array called "P". Do not use symbol
names beginning with the underscore, "_".
The expression should calculate "model" Y values given
the X values and model parameters. Using the vector
notation of IDL, this can be quite easy to do. If your
expression gets complicated, you may wish to make an IDL
function which will improve performance and readibility.
The resulting array should be of the same size and
dimensions as the "measured" Y values.
X - Array of independent variable values.
Y - Array of "measured" dependent variable values. Y should have
the same data type as X. The function MYFUNCT should map
X->Y.
ERR - Array of "measured" 1-sigma uncertainties. ERR should have
the same data type as Y. ERR is ignored if the WEIGHTS
keyword is specified.
START_PARAMS - An array of starting values for each of the
parameters of the model. The number of parameters
should be fewer than the number of measurements.
Also, the parameters should have the same data type
as the measurements (double is preferred).
This parameter is optional if the PARINFO keyword
is used (see MPFIT). The PARINFO keyword provides
a mechanism to fix or constrain individual
parameters. If both START_PARAMS and PARINFO are
passed, then the starting *value* is taken from
START_PARAMS, but the *constraints* are taken from
PARINFO.
If no parameters are given, then MPFITEXPR attempts
to determine the number of parameters by scanning
the expression. Parameters determined this way are
initialized to zero. This technique is not fully
reliable, so users are advised to pass explicit
parameter starting values.
RETURNS:
Returns the array of best-fit parameters.
KEYWORD PARAMETERS:
BESTNORM - the value of the summed squared residuals for the
returned parameter values.
COVAR - the covariance matrix for the set of parameters returned
by MPFIT. The matrix is NxN where N is the number of
parameters. The square root of the diagonal elements
gives the formal 1-sigma statistical errors on the
parameters IF errors were treated "properly" in MYFUNC.
Parameter errors are also returned in PERROR.
To compute the correlation matrix, PCOR, use this:
IDL> PCOR = COV * 0
IDL> FOR i = 0, n-1 DO FOR j = 0, n-1 DO $
PCOR(i,j) = COV(i,j)/sqrt(COV(i,i)*COV(j,j))
If NOCOVAR is set or MPFIT terminated abnormally, then
COVAR is set to a scalar with value !VALUES.D_NAN.
ERRMSG - a string error or warning message is returned.
FTOL - a nonnegative input variable. Termination occurs when both
the actual and predicted relative reductions in the sum of
squares are at most FTOL (and STATUS is accordingly set to
1 or 3). Therefore, FTOL measures the relative error
desired in the sum of squares. Default: 1D-10
FUNCTARGS - passed directly to the expression as the variable
PRIVATE. Any user-private data can be communicated to
the user expression using this keyword.
Default: PRIVATE is undefined in user expression
GTOL - a nonnegative input variable. Termination occurs when the
cosine of the angle between fvec and any column of the
jacobian is at most GTOL in absolute value (and STATUS is
accordingly set to 4). Therefore, GTOL measures the
orthogonality desired between the function vector and the
columns of the jacobian. Default: 1D-10
ITERARGS - The keyword arguments to be passed to ITERPROC via the
_EXTRA mechanism. This should be a structure, and is
similar in operation to FUNCTARGS.
Default: no arguments are passed.
ITERPROC - The name of a procedure to be called upon each NPRINT
iteration of the MPFIT routine. It should be declared
in the following way:
PRO ITERPROC, MYFUNCT, p, iter, fnorm, FUNCTARGS=fcnargs, $
PARINFO=parinfo, QUIET=quiet, ...
; perform custom iteration update
END
ITERPROC must either accept all three keyword
parameters (FUNCTARGS, PARINFO and QUIET), or at least
accept them via the _EXTRA keyword.
MYFUNCT is the user-supplied function to be minimized,
P is the current set of model parameters, ITER is the
iteration number, and FUNCTARGS are the arguments to be
passed to MYFUNCT. FNORM should be the
chi-squared value. QUIET is set when no textual output
should be printed. See below for documentation of
PARINFO.
In implementation, ITERPROC can perform updates to the
terminal or graphical user interface, to provide
feedback while the fit proceeds. If the fit is to be
stopped for any reason, then ITERPROC should set the
common block variable ERROR_CODE to negative value (see
MPFIT_ERROR common block below). In principle,
ITERPROC should probably not modify the parameter
values, because it may interfere with the algorithm's
stability. In practice it is allowed.
Default: an internal routine is used to print the
parameter values.
MAXITER - The maximum number of iterations to perform. If the
number is exceeded, then the STATUS value is set to 5
and MPFIT returns.
Default: 200 iterations
NFEV - the number of MYFUNCT function evaluations performed.
NFREE - the number of free parameters in the fit. This includes
parameters which are not FIXED and not TIED, but it does
include parameters which are pegged at LIMITS.
NITER - the number of iterations completed.
NOCOVAR - set this keyword to prevent the calculation of the
covariance matrix before returning (see COVAR)
NPEGGED - the number of free parameters which are pegged at a
LIMIT.
NPRINT - The frequency with which ITERPROC is called. A value of
1 indicates that ITERPROC is called with every iteration,
while 2 indicates every other iteration, etc. Note that
several Levenberg-Marquardt attempts can be made in a
single iteration.
Default value: 1
PARINFO - Provides a mechanism for more sophisticated constraints
to be placed on parameter values. When PARINFO is not
passed, then it is assumed that all parameters are free
and unconstrained. Values in PARINFO are never
modified during a call to MPFIT.
See description above for the structure of PARINFO.
Default value: all parameters are free and unconstrained.
PERROR - The formal 1-sigma errors in each parameter, computed
from the covariance matrix. If a parameter is held
fixed, or if it touches a boundary, then the error is
reported as zero.
If the fit is unweighted (i.e. no errors were given, or
the weights were uniformly set to unity), then PERROR
will probably not represent the true parameter
uncertainties.
*If* you can assume that the true reduced chi-squared
value is unity -- meaning that the fit is implicitly
assumed to be of good quality -- then the estimated
parameter uncertainties can be computed by scaling PERROR
by the measured chi-squared value.
DOF = N_ELEMENTS(X) - N_ELEMENTS(PARMS) ; deg of freedom
PCERROR = PERROR * SQRT(BESTNORM / DOF) ; scaled uncertainties
QUIET - set this keyword when no textual output should be printed
by MPFIT
STATUS - an integer status code is returned. All values other
than zero can represent success. It can have one of the
following values:
0 improper input parameters.
1 both actual and predicted relative reductions
in the sum of squares are at most FTOL.
2 relative error between two consecutive iterates
is at most XTOL
3 conditions for STATUS = 1 and STATUS = 2 both hold.
4 the cosine of the angle between fvec and any
column of the jacobian is at most GTOL in
absolute value.
5 the maximum number of iterations has been reached
6 FTOL is too small. no further reduction in
the sum of squares is possible.
7 XTOL is too small. no further improvement in
the approximate solution x is possible.
8 GTOL is too small. fvec is orthogonal to the
columns of the jacobian to machine precision.
WEIGHTS - Array of weights to be used in calculating the
chi-squared value. If WEIGHTS is specified then the ERR
parameter is ignored. The chi-squared value is computed
as follows:
CHISQ = TOTAL( (Y-MYFUNCT(X,P))^2 * ABS(WEIGHTS) )
Here are common values of WEIGHTS:
1D/ERR^2 - Normal weighting (ERR is the measurement error)
1D/Y - Poisson weighting (counting statistics)
1D - Unweighted
XTOL - a nonnegative input variable. Termination occurs when the
relative error between two consecutive iterates is at most
XTOL (and STATUS is accordingly set to 2 or 3). Therefore,
XTOL measures the relative error desired in the approximate
solution. Default: 1D-10
YFIT - the best-fit model function, as returned by MYFUNCT.
EXAMPLE:
; First, generate some synthetic data
x = dindgen(200) * 0.1 - 10. ; Independent variable
yi = gauss1(x, [2.2D, 1.4, 3000.]) + 1000 ; "Ideal" Y variable
y = yi + randomn(seed, 200) * sqrt(yi) ; Measured, w/ noise
sy = sqrt(y) ; Poisson errors
; Now fit a Gaussian to see how well we can recover
expr = 'P(0) + GAUSS1(X, P(1:3))' ; fitting function
p0 = [800.D, 1.D, 1., 500.] ; Initial guess
p = mpfitexpr(expr, x, y, sy, p0) ; Fit the expression
print, p
plot, x, y ; Plot data
oplot, x, mpevalexpr(expr, x, p) ; Plot model
Generates a synthetic data set with a Gaussian peak, and Poisson
statistical uncertainty. Then a model consisting of a constant
plus Gaussian is fit to the data.
COMMON BLOCKS:
COMMON MPFIT_ERROR, ERROR_CODE
User routines may stop the fitting process at any time by
setting an error condition. This condition may be set in either
the user's model computation routine (MYFUNCT), or in the
iteration procedure (ITERPROC).
To stop the fitting, the above common block must be declared,
and ERROR_CODE must be set to a negative number. After the user
procedure or function returns, MPFIT checks the value of this
common block variable and exits immediately if the error
condition has been set. By default the value of ERROR_CODE is
zero, indicating a successful function/procedure call.
REFERENCES:
MINPACK-1, Jorge More', available from netlib (www.netlib.org).
"Optimization Software Guide," Jorge More' and Stephen Wright,
SIAM, *Frontiers in Applied Mathematics*, Number 14.
MODIFICATION HISTORY:
Written, Apr-Jul 1998, CM
Added PERROR keyword, 04 Aug 1998, CM
Added COVAR keyword, 20 Aug 1998, CM
Added NITER output keyword, 05 Oct 1998
D.L Windt, Bell Labs, windt@bell-labs.com;
Added ability to return model function in YFIT, 09 Nov 1998
Parameter values can be tied to others, 09 Nov 1998
Added MPEVALEXPR utility function, 09 Dec 1998
Cosmetic documentation updates, 16 Apr 1999, CM
More cosmetic documentation updates, 14 May 1999, CM
Made sure to update STATUS value, 25 Sep 1999, CM
Added WEIGHTS keyword, 25 Sep 1999, CM
Changed from handles to common blocks, 25 Sep 1999, CM
- commons seem much cleaner and more logical in this case.
Alphabetized documented keywords, 02 Oct 1999, CM
Added QUERY keyword and query checking of MPFIT, 29 Oct 1999, CM
Check to be sure that X and Y are present, 02 Nov 1999, CM
Documented PERROR for unweighted fits, 03 Nov 1999, CM
Removed ITERPROC='' when quiet EQ 1, 10 Jan 2000, CM
Changed to ERROR_CODE for error condition, 28 Jan 2000, CM
Updated the EXAMPLE, 26 Mar 2000, CM
Copying permission terms have been liberalized, 26 Mar 2000, CM
Propagated improvements from MPFIT, 17 Dec 2000, CM
Correct reference to _WTS in MPFITEXPR_EVAL, 25 May 2001, CM
(thanks to Mark Fardal)
Added useful FUNCTARGS behavior (as yet undocumented), 04 Jul
2002, CM
Documented FUNCTARGS/PRIVATE behavior, 22 Jul 2002, CM
Added NFREE and NPEGGED keywords, 13 Sep 2002, CM
Documented RELSTEP field of PARINFO (!!), CM, 25 Oct 2002
$Id: mpfitexpr.pro,v 1.8 2002/11/07 00:12:54 craigm Exp $
(See mpfitexpr.pro)
NAME:
MPFITFUN
AUTHOR:
Craig B. Markwardt, NASA/GSFC Code 662, Greenbelt, MD 20770
craigm@lheamail.gsfc.nasa.gov
UPDATED VERSIONs can be found on my WEB PAGE:
http://cow.physics.wisc.edu/~craigm/idl/idl.html
PURPOSE:
Perform Levenberg-Marquardt least-squares fit to IDL function
MAJOR TOPICS:
Curve and Surface Fitting
CALLING SEQUENCE:
parms = MPFITFUN(MYFUNCT, X, Y, ERR, start_params, ...)
DESCRIPTION:
MPFITFUN fits a user-supplied model -- in the form of an IDL
function -- to a set of user-supplied data. MPFITFUN calls
MPFIT, the MINPACK-1 least-squares minimizer, to do the main
work.
Given the data and their uncertainties, MPFITFUN finds the best set
of model parameters which match the data (in a least-squares
sense) and returns them in an array.
The user must supply the following items:
- An array of independent variable values ("X").
- An array of "measured" *dependent* variable values ("Y").
- An array of "measured" 1-sigma uncertainty values ("ERR").
- The name of an IDL function which computes Y given X ("MYFUNCT").
- Starting guesses for all of the parameters ("START_PARAMS").
There are very few restrictions placed on X, Y or MYFUNCT. Simply
put, MYFUNCT must map the "X" values into "Y" values given the
model parameters. The "X" values may represent any independent
variable (not just Cartesian X), and indeed may be multidimensional
themselves. For example, in the application of image fitting, X
may be a 2xN array of image positions.
MPFITFUN carefully avoids passing large arrays where possible to
improve performance.
See below for an example of usage.
USER FUNCTION
The user must define a function which returns the model value. For
applications which use finite-difference derivatives -- the default
-- the user function should be declared in the following way:
FUNCTION MYFUNCT, X, P
; The independent variable is X
; Parameter values are passed in "P"
YMOD = ... computed model values at X ...
return, YMOD
END
The returned array YMOD must have the same dimensions and type as
the "measured" Y values.
User functions may also indicate a fatal error condition
using the ERROR_CODE common block variable, as described
below under the MPFIT_ERROR common block definition.
See the discussion under "ANALYTIC DERIVATIVES" and AUTODERIVATIVE
in MPFIT.PRO if you wish to compute the derivatives for yourself.
AUTODERIVATIVE is accepted by MPFITFUN and passed directly to
MPFIT. The user function must accept one additional parameter, DP,
which contains the derivative of the user function with respect to
each parameter at each data point, as described in MPFIT.PRO.
CONSTRAINING PARAMETER VALUES WITH THE PARINFO KEYWORD
The behavior of MPFIT can be modified with respect to each
parameter to be fitted. A parameter value can be fixed; simple
boundary constraints can be imposed; limitations on the parameter
changes can be imposed; properties of the automatic derivative can
be modified; and parameters can be tied to one another.
These properties are governed by the PARINFO structure, which is
passed as a keyword parameter to MPFIT.
PARINFO should be an array of structures, one for each parameter.
Each parameter is associated with one element of the array, in
numerical order. The structure can have the following entries
(none are required):
.VALUE - the starting parameter value (but see the START_PARAMS
parameter for more information).
.FIXED - a boolean value, whether the parameter is to be held
fixed or not. Fixed parameters are not varied by
MPFIT, but are passed on to MYFUNCT for evaluation.
.LIMITED - a two-element boolean array. If the first/second
element is set, then the parameter is bounded on the
lower/upper side. A parameter can be bounded on both
sides. Both LIMITED and LIMITS must be given
together.
.LIMITS - a two-element float or double array. Gives the
parameter limits on the lower and upper sides,
respectively. Zero, one or two of these values can be
set, depending on the values of LIMITED. Both LIMITED
and LIMITS must be given together.
.PARNAME - a string, giving the name of the parameter. The
fitting code of MPFIT does not use this tag in any
way. However, the default ITERPROC will print the
parameter name if available.
.STEP - the step size to be used in calculating the numerical
derivatives. If set to zero, then the step size is
computed automatically. Ignored when AUTODERIVATIVE=0.
This value is superceded by the RELSTEP value.
.RELSTEP - the *relative* step size to be used in calculating
the numerical derivatives. This number is the
fractional size of the step, compared to the
parameter value. This value supercedes the STEP
setting. If the parameter is zero, then a default
step size is chosen.
.MPSIDE - the sidedness of the finite difference when computing
numerical derivatives. This field can take four
values:
0 - one-sided derivative computed automatically
1 - one-sided derivative (f(x+h) - f(x) )/h
-1 - one-sided derivative (f(x) - f(x-h))/h
2 - two-sided derivative (f(x+h) - f(x-h))/(2*h)
Where H is the STEP parameter described above. The
"automatic" one-sided derivative method will chose a
direction for the finite difference which does not
violate any constraints. The other methods do not
perform this check. The two-sided method is in
principle more precise, but requires twice as many
function evaluations. Default: 0.
.MPMINSTEP - the minimum change to be made in the parameter
value. During the fitting process, the parameter
will be changed by multiples of this value. The
actual step is computed as:
DELTA1 = MPMINSTEP*ROUND(DELTA0/MPMINSTEP)
where DELTA0 and DELTA1 are the estimated parameter
changes before and after this constraint is
applied. Note that this constraint should be used
with care since it may cause non-converging,
oscillating solutions.
A value of 0 indicates no minimum. Default: 0.
.MPMAXSTEP - the maximum change to be made in the parameter
value. During the fitting process, the parameter
will never be changed by more than this value.
A value of 0 indicates no maximum. Default: 0.
.TIED - a string expression which "ties" the parameter to other
free or fixed parameters. Any expression involving
constants and the parameter array P are permitted.
Example: if parameter 2 is always to be twice parameter
1 then use the following: parinfo(2).tied = '2 * P(1)'.
Since they are totally constrained, tied parameters are
considered to be fixed; no errors are computed for them.
[ NOTE: the PARNAME can't be used in expressions. ]
Future modifications to the PARINFO structure, if any, will involve
adding structure tags beginning with the two letters "MP".
Therefore programmers are urged to avoid using tags starting with
the same letters; otherwise they are free to include their own
fields within the PARINFO structure, and they will be ignored.
PARINFO Example:
parinfo = replicate({value:0.D, fixed:0, limited:[0,0], $
limits:[0.D,0]}, 5)
parinfo(0).fixed = 1
parinfo(4).limited(0) = 1
parinfo(4).limits(0) = 50.D
parinfo(*).value = [5.7D, 2.2, 500., 1.5, 2000.]
A total of 5 parameters, with starting values of 5.7,
2.2, 500, 1.5, and 2000 are given. The first parameter
is fixed at a value of 5.7, and the last parameter is
constrained to be above 50.
INPUTS:
MYFUNCT - a string variable containing the name of an IDL function.
This function computes the "model" Y values given the
X values and model parameters, as desribed above.
X - Array of independent variable values.
Y - Array of "measured" dependent variable values. Y should have
the same data type as X. The function MYFUNCT should map
X->Y.
ERR - Array of "measured" 1-sigma uncertainties. ERR should have
the same data type as Y. ERR is ignored if the WEIGHTS
keyword is specified.
START_PARAMS - An array of starting values for each of the
parameters of the model. The number of parameters
should be fewer than the number of measurements.
Also, the parameters should have the same data type
as the measurements (double is preferred).
This parameter is optional if the PARINFO keyword
is used (see MPFIT). The PARINFO keyword provides
a mechanism to fix or constrain individual
parameters. If both START_PARAMS and PARINFO are
passed, then the starting *value* is taken from
START_PARAMS, but the *constraints* are taken from
PARINFO.
RETURNS:
Returns the array of best-fit parameters.
KEYWORD PARAMETERS:
BESTNORM - the value of the summed squared residuals for the
returned parameter values.
COVAR - the covariance matrix for the set of parameters returned
by MPFIT. The matrix is NxN where N is the number of
parameters. The square root of the diagonal elements
gives the formal 1-sigma statistical errors on the
parameters IF errors were treated "properly" in MYFUNC.
Parameter errors are also returned in PERROR.
To compute the correlation matrix, PCOR, use this:
IDL> PCOR = COV * 0
IDL> FOR i = 0, n-1 DO FOR j = 0, n-1 DO $
PCOR(i,j) = COV(i,j)/sqrt(COV(i,i)*COV(j,j))
If NOCOVAR is set or MPFIT terminated abnormally, then
COVAR is set to a scalar with value !VALUES.D_NAN.
ERRMSG - a string error or warning message is returned.
CASH - when set, the fit statistic is changed to a derivative of
the CASH statistic. The model function must be strictly
positive.
FTOL - a nonnegative input variable. Termination occurs when both
the actual and predicted relative reductions in the sum of
squares are at most FTOL (and STATUS is accordingly set to
1 or 3). Therefore, FTOL measures the relative error
desired in the sum of squares. Default: 1D-10
FUNCTARGS - A structure which contains the parameters to be passed
to the user-supplied function specified by MYFUNCT via
the _EXTRA mechanism. This is the way you can pass
additional data to your user-supplied function without
using common blocks.
By default, no extra parameters are passed to the
user-supplied function.
GTOL - a nonnegative input variable. Termination occurs when the
cosine of the angle between fvec and any column of the
jacobian is at most GTOL in absolute value (and STATUS is
accordingly set to 4). Therefore, GTOL measures the
orthogonality desired between the function vector and the
columns of the jacobian. Default: 1D-10
ITERARGS - The keyword arguments to be passed to ITERPROC via the
_EXTRA mechanism. This should be a structure, and is
similar in operation to FUNCTARGS.
Default: no arguments are passed.
ITERPROC - The name of a procedure to be called upon each NPRINT
iteration of the MPFIT routine. It should be declared
in the following way:
PRO ITERPROC, MYFUNCT, p, iter, fnorm, FUNCTARGS=fcnargs, $
PARINFO=parinfo, QUIET=quiet, ...
; perform custom iteration update
END
ITERPROC must either accept all three keyword
parameters (FUNCTARGS, PARINFO and QUIET), or at least
accept them via the _EXTRA keyword.
MYFUNCT is the user-supplied function to be minimized,
P is the current set of model parameters, ITER is the
iteration number, and FUNCTARGS are the arguments to be
passed to MYFUNCT. FNORM should be the
chi-squared value. QUIET is set when no textual output
should be printed. See below for documentation of
PARINFO.
In implementation, ITERPROC can perform updates to the
terminal or graphical user interface, to provide
feedback while the fit proceeds. If the fit is to be
stopped for any reason, then ITERPROC should set the
common block variable ERROR_CODE to negative value (see
MPFIT_ERROR common block below). In principle,
ITERPROC should probably not modify the parameter
values, because it may interfere with the algorithm's
stability. In practice it is allowed.
Default: an internal routine is used to print the
parameter values.
MAXITER - The maximum number of iterations to perform. If the
number is exceeded, then the STATUS value is set to 5
and MPFIT returns.
Default: 200 iterations
NFEV - the number of MYFUNCT function evaluations performed.
NFREE - the number of free parameters in the fit. This includes
parameters which are not FIXED and not TIED, but it does
include parameters which are pegged at LIMITS.
NITER - the number of iterations completed.
NOCOVAR - set this keyword to prevent the calculation of the
covariance matrix before returning (see COVAR)
NPEGGED - the number of free parameters which are pegged at a
LIMIT.
NPRINT - The frequency with which ITERPROC is called. A value of
1 indicates that ITERPROC is called with every iteration,
while 2 indicates every other iteration, etc. Note that
several Levenberg-Marquardt attempts can be made in a
single iteration.
Default value: 1
PARINFO - Provides a mechanism for more sophisticated constraints
to be placed on parameter values. When PARINFO is not
passed, then it is assumed that all parameters are free
and unconstrained. Values in PARINFO are never
modified during a call to MPFIT.
See description above for the structure of PARINFO.
Default value: all parameters are free and unconstrained.
PERROR - The formal 1-sigma errors in each parameter, computed
from the covariance matrix. If a parameter is held
fixed, or if it touches a boundary, then the error is
reported as zero.
If the fit is unweighted (i.e. no errors were given, or
the weights were uniformly set to unity), then PERROR
will probably not represent the true parameter
uncertainties.
*If* you can assume that the true reduced chi-squared
value is unity -- meaning that the fit is implicitly
assumed to be of good quality -- then the estimated
parameter uncertainties can be computed by scaling PERROR
by the measured chi-squared value.
DOF = N_ELEMENTS(X) - N_ELEMENTS(PARMS) ; deg of freedom
PCERROR = PERROR * SQRT(BESTNORM / DOF) ; scaled uncertainties
QUIET - set this keyword when no textual output should be printed
by MPFIT
STATUS - an integer status code is returned. All values other
than zero can represent success. It can have one of the
following values:
0 improper input parameters.
1 both actual and predicted relative reductions
in the sum of squares are at most FTOL.
2 relative error between two consecutive iterates
is at most XTOL
3 conditions for STATUS = 1 and STATUS = 2 both hold.
4 the cosine of the angle between fvec and any
column of the jacobian is at most GTOL in
absolute value.
5 the maximum number of iterations has been reached
6 FTOL is too small. no further reduction in
the sum of squares is possible.
7 XTOL is too small. no further improvement in
the approximate solution x is possible.
8 GTOL is too small. fvec is orthogonal to the
columns of the jacobian to machine precision.
WEIGHTS - Array of weights to be used in calculating the
chi-squared value. If WEIGHTS is specified then the ERR
parameter is ignored. The chi-squared value is computed
as follows:
CHISQ = TOTAL( (Y-MYFUNCT(X,P))^2 * ABS(WEIGHTS) )
Here are common values of WEIGHTS:
1D/ERR^2 - Normal weighting (ERR is the measurement error)
1D/Y - Poisson weighting (counting statistics)
1D - Unweighted
XTOL - a nonnegative input variable. Termination occurs when the
relative error between two consecutive iterates is at most
XTOL (and STATUS is accordingly set to 2 or 3). Therefore,
XTOL measures the relative error desired in the approximate
solution. Default: 1D-10
YFIT - the best-fit model function, as returned by MYFUNCT.
EXAMPLE:
; First, generate some synthetic data
npts = 200
x = dindgen(npts) * 0.1 - 10. ; Independent variable
yi = gauss1(x, [2.2D, 1.4, 3000.]) ; "Ideal" Y variable
y = yi + randomn(seed, npts) * sqrt(1000. + yi); Measured, w/ noise
sy = sqrt(1000.D + y) ; Poisson errors
; Now fit a Gaussian to see how well we can recover
p0 = [1.D, 1., 1000.] ; Initial guess (cent, width, area)
p = mpfitfun('GAUSS1', x, y, sy, p0) ; Fit a function
print, p
Generates a synthetic data set with a Gaussian peak, and Poisson
statistical uncertainty. Then the same function is fitted to the
data (with different starting parameters) to see how close we can
get.
COMMON BLOCKS:
COMMON MPFIT_ERROR, ERROR_CODE
User routines may stop the fitting process at any time by
setting an error condition. This condition may be set in either
the user's model computation routine (MYFUNCT), or in the
iteration procedure (ITERPROC).
To stop the fitting, the above common block must be declared,
and ERROR_CODE must be set to a negative number. After the user
procedure or function returns, MPFIT checks the value of this
common block variable and exits immediately if the error
condition has been set. By default the value of ERROR_CODE is
zero, indicating a successful function/procedure call.
REFERENCES:
MINPACK-1, Jorge More', available from netlib (www.netlib.org).
"Optimization Software Guide," Jorge More' and Stephen Wright,
SIAM, *Frontiers in Applied Mathematics*, Number 14.
MODIFICATION HISTORY:
Written, Apr-Jul 1998, CM
Added PERROR keyword, 04 Aug 1998, CM
Added COVAR keyword, 20 Aug 1998, CM
Added ITER output keyword, 05 Oct 1998
D.L Windt, Bell Labs, windt@bell-labs.com;
Added ability to return model function in YFIT, 09 Nov 1998
Analytical derivatives allowed via AUTODERIVATIVE keyword, 09 Nov 1998
Parameter values can be tied to others, 09 Nov 1998
Cosmetic documentation updates, 16 Apr 1999, CM
More cosmetic documentation updates, 14 May 1999, CM
Made sure to update STATUS, 25 Sep 1999, CM
Added WEIGHTS keyword, 25 Sep 1999, CM
Changed from handles to common blocks, 25 Sep 1999, CM
- commons seem much cleaner and more logical in this case.
Alphabetized documented keywords, 02 Oct 1999, CM
Added QUERY keyword and query checking of MPFIT, 29 Oct 1999, CM
Corrected EXAMPLE (offset of 1000), 30 Oct 1999, CM
Check to be sure that X and Y are present, 02 Nov 1999, CM
Documented PERROR for unweighted fits, 03 Nov 1999, CM
Changed to ERROR_CODE for error condition, 28 Jan 2000, CM
Corrected errors in EXAMPLE, 26 Mar 2000, CM
Copying permission terms have been liberalized, 26 Mar 2000, CM
Propagated improvements from MPFIT, 17 Dec 2000, CM
Added CASH statistic, 10 Jan 2001
Added NFREE and NPEGGED keywords, 11 Sep 2002, CM
Documented RELSTEP field of PARINFO (!!), CM, 25 Oct 2002
$Id: mpfitfun.pro,v 1.5 2002/11/07 00:12:54 craigm Exp $
(See mpfitfun.pro)
NAME:
MPFITPEAK
AUTHOR:
Craig B. Markwardt, NASA/GSFC Code 662, Greenbelt, MD 20770
craigm@lheamail.gsfc.nasa.gov
UPDATED VERSIONs can be found on my WEB PAGE:
http://cow.physics.wisc.edu/~craigm/idl/idl.html
PURPOSE:
Fit a gaussian, lorentzian or Moffat model to data
MAJOR TOPICS:
Curve and Surface Fitting
CALLING SEQUENCE:
yfit = MPFITPEAK(X, Y, A, NTERMS=nterms, ...)
DESCRIPTION:
MPFITPEAK fits a gaussian, lorentzian or Moffat model using the
non-linear least squares fitter MPFIT. MPFITPEAK is meant to be a
drop-in replacement for IDL's GAUSSFIT function (and requires
MPFIT and MPFITFUN).
The choice of the fitting function is determined by the keywords
GAUSSIAN, LORENTZIAN and MOFFAT. By default the gaussian model
function is used. [ The Moffat function is a modified Lorentzian
with variable power law index. (Moffat, A. F. J. 1969, Astronomy &
Astrophysics, v. 3, p. 455-461) ]
The functional form of the baseline is determined by NTERMS and
the function to be fitted. NTERMS represents the total number of
parameters, A, to be fitted. The functional forms and the
meanings of the parameters are described in this table:
GAUSSIAN# Lorentzian# Moffat#
Model A(0)*exp(-0.5*u^2) A(0)/(u^2 + 1) A(0)/(u^2 + 1)^A(3)
A(0) Peak Value Peak Value Peak Value
A(1) Peak Centroid Peak Centroid Peak Centroid
A(2) Gaussian Sigma HWHM% HWHM%
A(3) + A(3) * + A(3) * Moffat Index
A(4) + A(4)*x * + A(4)*x * + A(4) *
A(5) + A(5)*x *
Notes: # u = (x - A(1))/A(2)
% Half-width at half maximum
* Optional depending on NTERMS
By default the initial starting values for the parameters A are
estimated from the data. However, explicit starting values can be
supplied using the ESTIMATES keyword. Also, error or weighting
values can optionally be provided; otherwise the fit is
unweighted.
MPFITPEAK fits the peak value of the curve. The area under a
gaussian peak is A(0)*A(2)*SQRT(2*!DPI); the area under a
lorentzian peak is A(0)*A(2)*!DPI.
RESTRICTIONS:
If no starting parameter ESTIMATES are provided, then MPFITPEAK
attempts to estimate them from the data. This is not a perfect
science; however, the author believes that the technique
implemented here is more robust than the one used in IDL's
GAUSSFIT. The author has tested cases of strong peaks, noisy
peaks and broad peaks, all with success.
Users should be aware that if the baseline term contains a strong
linear component then the automatic estimation may fail. For
automatic estimation to work the peak amplitude should dominate
over the the maximum baseline.
INPUTS:
X - Array of independent variable values, whose values should
monotonically increase.
Y - Array of "measured" dependent variable values. Y should have
the same data type and dimension as X.
OUTPUTS:
A - Upon return, an array of NTERMS best fit parameter values.
See the table above for the meanings of each parameter
element.
RETURNS:
Returns the best fitting model function.
KEYWORDS:
** NOTE ** Additional keywords such as PARINFO, BESTNORM, and
STATUS are accepted by MPFITPEAK but not documented
here. Please see the documentation for MPFIT for the
description of these advanced options.
ERROR - upon input, the measured 1-sigma uncertainties in the "Y"
values. If no ERROR or WEIGHTS are given, then the fit is
unweighted.
ESTIMATES - Array of starting values for each parameter of the
model. The number of parameters should at least be
three (four for Moffat), and if less than NTERMS, will
be extended with zeroes.
Default: parameter values are estimated from data.
GAUSSIAN - if set, fit a gaussian model function. The Default.
LORENTZIAN - if set, fit a lorentzian model function.
MOFFAT - if set, fit a Moffat model function.
NEGATIVE / POSITIVE - if set, and ESTIMATES is not provided, then
MPFITPEAK will assume that a
negative/positive peak is present.
Default: determined automatically
NTERMS - An integer describing the number of fitting terms.
NTERMS must have a minimum value, but can optionally be
larger depending on the desired baseline.
For gaussian and lorentzian models, NTERMS must be three
(zero baseline), four (constant baseline) or five (linear
baseline). Default: 4
For the Moffat model, NTERMS must be four (zero
baseline), five (constant baseline), or six (linear
baseline). Default: 5
PERROR - upon return, the 1-sigma uncertainties of the parameter
values A. These values are only meaningful if the ERRORS
or WEIGHTS keywords are specified properly.
If the fit is unweighted (i.e. no errors were given, or
the weights were uniformly set to unity), then PERROR
will probably not represent the true parameter
uncertainties.
*If* you can assume that the true reduced chi-squared
value is unity -- meaning that the fit is implicitly
assumed to be of good quality -- then the estimated
parameter uncertainties can be computed by scaling PERROR
by the measured chi-squared value.
DOF = N_ELEMENTS(X) - N_ELEMENTS(PARMS) ; deg of freedom
PCERROR = PERROR * SQRT(BESTNORM / DOF) ; scaled uncertainties
QUIET - if set then diagnostic fitting messages are suppressed.
Default: QUIET=1 (i.e., no diagnostics)
WEIGHTS - Array of weights to be used in calculating the
chi-squared value. If WEIGHTS is specified then the ERR
parameter is ignored. The chi-squared value is computed
as follows:
CHISQ = TOTAL( (Y-MYFUNCT(X,P))^2 * ABS(WEIGHTS) )
Here are common values of WEIGHTS:
1D/ERR^2 - Normal weighting (ERR is the measurement error)
1D/Y - Poisson weighting (counting statistics)
1D - Unweighted
The ERROR keyword takes precedence over any WEIGHTS
keyword values. If no ERROR or WEIGHTS are given, then
the fit is unweighted.
EXAMPLE:
; First, generate some synthetic data
npts = 200
x = dindgen(npts) * 0.1 - 10. ; Independent variable
yi = gauss1(x, [2.2D, 1.4, 3000.]) + 1000 ; "Ideal" Y variable
y = yi + randomn(seed, npts) * sqrt(1000. + yi); Measured, w/ noise
sy = sqrt(1000.D + y) ; Poisson errors
; Now fit a Gaussian to see how well we can recover the original
yfit = mpfitpeak(x, y, a, error=sy)
print, p
Generates a synthetic data set with a Gaussian peak, and Poisson
statistical uncertainty. Then the same function is fitted to the
data.
REFERENCES:
MINPACK-1, Jorge More', available from netlib (www.netlib.org).
"Optimization Software Guide," Jorge More' and Stephen Wright,
SIAM, *Frontiers in Applied Mathematics*, Number 14.
MODIFICATION HISTORY:
New algorithm for estimating starting values, CM, 31 Oct 1999
Documented, 02 Nov 1999
Small documentation fixes, 02 Nov 1999
Slight correction to calculation of dx, CM, 02 Nov 1999
Documented PERROR for unweighted fits, 03 Nov 1999, CM
Copying permission terms have been liberalized, 26 Mar 2000, CM
Change requirements on # elements in X and Y, 20 Jul 2000, CM
(thanks to David Schlegel )
Added documentation on area under curve, 29 Aug 2000, CM
Added POSITIVE and NEGATIVE keywords, 17 Nov 2000, CM
Added reference to Moffat paper, 10 Jan 2001, CM
Added usage message, 26 Jul 2001, CM
Documentation clarification, 05 Sep 2001, CM
$Id: mpfitpeak.pro,v 1.5 2001/09/18 00:12:39 craigm Exp $
(See mpfitpeak.pro)
NAME:
MPFTEST
AUTHOR:
Craig B. Markwardt, NASA/GSFC Code 662, Greenbelt, MD 20770
craigm@lheamail.gsfc.nasa.gov
UPDATED VERSIONs can be found on my WEB PAGE:
http://cow.physics.wisc.edu/~craigm/idl/idl.html
PURPOSE:
Compute the probability of a given F value
MAJOR TOPICS:
Curve and Surface Fitting, Statistics
CALLING SEQUENCE:
PROB = MPFTEST(F, DOF1, DOF2, [/SIGMA, /CLEVEL, /SLEVEL ])
DESCRIPTION:
The function MPFTEST() computes the probability for a value drawn
from the F-distribution to equal or exceed the given value of F.
This can be used for confidence testing of a measured value obeying
the F-distribution (i.e., for testing the ratio of variances, or
equivalently for the addition of parameters to a fitted model).
P_F(X > F; DOF1, DOF2) = PROB
In specifying the returned probability level the user has three
choices:
* return the confidence level when the /CLEVEL keyword is passed;
OR
* return the significance level (i.e., 1 - confidence level) when
the /SLEVEL keyword is passed (default); OR
* return the "sigma" of the probability (i.e., compute the
probability based on the normal distribution) when the /SIGMA
keyword is passed.
Note that /SLEVEL, /CLEVEL and /SIGMA are mutually exclusive.
For the ratio of variance test, the two variances, VAR1 and VAR2,
should be distributed according to the chi-squared distribution
with degrees of freedom DOF1 and DOF2 respectively. The F-value is
computed as:
F = (VAR1/DOF1) / (VAR2/DOF2)
and then the probability is computed as:
PROB = MPFTEST(F, DOF1, DOF2, ... )
For the test of additional parameters in least squares fitting, the
user should perform two separate fits, and have two chi-squared
values. One fit should be the "original" fit with no additional
parameters, and one fit should be the "new" fit with M additional
parameters.
CHI1 - chi-squared value for original fit
DOF1 - number of degrees of freedom of CHI1 (number of data
points minus number of original parameters)
CHI2 - chi-squared value for new fit
DOF2 - number of degrees of freedom of CHI2
Note that according to this formalism, the number of degrees of
freedom in the "new" fit, DOF2, should be less than the number of
degrees of freedom in the "original" fit, DOF1 (DOF2 < DOF1); and
also CHI2 < CHI1.
With the above definition, the F value is computed as:
F = ( (CHI1-CHI2)/(DOF1-DOF2) ) / (CHI2/DOF2)
where DOF1-DOF2 is equal to M, and then the F-test probability is
computed as:
PROB = MPFTEST(F, DOF1-DOF2, DOF2, ... )
Note that this formalism assumes that the addition of the M
parameters is a small peturbation to the overall fit. If the
additional parameters dramatically changes the character of the
model, then the first "ratio of variance" test is more appropriate,
where F = (CHI1/DOF1) / (CHI2/DOF2).
INPUTS:
F - ratio of variances as defined above.
DOF1 - number of degrees of freedom in first variance component.
DOF2 - number of degrees of freedom in second variance component.
RETURNS:
Returns a scalar or vector of probabilities, as described above,
and according to the /SLEVEL, /CLEVEL and /SIGMA keywords.
KEYWORD PARAMETERS:
SLEVEL - if set, then PROB describes the significance level
(default).
CLEVEL - if set, then PROB describes the confidence level.
SIGMA - if set, then PROB is the number of "sigma" away from the
mean in the normal distribution.
EXAMPLE:
chi1 = 62.3D & dof1 = 42d
chi2 = 54.6D & dof2 = 40d
f = ((chi1-chi2)/(dof1-dof2)) / (chi2/dof2)
print, mpftest(f, dof1-dof2, dof2)
This is a test for addition of parameters. The "original"
chi-squared value was 62.3 with 42 degrees of freedom, and the
"new" chi-squared value was 54.6 with 40 degrees of freedom.
These values reflect the addition of 2 parameters and the
reduction of the chi-squared value by 7.7. The significance of
this set of circumstances is 0.071464757.
REFERENCES:
Algorithms taken from CEPHES special function library, by Stephen
Moshier. (http://www.netlib.org/cephes/)
MODIFICATION HISTORY:
Completed, 1999, CM
Documented, 16 Nov 2001, CM
Reduced obtrusiveness of common block and math error handling, 18
Nov 2001, CM
Added documentation, 30 Dec 2001, CM
Documentation corrections (thanks W. Landsman), 17 Jan 2002, CM
Example docs were corrected (Thanks M. Perez-Torres), 17 Feb 2002,
CM
Example corrected again (sigh...), 13 Feb 2003, CM
$Id: mpftest.pro,v 1.7 2003/02/13 23:41:16 craigm Exp $
(See mpftest.pro)
NAME:
MPNORMLIM
AUTHOR:
Craig B. Markwardt, NASA/GSFC Code 662, Greenbelt, MD 20770
craigm@lheamail.gsfc.nasa.gov
UPDATED VERSIONs can be found on my WEB PAGE:
http://cow.physics.wisc.edu/~craigm/idl/idl.html
PURPOSE:
Compute confidence limits for normally distributed variable
MAJOR TOPICS:
Curve and Surface Fitting, Statistics
CALLING SEQUENCE:
Z = MPNORMLIM(PROB, [/CLEVEL, /SLEVEL ])
DESCRIPTION:
The function MPNORMLIM() computes confidence limits of a normally
distributed variable (with zero mean and unit variance), for a
desired probability level. The returned values, Z, are the
limiting values: a the magnitude of a normally distributed value
is greater than Z by chance with a probability PROB:
P_NORM(ABS(X) > Z) = PROB
In specifying the probability level the user has two choices:
* give the confidence level (default);
* give the significance level (i.e., 1 - confidence level) and
pass the /SLEVEL keyword; OR
Note that /SLEVEL and /CLEVEL are mutually exclusive.
INPUTS:
PROB - scalar or vector number, giving the desired probability
level as described above.
RETURNS:
Returns a scalar or vector of normal confidence limits.
KEYWORD PARAMETERS:
SLEVEL - if set, then PROB describes the significance level.
CLEVEL - if set, then PROB describes the confidence level
(default).
EXAMPLE:
print, mpnormlim(0.99d, /clevel)
Print the 99% confidence limit for a normally distributed
variable. In this case it is about 2.58 sigma.
REFERENCES:
Algorithms taken from CEPHES special function library, by Stephen
Moshier. (http://www.netlib.org/cephes/)
MODIFICATION HISTORY:
Completed, 1999, CM
Documented, 16 Nov 2001, CM
Reduced obtrusiveness of common block and math error handling, 18
Nov 2001, CM
$Id: mpnormlim.pro,v 1.3 2001/11/18 12:59:17 craigm Exp $
(See mpnormlim.pro)
NAME:
MPNORMTEST
AUTHOR:
Craig B. Markwardt, NASA/GSFC Code 662, Greenbelt, MD 20770
craigm@lheamail.gsfc.nasa.gov
UPDATED VERSIONs can be found on my WEB PAGE:
http://cow.physics.wisc.edu/~craigm/idl/idl.html
PURPOSE:
Compute the probability of a given normally distributed Z value
MAJOR TOPICS:
Curve and Surface Fitting, Statistics
CALLING SEQUENCE:
PROB = MPNORMTEST(Z, [/CLEVEL, /SLEVEL ])
DESCRIPTION:
The function MPNORMTEST() computes the probability for the
magnitude of a value drawn from the normal distribution to equal or
exceed the given value Z. This can be used for confidence testing
of a measured value obeying the normal distribution.
P_NORM(ABS(X) > Z) = PROB
In specifying the returned probability level the user has two
choices:
* return the confidence level when the /CLEVEL keyword is passed;
OR
* return the significance level (i.e., 1 - confidence level) when
the /SLEVEL keyword is passed (default).
Note that /SLEVEL and /CLEVEL are mutually exclusive.
INPUTS:
Z - the value to best tested. Z should be drawn from a normal
distribution with zero mean and unit variance. If a given
quantity Y has mean MU and standard deviation STD, then Z can
be computed as Z = (Y-MU)/STD.
RETURNS:
Returns a scalar or vector of probabilities, as described above,
and according to the /SLEVEL and /CLEVEL keywords.
KEYWORD PARAMETERS:
SLEVEL - if set, then PROB describes the significance level
(default).
CLEVEL - if set, then PROB describes the confidence level.
EXAMPLES:
print, mpnormtest(5d, /slevel)
Print the probability for the magnitude of a randomly distributed
variable with zero mean and unit variance to exceed 5, as a
significance level.
REFERENCES:
Algorithms taken from CEPHES special function library, by Stephen
Moshier. (http://www.netlib.org/cephes/)
MODIFICATION HISTORY:
Completed, 1999, CM
Documented, 16 Nov 2001, CM
Reduced obtrusiveness of common block and math error handling, 18
Nov 2001, CM
$Id: mpnormtest.pro,v 1.4 2001/11/18 12:59:17 craigm Exp $
(See mpnormtest.pro)
NAME:
SetFitParm.pro
AUTHOR:
F.Bringezu, denet - Internetservice, Halle Germany,
bringezu@denet.de
PURPOSE:
Provide a widget interface for creating a parinfo structure.
This parinfo structure can by used by mpfit routines of Craig B. Markwardt.
MAJOR TOPICS:
Widget, mpfit.
CALLING SEQUENCE:
parinfo=SetFitParm(used_parinfo)
DESCRIPTION:
SetFitParm creates PARINFO using a widget interface.
PARINFO provides constraints for paramters used by the mpfit routines.
PARINFO is an array of structures, one for each parameter.
A detailed description can be found in the documentation of mpcurvefit.pro
This routine creates an array that contains a structure for each element.
The structure has the following entries.
- VALUE (DOUBLE): The starting parameter
- FIXED (BOOLEAN): 1 fix the parameter, 0 don't fix it at the
point given in VALUE.
- LIMITS (DBLARRAY(2)): Set upper and lower limit.
- LIMITED (BOOLEAN ARRAY 2): Fix the limit.
The parameter OLDPARINFO is optional. OLDPARINFO is used to set
the default values in the widget.
You can simply run:
test=SetFitParm() to create the array for the first time.
Once the array is created it can be used to set the default values
in the widget by calling
test2=SetFitParm(test)
INPUTS:
OPTIONAL INPUTS:
OLDFITPARM - The default values of the new array
INPUT KEYWORD PARAMETERS:
PARENT - if this widget is to be a child, set this keyword to the
parent widget ID.
OUTPUT KEYWORD PARAMETERS:
CANCEL - if the user selected the cancel button on the SETFITPARM
widget, then this keyword will be set upon exit.
OUTPUTS:
PARINFO array of structures
SEE ALSO:
mpcurvefit
MODIFICATION HISTORY:
Written, FB, 12/1999
Documented, FB, Jan 2000
Generalized positioning code, CM 01 Feb 2000
(See setfitparm.pro)
NAME:
TNMIN
AUTHOR:
Craig B. Markwardt, NASA/GSFC Code 662, Greenbelt, MD 20770
craigm@lheamail.gsfc.nasa.gov
UPDATED VERSIONs can be found on my WEB PAGE:
http://cow.physics.wisc.edu/~craigm/idl/idl.html
PURPOSE:
Performs function minimization (Truncated-Newton Method)
MAJOR TOPICS:
Optimization and Minimization
CALLING SEQUENCE:
parms = TNMIN(MYFUNCT, X, FUNCTARGS=fcnargs, NFEV=nfev,
MAXITER=maxiter, ERRMSG=errmsg, NPRINT=nprint,
QUIET=quiet, XTOL=xtol, STATUS=status,
FGUESS=fguess, PARINFO=parinfo, BESTMIN=bestmin,
ITERPROC=iterproc, ITERARGS=iterargs, niter=niter)
DESCRIPTION:
TNMIN uses the Truncated-Newton method to minimize an arbitrary IDL
function with respect to a given set of free parameters. The
user-supplied function must compute the gradient with respect to
each parameter. TNMIN is based on TN.F (TNBC) by Stephen Nash.
If you want to solve a least-squares problem, to perform *curve*
*fitting*, then you will probably want to use the routines MPFIT,
MPFITFUN and MPFITEXPR. Those routines are specifically optimized
for the least-squares problem. TNMIN is suitable for constrained
and unconstrained optimization problems with a medium number of
variables. Function *maximization* can be performed using the
MAXIMIZE keyword.
TNMIN is similar to MPFIT in that it allows parameters to be fixed,
simple bounding limits to be placed on parameter values, and
parameters to be tied to other parameters. See PARINFO below for
discussion and examples.
USER FUNCTION
The user must define an IDL function which returns the desired
value as a single scalar. The IDL function must accept a list of
numerical parameters, P. Additionally, keyword parameters may be
used to pass more data or information to the user function, via the
FUNCTARGS keyword.
The user function should be declared in the following way:
FUNCTION MYFUNCT, p, dp [, keywords permitted ]
; Parameter values are passed in "p"
f = .... ; Compute function value
dp = .... ; Compute partial derivatives (optional)
return, f
END
The function *must* accept at least one argument, the parameter
list P. The vector P is implicitly assumed to be a one-dimensional
array. Users may pass additional information to the function by
composing and _EXTRA structure and passing it in the FUNCTARGS
keyword.
User functions may also indicate a fatal error condition using the
ERROR_CODE common block variable, as described below under the
TNMIN_ERROR common block definition (by setting ERROR_CODE to a
number between -15 and -1).
ANALYTIC vs. NUMERICAL DERIVATIVES
By default, the user must compute gradient components analytically
using AUTODERIVATIVE=0. As explained below, numerical derivatives
can also be calculated using AUTODERIVATIVE=1.
For analytic derivatives, the user should call TNMIN using the
default keyword value AUTODERIVATIVE=0. [ This is different
behavior from MPFIT, where AUTODERIVATIVE=1 is the default. ] The
IDL user routine should compute the gradient of the function as a
one-dimensional array of values, one for each of the parameters.
They are passed back to TNMIN via "dp" as shown above.
The derivatives with respect to fixed parameters are ignored; zero
is an appropriate value to insert for those derivatives. Upon
input to the user function, DP is set to a vector with the same
length as P, with a value of 1 for a parameter which is free, and a
value of zero for a parameter which is fixed (and hence no
derivative needs to be calculated). This input vector may be
overwritten as needed.
For numerical derivatives, a finite differencing approximation is
used to estimate the gradient values. Users can activate this
feature by passing the keyword AUTODERIVATIVE=1. Fine control over
this behavior can be achieved using the STEP, RELSTEP and TNSIDE
fields of the PARINFO structure.
When finite differencing is used for computing derivatives (ie,
when AUTODERIVATIVE=1), the parameter DP is not passed. Therefore
functions can use N_PARAMS() to indicate whether they must compute
the derivatives or not. However there is no penalty (other than
computation time) for computing the gradient values and users may
switch between AUTODERIVATIVE=0 or =1 in order to test both
scenarios.
CONSTRAINING PARAMETER VALUES WITH THE PARINFO KEYWORD
The behavior of TNMIN can be modified with respect to each
parameter to be optimized. A parameter value can be fixed; simple
boundary constraints can be imposed; limitations on the parameter
changes can be imposed; properties of the automatic derivative can
be modified; and parameters can be tied to one another.
These properties are governed by the PARINFO structure, which is
passed as a keyword parameter to TNMIN.
PARINFO should be an array of structures, one for each parameter.
Each parameter is associated with one element of the array, in
numerical order. The structure can have the following entries
(none are required):
.VALUE - the starting parameter value (but see the START_PARAMS
parameter for more information).
.FIXED - a boolean value, whether the parameter is to be held
fixed or not. Fixed parameters are not varied by
TNMIN, but are passed on to MYFUNCT for evaluation.
.LIMITED - a two-element boolean array. If the first/second
element is set, then the parameter is bounded on the
lower/upper side. A parameter can be bounded on both
sides. Both LIMITED and LIMITS must be given
together.
.LIMITS - a two-element float or double array. Gives the
parameter limits on the lower and upper sides,
respectively. Zero, one or two of these values can be
set, depending on the values of LIMITED. Both LIMITED
and LIMITS must be given together.
.PARNAME - a string, giving the name of the parameter. The
fitting code of TNMIN does not use this tag in any
way.
.STEP - the step size to be used in calculating the numerical
derivatives. If set to zero, then the step size is
computed automatically. Ignored when AUTODERIVATIVE=0.
.MPSIDE - the sidedness of the finite difference when computing
numerical derivatives. This field can take four
values:
0 - one-sided derivative computed automatically
1 - one-sided derivative (f(x+h) - f(x) )/h
-1 - one-sided derivative (f(x) - f(x-h))/h
2 - two-sided derivative (f(x+h) - f(x-h))/(2*h)
Where H is the STEP parameter described above. The
"automatic" one-sided derivative method will chose a
direction for the finite difference which does not
violate any constraints. The other methods do not
perform this check. The two-sided method is in
principle more precise, but requires twice as many
function evaluations. Default: 0.
.TIED - a string expression which "ties" the parameter to other
free or fixed parameters. Any expression involving
constants and the parameter array P are permitted.
Example: if parameter 2 is always to be twice parameter
1 then use the following: parinfo(2).tied = '2 * P(1)'.
Since they are totally constrained, tied parameters are
considered to be fixed; no errors are computed for them.
[ NOTE: the PARNAME can't be used in expressions. ]
Future modifications to the PARINFO structure, if any, will involve
adding structure tags beginning with the two letters "MP" or "TN".
Therefore programmers are urged to avoid using tags starting with
these two combinations of letters; otherwise they are free to
include their own fields within the PARINFO structure, and they
will be ignored.
PARINFO Example:
parinfo = replicate({value:0.D, fixed:0, limited:[0,0], $
limits:[0.D,0]}, 5)
parinfo(0).fixed = 1
parinfo(4).limited(0) = 1
parinfo(4).limits(0) = 50.D
parinfo(*).value = [5.7D, 2.2, 500., 1.5, 2000.]
A total of 5 parameters, with starting values of 5.7,
2.2, 500, 1.5, and 2000 are given. The first parameter
is fixed at a value of 5.7, and the last parameter is
constrained to be above 50.
INPUTS:
MYFUNCT - a string variable containing the name of the function to
be minimized (see USER FUNCTION above). The IDL routine
should return the value of the function and optionally
its gradients.
X - An array of starting values for each of the parameters of the
model.
This parameter is optional if the PARINFO keyword is used.
See above. The PARINFO keyword provides a mechanism to fix or
constrain individual parameters. If both X and PARINFO are
passed, then the starting *value* is taken from X, but the
*constraints* are taken from PARINFO.
RETURNS:
Returns the array of best-fit parameters.
KEYWORD PARAMETERS:
AUTODERIVATIVE - If this is set, derivatives of the function will
be computed automatically via a finite
differencing procedure. If not set, then MYFUNCT
must provide the (analytical) derivatives.
Default: 0 (analytical derivatives required)
BESTMIN - upon return, the best minimum function value that TNMIN
could find.
EPSABS - a nonnegative real variable. Termination occurs when the
absolute error between consecutive iterates is at most
EPSABS. Note that using EPSREL is normally preferable
over EPSABS, except in cases where ABS(F) is much larger
than 1 at the optimal point. A value of zero indicates
the absolute error test is not applied. If EPSABS is
specified, then both EPSREL and EPSABS tests are applied;
if either succeeds then termination occurs.
Default: 0 (i.e., only EPSREL is applied).
EPSREL - a nonnegative input variable. Termination occurs when the
relative error between two consecutive iterates is at
most EPSREL. Therefore, EPSREL measures the relative
error desired in the function. An additional, more
lenient, stopping condition can be applied by specifying
the EPSABS keyword.
Default: 100 * Machine Precision
ERRMSG - a string error or warning message is returned.
FGUESS - provides the routine a guess to the minimum value.
Default: 0
FUNCTARGS - A structure which contains the parameters to be passed
to the user-supplied function specified by MYFUNCT via
the _EXTRA mechanism. This is the way you can pass
additional data to your user-supplied function without
using common blocks.
Consider the following example:
if FUNCTARGS = { XVAL:[1.D,2,3], YVAL:[1.D,4,9]}
then the user supplied function should be declared
like this:
FUNCTION MYFUNCT, P, XVAL=x, YVAL=y
By default, no extra parameters are passed to the
user-supplied function.
ITERARGS - The keyword arguments to be passed to ITERPROC via the
_EXTRA mechanism. This should be a structure, and is
similar in operation to FUNCTARGS.
Default: no arguments are passed.
ITERDERIV - Intended to print function gradient information. If
set, then the ITERDERIV keyword is set in each call to
ITERPROC. In the default ITERPROC, parameter values
and gradient information are both printed when this
keyword is set.
ITERPROC - The name of a procedure to be called upon each NPRINT
iteration of the TNMIN routine. It should be declared
in the following way:
PRO ITERPROC, MYFUNCT, p, iter, fnorm, FUNCTARGS=fcnargs, $
PARINFO=parinfo, QUIET=quiet, _EXTRA=extra
; perform custom iteration update
END
ITERPROC must accept the _EXTRA keyword, in case of
future changes to the calling procedure.
MYFUNCT is the user-supplied function to be minimized,
P is the current set of model parameters, ITER is the
iteration number, and FUNCTARGS are the arguments to be
passed to MYFUNCT. FNORM is should be the function
value. QUIET is set when no textual output should be
printed. See below for documentation of PARINFO.
In implementation, ITERPROC can perform updates to the
terminal or graphical user interface, to provide
feedback while the fit proceeds. If the fit is to be
stopped for any reason, then ITERPROC should set the
common block variable ERROR_CODE to negative value
between -15 and -1 (see TNMIN_ERROR common block
below). In principle, ITERPROC should probably not
modify the parameter values, because it may interfere
with the algorithm's stability. In practice it is
allowed.
Default: an internal routine is used to print the
parameter values.
MAXITER - The maximum number of iterations to perform. If the
number is exceeded, then the STATUS value is set to 5
and TNMIN returns.
Default: 200 iterations
MAXIMIZE - If set, the function is maximized instead of
minimized.
MAXNFEV - The maximum number of function evaluations to perform.
If the number is exceeded, then the STATUS value is set
to -17 and TNMIN returns. A value of zero indicates no
maximum.
Default: 0 (no maximum)
NFEV - upon return, the number of MYFUNCT function evaluations
performed.
NITER - upon return, number of iterations completed.
NPRINT - The frequency with which ITERPROC is called. A value of
1 indicates that ITERPROC is called with every iteration,
while 2 indicates every other iteration, etc.
Default value: 1
PARINFO - Provides a mechanism for more sophisticated constraints
to be placed on parameter values. When PARINFO is not
passed, then it is assumed that all parameters are free
and unconstrained. Values in PARINFO are never modified
during a call to TNMIN.
See description above for the structure of PARINFO.
Default value: all parameters are free and unconstrained.
QUIET - set this keyword when no textual output should be printed
by TNMIN
STATUS - an integer status code is returned. All values greater
than zero can represent success (however STATUS EQ 5 may
indicate failure to converge). Gaps in the numbering
system below are to maintain compatibility with MPFIT.
Upon return, STATUS can have one of the following values:
-18 a fatal internal error occurred during optimization.
-17 the maximum number of function evaluations has been
reached without convergence.
-16 a parameter or function value has become infinite or an
undefined number. This is usually a consequence of
numerical overflow in the user's function, which must be
avoided.
-15 to -1
these are error codes that either MYFUNCT or ITERPROC
may return to terminate the fitting process (see
description of MPFIT_ERROR common below). If either
MYFUNCT or ITERPROC set ERROR_CODE to a negative number,
then that number is returned in STATUS. Values from -15
to -1 are reserved for the user functions and will not
clash with MPFIT.
0 improper input parameters.
1 convergence was reached.
2-4 (RESERVED)
5 the maximum number of iterations has been reached
6-8 (RESERVED)
EXAMPLE:
FUNCTION F, X, DF, _EXTRA=extra ;; *** MUST ACCEPT KEYWORDS
F = (X(0)-1)^2 + (X(1)+7)^2
DF = [ 2D * (X(0)-1), 2D * (X(1)+7) ] ; Gradient
RETURN, F
END
P = TNMIN('F', [0D, 0D], BESTMIN=F0)
Minimizes the function F(x0,x1) = (x0-1)^2 + (x1+7)^2.
COMMON BLOCKS:
COMMON TNMIN_ERROR, ERROR_CODE
User routines may stop the fitting process at any time by
setting an error condition. This condition may be set in either
the user's model computation routine (MYFUNCT), or in the
iteration procedure (ITERPROC).
To stop the fitting, the above common block must be declared,
and ERROR_CODE must be set to a negative number. After the user
procedure or function returns, TNMIN checks the value of this
common block variable and exits immediately if the error
condition has been set. By default the value of ERROR_CODE is
zero, indicating a successful function/procedure call.
REFERENCES:
TRUNCATED-NEWTON METHOD, TN.F
Stephen G. Nash, Operations Research and Applied Statistics
Department
http://www.netlib.org/opt/tn
Nash, S. G. 1984, "Newton-Type Minimization via the Lanczos
Method," SIAM J. Numerical Analysis, 21, p. 770-778
MODIFICATION HISTORY:
Derived from TN.F by Stephen Nash with many changes and additions,
to conform to MPFIT paradigm, 19 Jan 1999, CM
Changed web address to COW, 25 Sep 1999, CM
Alphabetized documented keyword parameters, 02 Oct 1999, CM
Changed to ERROR_CODE for error condition, 28 Jan 2000, CM
Continued and fairly major improvements (CM, 08 Jan 2001):
- calling of user procedure is now concentrated in TNMIN_CALL,
and arguments are reduced by storing a large number of them
in common blocks;
- finite differencing is done within TNMIN_CALL; added
AUTODERIVATIVE=1 so that finite differencing can be enabled,
both one and two sided;
- a new procedure to parse PARINFO fields, borrowed from MPFIT;
brought PARINFO keywords up to date with MPFIT;
- go through and check for float vs. double discrepancies;
- add explicit MAXIMIZE keyword, and support in TNMIN_CALL and
TNMIN_DEFITER to print the correct values in that case;
TNMIN_DEFITER now prints function gradient values if
requested;
- convert to common-based system of MPFIT for storing machine
constants; revert TNMIN_ENORM to simple sum of squares, at
least for now;
- remove limit on number of function evaluations, at least for
now, and until I can understand what happens when we do
numerical derivatives.
Further changes: more float vs double; disable TNMINSTEP for now;
experimented with convergence test in case of function
maximization, 11 Jan 2001, CM
TNMINSTEP is parsed but not enabled, 13 Mar 2001
Major code cleanups; internal docs; reduced commons, CM, 08 Apr
2001
Continued code cleanups; documentation; the STATUS keyword
actually means something, CM, 10 Apr 2001
Added reference to Nash paper, CM, 08 Feb 2002
TODO
- scale derivatives semi-automatically;
- ability to scale and offset parameters;
$Id: tnmin.pro,v 1.11 2003/01/11 03:22:56 craigm Exp $
(See tnmin.pro)