Search   Memberlist   Usergroups
 Page 1 of 1 [10 Posts]
Author Message
C6L1V@shaw.ca
science forum Guru

Joined: 23 May 2005
Posts: 628

Posted: Fri Jul 07, 2006 10:28 pm    Post subject: Re: Questions on Least Squares Fitting for Logarithmic Relations

draccarlawpet wrote:
 Quote: hello, i tried doing the transformations, and they are FIENDISHLY DIFFICULT TO DO!!! I totally see your point now. If there are 2 variables (c1 - the constant, and c2 - the number associated with the variable), the equation is hard enough to derive. However, it becomes much much harder if there are 3 variables!!! I'd rather have the improper fit than go through this mess.

No need to panic. Just use a numerical nonlinear optimization package,
such as the EXCEL Solver (which is built-in in EXCEL, but might need to
be "installed"). Alternatively, you can use one of the many freely
downloadable nonlinear optimization packages. Some of them are designed
specifically to handle nonlinear least-squares problems. If your
problem is important, and if getting a correct fit is important, it
would be worthwile using available tools to do the job properly, but
you must be the judge of that.

RGV
draccarlawpet
science forum beginner

Joined: 09 Jan 2006
Posts: 41

 Posted: Fri Jul 07, 2006 9:26 pm    Post subject: Re: Questions on Least Squares Fitting for Logarithmic Relations hello, i tried doing the transformations, and they are FIENDISHLY DIFFICULT TO DO!!! I totally see your point now. If there are 2 variables (c1 - the constant, and c2 - the number associated with the variable), the equation is hard enough to derive. However, it becomes much much harder if there are 3 variables!!! I'd rather have the improper fit than go through this mess.
Robert B. Israel
science forum Guru

Joined: 24 Mar 2005
Posts: 2151

Posted: Fri Jul 07, 2006 4:27 am    Post subject: Re: Questions on Least Squares Fitting for Logarithmic Relations

C6L1V@shaw.ca <C6L1V@shaw.ca> wrote:
 Quote: draccarlawpet wrote: I have a question on LSFfLR. Why isn't a constant term included in the logarithmic portion of the model? Another words, why isn't the model of this form: Y = c1 + c2*ln (x + c3) ? In order to do "linear" regression (that is, doing a fit that is linear in c1, c2 and c3---or functions of them---but not necessarily linear in x!), you would either need to know (i) c1, in which case you could do the fit ln(Y-c1) = ln c2 + x+c3,

No. There's no equation linear in c2 and c3. However, if you know c2
you can transform the equation to
exp(Y/c2) = p x + q
and then after determining p and q, assuming p > 0,
take c1 = c2 ln(p) and c3 = q/p.

 Quote: so can determine ln c2 and c3 by linear regression; or to know (ii) c3, in which case you would do a simple linear fit y = c1 + c2*z, where z =ln ( x + c3). If both c1 and c3 are unknown, there is no way to "linearize" the problem. In that case you must resort to a general nonlinear least-squares minimization technique, and there are no longer any nice closed-form equations to give you the answer. That's why the Mathworld article does not give you a formula to cover this case.

The equations for least-squares optimization would be

c1 n + c2 sum_i ln(x_i + c3) = sum_i y_i

c1 sum_i ln(x_i + c3) + c2 sum_i ln(x_i + c3)^2 = sum_i y_i ln(x_i + c3)

c1 sum_i 1/(x_i + c3) + c2 sum_i ln(x_i + c3)/(x_i + c3)
= sum_i y_i/(x_i + c3)

Write these as

a_{11} c1 + a_{12} c2 = b_1
a_{21} c1 + a_{22} c2 = b_2
a_{31} c1 + a_{32} c2 = b_3

where the a's and b's are functions of c3.
In order to have a solution, you need the 3 x 3 determinant

| a_{11} a_{12} b_1 |
| a_{21} a_{22} b_2 |
| a_{31} a_{32} b_3 | = 0

This is a rather complicated nonlinear equation in c3. Try solving
it using standard numerical methods, such as Newton's, then determine
c1 and c2 from two of the linear equations.

Robert Israel israel@math.ubc.ca
Department of Mathematics http://www.math.ubc.ca/~israel
University of British Columbia Vancouver, BC, Canada
C6L1V@shaw.ca
science forum Guru

Joined: 23 May 2005
Posts: 628

Posted: Fri Jul 07, 2006 4:02 am    Post subject: Re: Questions on Least Squares Fitting for Logarithmic Relations

Brablo wrote:
 Quote: Are you suggesting that there is no general least squares method to fit data to the equation of Y = c1 + c2 * LN ( X + c3 ) ?

No, I am not suggesting it; I am claiming it as a fact. Also: even if
you can take logarithms of Y (for example), you would be getting a fit
that minimizes the mean square of LOG Y, not of Y itself. So,if you
really want to minimize
S = sum [Y_i - c1 - c2* ln (c3 + x_i)]^2,
you need to solve a nonlinear problem in the "variables" c1, c2 and c3.
There just are no formulas that will do the job. Note, however, that
there are numerous good packages available to solve such problems, and
some of them are even available free on-line.

RGV

 Quote: I would have thought that the transformation exists. Thanks for all your help and guidance. C6L1V@shaw.ca wrote: draccarlawpet wrote: I have a question on LSFfLR. Why isn't a constant term included in the logarithmic portion of the model? Another words, why isn't the model of this form: Y = c1 + c2*ln (x + c3) ? In order to do "linear" regression (that is, doing a fit that is linear in c1, c2 and c3---or functions of them---but not necessarily linear in x!), you would either need to know (i) c1, in which case you could do the fit ln(Y-c1) = ln c2 + x+c3, so can determine ln c2 and c3 by linear regression; or to know (ii) c3, in which case you would do a simple linear fit y = c1 + c2*z, where z =ln ( x + c3). If both c1 and c3 are unknown, there is no way to "linearize" the problem. In that case you must resort to a general nonlinear least-squares minimization technique, and there are no longer any nice closed-form equations to give you the answer. That's why the Mathworld article does not give you a formula to cover this case. R.G. Vickson How would I deduce to find the C1, C2, and C3 of this relationship? Least Squares Fitting--Logarithmic Given a function of the form (1) FORMULA CAN BE FOUND AT: http://mathworld.wolfram.com/LeastSquaresFittingLogarithmic.html the coefficients can be found from least squares fitting as (2) http://mathworld.wolfram.com/LeastSquaresFittingLogarithmic.html (3) http://mathworld.wolfram.com/LeastSquaresFittingLogarithmic.html SEE ALSO: Least Squares Fitting, Least Squares Fitting--Exponential, Least Squares Fitting--Power Law. [Pages Linking Here]
Brablo

Joined: 05 May 2005
Posts: 61

Posted: Fri Jul 07, 2006 2:30 am    Post subject: Re: Questions on Least Squares Fitting for Logarithmic Relations

Are you suggesting that there is no general least squares method to fit
data to the equation of

Y = c1 + c2 * LN ( X + c3 ) ?

I would have thought that the transformation exists.

Thanks for all your help and guidance.

C6L1V@shaw.ca wrote:
 Quote: draccarlawpet wrote: I have a question on LSFfLR. Why isn't a constant term included in the logarithmic portion of the model? Another words, why isn't the model of this form: Y = c1 + c2*ln (x + c3) ? In order to do "linear" regression (that is, doing a fit that is linear in c1, c2 and c3---or functions of them---but not necessarily linear in x!), you would either need to know (i) c1, in which case you could do the fit ln(Y-c1) = ln c2 + x+c3, so can determine ln c2 and c3 by linear regression; or to know (ii) c3, in which case you would do a simple linear fit y = c1 + c2*z, where z =ln ( x + c3). If both c1 and c3 are unknown, there is no way to "linearize" the problem. In that case you must resort to a general nonlinear least-squares minimization technique, and there are no longer any nice closed-form equations to give you the answer. That's why the Mathworld article does not give you a formula to cover this case. R.G. Vickson How would I deduce to find the C1, C2, and C3 of this relationship? Least Squares Fitting--Logarithmic Given a function of the form (1) FORMULA CAN BE FOUND AT: http://mathworld.wolfram.com/LeastSquaresFittingLogarithmic.html the coefficients can be found from least squares fitting as (2) http://mathworld.wolfram.com/LeastSquaresFittingLogarithmic.html (3) http://mathworld.wolfram.com/LeastSquaresFittingLogarithmic.html SEE ALSO: Least Squares Fitting, Least Squares Fitting--Exponential, Least Squares Fitting--Power Law. [Pages Linking Here]
C6L1V@shaw.ca
science forum Guru

Joined: 23 May 2005
Posts: 628

Posted: Fri Jul 07, 2006 12:47 am    Post subject: Re: Questions on Least Squares Fitting for Logarithmic Relations

Ray Koopman wrote:
 Quote: C6L1V@shaw.ca wrote: draccarlawpet wrote: I have a question on LSFfLR. Why isn't a constant term included in the logarithmic portion of the model? Another words, why isn't the model of this form: Y = c1 + c2*ln (x + c3) ? In order to do "linear" regression (that is, doing a fit that is linear in c1, c2 and c3---or functions of them---but not necessarily linear in x!), you would either need to know (i) c1, in which case you could do the fit ln(Y-c1) = ln c2 + x+c3, so can determine ln c2 and c3 by linear regression; [...] How did you get from Y = c1 + c2*ln (x + c3) to ln(Y-c1) = ln c2 + x+c3 ?

By making a stupid mistake.

R.G.V.
Ray Koopman
science forum Guru Wannabe

Joined: 25 Mar 2005
Posts: 216

Posted: Thu Jul 06, 2006 10:34 pm    Post subject: Re: Questions on Least Squares Fitting for Logarithmic Relations

C6L1V@shaw.ca wrote:
 Quote: draccarlawpet wrote: I have a question on LSFfLR. Why isn't a constant term included in the logarithmic portion of the model? Another words, why isn't the model of this form: Y = c1 + c2*ln (x + c3) ? In order to do "linear" regression (that is, doing a fit that is linear in c1, c2 and c3---or functions of them---but not necessarily linear in x!), you would either need to know (i) c1, in which case you could do the fit ln(Y-c1) = ln c2 + x+c3, so can determine ln c2 and c3 by linear regression; [...]

How did you get from
Y = c1 + c2*ln (x + c3)
to
ln(Y-c1) = ln c2 + x+c3 ?
C6L1V@shaw.ca
science forum Guru

Joined: 23 May 2005
Posts: 628

Posted: Thu Jul 06, 2006 10:16 pm    Post subject: Re: Questions on Least Squares Fitting for Logarithmic Relations

draccarlawpet wrote:
 Quote: I have a question on LSFfLR. Why isn't a constant term included in the logarithmic portion of the model? Another words, why isn't the model of this form: Y = c1 + c2*ln (x + c3) ?

In order to do "linear" regression (that is, doing a fit that is linear
in c1, c2 and c3---or functions of them---but not necessarily linear in
x!), you would either need to know (i) c1, in which case you could do
the fit ln(Y-c1) = ln c2 + x+c3, so can determine ln c2 and c3 by
linear regression; or to know (ii) c3, in which case you would do a
simple linear fit y = c1 + c2*z, where z =ln ( x + c3). If both c1 and
c3 are unknown, there is no way to "linearize" the problem. In that
case you must resort to a general nonlinear least-squares minimization
technique, and there are no longer any nice closed-form equations to
give you the answer. That's why the Mathworld article does not give you
a formula to cover this case.

R.G. Vickson
 Quote: How would I deduce to find the C1, C2, and C3 of this relationship? Least Squares Fitting--Logarithmic Given a function of the form (1) FORMULA CAN BE FOUND AT: http://mathworld.wolfram.com/LeastSquaresFittingLogarithmic.html the coefficients can be found from least squares fitting as (2) http://mathworld.wolfram.com/LeastSquaresFittingLogarithmic.html (3) http://mathworld.wolfram.com/LeastSquaresFittingLogarithmic.html SEE ALSO: Least Squares Fitting, Least Squares Fitting--Exponential, Least Squares Fitting--Power Law. [Pages Linking Here]
Ray Koopman
science forum Guru Wannabe

Joined: 25 Mar 2005
Posts: 216

Posted: Thu Jul 06, 2006 7:48 pm    Post subject: Re: Questions on Least Squares Fitting for Logarithmic Relations

draccarlawpet wrote:
 Quote: I have a question on LSFfLR. Why isn't a constant term included in the logarithmic portion of the model? Another words, why isn't the model of this form: Y = c1 + c2*ln (x + c3) ? How would I deduce to find the C1, C2, and C3 of this relationship?

For any given c3 > -min[x] let z = ln[x + c3].
Then the optimum c1 and c2 are easily found as functions of y and z,
and the error sum of squares is
sum{(y-M[y])^2} - (sum{(y-M[y])(z-M[z])})^2 / sum{(z-M[z])^2},
where M[.] is the mean.
Treat the error sum of squares as a function of c3, and minimize it.
draccarlawpet
science forum beginner

Joined: 09 Jan 2006
Posts: 41

 Posted: Thu Jul 06, 2006 6:08 pm    Post subject: Questions on Least Squares Fitting for Logarithmic Relations I have a question on LSFfLR. Why isn't a constant term included in the logarithmic portion of the model? Another words, why isn't the model of this form: Y = c1 + c2*ln (x + c3) ? How would I deduce to find the C1, C2, and C3 of this relationship? Least Squares Fitting--Logarithmic Given a function of the form (1) FORMULA CAN BE FOUND AT: http://mathworld.wolfram.com/LeastSquaresFittingLogarithmic.html the coefficients can be found from least squares fitting as (2) http://mathworld.wolfram.com/LeastSquaresFittingLogarithmic.html (3) http://mathworld.wolfram.com/LeastSquaresFittingLogarithmic.html SEE ALSO: Least Squares Fitting, Least Squares Fitting--Exponential, Least Squares Fitting--Power Law. [Pages Linking Here]

 Display posts from previous: All Posts1 Day7 Days2 Weeks1 Month3 Months6 Months1 Year Oldest FirstNewest First
 Page 1 of 1 [10 Posts]
 The time now is Sun Jun 26, 2011 8:22 am | All times are GMT
 Jump to: Select a forum-------------------Forum index|___Science and Technology    |___Math    |   |___Research    |   |___num-analysis    |   |___Symbolic    |   |___Combinatorics    |   |___Probability    |   |   |___Prediction    |   |       |   |___Undergraduate    |   |___Recreational    |       |___Physics    |   |___Research    |   |___New Theories    |   |___Acoustics    |   |___Electromagnetics    |   |___Strings    |   |___Particle    |   |___Fusion    |   |___Relativity    |       |___Chem    |   |___Analytical    |   |___Electrochem    |   |   |___Battery    |   |       |   |___Coatings    |       |___Engineering        |___Control        |___Mechanics        |___Chemical

 Topic Author Forum Replies Last Post Similar Topics Help with Boolean Algebra Questions!!! mcbivens@umes.edu Math 0 Thu Sep 14, 2006 6:58 pm Help with Boolean Algebra Questions!!! mcbivens@umes.edu Math 0 Thu Sep 14, 2006 6:58 pm M.C. Escher logarithmic image transformation Mark Spahn Recreational 1 Fri Jul 21, 2006 8:35 am A mathematical game - probability questions Alun Math 8 Sun Jul 16, 2006 12:25 pm My Questions Euler Cheung Electromagnetics 1 Sat Jul 15, 2006 12:43 pm