|
Author |
Message |
hmobahi@gmail.com science forum beginner
Joined: 14 Sep 2005
Posts: 12
|
Posted: Mon May 22, 2006 7:28 am Post subject:
Highest precision of finite difference, given machine's epsilon
|
|
|
Hi,
Computing derivative using finite difference is
df(x)/dx=(f(x+h)-f(x))/h . Of course, the smaller the h is the higher
precision is achieved, but if h gets too small, there is a risk of
losing precision due to machine's limitation in representing very small
numbers.
Somewhere I read a rule of thumb for choosing h is as
h=|x|*sqrt(epsilon) , where epsilon is the smallest number that the
machine can represent. For instance, if using double numbers in c
language, one could put DBL_EPSILON from "float.h" in place of the
epsilon.
1. The first question: is this rule of thumb correct? I just read it in
a web page and I cannot trust it so easily.
2. What if I am interested in the second derivative, i.e. (
f(x-h)-2*f(x)+f(x+h) ) / (h^2) ... now how the rule works? Should it be
h=|x|*sqrt(epsilon) or (h^2)=|x|*sqrt(epsilon) ?
Thnx |
|
Back to top |
|
 |
Alois Steindl science forum beginner
Joined: 10 May 2005
Posts: 25
|
Posted: Mon May 22, 2006 7:47 am Post subject:
Re: Highest precision of finite difference, given machine's epsilon
|
|
|
Hello,
you can find out the solution easily: As you already noticed, the
error consists of two parts: a) f(x) is calculated with some error
eps, which might be considerably larger than the machine-epsilon.
(After rereading your post, it seems that you didn't formulate that
point correctly: The main reason for the deviation for small h are
rounding errors in evaluating f(x).)
b) The finite difference approximation leads to an error, which can be
estimated by inserting the Taylor series for f into the difference
formula.
Just add these two terms and find the optimal value for h.
Once you have done this for a simple example, you should be able to
adapt it for many different cases. And you should be able to improve
the rule of thumb.
Alois |
|
Back to top |
|
 |
hmobahi@gmail.com science forum beginner
Joined: 14 Sep 2005
Posts: 12
|
Posted: Mon May 22, 2006 7:59 am Post subject:
Re: Highest precision of finite difference, given machine's epsilon
|
|
|
Thanks, but I am too dumb to get it Can you please clarify a bit
more how the optimal h is obtained using Taylor's serries given machine
epsilon? |
|
Back to top |
|
 |
Alois Steindl science forum beginner
Joined: 10 May 2005
Posts: 25
|
Posted: Mon May 22, 2006 8:05 am Post subject:
Re: Highest precision of finite difference, given machine's epsilon
|
|
|
Hello,
you should definitively try it yourself, but here is some hint:
The rounding error term can be estimated by 2*eps/h.
Write the first 3 terms of the Taylor series of f(x+h) at x and
compare the term (f(x+h)-f(x))/h with f'(x).
If you intend to do numerical calculations, you should be able to work
that out.
Alois |
|
Back to top |
|
 |
Martin Eisenberg science forum beginner
Joined: 11 May 2005
Posts: 28
|
Posted: Mon May 22, 2006 1:29 pm Post subject:
Re: Highest precision of finite difference, given machine's epsilon
|
|
|
hmobahi@gmail.com wrote:
Quote: | Somewhere I read a rule of thumb for choosing h is as
h=|x|*sqrt(epsilon) , where epsilon is the smallest number that
the machine can represent.
|
That's not what DBL_EPSILON means, at least.
Quote: | For instance, if using double numbers
in c language, one could put DBL_EPSILON from "float.h" in place
of the epsilon.
|
Martin
--
Quidquid latine scriptum sit, altum viditur. |
|
Back to top |
|
 |
bv science forum addict
Joined: 16 May 2005
Posts: 59
|
Posted: Fri Jul 14, 2006 9:53 pm Post subject:
Re: Highest precision of finite difference, given machine's epsilon
|
|
|
hmobahi@gmail.com wrote:
Quote: |
Computing derivative using finite difference is
df(x)/dx=(f(x+h)-f(x))/h . Of course, the smaller the h is the higher
precision is achieved, but if h gets too small, there is a risk of
losing precision due to machine's limitation in representing very small
numbers.
|
You're to be commended for the astute observation that even n e t l i b
ode codes apparently *missed*. e.g. look at their clumsy attempts to
estimate initial steps involving finite difference eqns.
Quote: | Somewhere I read a rule of thumb for choosing h is as
h=|x|*sqrt(epsilon) , where epsilon is the smallest number that the
machine can represent.
|
Correct, although you have to be mindful of the variations applied
to higher order difference schemes. The formula is derived from a
general error model first postulated by Henrici in his classic textbook.
Quote: | 1. The first question: is this rule of thumb correct? I just read it in
a web page and I cannot trust it so easily.
|
You can trust, but you must verify, as you've done. btw, it's not a rule
of thumb, it's numerical analysis equivalent of "mc^2".
Quote: | 2. What if I am interested in the second derivative, i.e. (
f(x-h)-2*f(x)+f(x+h) ) / (h^2) ... now how the rule works? Should it be
h=|x|*sqrt(epsilon) or (h^2)=|x|*sqrt(epsilon) ?
|
Actually, it's neither. If it was, then the former squared would be
correct, that is, one formula, one h. For actual h in this case take a
look at, http://www.uc.edu/sashtml/ormp/chap5/sect28.htm
--
sdx
http://www.sdynamix.com |
|
Back to top |
|
 |
Google
|
|
Back to top |
|
 |
|
The time now is Fri Apr 20, 2018 6:30 pm | All times are GMT
|
Copyright © 2004-2005 DeniX Solutions SRL
|
Other DeniX Solutions sites:
Electronics forum |
Medicine forum |
Unix/Linux blog |
Unix/Linux documentation |
Unix/Linux forums |
send newsletters
|
|
Powered by phpBB © 2001, 2005 phpBB Group
|
|