Author 
Message 
hmobahi@gmail.com science forum beginner
Joined: 14 Sep 2005
Posts: 12

Posted: Mon May 22, 2006 7:28 am Post subject:
Highest precision of finite difference, given machine's epsilon



Hi,
Computing derivative using finite difference is
df(x)/dx=(f(x+h)f(x))/h . Of course, the smaller the h is the higher
precision is achieved, but if h gets too small, there is a risk of
losing precision due to machine's limitation in representing very small
numbers.
Somewhere I read a rule of thumb for choosing h is as
h=x*sqrt(epsilon) , where epsilon is the smallest number that the
machine can represent. For instance, if using double numbers in c
language, one could put DBL_EPSILON from "float.h" in place of the
epsilon.
1. The first question: is this rule of thumb correct? I just read it in
a web page and I cannot trust it so easily.
2. What if I am interested in the second derivative, i.e. (
f(xh)2*f(x)+f(x+h) ) / (h^2) ... now how the rule works? Should it be
h=x*sqrt(epsilon) or (h^2)=x*sqrt(epsilon) ?
Thnx 

Back to top 


Alois Steindl science forum beginner
Joined: 10 May 2005
Posts: 25

Posted: Mon May 22, 2006 7:47 am Post subject:
Re: Highest precision of finite difference, given machine's epsilon



Hello,
you can find out the solution easily: As you already noticed, the
error consists of two parts: a) f(x) is calculated with some error
eps, which might be considerably larger than the machineepsilon.
(After rereading your post, it seems that you didn't formulate that
point correctly: The main reason for the deviation for small h are
rounding errors in evaluating f(x).)
b) The finite difference approximation leads to an error, which can be
estimated by inserting the Taylor series for f into the difference
formula.
Just add these two terms and find the optimal value for h.
Once you have done this for a simple example, you should be able to
adapt it for many different cases. And you should be able to improve
the rule of thumb.
Alois 

Back to top 


hmobahi@gmail.com science forum beginner
Joined: 14 Sep 2005
Posts: 12

Posted: Mon May 22, 2006 7:59 am Post subject:
Re: Highest precision of finite difference, given machine's epsilon



Thanks, but I am too dumb to get it Can you please clarify a bit
more how the optimal h is obtained using Taylor's serries given machine
epsilon? 

Back to top 


Alois Steindl science forum beginner
Joined: 10 May 2005
Posts: 25

Posted: Mon May 22, 2006 8:05 am Post subject:
Re: Highest precision of finite difference, given machine's epsilon



Hello,
you should definitively try it yourself, but here is some hint:
The rounding error term can be estimated by 2*eps/h.
Write the first 3 terms of the Taylor series of f(x+h) at x and
compare the term (f(x+h)f(x))/h with f'(x).
If you intend to do numerical calculations, you should be able to work
that out.
Alois 

Back to top 


Martin Eisenberg science forum beginner
Joined: 11 May 2005
Posts: 28

Posted: Mon May 22, 2006 1:29 pm Post subject:
Re: Highest precision of finite difference, given machine's epsilon



hmobahi@gmail.com wrote:
Quote:  Somewhere I read a rule of thumb for choosing h is as
h=x*sqrt(epsilon) , where epsilon is the smallest number that
the machine can represent.

That's not what DBL_EPSILON means, at least.
Quote:  For instance, if using double numbers
in c language, one could put DBL_EPSILON from "float.h" in place
of the epsilon.

Martin

Quidquid latine scriptum sit, altum viditur. 

Back to top 


bv science forum addict
Joined: 16 May 2005
Posts: 59

Posted: Fri Jul 14, 2006 9:53 pm Post subject:
Re: Highest precision of finite difference, given machine's epsilon



hmobahi@gmail.com wrote:
Quote: 
Computing derivative using finite difference is
df(x)/dx=(f(x+h)f(x))/h . Of course, the smaller the h is the higher
precision is achieved, but if h gets too small, there is a risk of
losing precision due to machine's limitation in representing very small
numbers.

You're to be commended for the astute observation that even n e t l i b
ode codes apparently *missed*. e.g. look at their clumsy attempts to
estimate initial steps involving finite difference eqns.
Quote:  Somewhere I read a rule of thumb for choosing h is as
h=x*sqrt(epsilon) , where epsilon is the smallest number that the
machine can represent.

Correct, although you have to be mindful of the variations applied
to higher order difference schemes. The formula is derived from a
general error model first postulated by Henrici in his classic textbook.
Quote:  1. The first question: is this rule of thumb correct? I just read it in
a web page and I cannot trust it so easily.

You can trust, but you must verify, as you've done. btw, it's not a rule
of thumb, it's numerical analysis equivalent of "mc^2".
Quote:  2. What if I am interested in the second derivative, i.e. (
f(xh)2*f(x)+f(x+h) ) / (h^2) ... now how the rule works? Should it be
h=x*sqrt(epsilon) or (h^2)=x*sqrt(epsilon) ?

Actually, it's neither. If it was, then the former squared would be
correct, that is, one formula, one h. For actual h in this case take a
look at, http://www.uc.edu/sashtml/ormp/chap5/sect28.htm

sdx
http://www.sdynamix.com 

Back to top 


Google


Back to top 



The time now is Mon Sep 24, 2018 5:54 pm  All times are GMT

