Highest precision of finite difference, given machine's epsilon

hmobahi@gmail.com · science forum beginner Joined: 14 Sep 2005 Posts: 12

Hi,

Computing derivative using finite difference is
df(x)/dx=(f(x+h)-f(x))/h . Of course, the smaller the h is the higher
precision is achieved, but if h gets too small, there is a risk of
losing precision due to machine's limitation in representing very small
numbers.

Somewhere I read a rule of thumb for choosing h is as
h=|x|*sqrt(epsilon) , where epsilon is the smallest number that the
machine can represent. For instance, if using double numbers in c
language, one could put DBL_EPSILON from "float.h" in place of the
epsilon.

1. The first question: is this rule of thumb correct? I just read it in
a web page and I cannot trust it so easily.

2. What if I am interested in the second derivative, i.e. (
f(x-h)-2*f(x)+f(x+h) ) / (h^2) ... now how the rule works? Should it be
h=|x|*sqrt(epsilon) or (h^2)=|x|*sqrt(epsilon) ?

Thnx

Alois Steindl · science forum beginner Joined: 10 May 2005 Posts: 25

Hello,
you can find out the solution easily: As you already noticed, the
error consists of two parts: a) f(x) is calculated with some error
eps, which might be considerably larger than the machine-epsilon.
(After rereading your post, it seems that you didn't formulate that
point correctly: The main reason for the deviation for small h are
rounding errors in evaluating f(x).)
b) The finite difference approximation leads to an error, which can be
estimated by inserting the Taylor series for f into the difference
formula.
Just add these two terms and find the optimal value for h.
Once you have done this for a simple example, you should be able to
adapt it for many different cases. And you should be able to improve
the rule of thumb.

Alois

hmobahi@gmail.com · science forum beginner Joined: 14 Sep 2005 Posts: 12

Thanks, but I am too dumb to get it Smile

Can you please clarify a bit
more how the optimal h is obtained using Taylor's serries given machine
epsilon?

Alois Steindl · science forum beginner Joined: 10 May 2005 Posts: 25

Hello,
you should definitively try it yourself, but here is some hint:
The rounding error term can be estimated by 2*eps/h.

Write the first 3 terms of the Taylor series of f(x+h) at x and
compare the term (f(x+h)-f(x))/h with f'(x).

If you intend to do numerical calculations, you should be able to work
that out.

Alois

Martin Eisenberg · science forum beginner Joined: 11 May 2005 Posts: 28

hmobahi@gmail.com wrote:

bv · science forum addict Joined: 16 May 2005 Posts: 59

hmobahi@gmail.com wrote:

Google