Search   Memberlist   Usergroups
 Page 1 of 1 [6 Posts]
Author Message
hmobahi@gmail.com
science forum beginner

Joined: 14 Sep 2005
Posts: 12

Posted: Mon May 22, 2006 7:28 am    Post subject: Highest precision of finite difference, given machine's epsilon

Hi,

Computing derivative using finite difference is
df(x)/dx=(f(x+h)-f(x))/h . Of course, the smaller the h is the higher
precision is achieved, but if h gets too small, there is a risk of
losing precision due to machine's limitation in representing very small
numbers.

Somewhere I read a rule of thumb for choosing h is as
h=|x|*sqrt(epsilon) , where epsilon is the smallest number that the
machine can represent. For instance, if using double numbers in c
language, one could put DBL_EPSILON from "float.h" in place of the
epsilon.

1. The first question: is this rule of thumb correct? I just read it in
a web page and I cannot trust it so easily.

2. What if I am interested in the second derivative, i.e. (
f(x-h)-2*f(x)+f(x+h) ) / (h^2) ... now how the rule works? Should it be
h=|x|*sqrt(epsilon) or (h^2)=|x|*sqrt(epsilon) ?

Thnx
Alois Steindl
science forum beginner

Joined: 10 May 2005
Posts: 25

 Posted: Mon May 22, 2006 7:47 am    Post subject: Re: Highest precision of finite difference, given machine's epsilon Hello, you can find out the solution easily: As you already noticed, the error consists of two parts: a) f(x) is calculated with some error eps, which might be considerably larger than the machine-epsilon. (After rereading your post, it seems that you didn't formulate that point correctly: The main reason for the deviation for small h are rounding errors in evaluating f(x).) b) The finite difference approximation leads to an error, which can be estimated by inserting the Taylor series for f into the difference formula. Just add these two terms and find the optimal value for h. Once you have done this for a simple example, you should be able to adapt it for many different cases. And you should be able to improve the rule of thumb. Alois
hmobahi@gmail.com
science forum beginner

Joined: 14 Sep 2005
Posts: 12

 Posted: Mon May 22, 2006 7:59 am    Post subject: Re: Highest precision of finite difference, given machine's epsilon Thanks, but I am too dumb to get it Can you please clarify a bit more how the optimal h is obtained using Taylor's serries given machine epsilon?
Alois Steindl
science forum beginner

Joined: 10 May 2005
Posts: 25

Posted: Mon May 22, 2006 8:05 am    Post subject: Re: Highest precision of finite difference, given machine's epsilon

Hello,
you should definitively try it yourself, but here is some hint:
The rounding error term can be estimated by 2*eps/h.

Write the first 3 terms of the Taylor series of f(x+h) at x and
compare the term (f(x+h)-f(x))/h with f'(x).

If you intend to do numerical calculations, you should be able to work
that out.

Alois
Martin Eisenberg
science forum beginner

Joined: 11 May 2005
Posts: 28

Posted: Mon May 22, 2006 1:29 pm    Post subject: Re: Highest precision of finite difference, given machine's epsilon

hmobahi@gmail.com wrote:

 Quote: Somewhere I read a rule of thumb for choosing h is as h=|x|*sqrt(epsilon) , where epsilon is the smallest number that the machine can represent.

That's not what DBL_EPSILON means, at least.

 Quote: For instance, if using double numbers in c language, one could put DBL_EPSILON from "float.h" in place of the epsilon.

Martin

--
Quidquid latine scriptum sit, altum viditur.
bv

Joined: 16 May 2005
Posts: 59

Posted: Fri Jul 14, 2006 9:53 pm    Post subject: Re: Highest precision of finite difference, given machine's epsilon

hmobahi@gmail.com wrote:
 Quote: Computing derivative using finite difference is df(x)/dx=(f(x+h)-f(x))/h . Of course, the smaller the h is the higher precision is achieved, but if h gets too small, there is a risk of losing precision due to machine's limitation in representing very small numbers.

You're to be commended for the astute observation that even n e t l i b
ode codes apparently *missed*. e.g. look at their clumsy attempts to
estimate initial steps involving finite difference eqns.

 Quote: Somewhere I read a rule of thumb for choosing h is as h=|x|*sqrt(epsilon) , where epsilon is the smallest number that the machine can represent.

Correct, although you have to be mindful of the variations applied
to higher order difference schemes. The formula is derived from a
general error model first postulated by Henrici in his classic textbook.

 Quote: 1. The first question: is this rule of thumb correct? I just read it in a web page and I cannot trust it so easily.

You can trust, but you must verify, as you've done. btw, it's not a rule
of thumb, it's numerical analysis equivalent of "mc^2".

 Quote: 2. What if I am interested in the second derivative, i.e. ( f(x-h)-2*f(x)+f(x+h) ) / (h^2) ... now how the rule works? Should it be h=|x|*sqrt(epsilon) or (h^2)=|x|*sqrt(epsilon) ?

Actually, it's neither. If it was, then the former squared would be
correct, that is, one formula, one h. For actual h in this case take a
look at, http://www.uc.edu/sashtml/ormp/chap5/sect28.htm

--
sdx
http://www.sdynamix.com

 Display posts from previous: All Posts1 Day7 Days2 Weeks1 Month3 Months6 Months1 Year Oldest FirstNewest First
 Page 1 of 1 [6 Posts]
 The time now is Tue Mar 26, 2019 4:04 am | All times are GMT
 Jump to: Select a forum-------------------Forum index|___Science and Technology    |___Math    |   |___Research    |   |___num-analysis    |   |___Symbolic    |   |___Combinatorics    |   |___Probability    |   |   |___Prediction    |   |       |   |___Undergraduate    |   |___Recreational    |       |___Physics    |   |___Research    |   |___New Theories    |   |___Acoustics    |   |___Electromagnetics    |   |___Strings    |   |___Particle    |   |___Fusion    |   |___Relativity    |       |___Chem    |   |___Analytical    |   |___Electrochem    |   |   |___Battery    |   |       |   |___Coatings    |       |___Engineering        |___Control        |___Mechanics        |___Chemical

 Topic Author Forum Replies Last Post Similar Topics Choice function over finite sets Peter Webb Math 5 Fri Jul 21, 2006 3:28 am Finite fields and Splitting fields vedmundson@hotmail.com Math 8 Thu Jul 20, 2006 6:35 pm What is the difference between unit h and unit h = h / 2pi? socratus Relativity 0 Thu Jul 20, 2006 1:22 pm JSH: Truth is the highest ideal jstevh@msn.com Math 9 Thu Jul 20, 2006 5:18 am Finite # of subgroups -> Finite group tppytel@gmail.com Undergraduate 9 Sun Jul 16, 2006 1:47 am