FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups 
 ProfileProfile   PreferencesPreferences   Log in to check your private messagesLog in to check your private messages   Log inLog in 
Forum index » Science and Technology » Math » num-analysis
Highest precision of finite difference, given machine's epsilon
Post new topic   Reply to topic Page 1 of 1 [6 Posts] View previous topic :: View next topic
Author Message
hmobahi@gmail.com
science forum beginner


Joined: 14 Sep 2005
Posts: 12

PostPosted: Mon May 22, 2006 7:28 am    Post subject: Highest precision of finite difference, given machine's epsilon Reply with quote

Hi,

Computing derivative using finite difference is
df(x)/dx=(f(x+h)-f(x))/h . Of course, the smaller the h is the higher
precision is achieved, but if h gets too small, there is a risk of
losing precision due to machine's limitation in representing very small
numbers.

Somewhere I read a rule of thumb for choosing h is as
h=|x|*sqrt(epsilon) , where epsilon is the smallest number that the
machine can represent. For instance, if using double numbers in c
language, one could put DBL_EPSILON from "float.h" in place of the
epsilon.

1. The first question: is this rule of thumb correct? I just read it in
a web page and I cannot trust it so easily.

2. What if I am interested in the second derivative, i.e. (
f(x-h)-2*f(x)+f(x+h) ) / (h^2) ... now how the rule works? Should it be
h=|x|*sqrt(epsilon) or (h^2)=|x|*sqrt(epsilon) ?

Thnx
Back to top
Alois Steindl
science forum beginner


Joined: 10 May 2005
Posts: 25

PostPosted: Mon May 22, 2006 7:47 am    Post subject: Re: Highest precision of finite difference, given machine's epsilon Reply with quote

Hello,
you can find out the solution easily: As you already noticed, the
error consists of two parts: a) f(x) is calculated with some error
eps, which might be considerably larger than the machine-epsilon.
(After rereading your post, it seems that you didn't formulate that
point correctly: The main reason for the deviation for small h are
rounding errors in evaluating f(x).)
b) The finite difference approximation leads to an error, which can be
estimated by inserting the Taylor series for f into the difference
formula.
Just add these two terms and find the optimal value for h.
Once you have done this for a simple example, you should be able to
adapt it for many different cases. And you should be able to improve
the rule of thumb.

Alois
Back to top
hmobahi@gmail.com
science forum beginner


Joined: 14 Sep 2005
Posts: 12

PostPosted: Mon May 22, 2006 7:59 am    Post subject: Re: Highest precision of finite difference, given machine's epsilon Reply with quote

Thanks, but I am too dumb to get it Smile Can you please clarify a bit
more how the optimal h is obtained using Taylor's serries given machine
epsilon?
Back to top
Alois Steindl
science forum beginner


Joined: 10 May 2005
Posts: 25

PostPosted: Mon May 22, 2006 8:05 am    Post subject: Re: Highest precision of finite difference, given machine's epsilon Reply with quote

Hello,
you should definitively try it yourself, but here is some hint:
The rounding error term can be estimated by 2*eps/h.

Write the first 3 terms of the Taylor series of f(x+h) at x and
compare the term (f(x+h)-f(x))/h with f'(x).

If you intend to do numerical calculations, you should be able to work
that out.

Alois
Back to top
Martin Eisenberg
science forum beginner


Joined: 11 May 2005
Posts: 28

PostPosted: Mon May 22, 2006 1:29 pm    Post subject: Re: Highest precision of finite difference, given machine's epsilon Reply with quote

hmobahi@gmail.com wrote:

Quote:
Somewhere I read a rule of thumb for choosing h is as
h=|x|*sqrt(epsilon) , where epsilon is the smallest number that
the machine can represent.

That's not what DBL_EPSILON means, at least.

Quote:
For instance, if using double numbers
in c language, one could put DBL_EPSILON from "float.h" in place
of the epsilon.


Martin

--
Quidquid latine scriptum sit, altum viditur.
Back to top
bv
science forum addict


Joined: 16 May 2005
Posts: 59

PostPosted: Fri Jul 14, 2006 9:53 pm    Post subject: Re: Highest precision of finite difference, given machine's epsilon Reply with quote

hmobahi@gmail.com wrote:
Quote:

Computing derivative using finite difference is
df(x)/dx=(f(x+h)-f(x))/h . Of course, the smaller the h is the higher
precision is achieved, but if h gets too small, there is a risk of
losing precision due to machine's limitation in representing very small
numbers.

You're to be commended for the astute observation that even n e t l i b
ode codes apparently *missed*. e.g. look at their clumsy attempts to
estimate initial steps involving finite difference eqns.

Quote:
Somewhere I read a rule of thumb for choosing h is as
h=|x|*sqrt(epsilon) , where epsilon is the smallest number that the
machine can represent.

Correct, although you have to be mindful of the variations applied
to higher order difference schemes. The formula is derived from a
general error model first postulated by Henrici in his classic textbook.

Quote:
1. The first question: is this rule of thumb correct? I just read it in
a web page and I cannot trust it so easily.

You can trust, but you must verify, as you've done. btw, it's not a rule
of thumb, it's numerical analysis equivalent of "mc^2".

Quote:
2. What if I am interested in the second derivative, i.e. (
f(x-h)-2*f(x)+f(x+h) ) / (h^2) ... now how the rule works? Should it be
h=|x|*sqrt(epsilon) or (h^2)=|x|*sqrt(epsilon) ?

Actually, it's neither. If it was, then the former squared would be
correct, that is, one formula, one h. For actual h in this case take a
look at, http://www.uc.edu/sashtml/ormp/chap5/sect28.htm

--
sdx
http://www.sdynamix.com
Back to top
Google

Back to top
Display posts from previous:   
Post new topic   Reply to topic Page 1 of 1 [6 Posts] View previous topic :: View next topic
The time now is Wed Oct 18, 2017 4:43 pm | All times are GMT
Forum index » Science and Technology » Math » num-analysis
Jump to:  

Similar Topics
Topic Author Forum Replies Last Post
No new posts Choice function over finite sets Peter Webb Math 5 Fri Jul 21, 2006 3:28 am
No new posts Finite fields and Splitting fields vedmundson@hotmail.com Math 8 Thu Jul 20, 2006 6:35 pm
No new posts What is the difference between unit h and unit h = h / 2pi? socratus Relativity 0 Thu Jul 20, 2006 1:22 pm
No new posts JSH: Truth is the highest ideal jstevh@msn.com Math 9 Thu Jul 20, 2006 5:18 am
No new posts Finite # of subgroups -> Finite group tppytel@gmail.com Undergraduate 9 Sun Jul 16, 2006 1:47 am

Copyright © 2004-2005 DeniX Solutions SRL
Other DeniX Solutions sites: Electronics forum |  Medicine forum |  Unix/Linux blog |  Unix/Linux documentation |  Unix/Linux forums  |  send newsletters
 


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.0416s ][ Queries: 16 (0.0235s) ][ GZIP on - Debug on ]