|
|
| Author |
Message |
Jake Mulch
science forum beginner
Joined: 20 Jun 2006
Posts: 2
|
 Posted: Tue Jun 20, 2006 1:07 pm Post subject:
Number of Monte-Carlo evaluations
|
|
|
|
When carrying out an Uncertainty Analysis of a model using a Monte Carlo method how do I decide upon the number of evaluations required? |
|
| Back to top |
|
 |
david250@gmail.com
science forum beginner
Joined: 28 Apr 2006
Posts: 7
|
| Posted: Wed Jun 21, 2006 10:10 am Post subject:
Re: Number of Monte-Carlo evaluations
|
|
|
Jake Mulch wrote:
| Quote: | When carrying out an Uncertainty Analysis of a model using a Monte Carlo method how do I decide upon the number of evaluations required?
|
In many applications of the Monte Carlo method, what is wanted
is an estimate of the probability of success (p) of a random
experiment.
For exemple, two independent real numbers are chosen, each
following a uniform probability density function on the interval
[-1, 1]. If we call the random variables X and Y, we define
a success as:
X^2 + Y^2 <=1 and a failure otherwise.
The probability of success, p, is pi/4 or about 0.785.
If p were easly computable from known results in probability, there
would be no need to resort to the Monte Carlo method.
If we pretend we don't know 'p' in the above situation, then one
criterion that can help in deciding how long to run the Monte Carlo
method is: "How well does one need to know 'p'?" and
"Why is that level of certainty required?"
If 'f' is the observed frequency of success, and N is the number
of Monte Carlo trials, then the variance of 'f' is:
p*(1-p)/N . Since 'p' is presumably unknown, so
is the variance of 'f' ...
Unless one Monte Carlo trial costs a lot of computer time, I would
argue for (say) a few million trials.
Another angle is that a model is usually set up to approximate
some real-life (or external world) situation. If the model
is flawed, no amount of Monte Carlo experiments can
compensate for a flawed model.
Perhaps a first step would be to describe the real-life situation and
the
proposed model you have in mind. I'm not a statistician; my
impression is that statiscians would know about good/bad models.
David Bernier |
|
| Back to top |
|
|
Jyrki Lahtonen
science forum Guru Wannabe
Joined: 02 May 2005
Posts: 190
|
| Posted: Wed Jun 21, 2006 1:29 pm Post subject:
Re: Number of Monte-Carlo evaluations
|
|
|
David Bernier wrote:
| Quote: | If we pretend we don't know 'p' in the above situation, then one
criterion that can help in deciding how long to run the Monte Carlo
method is: "How well does one need to know 'p'?" and
"Why is that level of certainty required?"
If 'f' is the observed frequency of success, and N is the number
of Monte Carlo trials, then the variance of 'f' is:
p*(1-p)/N . Since 'p' is presumably unknown, so
is the variance of 'f' ...
|
Quite.
| Quote: | Unless one Monte Carlo trial costs a lot of computer time, I would
argue for (say) a few million trials.
|
A blanket answer is not good enough, as it obviously depends on the
value of p. If p is, say, in the ballpark of ten percent, then
a few million trials is probably excessive, unless you want to
get many significant digits.
When I simulate the performance of a candidate code in a data
transmission application, I seek to set the loop up so that the
number of 'bad' events is at least 400. This is because I need
to do several loops for different values of an auxiliary parameter
(more often than not the signal-to-noise ratio). So as this parameter
varies, the failure probability p that I want to estimate may
become quite small. Approximating 1-p by 1 gives us then a standard
deviation of approximately sqrt(p*N), IOW the square root of
the number of failures. If I require p*N to be at least 400, then
its square root is at most one twentieth of the number of failures.
This puts the +/- 2SD at exactly 10 per cent. So I can be 95 per cent
sure that I have "one significant digit" of accuracy. This seems
to be a relatively practical place for ending the loop in my case.
Of course, there is an expected behaviour of 'p' as a function of
the auxiliary parameter, so taken together the Monte Carlo values
give slightly better reliability range.
Am I making any sense?
Cheers,
Jyrki |
|
| Back to top |
|
|
Herman Rubin
science forum Guru
Joined: 25 Mar 2005
Posts: 730
|
| Posted: Wed Jun 21, 2006 3:59 pm Post subject:
Re: Number of Monte-Carlo evaluations
|
|
|
In article <1150884637.197932.16200@g10g2000cwb.googlegroups.com>,
David Bernier <david250@gmail.com> wrote:
| Quote: | Jake Mulch wrote:
When carrying out an Uncertainty Analysis of a model using a Monte Carlo method how do I decide upon the number of evaluations required?
In many applications of the Monte Carlo method, what is wanted
is an estimate of the probability of success (p) of a random
experiment.
|
Even more generally, one can use the variance approach for
straight Monte Carlo.
If one is estimating an expectation by random sums, which
includes success probabilities, one can use the sample
variance of a fair number to estimate the variance, and
use this to decide how many evaluations to use to get a
given standard deviation for the average.
This does NOT work for MCMC. In some cases, there is an
equivalent method, but it is usually not apparent.
--
This address is for information only. I do not claim that these views
are those of the Statistics Department or of Purdue University.
Herman Rubin, Department of Statistics, Purdue University
hrubin@stat.purdue.edu Phone: (765)494-6054 FAX: (765)494-0558 |
|
| Back to top |
|
|
Google
|
|
| Back to top |
|
|
|
|
The time now is Sat Jan 10, 2009 1:17 am | All times are GMT
|
|
Home Loan | Keeping Aquarium Fish | Northern Rock | MPAA | Secured Loan
|
|
Copyright © 2004-2005 DeniX Solutions SRL
|
|
Other DeniX Solutions sites:
Electronics forum |
Medicine forum |
Unix/Linux blog |
Unix/Linux documentation |
Unix/Linux forums
|
Powered by phpBB © 2001, 2005 phpBB Group
|
|
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
Forum index
»
Science and Technology
»
Math
