Author 
Message 
Don Taylor From: science forum beginner
Joined: 18 May 2005
Posts: 4

Posted: Wed May 18, 2005 7:26 pm Post subject:
Re: Fractals and Stock Markets



"Pavel314" <Pavel314@comcast.net> writes:
Quote:  A good book on stock market behavior is "The Misbehavior of Markets" by
Benoit Mandelbrot and Richard L. Hudson. They apply fractal theory to the
financial markets, reviewing historical stock activity and exposing some
flaws in modern financial theory. Very readable and well researched.

I read that carefully. Some of the flaws they point out certainly
seem valid. For example, the market data seems to support their
claim that market movement cannot be strictly normally distributed
because there are too many sudden large changes to be accounted for
by the statistics calculated from the overall dataset.
But then when they try to counter these with their own explanations
I came to the conclusion that there were serious gaps and inconsistencies
in the arguments they were making. Am I just misunderstanding how
good that book was?
I'd have to go find it again and page through the chapters to come
up with a specific list of objections again. But I did mention
some of these to a person who has been fascinated with fractals for
decades. He didn't immediately refute what I was saying but did
say that he didn't want to criticize something that Mandelbrot might
have written in his old age. 

Back to top 


Pavel314 science forum addict
Joined: 29 Apr 2005
Posts: 78

Posted: Sat May 21, 2005 6:56 pm Post subject:
Re: A probability problem



"wasknijper" <wasknijper@x.y> wrote in message
news:%opje.4197$184.3498@amstwist00...
Quote:  Here is a probability problem from real life (I am now running the program
that the problem is about).
A computer program goes through 720 loops to find the best score (no need
to elaborate on what score that is). The program has now finished 25
loops; the best score so far was found in loop 7.
How do you calculate the probability p that this score will remain the
best, if, at the start, each loop is equally likely to find the best
score, and the chance of more than 1 loop both/all finding it is zero? Is
the number 7 relevant at all, or are 720 and 25 enough to find p? I hope
my wording is clear.

What does each loop of the program do? 720 seems like the number of
combinations of n things taken k at a time, but I'm on my second
ginandtonic of the afternoon and can't remember what nCk gives 720. From
your description, there are several alternatives.
1. There is a file of scores and each loop takes a sample of those scores
and reports the best score in the sample. You mention "find the best score"
frequently, which leads me to believe that this is what is happening.
If you have 25 samples (how many scores do you take in each sample?) from
the score universe, you can take the mean and standard deviation of each and
establish a confidence interval for the mean and standard deviation of the
universe as a whole. This will tell you how many standard deviations the
best score in loop 7 is from the mean and hence the probability of finding a
higher score.
2. Each loop of the program generates a score. No scores exist or can be
known before a program loop generates it.
Here you can take your 25 scores as generated by the machine as a sample to
get the mean and standard deviation of your sample. Again, the best score
from loop 7 can be plotted on the distribution and the probability of a
higher score computed.
I'm assuming that the scores fall in a normal distribution.
Paul 

Back to top 


Ray Koopman science forum Guru Wannabe
Joined: 25 Mar 2005
Posts: 216

Posted: Sun May 22, 2005 5:03 am Post subject:
Re: A probability problem



Pavel314 wrote:
Quote:  "wasknijper" <wasknij...@x.y> wrote in message
news:%opje.4197$184.3498@amstwist00...
Here is a probability problem from real life (I am now running the
program that the problem is about).
A computer program goes through 720 loops to find the best score
(no need to elaborate on what score that is). The program has now
finished 25 loops; the best score so far was found in loop 7.
How do you calculate the probability p that this score will remain
the best, if, at the start, each loop is equally likely to find the
best score, and the chance of more than 1 loop both/all finding it
is zero? Is the number 7 relevant at all, or are 720 and 25 enough
to find p? I hope my wording is clear.

If the program does N loops, the probability that the best score is
found in the first n loops, or any particular set of n loops, is n/N. 

Back to top 


wasknijper science forum beginner
Joined: 03 May 2005
Posts: 16

Posted: Sun May 22, 2005 1:29 pm Post subject:
Re: A probability problem



Pavel314 wrote:
Quote:  What does each loop of the program do? 720 seems like the number of
combinations of n things taken k at a time, but I'm on my second
ginandtonic of the afternoon and can't remember what nCk gives 720. From
your description, there are several alternatives.

What it does is approximating a number, like pi or e, using digits 1
to n (n in [1,9]), the operations +*/^, decimal dot, (), and
concatenation of digits. (Both binary and unary minus.) In a number of
embedded loops, the program generates all wellformed postfix strings
and evaluates them (only for lower n).
See http://www.stetson.edu/%7Eefriedma/mathmagic/0804.html
(I know that certain kinds of shortcuts are possible, which will not
take endanger the exhaustive nature of the search.) 

Back to top 


wasknijper science forum beginner
Joined: 03 May 2005
Posts: 16

Posted: Sun May 22, 2005 1:34 pm Post subject:
Re: A probability problem



Pavel314 wrote:
Quote:  What does each loop of the program do? 720 seems like the number of
combinations of n things taken k at a time, but I'm on my second
ginandtonic of the afternoon and can't remember what nCk gives 720. From
your description, there are several alternatives.

What it does is approximating a number, like pi or e, using digits 1
to n (n in [1,9]), the operations +*/^, decimal dot, (), and
concatenation of digits. (Both binary and unary minus.) In a number of
embedded loops, the program generates all wellformed postfix strings
and evaluates them (only for lower n).
See http://www.stetson.edu/%7Eefriedma/mathmagic/0804.html
(I know that certain kinds of shortcuts are possible, which will not
endanger the exhaustive nature of the search.) 

Back to top 


Leonid Gavrilov science forum beginner
Joined: 25 May 2005
Posts: 6


Back to top 


Pavel314 science forum addict
Joined: 29 Apr 2005
Posts: 78

Posted: Wed May 25, 2005 8:51 pm Post subject:
Re: A probability problem



"wasknijper" <wasknijper@x.y> wrote in message
news:vE1ke.4322$184.27@amstwist00...
Quote:  Pavel314 wrote:
What does each loop of the program do? 720 seems like the number of
combinations of n things taken k at a time, but I'm on my second
ginandtonic of the afternoon and can't remember what nCk gives 720.
From your description, there are several alternatives.
What it does is approximating a number, like pi or e, using digits 1 to n
(n in [1,9]), the operations +*/^, decimal dot, (), and concatenation of
digits. (Both binary and unary minus.) In a number of embedded loops, the
program generates all wellformed postfix strings and evaluates them (only
for lower n).
See http://www.stetson.edu/%7Eefriedma/mathmagic/0804.html
(I know that certain kinds of shortcuts are possible, which will not
endanger the exhaustive nature of the search.)

I have a possible solution at
http://mywebpages.comcast.net/pavel314/betscore.html
Let me know if you agree. Or not.
Paul 

Back to top 


wasknijper science forum beginner
Joined: 03 May 2005
Posts: 16

Posted: Thu May 26, 2005 3:14 pm Post subject:
Re: A probability problem



Pavel314 wrote:
It seems a good solution to me, as far as I'm able to judge (which, in
all honesty, isn't terribly far). 

Back to top 


MJ science forum beginner
Joined: 29 Apr 2005
Posts: 32

Posted: Fri May 27, 2005 12:54 pm Post subject:
Re: A probability problem



"wasknijper" <wasknijper@x.y> wrote in message
news:%opje.4197$184.3498@amstwist00...
Quote:  Here is a probability problem from real life
(I am now running the program that the problem is about).
A computer program goes through 720 loops to find the best score (no
need to elaborate on what score that is). The program has now finished
25 loops; the best score so far was found in loop 7.
How do you calculate the probability p that this score will remain the
best, if, at the start, each loop is equally likely to find the best
score, and the chance of more than 1 loop both/all finding it is zero?
Is the number 7 relevant at all, or are 720 and 25 enough to find p? I
hope my wording is clear.

I interpret your problem as follows:
"A sample of n independent observations,
is drawn from an unknown distribution,
over a finite range [0 <= x <= A], A unknown.
The largest x in the sample is x(n).
What is the best estimate for A? "
========
A Guideline
========
Assume the distribution of x to be uniform, that is,
pdf[x] = 1/A on [0, A], zero elsewhere.
Let the largest x in the sample of size n be x(n).
Then E[x(n)] = A*[n / (n + 1],
Hence y(n) = [(n + 1)/n]*x(n) is an unbiased
estimator of A, that is, E[y(n)] = A [1]
Variance[y(n)] = a^2 / [n(n + 2)]
sd[y(n)] = A / sqrt[n(n + 2)] = a / (n+1) (approx) [2]
For [1] and [2], for instance, see
http://www.ds.unifi.it/VL/VL_EN/point/point3.html
Example:
Say the value of the largest element in a sample of 25 is 100.
Then an unbiased estimate for A, the largest possible value
of x is 100*(26/25) = 104. [From [1]]
The standard error of this estimate is 100/26 = 3.8. [From [2]]
Using say (3*standard_error), the confidence interval for
A is estimated at {104, 104 + 3*3.8} = {104, 115.4}
which interval is quite narrow.
This is so because the (n+1) in the denominator of sd[y(n)] in [2] makes
the estimator y(n) "very efficient", that is, the standard error of the
estimator y(n) decreases as 1/n , compared with the usual 1/sqrt[n].
MJ 

Back to top 


Mark science forum addict
Joined: 02 May 2005
Posts: 51

Posted: Thu Jun 09, 2005 9:56 am Post subject:
Probability question, was Re: Mensa Forgot Another Possibility!



"BruceS" <bruces42@hotmail.com> wrote in message
news:1118264621.238754.254340@f14g2000cwb.googlegroups.com...
Quote:  I wouldn't be at all offended if you reposted there and returned here
with any answers. I try to avoid crossposting, especially to groups I
don't follow.

Here then is the question as I remember it:
on consideration this may not be precisely on topic for sci.stat.math
although I suppose that in general people who are up on stats are up on
probability too.
A student sits a test that has 20 questions; every question has precisely 1
matching answer. There are 20 answers, no duplication. The student answers
every question with a different answer from the 20.
What is the probability, of a student answering randomly, answering
precisely 1 right? precisely 10 right?
Mark 

Back to top 


Guest

Posted: Thu Jun 09, 2005 11:50 am Post subject:
Re: Probability question, was Re: Mensa Forgot Another Possibility!



Disclaimer: I'm no expert statistician but that rarely shuts me up.
This is a matching problem. I had reason to calculate probabilities for
a matching problem during a discussion on a "hoax" discussion board
about Natalya Demkina, the socalled "Girl with XRay Eyes" who claimed
that she could diagnose people's illness by looking inside their
bodies. She was given 7 people to diagnose, was told what she should be
looking for, and asked to match the people to their illnesses (e.g.
metal plate in the head). The TV experiment gave a predefined
definition of success if she managed to identify 5 or more of the 7
ailments correctly.
There was considerable discussion on the hoax discussion board about
the probability of 5 or more correct matches occurring by chance as
described during the television program compared to what a few people
found by using a computer program to enumerate all possible matchings.
This discussion ended up with me (hopefully I got it right) reproducing
the probabilities found by exhaustive computer program search using the
formulae on the web page:
http://www.ds.unifi.it/VL/VL_EN/urn/urn6.html
Someone else calculated the Poisson approximation to the true
distribution, and got the same number as described on the television
program. The trivial differences were sufficient to cause a significant
amount of argument.
I believe that using the formulae on the above page will allow you to
calculate the answers you want. But, I'm too lazy to do it for you
myself.
Cheers,
Rossc 

Back to top 


Mark science forum addict
Joined: 02 May 2005
Posts: 51

Posted: Thu Jun 09, 2005 3:45 pm Post subject:
Re: Probability question, was Re: Mensa Forgot Another Possibility!



<clemenr@wmin.ac.uk> wrote in message
news:1118325015.324076.228310@o13g2000cwo.googlegroups.com...
Quote:  This discussion ended up with me (hopefully I got it right) reproducing
the probabilities found by exhaustive computer program search using the
formulae on the web page:
http://www.ds.unifi.it/VL/VL_EN/urn/urn6.html
Someone else calculated the Poisson approximation to the true
distribution, and got the same number as described on the television
program. The trivial differences were sufficient to cause a significant
amount of argument.
I believe that using the formulae on the above page will allow you to
calculate the answers you want. But, I'm too lazy to do it for you
myself.
Cheers,
Rossc

That looks right; I'll stick with the Poisson approximation for now, as 1.
it sounds very close to what I was reaching for before, 2. There's quite a
bit of Maths on the site that I haven't seen before. Making the probability
of 1 correct answer ~= e^1 ~= 0.36788, of 10 correct answers ~=
(e^1)/(10!) ~= 1.0138E7
Cheers
Mark 

Back to top 


Scott Hemphill science forum beginner
Joined: 09 Jun 2005
Posts: 21

Posted: Thu Jun 09, 2005 4:30 pm Post subject:
Re: Probability question, was Re: Mensa Forgot Another Possibility!



"Mark" <Markiehatesspam@notelespam2.fr> writes:
Quote:  "BruceS" <bruces42@hotmail.com> wrote in message
news:1118264621.238754.254340@f14g2000cwb.googlegroups.com...
I wouldn't be at all offended if you reposted there and returned here
with any answers. I try to avoid crossposting, especially to groups I
don't follow.
Here then is the question as I remember it:
on consideration this may not be precisely on topic for sci.stat.math
although I suppose that in general people who are up on stats are up on
probability too.
A student sits a test that has 20 questions; every question has precisely 1
matching answer. There are 20 answers, no duplication. The student answers
every question with a different answer from the 20.
What is the probability, of a student answering randomly, answering
precisely 1 right? precisely 10 right?

This problem is treated in _Concrete Mathematics_ by Graham, Knuth and
Patashnik. They call it the "'football victory problem': a group of n fans
of the winning football team throw their hats high into the air. The
hats come back randomly, one hat to each of the n fans. How many ways
h(n,k) are there for exactly k fans to get their own hats back?"
They derive a closed form solution for the answer:
h(n,k) = choose(n,k) * subfactorial(nk)
where choose(n,k) = nCk = n!/((nk)!k!)
and subfactorial(n) is the number of derangements of n objects, i.e.
the number of permutations for which none of objects are in their
original position.
subfactorial(n) is further developed:
subfactorial(n) = round(n!/e) + [n==0]
where round is the function which rounds to the nearest integer
and the bracket notation yields 1 if the boolean expression inside
is true and 0 if the boolean expression is false. I've used ==
for the equality operator, and e is Euler's constant e = 2.71828....
The values of h(n,k) for n = 20:
k h(20,k)
 
0 895014631192902121
1 895014631192902120
2 447507315596451070
3 149169105198816960
4 37292276299704525
5 7458455259939936
6 1243075876659240
7 177582268088640
8 22197783520770
9 2466420377200
10 246642054516
11 22421988160
12 1868513010
13 143722080
14 10271400
15 682176
16 43605
17 2280
18 190
19 0
20 1
The exact probability of each "k" occurring is calculated by dividing each
of these values by 20!.
Scott

Scott Hemphill hemphill@alumni.caltech.edu
"This isn't flying. This is falling, with style."  Buzz Lightyear 

Back to top 


Scott Hemphill science forum beginner
Joined: 09 Jun 2005
Posts: 21

Posted: Thu Jun 09, 2005 4:37 pm Post subject:
Re: Probability question, was Re: Mensa Forgot Another Possibility!



"Mark" <Markiehatesspam@notelespam2.fr> writes:
Quote:  clemenr@wmin.ac.uk> wrote in message
news:1118325015.324076.228310@o13g2000cwo.googlegroups.com...
This discussion ended up with me (hopefully I got it right) reproducing
the probabilities found by exhaustive computer program search using the
formulae on the web page:
http://www.ds.unifi.it/VL/VL_EN/urn/urn6.html
Someone else calculated the Poisson approximation to the true
distribution, and got the same number as described on the television
program. The trivial differences were sufficient to cause a significant
amount of argument.
I believe that using the formulae on the above page will allow you to
calculate the answers you want. But, I'm too lazy to do it for you
myself.
Cheers,
Rossc
That looks right; I'll stick with the Poisson approximation for now, as 1.
it sounds very close to what I was reaching for before, 2. There's quite a
bit of Maths on the site that I haven't seen before. Making the probability
of 1 correct answer ~= e^1 ~= 0.36788, of 10 correct answers ~=
(e^1)/(10!) ~= 1.0138E7

I posted a formula for the exact answers separately.
The differences between these answers and the exact answers are small:
1 correct: h[20,1]/20!  e^1 ~= 4E19
10 correct: h[20,10]/20!  e^1/10! ~= 6E15
Scott

Scott Hemphill hemphill@alumni.caltech.edu
"This isn't flying. This is falling, with style."  Buzz Lightyear 

Back to top 


Mark science forum addict
Joined: 02 May 2005
Posts: 51

Posted: Thu Jun 09, 2005 4:59 pm Post subject:
Re: Probability question, was Re: Mensa Forgot Another Possibility!



XNoArchive: Yes
"Scott Hemphill" <hemphill@hemphills.net> wrote in message
news:m3mzpzwfg9.fsf@pearl.local...
Quote:  They derive a closed form solution for the answer:
h(n,k) = choose(n,k) * subfactorial(nk)
where choose(n,k) = nCk = n!/((nk)!k!)
and subfactorial(n) is the number of derangements of n objects, i.e.
the number of permutations for which none of objects are in their
original position.
subfactorial(n) is further developed:
subfactorial(n) = round(n!/e) + [n==0]
where round is the function which rounds to the nearest integer
and the bracket notation yields 1 if the boolean expression inside
is true and 0 if the boolean expression is false. I've used ==
for the equality operator, and e is Euler's constant e = 2.71828....

Hi Scott,
Thanks for that very lucid description of the precise answer, too.
Mark 

Back to top 


Google


Back to top 



The time now is Sat Nov 17, 2018 11:42 pm  All times are GMT

