Determining the probability of ties in a performance appraisal process

HanSolo · science forum beginner Joined: 07 Apr 2006 Posts: 3

I've been tasked with determining the probability of ties in a
multi-reviewer performance appraisel process and I was hoping for some
advice on how best to calculate this. I have a weak stats background
so I'm not only looking for logic, but also the formula to run the
calculation. Thanks!!

HERE'S THE BACKGROUND:

- Five reviewers each independantly review the performance appraisel of

every incumbent within a group, then assign a score to that incumbent
using a scale of 1 through 30.

- Once all reviewers are finished scoring every incumbent in the group,

we normalize the ranges of all five of the reviewers so that all of
their scores are within a common max score and min score.

- We average these normalized scores for each incumbent to come up with

a final score.

- Ties occur if all five reviewers think that the same two (or more)
people are equal.

MY QUESTION:

- What are the odds of this occuring?

MY THOUGHTS AND CHALLENGES IN THINKING THIS OUT:

- I imagine we would first calculate the odds of each respective
reviewer thinking that more than 1 people are equal, and this would
depend upon the size of the population they are reviewing (e.g. the
larger the group, the more of a chance). The reviewer is not picking a

random number between 1 and 30, rather they have a performance
appraisel to influence the number chosen. How do we account for this?
In addition, after an incumbent is scored, the population left for that

reviewer to score decreases. Do we need to account for this in the
formula?

- I imagine the second step would be to calculate the odds that the
other four reviewers thought that those same two (or more) people are
equal. Remember that ultimately an average is calc'd based on the 5
reviewers scores, so we're not looking at the odds of the other four
reviewers giving the same scores, just that they think that the same
two (or more) people are equal.

Google