Two baseball players are, Babe Ruth and Lou Gehrig, are compared during the first and second halves of the season.

• Ruth had the highest batting average for both the first half of the season and the last half of the season.
• Who had the higher batting average for the entire season (i.e., both halves put together)?

The first and second half statistics are as follows.

Ruth had the highest average during both the first and the second halves of the season.

Who has the best overall batting average?

Here are the overall results.

For the season, Gehrig had the highest average.

A paradox is a situation that seems to defy intuition.

It is possible to do any number of surveys, each of which indicates a certain conclusion (e.g., that there is more of a perceived bias in the South), and yet, when the data for all surveys are added together, the conclusion can switch (e.g., that there is more perceived bias in the North). Where is the problem?

Rule #1: Be very careful when comparing averages.

Never add, multiply, or average averages. Averages are a measure of central tendency, or how grouped a set of numbers are, in which the relative numbers of each sample have been factored out.

Rule #2: Make sure that differences are statistically significant.

This paradox is called Simpson's paradox (from a 1951 paper by E. Simpson), although the paradox itself is older.

Simpson's paradox is not related to the cartoon character Homer Simpson.

In fact, in an actual job discrimination survey, a small but statistically significant margin of error arrived at the same result as our assignment. That is, two surveys concluded job discrimination against woman. Added together, the surveys indicated job discrimination against men. And the result was statistically significant.

The classical case demonstrating Simpson's paradox took place in 1975, when UC Berkeley was investigated for sex bias in graduate admission. In this study, overall data showed a higher rate of admission among male applicants, but, broken down by departments, data showed a slight bias in favor of admitting female applicants. Judea Pearl The explanation is simple: female applicants tended to apply to more competitive departments than males, and in these departments, the rate of admission was low for both males and females. Judea Pearl Why is an average called a measure of central tendency?

Simpson's paradox first noticed by Karl Pearson in 1899, can be stated as follows: Every statistical relationship between two variables may be reversed by including additional factors in the analysis. (Judea Pearl, at http://bayes.cs.ucla.edu/jp_home.html) (as of 1974)

Simpson's paradox, first noticed by Karl Pearson in 1899, concerns the disturbing observation that every statistical relationship between two variables may be REVERSED by including additional factors in the analysis. http://bayes.cs.ucla.edu/jp_home.html (as of 1974)

For example, you might run a study and find that students who smoke get higher grades, however, if you adjust for AGE, the opposite is true in every AGE GROUP, namely, smoking predicts lower grades. If you further adjust for PARENT INCOME, you find that smoking predicts higher grades again, in every AGE-INCOME group, and so on. http://bayes.cs.ucla.edu/jp_home.html (as of 1974)

Imagine your family physician saying, "This drug seems to work on the population as a whole, but it has an adverse effect on males ... and an adverse effect on females." Only when you look at the numbers and agree to interpret the phrase "seems to have an effect" as a statement about a change in proportions do you begin to see that the calculus of proportions clashes with our intuitive predictions. Pearl, J. (1988). Probabilistic reasoning systems, 2nd revised printing. San Francisco, CA: Morgan Kaufman., p. 496. Equally disturbing is the fact that no one has been able to tell us which factors SHOULD be included in the analysis. Such factors can now be identified by simple graphical means. http://bayes.cs.ucla.edu/jp_home.html (as of 1974) Another example of Simpson's paradox is "reverse regression". In salary discrimination cases, should one compare salaries of equally qualified men and women or compare qualifications of equally paid men and women? Remarkably, the two choices led to opposite conclusions. It turned out that men earned a higher salary than equally qualified women, and SIMULTANEOUSLY, men were more qualified than equally paid women. (Judea Pearl)

Surveys (and petitions, etc.) tend not to be used to make decisions.

Instead, managers often tend to request the surveys and data that will support the decision they have already decided they want to make. (And ignore the results if the results do not fit what they expect or want).