Sampling Distributions Examples

What is a sampling distribution? How do we find it? What assumptions do we have to make?

A sampling distribution includes the possible results from sampling the population in a particular way. It includes how likely each result is as well. By "result," we mean whatever estimate we calculate from the sample, such as a sample mean.

The sampling distribution is different for different sample sizes, even when drawing from the same population. "The sampling distribution of the number of heads while flipping 5 coins" is different from "The sampling distribution of the number of heads while flipping 6 coins."

We have to make some kind of assumption, called the null hypothesis, when creating a sampling distribution. When flipping coins, the assumption is that the coin is fair, so that there is a 50% chance to get either side.

Repeatedly sampling the population, and calculating an estimate each time, can create a sampling distribution, but is very rarely done. Another method is to use simulation to model the sampling distribution. A third way (that we haven't covered) is to derive a sampling distribution using statistical theory. This is common, but pretty math heavy.

We went to a local fishing tournament and measured the average number of fish caught by each person, which was 13. Before the tournament started, our hypothesis was that people would only catch 5 fish on average. We ran some simulations to find the sampling distribution. Out of 250 trials, 12 of them were as extreme or more so than our sample mean. Can we reject the null hypothesis?

In order to reject the null hypothesis, our sample estimate has to be in the extreme tails of the sampling distribution. The most common cutoff used is a 5% significance level. So, if our estimate has less than a 5% chance of occurring, assuming the sampling distribution is true, then we reject both the null hypothesis and the sampling distribution we built from it.

Our fishing tournament data are pretty unlikely under the null hypothesis. 12 out of 250 trials equals 0.048, or 4.8% of the distribution. That's our P-value. That's less than 5%, so that means we can reject the null hypothesis. People caught an average of more than 5 fish at the tournament. Those poor fish.

Bob and Jim want to know if one of them is a faster runner than the other. They ran the same length of track 10 times each and found their average times. Turns out that, on average, Bob ran the track 6 seconds faster than Jim. Using resampling and simulation, they found that the margin of error is ±8 seconds. What can Bob and Jim conclude?

The margin of error covers from -2 to 14 seconds. A negative number, in this case, means that Jim is faster than Bob, not that someone has gone back in time.

The margin of error includes 0 seconds, or that there is no difference between the runners. That means their data isn't enough to reject the null hypothesis of no difference. That sentence has a lot of double negatives, but that's what statistical results look like.

That's because we can't accept the null hypothesis. The data is consistent with them being equally fast, but doesn't prove it. One of them might be faster, but there wasn't enough data to show it.