Recall that with a normally distributed random variable, we can measure the distance from the mean in three different units:

1. natural units (such as inches, dollars, or points on a test score);
2. standard deviations (the z transformation);
3. percentiles (normcdf)

The concepts used in computing confidence intervals and carrying out hypothesis tests use these various measures. It may help to think in terms of a concrete situation. Suppose that we were given that the standard deviation for the height of a full-grown golden retriever is 4 inches. We take a sample of 9 golden retrievers and find that the mean height is 35 inches. The natural units are inches.

• Confidence Level
• Confidence Interval
• Margin of Error
• Critical value z*
• P-value
• Probability of type I error
• Probability of rejecting the null hypothesis when in fact it is true
• Significance level
• Probability of type II error
• Probability of failing to reject the null hypothesis when in fact it is false
• Power

Also, for each concept above, indicate whether in a typical situation, with a given sample size, the magnitude is something selected by the investigator or something calculated based on the data.

The power of a test is measured against a specific alternative. We assume that the alternative hypothesis is true, and we calculate the probability that we nonetheless fail to reject the null hypothesis.

For example, suppose that we want to test the hypothesis that the mean height of a golden retriever is 36 inches, based on our sample of 9 that had a known standard deviation of 4 inches. The null hypothesis is that the mean is 36 inches. We need a specific alternative. For example, the alternative hypothesis could be that the mean height is 33 inches.

Suppose that we set a significance level of 5 percent. The cutoff for a significance level of 5 percent is 1.96. That is, we have to be 1.96 standard deviations away from the null hypothesis of 36 in order to reject the null hypothesis.

The standard deviation of the mean is 4/(square root of n) = 4/3. A distance of 1.96 standard deviations thus corresponds to about 2.6 inches. Therefore, we would need to have a sample mean of 33.4 inches or less in order to reject the null hypothesis.

To calculate the power of the test, we assume that the alternative hypothesis is true. That is, we assume that the true mean is 33 inches. If that is the case, what is the probability that we will find a sample mean of 33.4 inches or greater (in which case, we will fail to reject the null hypothesis in favor of the alternative)? Since 33.4 inches is only .4 inches greater than the presumed true mean, it is only .4/(4/3) = .3 standard deviations away. So we take one minus normcdf(.3) to find the probability that we will fail to reject the null hypothesis. The value we obtain is the power of the test.