Hypothesis Tests

 Teaching Tips

• The hypothesis test answers the question "Am I surprised by my sample?" The p-value answers the question "How surprising is my sample?" The confidence interval answers the question "What values of the population parameter would cause me not to be surprised by the sample?"

• Words are very important in this unit. Students will be penalized for sloppy language on the AP exam and throughout their lives.

• The AP readers will look for (a) clearly stated hypotheses (b) conditions and/or assumptions are stated and checked (c) the mechanics of calculations (d) conclusion stated in context. It's important that the conclusion (part d) refer back to the mechanics (part c) so that the student makes perfectly clear how his or her decision was made.

• Sometimes you can just look at the data and see the null hypothesis is clearly false. (For example, if a histogram shows that every observation is greater than the null hypothesis value.) Students should not be taught to distrust graphical evidence, but taught that the hypothesis test is a way of formalizing what they see.

• Students find the null hypothesis difficult to identify in context. They need to practice writing null and alternative hypotheses.

• Students should recognize when to use two-tailed versus one-tailed hypotheses.

• Neither the p-value nor the significance level is the probability that the null hypothesis is true. The null hypothesis is either true or false and probability statements don't apply. Remember that the p-value is a conditional probability: we assume the null hypothesis is true when computing it.

• The 1% and 5% significance levels are historical artifacts. (Fisher thought 5% was about right.) Let your students know that p-values are often better communication than simply stating "reject" or "fail to reject".

• If the evidence is insufficient to reject the null hypothesis, don't write or say that you "accept" the null hypothesis. The best you can say is that you "fail to reject" the null hypothesis. The null hypothesis might still be false.

• Power calculations are shown in some textbooks, and your understanding might be helped by struggling with a power calculation on your own. But students are not expected to do power caculations on the AP exam. Students are expected to understand what power is and how it is related to Type I and Type II error, and sample size and significance level and population standard deviation. Review Allan's batting average example and the flat tire example from the summer workshop.