Collecting Data

 Teaching Tips

General

• By the end of the course, students will be expected to design an experiment and explain their design clearly in writing.

• Don't get hung up on the difference between "lurking variables" and "confounding variables". It just doesn't matter.

Randomization

• Randomization is not meant to make the treatment groups *exactly* equal after the fact, but is, in a way we'll understand after studying probability, meant to help us quantify the reliability of our answers.

• Students mistakenly believe that randomization eliminates confounding effects rather than balancing treatment groups with regard to those effects.

Repetition

• It's useful for some to think of the process of comparing groups to be that of distinguishing the signal (treatment effect) from the noise (of variation). Collecting more data helps the signal stand out from the noise.

Control

• Researchers try to control for every conceivable confounding variable, but in practice there is no way to guarantee that every existing confounding variable has been considered and controlled.

Bias

• "Bias" is a complicated term, and will be refined in Unit 9.

• Students need to understand and be able to identify potential sources of bias. Don't get caught up in distinguishing among the different names. There might be one multiple choice question on the AP on this topic , but in all likelihood not more.

Sampling

• All of the main and important concepts about sampling can be understood in the context of simple random sampling. The other types of sampling are technical implementations, but do not affect the main concepts.

• Sampling with and without replacement: in practice we virtually always sample without replacement, but in classroom situations you might sample with replacement, which simulates sampling without replacement from a very large population.

• Simple random samples are very difficult to achieve in any but the most simplistic situations.

Blocking and Stratifying (which we'll address in Unit 15)

• Teaching blocking is hard, in part because the curriculum doesn't cover analysis of block designs and so students never see the payoff from this technique.

• However, later we cover analysis of matched pairs, which is a form of blocking, and students can see the payoff of blocking in this context.

• Students will confuse blocking and stratification. They have similar purposes but arise in different contexts, and students need experience in both.