Having examined 63 experimental studies of the effects of drugs on driving, the authors found that their formulation of statistical hypothesis testing was inadequate. This paper outlines the main topics of hypothesis testing, and discusses the relations between (1) sample size; (2) effect size; (3) the probabilities of errors of the first and second types; (4) the alpha-adjustment for multiple comparisons. In testing the effectiveness of a drug, the resulting test decision is liable to a `type one error', an incorrect decision about effectiveness that describes consumer risk, and a `type two error', a decision overlooking an effective drug that describes producer risk. Usually, the consumer risk is fixed in advance to 0.05, the 5% significance level, but it is important to control the producer risk also, to obtain reliable decisions. In comparing an actual treatment with a placebo treatment, the hypotheses to be compared are `no positive effect' versus `positive effect'. In typical clinical trials, where the outcome is evaluated by several variables, effectiveness is often decided by performing significance tests to all individual comparisons. `Equivalence tests' are used to decide whether the adverse effects of a drug are tolerable. The authors recommend finding an index to describe the effect of a drug on driving, then using it in equivalence testing.
Abstract