12.1: Introduction

Statistical testing is part of a much larger process known as the scientific method. This method was developed more than two centuries ago as the accepted way that new knowledge could be created. Until then—and unfortunately even today, among some—"knowledge" could be created simply by some authority saying something was so, ipso dicta. Superstition and pseudoscience were (and are?) often accepted uncritically.

The scientific method, briefly, states that only by following a careful and specific process can some assertion be included in the accepted body of knowledge. This process begins with a set of assumptions upon which a theory, sometimes called a model, is built. This theory, if it has any validity, will lead to predictions; what we call hypotheses.

As an example, in the field of Kinesiology, the theory of hypertrophy (muscle growth) begins with certain assumptions concerning human physiology and mechanical tension. From these assumptions, a theory of how muscle fibers respond to resistance training is built. This theory gives rise to a very important prediction: namely, that there is a positive relationship between training volume and increases in muscle cross-sectional area. This relationship is a prediction, or a hypothesis, that can be tested with statistical tools.

Unless hundreds and hundreds of statistical tests of this hypothesis had confirmed this relationship, the established protocols for strength training would have been discarded years ago. This is the role of statistics: to test the hypotheses of various theories to determine if they should be admitted into the accepted body of knowledge—how we understand human performance and health. Once admitted, however, they may be later discarded if new theories come along that make better predictions.

Not long ago, certain manufacturers claimed that wearing specific "holographic" wristbands could immediately improve a person’s balance and strength through "energy frequencies." This caused a tremendous stir and these products were worn by elite professional athletes and sold to millions. It was not long, however, until these claims were subjected to the rigorous tests of the scientific method and found to be a failure. In double-blind studies, no independent lab could replicate the findings of the manufacturers. Consequently, the "theory" was discarded and the claims were exposed as unfounded. It may surface again if someone can pass the tests of the hypotheses required by the scientific method, but until then, it is just a curiosity. Many fitness frauds have been attempted over time, but most have been found out by applying the process of the scientific method.

This discussion is meant to show just where in this process statistics falls. Statistics and statisticians are not necessarily in the business of developing theories, but in the business of testing others' theories. Hypotheses come from these theories based upon an explicit set of assumptions and sound logic. The hypothesis comes first, before any data are gathered. Data do not create hypotheses; they are used to test them. If we bear this in mind as we study this section, the process of forming and testing hypotheses will make more sense.

One job of a statistician is to make statistical inferences about populations based on samples taken from the population. Confidence intervals are one way to estimate a population parameter. Another way to make a statistical inference is to make a decision about the value of a specific parameter.

For instance:

• A supplement company advertises that its new pre-workout increases bench press max by 15 pounds, on average.
• A running coach claims that their specific interval training method helps 90% of runners achieve a personal best time.
• A public health study says that adults in a specific city walk an average of 7,000 steps per day.

A statistician will make a decision about these claims. This process is called "hypothesis testing." A hypothesis test involves collecting data from a sample and evaluating the data. Then, the statistician makes a decision as to whether or not there is sufficient evidence, based upon analyses of the data, to reject the null hypothesis.

In this chapter, you will conduct hypothesis tests on single means and single proportions. You will also learn about the errors associated with these tests.