2.4: Hypothesis Testing in Analytic Observational Studies

The three sampling methods -each denoting a type of analytic study described in this section differ in the amount of information they provide with respect to the population. Cross-sectional studies are based on a single sample of the population, whereas, in principle, cohort and case-control studies are based on two separate often purposive samples (Fleiss 1973).

To assist the description of these sampling methods, the basic population structure with respect to one exposure factor (often called the independent variable) and one disease (often called the dependent variable) both with two levels, present or absent, is shown below. The letters A, B, C, and D, represent the number of individuals (sampling units) in each factor-disease category in the population.

A variety of rates and proportion can be calculated if the numbers in each of the four cells (factor-disease combination) are known. The objective of analytic studies is to estimate these rates, although not all may be estimated from each study design. See Table 2.4.

For purposes of nomenclature, lowercase characters indicate that the values are derived from a sample, whereas uppercase characters indicate population values. Thus p indicates an estimate, that is a statistic, from a sample, whereas P indicates the corresponding population value or parameter. In discussing numbers of individuals as opposed to proportions, n will be substituted for p. For example, n(F +) is the number of exposed units in the sample which may also be indicated as (a + b).

To clarify the sampling strategy in each of the three analytic study methods, assume the investigator wishes to test if vaccination against selected viruses alters the risk of pneumonia in feedlot cattle. Although it is rare that the structure of the population to be sampled is known, a numerical example is given in Table 2.5. Although based on fictitious data, the example demonstrates the information that would be provided by each of the sampling methods, in comparison to the information that would be available if the population structure was known. With a few modifications, the same approaches to sampling could be used if disease was the independent variable and production the dependent variable (e.g., if the intention were to test the hypothesis that the presence of a disease alters the level of production).

2.4.1 Cross-Sectional Sampling

A sample, usually obtained by one of the previous probability sampling methods, is selected from the population, and each member (sampling unit) is classified according to its current status for the factor and the disease. AU of the disease rates in the population may be estimated, based on the results of a cross-sectional sample. Thus this method allows the investigator to learn about the population structure, as well as to test the null hypothesis that the factor (vaccination) and disease (pneumonia) are independent events in the population. However, this method of sampling may be impractical when disease frequency is low, because large sample sizes would be required to obtain a sufficient number of cases. In the example in Table 2.5, 120 vaccinated cattle with pneumonia were observed; whereas 180 would be expected if vaccination and pneumonia were independent events. The expected number is derived by multiplying the first row total by the first column total, and dividing by n (i.e., 600 x 300/ 1000). This calculation is based on statistical theory regarding probabilities of independent events and is the basis of the chi-square test, see 5.2. Since there are fewer observed vaccinated animals with pneumonia than expected, it appears that vaccination may protect against pneumonia.

An example of a cross-sectional study is presented in Table 2.6. This northern California study was designed to estimate the frequency of acute bovine pulmonary emphysema and to identify factors associated with this disease (Heron and Suther 1979). A list of all herds in three counties (the sampling frame) was obtained from the California Bureau of Animal Health. Then a stratified random sample was used-each county constituted a separate stratum-and a 10% random sample of herds (the sampling unit and the unit of concern) was selected within each county.

Farm owners were interviewed about their husbandry methods, particularly forage management practices. Based on the results of this study, it appeared that approximately 10% of the farms experienced an outbreak of acute bovine pulmonary emphysema during the 4-year study, and that approximately 35% of farm managers used pasture rotation but did nothing specific to prevent the problem. Approximately 2.5 farms (24 x 7/68) or 3.6% of farms would be expected to use pasture rotation and experience the disease if these were independent events; whereas 7 (10.3%) actually did. This suggested a strong association between pasture rotation with no preventive measures and the occurrence of pulmonary emphysema. Additional data indicated that about 3% of the cattle at risk on the affected farms developed pulmonary emphysema. The case fatality rate was 53.8%.

A cross-sectional design was used in a study of factors influencing morbidity and mortality in feedlot calves (Martin et al. 1982). However, since no formal sampling was used to select collaborators, it is not known how closely the distribution of various risk factors or the prevalence of disease found in the study might be to population values. Thus, although the associations found in the study may be valid, it is difficult to extrapolate certain results beyond the sample (i.e., beyond the groups of catt.le under study).

2.4.2 Cohort Sampling

In cohort sampling, a sample of exposed (F +) and a sample of unexposed (F-) sampling units are selected and observed for a period of time, and the rate of disease in each sample is used to estimate the corresponding rates of development of disease in the two populations. Usually when cohort sampling is used, one does not gain information about the frequency of the factor or of the disease in the population. Testing whether the rate of disease in the exposed group is equal to the rate in the unexposed group evaluates the null hypothesis that the factor and disease are independent events in the population. In the example in Table 2.5, a sample of 500 vaccinated animals and a comparison cohort of 500 unvaccinated animals were identified and observed for a specified time to determine the respective rates of pneumonia. In this fictitious data, since only 20% of vaccinated animals and 45 4 % of nonvaccinated animals developed pneumonia, it appears vaccination helped prevent the development of pneumonia.

The two cohorts (i.e., the two exposure groups) are only infrequently selected by a formal random sampling process. Usually they are purposively sampled specifically because of their exposure or nonexposure to the factor of interest. As long as the two groups are comparable in other respects, the effect of the exposure factor can still be evaluated. However, the groups should be demonstratively representative of the exposed and unexposed segments of the population if the results are to be extrapolated beyond the sampling units in the study. An example of the use of cohort sampling is shown in Table 2. 7. The objective was to contrast the rate of pulmonary disease in rural (F-) and urban (F+) dogs in an attempt to estimate the impact of living in a relatively unpolluted (rural) versus a polluted (urban) environment (Reif and Cohen 1970). No differences were noted in young dogs. However, significant differences were seen in dogs 7-12 years of age; the highest rates being in urban dogs, suggesting a harmful effect of the polluted environment.

2.4.3 Case-Control Sampling

In case-control sampling, samples of diseased (D +) and nondiseased (D-) individuals are selected, and the proportion of each that has been exposed to the factor of interest is used to estimate the corresponding population proportion. Testing whether these two sample proportions are equal evaluates the null hypothesis that the factor and disease are independent events in the population. In the example in -fable 2.5, a group of 500 animals with pneumonia and a sample of 500 animals without pneumonia would be selected, and the proportion vaccinated in each group would be contrasted. If the proportion of cases that were vaccinated (40%) was significantly different than the proportion of controls that were vaccinated (69%), vaccination would be associated with pneumonia. Since the former proportion is smaller, it appears that vaccination protected against the development of pneumonia in this hypothetical example.

Only infrequently are the two groups (D + and D-) obtained by a formal random sampling procedure. Usually the cases arc obtained from one or more sources and essentially represent all of the available cases from the purposively selected sources. Often, the comparison group consists of all animals not having the disease of interest from the same source, be that a set of clinic or farm records. Sometimes, however, formal sampling is used. In a study of feline urological syndrome, the cases represented all cats with the disease in !he clinic records; whereas the controls were obtained by taking a 100'/o systematic random sample of cats without the urologic syndrome (Willeberg 1975). In another example, the characteristics of herds with reactors to brucellosis were contrasted with those with no reactors. 'f.he data were obtained from the records of a diagnostic laboratory. Since a large number of herd records were available, a 10% random sample of herds having reactors and a 6!1/o random sample of herds not having reactors to bovine brucellosis were selected. (These sampling fractions were selected because initial estimates indicated that they would provide the required number of reactor and nonreactor farms.) (S. W. Martin, pers. comm.)

In a study of factors associated with mastitis in dairy cows (Goodhope and Meek 1980), the case herds were the 550 with the highest milk-gel index in the province of Ontario. Each was matched to the closest herd in the same county with the lowest milk-gel index (i.e., the controls). (The latter selection method helped ensure that the case and control herds were comparable since they were geographically matched.) An example of case-control sampling is presented in Table 2.8 (Willeberg 1980). Herds with high levels ( > 5010) of enzootic pneumonia in swine at slaughter (cases) were compared to herds with low levels ( <5%) of enzootic pneumonia in their pigs (controls). While a number of characteristics of these herds were contrasted. Table 2.8 demonstrates the association of one factor (herd size) with level of pneumonia. Note that the sampling units are herds, not individual pigs. It is obvious from these data that larger herds (the exposure factor) occur much more frequently among herds with pneumonia problems than in herds with low levels of pneumonia. This suggests a harmful effect of the factor "large herds" on the level of pneumonia.

2.4.4 Sample Size Considerations

Because of the time and expense required to conduct a valid analytic study, careful consideration should be given to determining the number of animals or sampling units required. The formulas given in Table 2.9 provide a basis for estimating sample sizes when the study is designed to contrast two groups.

Example \(\PageIndex{1}\)

Two hypothetical examples will be presented to demonstrate the use of sample size formulas. In the first example, assume that the study is intended to compare the milk production of cows with clinical mastitis to cows not having mastitis (i.e., comparing the means of two quantitative variables). Suppose cows not experiencing clinical mastitis will produce 160 BCM units of milk with a standard deviation of 40 BCM units. (BCM is the breed class average for milk; see 3.6.1.) Further, assume clinical masticis will reduce milk production by 10% to 144 BCM units. How many cows are required in a cohort study to be 80% (l - type II error) certain of detecting a difference as large as this, if it exists? Substitution of the above estimates into the second formula for sample size determinations gives:

\[\begin{aligned}
n & =2[(1.96+0.84) 40 /(144-160)]^2=2(112 /-16)^2 \\
& =2(-7)^2=2 \times 49=98
\end{aligned}\]

Thus, the investigator should use approximately 100 mastitic and 100 nonmastitic cows for the study

Example \(\PageIndex{2}\)

As a second example, suppose a newly identified organism is present in 40% (P.) of nasal swabs of feedlot calves with pneumonia, and it is thought to occur in about 15% (P,) of swabs from feedlot calves without pneumonia. How many calves would have to be examined in a case-control study to be 80% sure of detecting this difference (or greater) if it existed? Note that P = 0.275 and Q = 0. 725. (This is contrasting the means of two qualitative variables. the means being expressed as rates or proportions.)

\[\begin{aligned}
n & =\frac{\left[1.96(2 \times 0.275 \times 0.725)^{1 / 2}+0.84(0.4 \times 0.6+0.15 \times 0.85)^{1 / 2}\right]^2}{(0.4-0.15)^2} \\
& =(1.24+0.51)^2 / 0.25^2 \\
& =3.06 / 0.063 \\
& =49
\end{aligned}\]

The investigator should plan to include approximately 50 calves with pneumonia (cases) and 50 calves without pneumonia (controls) in the study.

2.4.5 Cost Considerations in Analytic Studies

Under most practical field conditions, it can be shown that case-control studies require the fewest sampling units of all analytic observational studies to evaluate a specified hypothesis (Fleiss 1973). This and other features of study design make case-control studies a popular choice when selecting a study method (see Chapter 6).

In the previous discussions of sampling for hypothesis testing, equal size groups were used (i.e., the F+ and F - groups were of equal size in cohort studies and the D + and D- groups were of equal size in casecontrol studies). If the costs of obt.aining study subjet:ts differ between unexposed and exposed, or cases and controls, the study design can be modified to take this feature into consideration. Although straightforward in principal, the formulas are somewhat complex, and the interested reader should consult the appropriate references for details and examples (Meydrech and Kupper 1978; Pike and Casagrande 1979).