4.12: Quality control

In most intervention studies, members of the population are invited to participate, the intervention is applied, perhaps repeatedly, and the population is kept under surveillance, until the final trial outcomes are recorded. The quality of each step in this process must be monitored. The two major reasons, which hardly need stating, are first to ensure that each operation is being performed to an acceptable standard, and second to identify areas where attention is required. A third reason is to be able to ascertain, at the end of a trial that failed to show anticipated effects, the possible reasons for failure. The damage done by a misleading ‘negative’ result can be serious. The following are major aspects of quality control (QC) that need attention.

9.1 The intervention

Regular monitoring of the delivery of the intervention should be an integral part of the design to ensure that there is no change in the quality, as a trial goes on. For example, in a vaccination trial, continual review would be needed of the quality of the vaccination techniques being used by fieldworkers and of the quality of the vaccine(s) used in the intervention. For example, the potency of each batch of vaccine used should be assayed, together with monitoring of the maintenance of any required cold chain. Particularly relevant for trials where the intervention includes case management or counselling is monitoring the quality of these procedures through regular observation of a sample of provider–client interactions.

Short-term endpoints may be used for monitoring the quality of the intervention. At the individual level, repeated surveys of physiological measures of response to the intervention will provide an assessment of whether an effective intervention agent has been delivered. Examples would be antibody levels against a vaccine or levels of a micronutrient in serum. In trials including provider–client interactions, exit interviews with clients can be used to monitor their understanding of the advice that was provided. Such evaluations may have to be done or be evaluated by an independent trial monitor to ensure that those who will assess the main endpoints in the trial are kept blind—whenever possible—to the identity of those in intervention and control groups.

9.2 Follow-up

For many intervention studies, the endpoints of interest may not emerge until a lengthy period after the start of the intervention. It may not be necessary to keep the entire trial population under active observation, and this is often not feasible (for example, cases might be detected, as they report to clinics, rather than by conducting periodic surveys of the trial population), but it is essential that the trial is designed in such a way that losses to the trial population (for example, cases who do not go to clinics) will not distort the conclusions. The follow-up rate should be monitored, in order to identify potential problems at an early stage (for example, disgruntlement in a particular village or to identify a fieldworker whose work quality is declining). If possible, the reasons that individuals are lost to follow-up should be ascertained. Some losses may be inevitable, such as participants who die or who move out of the trial area, while it may be possible to take remedial action to prevent others such as participants who withdraw their participation or who are temporarily absent but could be found by repeated visits to their homes. The baseline characteristics of those who are lost to follow-up should be compared with those of participants who remain in the trial, and this information should be analysed to assess any effect that the losses might have on the interpretation of the results of the trial.

9.3 Assessment of trial outcomes

Mechanisms have to be established to ensure that the quality of information on all the trial outcomes is acceptable. Ongoing monitoring is required to establish that the data on trial outcomes are maintaining acceptable quality and that no biases are present in the way outcomes are recorded in different treatment arms. Attention needs to be paid to inter-observer variation in the assessment of the outcomes and changes that may occur in this variation, as the trial progresses.

9.4 Other field and laboratory procedures

QC should pervade all field activities, and the question as to how high quality is to be achieved and maintained should be addressed specifically for all activities. This is discussed in most of the chapters that follow, and specifically in Chapter 16, Section 7.

Laboratory procedures should be subject to constant scrutiny, and ‘blind’-coded duplicate samples or known positives or negatives should be introduced into the workload regularly to monitor performance.

In interview surveys, a proportion of respondents should be re-interviewed by a second interviewer, blind to the results of the first interviewer, to check on the repeatability of the responses. If the questionnaire is long, the re-interviews might focus on a subset of key questions, rather than repeating the full questionnaire, in order to avoid undue demands on participants.

It is important that all involved in the trial accept and understand the need for constant checking and re-checking. This is both so that any sanctions that are taken for repeated poor performance do not come ‘out of the blue’, but, more importantly, as a way of encouraging all trial staff to maintain high quality at all times, because they know that errors will be spotted reasonably quickly. On the other hand, errors are bound to occur, and their detection should usually result in support and, where necessary, additional training, with reprimands being reserved for where there is evidence of dishonesty or continual carelessness. Incentives or rewards to encourage high-quality work may be worthwhile.

All members of the field team are, and must be made to feel, important contributors to the research project. Feedback of results and progress should be continuous and frequent, so that they can appreciate where their contribution fits into the overall project. Neglect is a great stimulus to poor-quality work.