Results of economic evaluations in trials are subject to several sources of uncertainty.
6.1.1 Sampling uncertainty
Economic outcomes in trials are usually based on effectively a single sample drawn from the population. In general, there is uncertainty with respect to both costs and outcomes, and this variability should be reflected in CEAs to determine to what extent uncertainty in the estimates might influence the decisions that might be made as a result of the analyses. For example, if an intervention appears to be, on average, cost-effective, but the uncertainty interval includes instances of cost-ineffectiveness, then the confidence with which the intervention can be recommended might need to be tempered. Methods for taking into account uncertainty are not always straightforward and generally benefit from the input of a health economist.
6.1.2 Parameter uncertainty
Uncertainty related to parameter estimates, such as unit costs and the discount rate, should be assessed by the use of sensitivity analysis. For example, if a discount rate of 3% is used, it may be desirable to assess the impact of this assumption by repeating the analysis, but using a 0% or 5% rate. Analysts should evaluate the effect of varying all major cost parameters (such as the proportion of personnel time allocated to the intervention), as this may influence policy decisions.
6.2 Policy inferences
Policy inferences about the adoption of an intervention should be based on the level of confidence that the cost of the intervention for a unit of outcome, for example, a DALY, is affordable, with a threshold, or ceiling, beyond which it would be unacceptable to adopt it. Ranges of ceiling ICERs should be reported, for which the analyst: (1) is confident that the intervention is good value for the cost; (2) is confident that the intervention is not good value; or (3) is unsure that the cost-effectiveness of the two interventions differ from each another sufficiently to make a choice between them based on the ICER alone. Policymakers can then draw inferences by identifying into which of the ranges it falls. The ranges of ceiling ratios where the analyst can and cannot be confident about the value of a new intervention relative to the current intervention can be calculated by the use of confidence intervals (CIs) for the cost-effectiveness ratios, allowing for the various sensitivity analyses done.
6.3 External validity
Some field trials may have low external validity (i.e. they cannot be generalized easily, and the impact of the intervention may be different when applied in a public health setting). The threats to external validity come from:
- inclusion of study sites with access and availability of health care services which are not representative of the wider population that would be targeted in a public health programme
- restrictive inclusion and exclusion criteria (patient population, disease severity, co-morbidities)
- artificially enhanced compliance (for the purposes of the trial).
In such circumstances, it might be possible to test the potential cost-effectiveness of the new intervention in programmatic conditions within sensitivity analyses, after making assumptions about how each of these factors might differ in the routine programmatic situation relative to the situation within the trial.