Meta-analysis of small randomized controlled trials in surgery may be unreliable

Abstract Background Meta-analysis of randomized controlled trials (RCTs) should provide reliable evidence about the effects of interventions. This may be less reliable when only small trials are available. Methods The sample size was determined for all surgical RCTs included in Cochrane Collaboration systematic reviews. The difficulty in interpreting meta-analysis of small trials is illustrated using two specific reviews. Results The typical sample size for surgical RCTs was small with a median of only 87 participants. Only 39·8 per cent had adequate prerandomization treatment allocation concealment. In both systematic reviews that were assessed in detail, statistically significant early results from meta-analysis of several small RCTs did not reliably predict the results of subsequent RCTs. Conclusion Surgical RCTs tend to be small and underpowered. Meta-analysis of such trials does not necessarily produce reliable results.