Simpson's paradox and Pharmacovigilance

As summarized by Deepak plays Pharmacovigilance and Simpson's paradox an important role in drug design, which is explained by Derek for the PPAR drug Avandia.

"Meta-analyses are highly dependent, as you might expect, on which studies are included in the analysis. I'm not a statistician, but the reasoning given by the authors for excluding these studies seems specious to me based on fairly basic principles of the design of meta-analyses and basic biostatistics. In reality, you can estimate a risk from these studies. It's zero." [Respectful Insolence]

A similar conclusion is mentioned by a follow-up article on Derek's blog

"GlaxoSmithKline did similar meta-analyses in 2005 and 2006 and found hazard ratios in the same direction as Nissen and Wolski. However, all these results are highly dependent on the methods used and the studies included, given the small number of events reported. For example, the actual number of myocardial infarctions in the Nissen and Wolski meta-analysis yields a very low frequency of events (0.6%), and the absolute difference in rates of myocardial infarctions between rosiglitazone and controls is less than 0.1%" [Ronald Krall, GSK]

What have we learned so far? Statistics and drug design have a love-hate relationship and Simpson's paradox is still a paradox ;-)

Further reading

• Lievre M, Cucherat M, Leizorovicz A.,
  Pooling, meta-analysis, and the evaluation of drug safety,
  Curr Control Trials Cardiovasc Med., 19, 6, 2002. PMID 11991807
• Kvasz M, Allen IE, Gordon MJ, Ro EY, Estok R, Olkin I, Ross SD.,
  Adverse drug reactions in hospitalized patients: A critique of a meta-analysis,
  MedGenMed., 2, E3, 2000. PMID 11104449
• R. Krall,
  Cardiovascular safety of rosiglitazone,
  The Lancet, 369, 1995-1996, 2007. DOI 10.1016/S0140-6736(07)60824-1