Non inferiority trials

There’s been a recent rash of ‘non inferiority’ trials in the medical literature. The new oral anticoagulants are a great example. It’s hard to blame investigators. Proving that Drug B is better than Dug A is a very difficult thing to do, especially now that it all needs to be focused on poo (or Patient Orientated Outcomes – what did you think I meant…)

A superiority trial requires a massive sample size to find this difference reliably. Proving that drug B is ‘non inferior’ to Drug A is a lot easier to do. Though not without its problems.

I was doing some prep for one of our national EM training days for which we typically sit an FRCEM style critical appraisal. One of the questions was:

• What is a non-inferiority trial and when should it be used?

As usual, this is typical type of material that FOAMed has not covered especially well.

First off some reading:

Sedgwick P. What is a non-inferiority trial? BMJ 2013 Nov 15;347(nov15 1):f6853–3. [pdf]

Snapinn SM. Non inferiority trials. Curr Control Trials Cardiovasc Med. 2000. [FOAMed pdf]

Here are some take home headlines:

• These are not equivalence trials, though people have used the terms equivalence and non inferiority interchangeable. Bioequivalence trials of drugs are probably the only true equivalence trials
• You must specify a noninferiority margin – for it to be a positive trial the primary outcome has to be within a certain margin of the existing treatment. Selection of this margin is unavoidably subjective and is open to all kinds of fudging.
• These trials are suitable when you are testing something that you assume will be at best non inferior. For example face to face versus telephone consultation – no one expects telephone to be better but it might be non inferior. So instead of assuming the null hypothesis (that there is no difference between treatments – this is what we normally assume in superiority trials) and trying to disprove it; in a non inferiority trial the null hypothesis is that the new intervention is not as good and you are trying to prove that is non inferior.
• Of note, blinding is much less use in a non inferiority trial as a biased but still blinded end point assessor can just give everyone equivalent outcomes and the new drug being assessed comes out as non inferior.
• ‘intention to treat’ and ‘per protocol’ analyses can both bias the results. Typically ITT is the way to go but in non inferiority trials you typically need both ITT and per protocol analyses to be positive for the intervention to be truly non inferior.
• non inferior trials are typically analysed on a confidence interval but without the p value. This is in the Sedgwick paper but I still struggle to understand why they don’t use the p value.

UPDATE 2-12-15

Rory Spiegel made some great comments on twitter regarding my confusion regarding the use of CI rather than p values. To quote several of his tweets:

the reason the 2-sided p values is not traditionally used is it can only tell us the probability that the results are compatible with the null hypothesis. P> .05 does not prove equality but rather just an inability to prove a difference. Using CI allows us to determine if the difference crosses the non inferiority margin.

He also provided a nice reference too.