Time to treatment for stroke and problems with observational data

Here’s a big study suggesting earlier TPA treatment for stroke results in better outcomes. This of course isn’t a new idea but this is new data here to support it:

Saver JL, Fonarow GC, Smith EE, Reeves MJ. Time to treatment with intravenous tissue plasminogen activator and outcome from acute ischemic stroke. JAMA. 2013. PMID 23780461

METHODS

  • data from the get with the guidelines registry (ran by American Heart and American Stroke Associations)
  • web based data collection tool for providers to enter patients into
  • from 2003-2012 which should quite the spectrum to include pre ECASS III and post ECASS III data
  • patients got into this either prospectively as they got treated or retrospectively by someone identifying a stroke on discharge documentation and entering them into the regsitry . The retrospective data is what we should be particularly suspicious of - you could easily pick and choose the ones you want to enter into the registry. Ultimately this is a study of patients that someone decided to enter into a registry. Some patients with data relevant to this question may well not be entered and therefore we don’t know what happened to them.
  • excluded patients (for this analysis): poor documentation, sites that had few stroke pts in the registry and those who got intra arterial treatments
  • there are, as usual, a lot of conflicts of interest and indeed the registry itself is funded by a pharmaceutical company (and has been funded by several others in the past)

RESULTS

  • 2000 hospitals submitting 1.2 million patients. So there’s a lot of data in this registry
  • they report 6% of these getting TPA
  • they further examine this 6% but exclude all the ones with dubious times, all the ones post 4.5 hrs and all those discharged to other hospitals (for whom they say they couldn’t get good outcome data). There are various other exclusions along the way.
  • note they don’t use the standard stroke outcomes of modified Rankin scale here. They use “ambulatory at discharge” or “discharge home” as surrogates of good outcome. This is even more dubious than the usual mRS assessed by postal questionnaire.
  • ultimately after all the slicing and dicing we get 58300 pts.
  • median time at treatment was 144 mins. 10% treated before 90 mins
  • they report 8.8% mortality, 4.9% ICH, 33% walking at discharge and 38% discharged home. Remember all the exclusions that went into this before hand so these numbers may be on the slightly optimistic side.
  • by comparing all these they find +ve associations between time to treatment and all of the outcomes. Because of the massive numbers all of these are statistically significant.
  • they even calculate NNTs to compare how much better people do when they are treated quickly.

THOUGHTS

  • The basic demographic data and the number generated have important audit, governance and even actuarial importance.
  • Registry data that is only as good as what people put into it, and it’s not clear how good that data is.
  • From the colossal numbers involved any difference will be statistically significant. Numerically significant and clinically significant are hardly the same.
  • The silliness comes with the comparisons and the implication of causation.
  • The study has no way to account for the confounding factors invovled in why someone might present to the hospital earlier than someone else, or why they might get treated earlier than someone else.
  • For example, it may well be that those who presented earlier were more likely to be having a TIA rather than an established stroke and it may be natural autoregulation and fibrinolysis that resolved their stroke rather than the TIA. Indeed if there was no actual infarcted brain then it is hardly a surprise that rates of ICH were lower in the TIA…ahem… I mean earlier treated group.
  • The point is that this data cannot answer the question they have asked of it. The only data that can answer this question is the data from the RCTs – the very data that is so contentious and controversial (among emergency physicians at least).
  • The associations presented here are all very interesting but add little real science. Rather it reinforces the rhetoric that makes acute stroke the interesting, frantic and emergent condition that we see every day.

The study did, however, remind of two of my favourite bits on causation. The first from XKCD that gets rolled out every month or so on this blog:

correlation

And the most thoughtful and sensible of science writers, the late, great Stephen Jay Gould. This is from an essay critiquing the slightly naive narrative we’ve told of Galileo and his achievements in proving heliocentrism. He interestingly got a little bit carried away and claimed that Saturn didn’t have rings but had two tiny planets at either end. His supportive proof was that he had “observed” it and therefore it must be true. All a touch embarrassing really…

Utterly unbiased observation must rank as a primary myth and shibboleth of science, for we can only see what fits into our mental space, and all description includes interpretation as well as sensory reporting. Moreover, our mental spaces house a complex architecture built of social constraint, historical circumstance, and psychological hope.

P50 The lying stones of Marrakech

Stephen Jay Gould

Further Reading/Listening:

LITFL – CCC Stroke Thrombolysis

LITFL - Michelle Johnston on tPA for stroke

TheNNT.com

EMCrit Cage Match – Jagoda v Swaminathan

The SGEM

Busting the Clotbusters – Domhnall Brannigan

SMART EM – thrombolytics for stroke

 

Long term follow up on IST-3

I read this a few months back and forgot to post something on it. It’s not headline data the way IST 3 was but it’s worth knowing it’s out there. This is the long term (by which they mean 18 month) follow up on IST 3. The original trial has been written about and discussed fairly extensively on the FOAMed sites.

What they found:

  • they got around 2000 or so because only certain countries did follow up.
  • for those available they managed to follow up on the vast majority
  • the follow up was mainly phone and postal so not that great.
  • 30% knew which drug they got which they try to play down but given that the supposed benefit is so small then any recall may cause the difference
  • they do find about a 3.6% difference in alive and independent at 18 months favouring tPA (as pointed out by Brandon in the comments)
  • there’s a great Kaplan Meier curve showing early harm from TPA and then completely flat with placebo from then on. It’s behind the pay wall unfortunately but worth looking at
  • there was no difference in alive and independent at 18 months

“the unadjusted absolute difference in the number of patients alive and independent at 18 months was not significant”

So just to summarise that. A third of the patients knew which group they were in (nothing versus the fancy new drug) and ultimately there was no difference at 18 months.

Reference:

“Effect of Thrombolysis with Alteplase Within 6 H of Acute Ischaemic Stroke on Long-Term Outcomes (the Third International Stroke Trial [IST-3]): 18-Month Follow-Up of a Randomised Controlled Trial.” The Lancet Neurology (June 2013). PMID 23791822

Thrombolysis for stroke head to head in the BMJ

Hat tip to Domhnall for the heads up on this one.

The BMJ runs a series of head to head articles where two sides of a debate are presented. They tend to be written by the main players and always make for interesting reading. Following on from the recent controversy over their piece on guidelines for thrombolysis in stroke it seems that the BMJ have taken on stroke lytics as a suitably contested issue to justify a head to head. The head to head is definitely worth a read.

I’ve written before on lytics for stroke but find my sceptical position to be the minority  among my local colleagues and certainly amongst the wider medical establishment.

It’s nice to see the debate getting a wider audience

There’s also an associated poll where you can vote on whether harms outweigh benefits. As of today (wednesday 5th at 17:30) there’s a slight majority 52% suggesting harms outweigh benefits. This isn’t the x-factor remember but it’s nice to take your chance to have your say so I suggest head over now. (it’s at the bottom left corner of the home page)

PCI for stroke trials

OK, so I admit PCI for stroke is something I made up, but I think it’s a reasonable analogy. Given that treatment for STEMI moved from tPA to PCI then it’s hardly surprising to see a similar trend in the stroke world.

The idea is to remove (or sometimes lyse intrarterially) the cerebral arterial occlusion using interventional radiology. Whatever you make it’s certainly pretty cool and ambitious technology. There are a number of trials and devices out there for this type of thing, perhaps the most famous being the MERCI device.

Below is a review of two recent studies that compared new devices against the MERCI device.

Nogueira, Raul G, Helmi L Lutsep, Rishi Gupta, Tudor G Jovin, Gregory W Albers, Gary A Walker, David S Liebeskind, Wade S Smith, TREVO 2 Trialists. “Trevo Versus Merci Retrievers for Thrombectomy Revascularisation of Large Vessel Occlusions in Acute Ischaemic Stroke (TREVO 2): a Randomised Trial..” Lancet 380, no. 9849: 1231–1240. doi:10.1016/S0140-6736(12)61299-9.

METHODS

  • big sponsored trial to prove that the TREVO is better than MERCI device. Stryker makes both interestingly…
  • both are retrieval devices where the coil goes distal and the clot is withdrawn (there are others called micro aspiration)
  • all had to have failed tPA or be ineligible for it. (ALERT – lots of cherry picking can be done here)
  • lots of exclusion criteria (excluded 80% of those screened)
  • end point was revascualrisation NOT clinical outcome; hardly unsurprising though

RESULTS

  • 180 pts
  • The TREVO (the new device) did better in both the reperfusion and the clinical outcomes.
  • more died in the TREVO group at 90 days (24% v 34%) they somehow neglect to mention this…

———

Saver, Jeffrey L, Reza Jahan, Elad I Levy, Tudor G Jovin, Blaise Baxter, Raul G Nogueira, Wayne Clark, Ronald Budzik, Osama O Zaidat, SWIFT Trialists. “Solitaire Flow Restoration Device Versus the Merci Retriever in Patients with Acute Ischaemic Stroke (SWIFT): a Randomised, Parallel-Group, Non-Inferiority Trial..” Lancet 380, no. 9849: 1241–1249. doi:10.1016/S0140-6736(12)61384-1.

METHODS

  • a Dublin made device being trialled in the US against the only currently approved device
  • similar criteria as the previous trial – either failed tPA or contraindications and within 8 hrs
  • primary outcome was recanalisation NOT clinical outcome

RESULTS

  • total 120 pts
  • excluded 80% assessed
  • stopped early for benefit (always a shame…)
  • solitaire device was better on everything (even mortality in this tiny trial)
  • of note mortality was 44% in the MERCI group and 18% in the Solitaire group. Think about that for a second – in the MERCI group almost half died – that seems a bit off to me – most strokes don’t die in numbers like this – at least not by 90 days. They certainly didn’t die in these rates in the lytic trials.

 

Some thoughts

These are all very selected patients – so this is the best possible picture of the results. This is NOT a treatment that will be available to all your stroke patients. Often it will be limited by anatomy but it’s also only gonna be done in the younger patients with a better chance of outcome. The key will be (as with most things) being able to work out who might actually benefit from this sort of thing.

This may over the next 20 years become like PCI for STEMIs. But i doubt it, for the same reasons I’m dubious about lytics in stroke.

  1. diagnostically stroke is a much more difficult disease than STEMI
  2. the brain and its circulation is a lot more complex and tenuous than the heart (we have a circle of willis for a reason  - a built in collateral circuit in case of failure of flow.)

For added value I’ve added a demonstration video for all 3 devices below. See if you can spot the differences.

On IST-3 and why we still don’t have the answer we were looking for…

OK so IST-3 is out. It’s a big and important trial so make sure and read it. Stroke lytics get a lot of attention from EM folks, if only because we seem the only ones not convinced of its efficacy. Jerry Hoffman is probably the most important contentious voice but he’s by no means alone. Ryan Radecki has also some good stuff online and in print about it. I’ve even got my own compendium on it.

When I heard IST-3 was underway and almost ready to report I was quite excited to see someone finally answering the question in a rigorous way – do lytics in acute stroke improve outcomes? The trial that is still cited is the original NINDS trial. It showed a benefit but it was small (comparatively) and there were baseline differences between the groups that may have biased the trial in favour of tPA. This could have been answered by repeating the trial and replicating the results (a fairly common practice). This has never been done and unfortunately IST-3 is not the one to do it either. Now that’s hardly surprising – it was never the question the authors set out to answer.

If you don’t fancy reading the more detailed analysis below here’s the three major problems my point of view.

  1. this is an open label trial – It was blinded at assessment but in hospital following treatment everyone knew who got tPA and who didn’t and they were treated differently. This is a potential source of bias
  2. it only randomised people currently outside the licence for tPA so it’s testing an entirely different bunch from NINDS. See here for which patients were randomised.
  3. it was a negative trial by its primary outcome. Something the authors don’t acknowledge in the conclusion in the abstract.

There are lots of other problems that I’ve highlighted below. I’d be interested in your thoughts.

METHODS

  • multicentre international RCT
  • a pilot phase (<10% total) was blinded and then in the main trial it was open label. They don’t explain why the main trial was open label but presumably it’s cheaper and easier to do.
  • Randomised on basis of uncertainty principle. Basically if you’re unsure whether or not you should give it based on current guidelines then randomise
  • all kinds of follow-up but mainly via GP or phone or mailed questionnaire
  • originally planned for 6000 but recruitment insufficient so recalculated their power and changed the statistical plan.
  • realised that there were big differences in the sub-groups at baseline (mainly on time and stroke severity) and had to apply logistic regression to adjust for them. Seeing as the only +ve results in the trial were in the sub-groups it makes me further question their significance if they were adjusted to compensate for base-line imbalances.

RESULTS

  • 3000 over 10 years (which is very slow) 300/yr split between 150 centres means 2/yr/centre
  • half over 80 years old
  • virtually all outside the current european licence (which is the point of the trial)
  • mainly treated at 4.2hrs
  • pts who got tPA more likely to go to HDU (24%v17%) than those who didn’t. For this you could perhaps make the assumption that pts who got tPA got better nursing care…
  • big spike in early deaths (11% v 7%) but then improved by 6 months (16% v 20%). Overall mortality was identical at 6 months
  • found a 2% benefit in primary outcome (alive and independent, 35% v 37%) at 6 months. A difference this small of course did not reach statistical significance. You could call this an NNT of 50 if it was real.

Click for source

  • significant ICH 7% v 1%
  • oddly an increase in fatal swelling (odd because if tPA works then the infarct would be smaller and the swelling would be less) of infarcts in tPA group of 47 pts v 25 pts. This is played down as inconsistent with prior studies in the paper. UPDATE: it was pointed out to me by a fine neurologist that when you lyse the clot you get reperfusion oedema so this is actually a sign the tPA does breakdown clot. It makes sense. It’s still a problem if the reperfusion oedema kills people, but it’s not oedema simply from a big infarct.
  • they have a whole ream of things in the forest plot of secondary outcomes (these are the adjusted ones) and only one approaches significance – age >80 yrs. Unfortunately as Ryan points out – if it’s better for those greater than 80 then it’s worse for those <80 – which is the precise opposite of prior trials.

Below is a video of the lead author talking about the results if you’re interested

The paper itself lives here

In the same issue the same authors have published an updated systematic review and meta-analysis that now includes these results. One of the concerns that has been pointed out before is perhaps this heterogeneous data set (now up to 12 very differently ran trials) aren’t actually appropriate to meta-analyse.

There’s a glowing editorial about the trial that makes the slightly odd and even reckless statement that:

Every stroke patient should therefore be classed as a candidate for thrombolysis

This seems a little bit of a stretch seeing as the IST-3 trial used these criteria and excluded lots of pts:

Click for source

My suspicion as always, is that tPA does work for some patients with stroke, but certainly not all and until we can pick out the ones who benefit then I suspect that we shouldn’t have adopted this as wholeheartedly as we already have.

UPDATE – if you look at page 4 of the supplementary appendix there’s a list of the drug treatments that varied between the 2 groups. The two groups were treated differently 

UPDATE – there’s been some discussion of the technique of ‘ordinal analysis’ that was used in the trial and pushed by the authors as a way of seeing this as a positive trial. EM Nerd has written a great little piece on it if you want to know more