Real world modified
Download as PPTX, PDF10 likes5,686 views
Stephen Senn argues that clinical trials are often unrepresentative of real-world clinical practice, and it's crucial to use appropriate analysis scales and real-world data for better decision-making. He emphasizes that trying to make trials more representative is not the solution; rather, we need effective modeling and auxiliary information to translate results into practical applications. The document highlights the importance of collaboration among medical professionals and statisticians to better evaluate treatment benefits and risks in real-world settings.
Real world modified
- 1. Clinical Trials are not Enough Stephen Senn (c) Stephen Senn 1
- 2. Acknowledgements (c) Stephen Senn 2 Acknowledgements This work is partly supported by the European Union’s 7th Framework Programme for research, technological development and demonstration under grant agreement no. 602552. “IDEAL”
- 3. Basic Thesis • Clinical trials are experiments • At every stage of drug development, even phase III, they are unrepresentative of ‘real world’ clinical practice • The key to using their results to inform ‘real world’ decisions is not to make the trial more representative • The key is to use appropriate scales for analysis and transfer the results into practical real-world decision making • This will require – A different attitude – More modelling – Extensive use of auxiliary real world data (c) Stephen Senn 3
- 4. An Example • Dan Moerman’s analysis more than 30 years ago of Smith-Kline-French’s development of Tagamet® – Tagamet® (cimetidine) was the best selling drug of its day • 31 trials in 1692 patients – in 17 countries – Duodenal, Gastric, Mixed – 13 significant according to Moerman • Used Chi-square test with Yates’s correction (c) Stephen Senn 4
- 5. (c) Stephen Senn 5 on the risk- difference scale and there is significant evidence of heterogeneity
- 6. What the EBM movement used to conclude from this sort of thing • The treatment effect varies according to the type of patient • We want RCTS that automatically deliver the right decision • We can’t rely on the results from RCTs which involve artificially selected patients • We need large simple trials in representative populations • They need to be reported using numbers needed to treat (NNTs) (c) Stephen Senn 6
- 7. Problems • At best you can hope to recommend treatments that are beneficial on average • But in practice you can never guarantee that trials are representative of practice anyway • You lose the opportunity to study issues more deeply • NNTs are a terrible scale for doing analysis (c) Stephen Senn 7
- 8. (c) Stephen Senn 8 Here the analysis is on the log-odds ratio scale and heterogeneity is much reduced and not even ‘significant’.
- 9. 9 “If you need statistics to prove it I don’t believe it” You can’t prove it with statistics but everybody believes it Thanks to Pat Ballew’s blog site (c) Stephen Senn 9 That is to say, we see non-random variation too easily. This example does NOT give strong evidence treatment by trial interaction
- 10. Lessons • If not carefully, studied random variation can be underestimated • Differences from trial to trial in true effect may be less than one thinks • Finding a good scale is important • BUT The additive scale is not necessarily the relevant one (c) Stephen Senn 10
- 11. What not to do • The solution is not to attempt to make trials more representative • The solution is to measure appropriately and translate appropriately • This requires the following – Good scales – Good analysis – Good modelling – Good supplementary real world data (c) Stephen Senn 11
- 12. (c) Stephen Senn 12 Chasing sub- groups leads nowhere
- 13. Solution? (c) Stephen Senn 14 “a possible resolution is to use the additive measure at the point of analysis and transform to the relevant scale at the point of implementation. This transformation at the point of medical decision-making will require auxiliary information on the level of background risk of the patient.” Senn, Statistics in Medicine, 2004
- 14. How we already use modelling, data and additive scales • Interspecies scaling • Bioequivalence – log relative bioavailability is additive but difference in absolute bioavailability is not • Dose proportionality • Use of additive scales in phase III – Log hazard – Log-odds ratio (c) Stephen Senn 15
- 15. (c) Stephen Senn 17Controlled Clinical Trials, 1989
- 16. Implications of the Lubsen-Tijssen Model • We need to study treatment benefit on disaggregated (of harm) additive scale • We will need real world data on harms • We will need real world data on background risk • We will need models • We will need cooperation between – Medics and statisticians working on clinical trials – Statisticians, epidemiologists, health economists, medics and others working in real world data (c) Stephen Senn 18
- 17. Example of Atrial Fibrillation • Such patients are at higher risk of stroke • Meta-analysis (reproduced in Hart et al 2007)concluded that warfarin has a beneficial protective effect • But there is a risk of intracranial bleeding • Who should get warfarin? (c) Stephen Senn 19
- 18. (c) Stephen Senn 20
- 19. (c) Stephen Senn 21
- 20. So you have atrial fibrillation • Should you take warfarin? • What else do you need to know? – The difference in risk taking warfarin or not – The rate of side effects – The consequences of side-effects • These cannot be answered (alone) by analysis of RCTs with pre-specified efficacy measures on the additive scale • The RCTs has to be translated and supplemented by real world data (c) Stephen Senn 22
- 21. (c) Stephen Senn 23
- 22. (c) Stephen Senn 24 Estimate based on 6 v 3 cases only
- 23. The reimburser’s perspective • What benefit and harm to the population will accrue from recommending warfarin prophylaxis? • How much will it cost? • How can its use be optimised? • Who should get it? (c) Stephen Senn 25
- 24. (c) Stephen Senn 26
- 25. Reimburser’s needs Requirements • A means of separating patients by risk • A means of establishing risk distribution in the population of patients above any threshold chosen • A means of determining expected benefits and costs Solutions • These figures cannot be delivered by clinical trials alone but will require – Cohort studies/case control studies – Health surveys – Economic modelling (c) Stephen Senn 27
- 26. We need to model background risk • Sort of data set we could use is that provided by the UK Clinical Practice Research Data Link CPRD • Could use this to model – A) Predictors of risk of stroke – B) Distribution of risk levels in the population • Former relevant to individuals to make decisions • Latter is relevant to reimbursers (c) Stephen Senn 28
- 27. Is there a Trust Problem? • Yes • Clinical trials provide a “template of trust” whereby regulators can mandate sponsors to provide the proof • Modelling + real world data cannot provide these guarantees • But this is no excuse – Whether or not you model, others will – You need to know as much as possible about your own drugs and where and when to use them (c) Stephen Senn 29
- 28. Finally I leave you with this though (c) Stephen Senn 30 Any damn fool can analyse a clinical trial and frequently does But doing it properly involves skilful analysis, understanding what the results mean requires intelligence insight and experience, and applying the results intelligently needs more of the same plus modelling and real world data And to whom do we look to provide these skills? Statisticians!