![• Looking for instances of:
- complexity (Remus, 1984; 1987)
- summary/overview needed (Tufte 2001)
- comparison needed (Spence 2007)
- time limited (Resnikoff, 1989)
- selective focussing needed (Thomas, 2005)
“In terms of [presenting a large] quantity of
information, then the problem is usually
with the clinical effectiveness, and
summarising that.”
Data must be split over several pages, or
slides.
Also sensitivity analysis of models
mentioned.](https://image.slidesharecdn.com/pentag17may11-110518081503-phpapp01/85/WST-PhD-presentation-for-PenTAG-17may11-41-320.jpg)
![Average Times (mean)
Numerical - Graphical “in this case, i think the
Graphical - Numerical
200 seconds
graphical plus the numerical
makes for more confustion.
one or the other is sufficient”
95% confidence
mean (g-n group)
“[the graphical presentation]
mean (n-g group)
seemed more confusing -
95% confidence too manty differnt elements
to look at - visual noise”
0 seconds “I found the screen
Y1 Y2 Y3 Y4 Y5 Y6 cluttered.”
N=78 N=71 N=69 N=65 N=63 N=61
N=77 N=70 N=67 N=54 N=54 N=52
numerical presentation graphical presentation](https://image.slidesharecdn.com/pentag17may11-110518081503-phpapp01/85/WST-PhD-presentation-for-PenTAG-17may11-70-320.jpg)
![Average Times (mean)
Numerical - Graphical “in this case, i think the
Graphical - Numerical
200 seconds
graphical plus the numerical
makes for more confustion.
one or the other is sufficient”
95% confidence
mean (g-n group)
“[the graphical presentation]
mean (n-g group)
seemed more confusing -
95% confidence too manty differnt elements
to look at - visual noise”
0 seconds “I found the screen
Y1 Y2 Y3 Y4 Y5 Y6 cluttered.”
N=78 N=71 N=69 N=65 N=63 N=61
N=77 N=70 N=67 N=54 N=54 N=52
numerical presentation graphical presentation](https://image.slidesharecdn.com/pentag17may11-110518081503-phpapp01/85/WST-PhD-presentation-for-PenTAG-17may11-71-320.jpg)
![Interview 8:
Those given COGS first “I would say I got much more of an
overview, just from looking at that
took a mean of 99.5% graphical summary”
of their COGS task time
with the report. Interview 3:
(pointing to graphic)
“This really helps to have all of those
Those given the report elements brought together, so you
can get a more holistic view of
first took a mean of where is it from and how big is it,
268.5% of their COGS what’s the study design.”
task time with the
report. Interview 1:
“I speculate that I would have had a
much, much less detailed idea of the
two-sample t(7) = 4.0, p = 0.005 quality of the evidence if I’d been
confronted with that [the report]
first.”](https://image.slidesharecdn.com/pentag17may11-110518081503-phpapp01/85/WST-PhD-presentation-for-PenTAG-17may11-82-320.jpg)
![stated preference for COGS
key stated preference for report
did not state preference during task
using
report display 1 using
COGS Interview 7:
familiar-
isation 1
task 1 “I do think [the overall quality of the
task 2 evidence is] easier to see with this,
task 3
actually. It’s a good way of
task 4
general presenting it.”
reliability 1
display 2
Interview 2:
familiar-
isation 2
task 5
task 6 “again, I’m going to use the
task 7
graphical summary because it’s far
task 8
general
reliability 2
more useful [for this task], I think.”
display 3
familiar-
isation 3
task 9
task 10
task 11
task 12
probe
general questions
useful for
this review?
useful for
other reviews?
validate tasks
interactive version](https://image.slidesharecdn.com/pentag17may11-110518081503-phpapp01/85/WST-PhD-presentation-for-PenTAG-17may11-84-320.jpg)
WST PhD presentation for PenTAG 17may11
- 1. Information Graphics in Health Technology Assessment Will Stahl-Timmins 17th May 2011
- 17. PhD research intro A journey My PhD (so far) Opportunities
- 18. PhD research intro A journey My PhD (so far) Opportunities
- 20. Health Policy Medical Practice EBM HTA Scientific Evidence
- 21. Information Graphics in Health Technology Assessment Virtual presentations of information, which use graphical elements (eg position, colour, size, etc) to present scientific evidence, informing health policy-making in terms of recommendations for the adoption of specific health interventions.
- 22. Research Question How should information graphics be designed, produced and used in health technology assessment?
- 23. TAR review NICE interviews Methods study Design & critique COGS SOC test test
- 24. TAR review NICE interviews Methods study Design & critique COGS SOC test test
- 25. TAR review • 50 of 98 NICE TAR reports reviewed • dated Oct 2003 - Nov 2007 • content analysis • graphics categorised
- 27. • 965 graphics used
- 28. • 965 graphics used • graphics in every report but one
- 29. • 965 graphics used • graphics in every report but one • 0.20 graphics per page
- 30. • 965 graphics used • graphics in every report but one • 0.20 graphics per page • 0.58 tables per page
- 31. 1.0 • 965 graphics used 0.8 • graphics in every report 0.6 but one 0.4 • 0.20 graphics per page 0.2 • 0.58 tables per page 0 GRAPHICS TABLES PER PAGE PER PAGE
- 33. TIME SERIES CEAC LINE THRESHOLD ANALYSIS 301 102 124 37 38 OTHER STATE TRANSITION 78 41 DECISON TREE FLOW 5 OTHER 124 BAR CHART SCATTER PLOT AREA /POSITION 88 55 44 OTHER 187 FOREST PLOT OTHER 331 22 OTHER 353
- 34. USED IN NICE-COMMISSIONED TECHNOLOGY ASSESSMENT REPORTS, 2003-2007 CIRCLES REPRESENT THE NUMBER OF REPORTS THAT USED A TYPE OF GRAPHIC AT LEAST ONCE, BY REPORT SECTION INTRO/ SYSTEM. SYSTEM. MODEL MODEL MODEL CONC. APPEN- BACKG. REVIEW REVIEW REVIEWS METHODS RESULTS DICES METHODS RESULTS LINE TIME SERIES 10 6 10 8 8 9 CEAC 31 2 2 THRESHOLD 1 1 4 1 1 1 8 1 OTHER 3 1 FLOW STATE TRANSITION 6 2 8 2 30 2 DECISION TREE 1 4 1 20 5 14 2 OTHER 3 5 AREA 10 10 1 BAR CHART 2 2 4 2 SCATTER PLOT 1 16 2 1 /POSITION OTHER 3 2 1 2 1 OTHER FOREST PLOT 25 7 1 2 OTHER 5 4 4 1 ALL CALCULATIONS FOR CIRCLE SIZES ARE AREA-BASED. SO, A CIRCLE REPRESENTING 50 REPORTS HAS A DIAMETER OF 10mm, AND AN AREA OF 7.9mm2. A CIRCLE REPRESENTING 25 REPORTS 10 20 30 40 50 HAS AN AREA OF 3.9mm2 AND A DIAMETER OF 7.1mm.
- 35. TAR review NICE interviews Methods study Design & critique COGS SOC test test
- 36. NICE technical advisors telephone interviews - needs assessment • 5 interviews • ~30 minutes • gist transcribed • framework analysis
- 37. • Looking for instances of: - complexity (Remus, 1984; 1987) - summary/overview needed (Tufte 2001) - comparison needed (Spence 2007) - time limited (Resnikoff, 1989) - selective focussing needed (Thomas, 2005)
- 38. • Looking for instances of: - complexity (Remus, 1984; 1987) - summary/overview needed (Tufte 2001) - comparison needed (Spence 2007) - time limited (Resnikoff, 1989) - selective focussing needed (Thomas, 2005)
- 39. • Looking for instances of: - complexity (Remus, 1984; 1987) - summary/overview needed (Tufte 2001) - comparison needed (Spence 2007) - time limited (Resnikoff, “It’s difficult 1989) - because it depends on the - individual appraisal” selective focussing needed (Thomas, 2005) Some things were thought to lead to complex data included: - Multiple outcome measures - Many subgroups - Sequential treatments - Mixed treatment comparisons - Many variables in SA - Many disease states in model
- 40. • Looking for instances of: - complexity (Remus, 1984; 1987) - summary/overview needed (Tufte 2001) - comparison needed (Spence 2007) - time limited (Resnikoff, 1989) - selective focussing needed (Thomas, 2005)
- 41. • Looking for instances of: - complexity (Remus, 1984; 1987) - summary/overview needed (Tufte 2001) - comparison needed (Spence 2007) - time limited (Resnikoff, 1989) - selective focussing needed (Thomas, 2005) “In terms of [presenting a large] quantity of information, then the problem is usually with the clinical effectiveness, and summarising that.” Data must be split over several pages, or slides. Also sensitivity analysis of models mentioned.
- 42. • Looking for instances of: - complexity (Remus, 1984; 1987) - summary/overview needed (Tufte 2001) - comparison needed (Spence 2007) - time limited (Resnikoff, 1989) - selective focussing needed (Thomas, 2005)
- 43. • Looking for instances of: - complexity (Remus, 1984; 1987) - summary/overview needed (Tufte 2001) - comparison needed (Spence 2007) - time limited (Resnikoff, 1989) - selective focussing needed (Thomas, 2005) The most commonly mentioned type of data that needed to be compared to another was the ICER, the overall measure of the cost-effectiveness of an intervention. One interviewee noticed that it was frequently necessary in committee meetings to “flip backwards and forwards” between slides when questions were asked about the certainty of evidence.
- 44. • Looking for instances of: - complexity (Remus, 1984; 1987) - summary/overview needed (Tufte 2001) - comparison needed (Spence 2007) - time limited (Resnikoff, 1989) - selective focussing needed (Thomas, 2005)
- 45. • Looking for instances of: - complexity (Remus, 1984; 1987) - summary/overview needed (Tufte 2001) - When asked comparison needed (Spence 2007) if time was limited (in any part - time limited (Resnikoff, 1989) appraisal process), interviewees of the - responded: selective focussing needed (Thomas, 2005) “Time is always limited” or “Yes, is the short answer!” All five interviewees stated that time was always limited for decision-makers to familiarise themselves with the necessary information before an appraisal committee.
- 46. • Looking for instances of: - complexity (Remus, 1984; 1987) - summary/overview needed (Tufte 2001) - comparison needed (Spence 2007) - time limited (Resnikoff, 1989) - selective focussing needed (Thomas, 2005)
- 47. • Looking for instances of: - complexity (Remus, 1984; 1987) - summary/overview needed (Tufte 2001) - comparison needed (Spence 2007) - time limited (Resnikoff, 1989) - selective focussing needed (Thomas, 2005) Most interviewees seemed very uncomfortable about the idea. “it’s really not fair, on- it’s not right to give some of them extra detail.”
- 48. TAR review NICE interviews Methods study Design & critique COGS SOC test test
- 49. 1. Small multiple techniques including 10 information graphics Sankey diagrams for overview of studies in a systematic review The Friday Information Graphic 2. Two-way sensitivity analysis matrix / bubble chart 16th Oct 2009 Link diagrams for showing 3. Parallel coordinates for probabilistic connections between search sensitivity analysis strategies in multiple systematic reviews 4. Technology assessment report graphical overview 5. Sankey Markov overview 6. ‘Whirlpool’ display for enhancing tornado diagram in deterministic sensitivity analysis 7. Survival synthesis bubble chart Peninsula Technology Information Graphics in Assessment Group Health Technology Assessment www.pms.ac.uk/pentag www.pms.ac.uk/infographics Noy Scott House Will Stahl-Timmins Barrack Road Exeter EX2 5DW [email protected] +44 (0) 1392 406 967 1 8. Distribution-based forest plot 9. Search strategy link diagram 10. Individual patient display for discrete event simulation
- 50. Evaluation?
- 51. TAR review NICE interviews Methods study Design & critique COGS SOC test test
- 52. measurements test (experiment) online decision gradually increasing task study decision complexity (subgroups) sample: general internet-using measurements: public - respondent decision accuracy, -driven sample time & preference numerical presentation graphical presentation
- 53. measurements test (experiment) online decision gradually increasing task study decision complexity (subgroups) sample: general internet-using measurements: public - respondent decision accuracy, -driven sample time & preference http://www.pms.ac.uk/ infographics/
- 54. measurements test (experiment) online decision gradually increasing task study decision complexity (subgroups) sample: general internet-using measurements: public - respondent decision accuracy, -driven sample time & preference http://www.pms.ac.uk/ infographics/
- 55. measurements test (experiment) online decision gradually increasing task study decision complexity (subgroups) sample: general internet-using measurements: public - respondent decision accuracy, -driven sample time & preference seed 1 seed 2 25+ participants... seed 3
- 56. measurements test (experiment) online decision gradually increasing task study decision complexity (subgroups) sample: general internet-using measurements: public - respondent decision accuracy, -driven sample time & preference increasing complexity single group males and females low, medium, high risk males and females
- 57. measurements test (experiment) online decision gradually increasing task study decision complexity (subgroups) sample: general internet-using measurements: public - respondent decision accuracy, -driven sample time & preference
- 58. measurements test (findings)
- 59. measurements test (findings) Study Duration: 36 days (15th June - 21st July 2009)
- 60. measurements test (findings) Study Duration: 36 days (15th June - 21st July 2009) 244 entries were recorded during this time.
- 61. measurements test (findings) Study Duration: 36 days (15th June - 21st July 2009) 244 entries were recorded during this time. 48 excluded as possible duplicates, leaving 196 for the analysis
- 62. measurements test (findings) Study Duration: 36 days (15th June - 21st July 2009) 244 entries were recorded during this time. 48 excluded as possible duplicates, leaving 196 for the analysis 99 participants received the graphical presentation first. 97 received the numerical one first.
- 63. Randomised to receive: Numerical first (N=97) Graphical first (N=99) did not did not complete complete task 1 (N=19) task 1 (N=22) Task 1 did not complete did not complete task 2 (N=7) Task 2 task 2 (N=7) did not complete did not complete task 3 (N=3) Task 3 task 3 (N=2) did not complete did not complete details collection (N=6) Details details collection (N=2) collection did not complete did not complete task 4 (N=7) Task 4 task 4 (N=2) did not complete Task 5 task 5 (N=2) did not complete did not complete task 6 (N=2) Task 6 task 6 (N=2) did not give preference (N=2) Preference did not give preference (N=5) collection 38 people gave a 25 people liked 43 people gave a preference for the the displays preference for the numerical display equally graphical display
- 64. Average Deaths (mean) Numerical - Graphical group 7787 7787 7787 Graphical - Numerical group Max. Possible Deaths 7224 7224 7224 95% confidence 5920 mean (g-n group) mean (n-g group) 5466 95% confidence Min. Possible Deaths 5074 5114 5015 4748 Y1 Y2 Y3 Y4 Y5 Y6 N=78 N=71 N=69 N=65 N=63 N=61 N=77 N=70 N=67 N=54 N=54 N=52
- 65. Average Times (mean) Numerical - Graphical Graphical - Numerical 200 seconds 95% confidence mean (g-n group) mean (n-g group) 95% confidence 0 seconds Y1 N=78 Y2 N=71 Y3 N=69 Y4 N=65 Y5 N=63 Y6 N=61 N=77 N=70 N=67 N=54 N=54 N=52
- 67. 20 mins 0 secs num. task graph. task num. task graph. task num. task graph. task Numerical Undecided Graphical Preference Preference
- 68. rs = .484 p < 0.05
- 69. Average Times (mean) Numerical - Graphical Graphical - Numerical 200 seconds 95% confidence mean (g-n group) mean (n-g group) 95% confidence 0 seconds Y1 N=78 Y2 N=71 Y3 N=69 Y4 N=65 Y5 N=63 Y6 N=61 N=77 N=70 N=67 N=54 N=54 N=52 numerical presentation graphical presentation
- 70. Average Times (mean) Numerical - Graphical “in this case, i think the Graphical - Numerical 200 seconds graphical plus the numerical makes for more confustion. one or the other is sufficient” 95% confidence mean (g-n group) “[the graphical presentation] mean (n-g group) seemed more confusing - 95% confidence too manty differnt elements to look at - visual noise” 0 seconds “I found the screen Y1 Y2 Y3 Y4 Y5 Y6 cluttered.” N=78 N=71 N=69 N=65 N=63 N=61 N=77 N=70 N=67 N=54 N=54 N=52 numerical presentation graphical presentation
- 71. Average Times (mean) Numerical - Graphical “in this case, i think the Graphical - Numerical 200 seconds graphical plus the numerical makes for more confustion. one or the other is sufficient” 95% confidence mean (g-n group) “[the graphical presentation] mean (n-g group) seemed more confusing - 95% confidence too manty differnt elements to look at - visual noise” 0 seconds “I found the screen Y1 Y2 Y3 Y4 Y5 Y6 cluttered.” N=78 N=71 N=69 N=65 N=63 N=61 N=77 N=70 N=67 N=54 N=54 N=52 numerical presentation graphical presentation
- 72. TAR review NICE interviews Methods study Design & critique COGS SOC test test
- 73. COGS (Clinical effectiveness Overview Graphical Summary
- 75. COGS test Task-based cognitive interviewing Speak-aloud protocol 9 expert users (HTA systematic reviewers) Randomised, sequencial comparison to report Quantitative results (time and accuracy) Qualitative results (actions and words of participants - framework analysis)
- 76. Randomised, crossover design report TASK TASK TASK TASK TASK TASK TASK TASK given first 1 2 3 4 5 6 7 8 TASK TASK TASK TASK graphic TASK TASK TASK TASK TASK TASK TASK TASK 9 10 11 12 given first 1 2 3 4 5 6 7 8 12 tasks
- 77. Task 4: Can you tell me about selection bias in the Peters et al. (2007) trial please? COGS display report section 1 4 5 8 9 2 3 6 7 6.9% 9.3% 12.0% 12.3% 13.5% 16.4% 17.0% 17.9% 24.1%
- 78. Task 8: Of the unilateral cochlear implants vs non-technological support trials, which reported at least one significant outcome measure, and which measures were these? report section COGS display 1 4 5 8 9 2 3 6 7 5.4% 5.1% 6.3% 7.9% 13.3% 15.0% 15.1% 18.2% 32.4%
- 79. +&,-.*,/0.*12& !"#$%#&$&'(#&')*#&+,-. /00$0#1'0)2#34"#56 345 675 two-sample t(69) = 4.4 p < 0.001 645 75 45 %&'()* !"#$
- 80. task accuracy COGS: 74.3% report: 46.4% c2 (1, N = 63) = 5.12, p = 0.024
- 81. Those given COGS first took a mean of 99.5% of their COGS task time with the report. Those given the report first took a mean of 268.5% of their COGS task time with the report. two-sample t(7) = 4.0, p = 0.005
- 82. Interview 8: Those given COGS first “I would say I got much more of an overview, just from looking at that took a mean of 99.5% graphical summary” of their COGS task time with the report. Interview 3: (pointing to graphic) “This really helps to have all of those Those given the report elements brought together, so you can get a more holistic view of first took a mean of where is it from and how big is it, 268.5% of their COGS what’s the study design.” task time with the report. Interview 1: “I speculate that I would have had a much, much less detailed idea of the two-sample t(7) = 4.0, p = 0.005 quality of the evidence if I’d been confronted with that [the report] first.”
- 83. stated preference for COGS key stated preference for report did not state preference during task using report display 1 using COGS familiar- isation 1 task 1 task 2 task 3 task 4 general reliability 1 display 2 familiar- isation 2 task 5 task 6 task 7 task 8 general reliability 2 display 3 familiar- isation 3 task 9 task 10 task 11 task 12 probe general questions useful for this review? useful for other reviews? validate tasks interactive version
- 84. stated preference for COGS key stated preference for report did not state preference during task using report display 1 using COGS Interview 7: familiar- isation 1 task 1 “I do think [the overall quality of the task 2 evidence is] easier to see with this, task 3 actually. It’s a good way of task 4 general presenting it.” reliability 1 display 2 Interview 2: familiar- isation 2 task 5 task 6 “again, I’m going to use the task 7 graphical summary because it’s far task 8 general reliability 2 more useful [for this task], I think.” display 3 familiar- isation 3 task 9 task 10 task 11 task 12 probe general questions useful for this review? useful for other reviews? validate tasks interactive version
- 85. stated preference for COGS key stated preference for report did not state preference during task using report display 1 using COGS familiar- isation 1 task 1 task 2 task 3 task 4 general reliability 1 display 2 familiar- isation 2 task 5 task 6 task 7 task 8 general reliability 2 display 3 familiar- isation 3 task 9 task 10 task 11 task 12 probe general questions useful for this review? useful for other reviews? validate tasks interactive version
- 86. stated preference for COGS key stated preference for report did not state preference during task using report display 1 using COGS Preferred elements (N): familiar- isation 1 task 1 The outcomes display (2) task 2 task 3 The quality grid (2) task 4 general Follow-up display (1) reliability 1 display 2 Being able to compare familiar- characteristics, quality and isation 2 task 5 outcomes together (1) task 6 task 7 Being able to compare task 8 characteristics between studies (1) general reliability 2 The study design symbols (3) display 3 familiar- Age display (1) isation 3 task 9 task 10 task 11 task 12 probe general questions useful for this review? useful for other reviews? validate tasks interactive version
- 87. COGS test - conclusions Search time reduced - however, there was less information available overall in COGS. Gives overview Failed to present study designs successfully - revisions to key needed Different intervention areas will need different data
- 88. Design/size arrows Height of arrow is proportional to N (number of people tested) larger study smaller study pre/post design (same cohort is measured before and a er intervention). pre-intervention N = 29 post-intervention N = 20 pre-intervention N=7 post-intervention N=2 cross-sectional / Intervention Intervention non-randomised N = 29 N=7 cohort design Control N = 20 Control N = 2 Intervention N = 29 Intervention N = 7 randomised design N=9 N = 49 Control N = 20 Control N = 2 Intervention N = 29 retrospective non-randomised Intervention N = 7 cohort study design Control N = 20 Control N = 2 survey design N = 49 N=9
- 89. length of follow-up 0 yrs 5 10 10 cross-sectional Intervention N = 29 design (no follow-up) Control N = 20 Intervention N = 29 5 year follow-up Control N = 20 Intervention 12 year follow-up N = 29 Control N = 20
- 90. follow-up outcome measures 0yr 5 10 15 GASP CUNY CAP CDT CID CNC CPT FMWT GSL ESP Intervention N = 21 N = 43 Control N = 22
- 91. outcome measures used no. of design, size baseline study cog func be glo author location centres & follow-up MMSE sex ages quality ADCS-ADL ADCS-CGIC ADAS-cog other other other CIBIC MMSE DAD PDS CDR QoL SIB NPI GDS 0yr 1 2 0 10 20 30 55 75 95 Donepezil 1mg N = 42 M F Rogers & Rand Donepezil 3mg N = 40 M F ? N = 161 Donepezil 5mg N = 39 M F Char Blind Analy 1mg 3mg 1996 F Placebo N = 40 M 5mg Donepezil 5mg N = 154 M F 5mg Rogers et al. Rand M F Char 10 Donepezil 10mg N = 157 Blind mg 1998 (A) N = 473 Analy M F Placebo N = 162 Donepezil 5mg N = 157 M F 5mg Rogers et al. Rand M F Char 10 Donepezil 10mg N = 158 mg Blind 1998 (B) N = 468 Analy M F Placebo N = 153 Donepezil 5mg N = 271 M F 5mg Burns et al. Rand M F Char 10mg Donepezil 10mg N = 273 Blind 1999 Analy N = 818 M F Placebo N = 274 Greenberg Donepezil 5mg (D) Rand et al. group 1 (p-D-p-p) N=30 M F Char group 2 (p-p-D-p) N=30 M F Blind N = 60 Analy 2000 Placebo (p) Donepezil 5mg N = 134 Homma et al. M F Rand Char Blind 2000 N = 268 M F Analy Placebo N = 129 Donepezil 10mg N = 214 Mohs et al. M F Rand Char ADCS-CGIC ADCS-ADL Blind ADAS-cog 2001 N = 431 M F Analy MMSE CIBIC Placebo N = 217 DAD other other other CDR GDS PDS QoL NPI SIB 0yr 1 2 0 10 20 30 55 75 95 cog func be glo
- 92. TAR review NICE interviews Methods study Design & critique COGS SOC test test
- 93. State Occupancy Charts (SOCs) temozolomide vs placebo for the treatment of newly diagnosed high-grade glioma 1 — State Occupancy Chart 2 — State Occupancy & Absolute Quality of Life 3 — State Occupancy & Absolute Costs Per Person 4 — Incremental State Occupancy 5 — Incremental QALYs 6 — Incremental Costs
- 94. State Occupancy Chart placebo arm treatment arm 1 state occupancy state occupancy week 1 surgery 0% 25% 50% 75% 100% 0% 25% 50% 75% 100% week 1 week 2 weeks 2-6 post-op recovery week 7 weeks 7-12 radiotherapy week week 13+ stable/progressive/death 13 week week 26 26 is graphic shows the number of simulated people in the six week week di erent states of the model, 39 39 over the 260 one-week cycles week week of the model. 52 52 week week During the rst week of treat- 65 65 ment, all of the people week week in the model were assumed 78 78 to undergo surgery, which is represented with a seperate week week state in the model. 91 91 week week From weeks 2-6, patients can 104 104 either be in a post-operation recovery (treatment-free) state, week week 117 117 or move to death in any of these ve weeks. week week 130 130 In weeks 7-12, patients will week week undergo radiotherapy, have 143 143 progressive disease or be dead. week week 156 156 From week 13 onwards, the model becomes a fairly typical week week three-state model, with patients 169 169 either in a stable state, having week week progressive disease, 182 182 or dead. week week 195 195 – surgery (week 1) week week 208 208 – post-op recovery (weeks 2-6) week week – radiotherapy 221 221 (weeks 7-12) – stable disease week week (week 13+) 234 234 – progressive week week disease 247 247 – death week week 260 260
- 95. State Occupancy & Absolute Quality of Life 2 placebo arm treatment arm week 1 surgery 0% 25% 50% 75% 100% 0% 25% 50% 75% 100% week 1 week 2 weeks 2-6 post-op recovery week 7 weeks 7-12 radiotherapy week week 13+ stable/progressive/death 13 week week 26 26 is information graphic shows the absolute quality of life week week experienced by the simulated 39 39 patients in the model. e week week shades of grey provide a scale 52 52 from black (a utility of 1 per week week person) to white (a utility of 0 65 65 per person). week week 78 78 ese shades of grey are presented in bars whose length week week correspond to the number 91 91 of people in that state in the week week model during that week, as in 104 104 graphic 1 — State Occupancy. week week 117 117 e slowly lightening e ect in the progressive state is caused week week by the gradual decomposition 130 130 of utility values in this state in week week the model. e simulated 143 143 patients experience less quality week week of life the longer they spend in 156 156 this state. e values presented here are the average (mean) of week week the utility scores experienced 169 169 by the cohort in that week of week week the model. 182 182 week week 195 195 utility of 1 week week (per person) 208 208 week week 221 221 utility of 0.5 week week 234 234 week week 247 247 utility of 0 week week 260 260
- 96. State Occupancy & Absolute Costs Per Person 3 placebo arm treatment arm 0% 25% 50% 75% 100% 0% 25% 50% 75% 100% week 1 surgery week 1 week 2 weeks 2-6 post-op recovery week 7 weeks 7-12 radiotherapy week week 13+ stable/progressive/death 13 week week 26 26 is information graphic shows the absolute costs incurred per week week person in the model, on a scale 39 39 from black (£3000 per person) week week to white (£0 per person). 52 52 week week ese shades of grey are 65 65 presented in bars whose length correspond to the number week week of people incurring that cost 78 78 in that state in the model, as in week week graphic 1 — State Occupancy. 91 91 e small dark bars appearing week week 104 104 between the progressive and death states represent the week week one-o costs assigned to death 117 117 in the model. e length of week week these bars is again proportional 130 130 to the number of people dying during that week of the model. week week 143 143 Similar dark boxes appear week week between the stable and 156 156 progressive states to indicate the higher costs assigned week week 169 169 to a patient’s rst week in the progressive state. week week 182 182 Costs for surgery in week week week 1 are o the scale at £5953 per 195 195 person, but this cost is identical week week in both arms of the model. 208 208 week week 221 221 £3000 per person week week 234 234 £1500 per person week week 247 247 £0 per person week week 260 260
- 97. Incremental State Occupancy 4 stable progressive death week 1 surgery week 1 week 2 weeks 2-6 post-op recovery week 7 weeks 7-12 radiotherapy week week 13+ stable/progressive/death 13 week week 26 26 e di erence in state occupancy between the two week week arms of the model are shown 39 39 here. week week 52 52 A bar extending to the week week le shows that there were more 65 65 people in that state during that week week week in the placebo arm than 78 78 in the temozolomide arm. Extending to the right indicates week week more people in the temozolo- 91 91 mide arm. week week 104 104 e length of the bars, indicat- ing the incremental di erence week week 117 117 between the state occupancy of the two arms, are proprtional week week to graphic 1: State Occupancy. 130 130 A 10% shi is indicated by a week week thin vertical white line. 143 143 week week 156 156 week week 169 169 week week 182 182 week week 195 195 week week 208 208 week week 221 221 5% shi week week 234 234 10% shi 15% shi week week 247 247 20% shi week week 260 260
- 98. Incremental QALYs 5 stable progressive total week 1 surgery week 1 week 2 weeks 2-6 post-op recovery week 7 weeks 7-12 radiotherapy week week 13+ stable/progressive/death 13 week week 26 26 is graphic shows the di erences between the two week week arms of the model in terms of 39 39 the quality adjusted life years week week (QALYs) that would be 52 52 experienced by a simulated week week cohort of 1000 people. 65 65 week week A bar extending to the le 78 78 shows that, during that week, more QALYs were experienced week week by the people in the placebo 91 91 arm than the temololomide week week arm. A bar extending to the 104 104 right represents more QALYs experienced in the temozolo- week week 117 117 mide arm. week week in vertical white lines 130 130 show the number of QALYs week week that would be experienced 143 143 in a cohort of 1000 simulated week week patients. A bar that reaches one 156 156 line represents one QALY. week week e death state is not shown, 169 169 as no QALYs are experienced week week in that state, as it was assigned 182 182 a utility of 0. week week 195 195 e “total” column on the far right shows a sum of the values week week 208 208 from the other two states. week week 221 221 incremental QALYs week week 234 234 1 2 week week 247 247 3 week week 260 260
- 99. Incremental Costs 6 stable progressive death total week 1 surgery week 1 week 2 weeks 2-6 post-op recovery week 7 weeks 7-12 radiotherapy week week 13+ stable/progressive/death 13 week week 26 26 is graphic shows the di erences in costs between week week the two arms of the model, 39 39 again with a cohort of 1000 week week simulated patients. 52 52 week week As before, a bar extending 65 65 to the le shows higher costs week week in the placebo arm, and 78 78 a bar extending to the right shows higher costs in week week the temozolomide arm. 91 91 week week e thin vertical white lines 104 104 show cost thresholds in increments of £100,000 week week 117 117 e “total” column on the week week far right shows a sum of the 130 130 values from all three states week week of the model. 143 143 week week 156 156 week week 169 169 week week 182 182 week week 195 195 week week 208 208 incremental costs week week £100,000 221 221 £200,000 week week £300,000 234 234 £400,000 week week £500,000 247 247 £600,000 week week 260 260
- 100. SOC test Task-based cognitive interviewing Probing protocol (Interviewer-led) 6 expert users (HTA modellers) Stand-alone evaluation (no comparator) Tasks used to assess understanding Qualitative results (participants asked for opinions - framework analysis)
- 101. Task 1 Q: How many people have progressive disease in week 52 of the temozolomide arm of the model? A: 32.7%
- 102. Task 1
- 103. Task 2 Q: Where do the costs tend to come from in each arm of the model?
- 104. Task 2
- 105. Task 6 Q: Where does the greatest difference between the costs of the two arms lie?
- 106. Task 6
- 107. SOC test - conclusions Participants largely understood meaning of displays Main function is to give overview, adding a temporal display to existing methods Considered useful by participants Could be used to display SA? Applicability to other models - with many more states?
- 108. placebo arm 0% 25% 50% 75% 100% £5.9m week 1 surgery week 26 week 39 £1.6m week 52 weeks 7-12 radiotherapy week 65 week 78 week £6.7m 91 week 104 weeks 2+ progressive disease week 117 week £3.1m 130 week 143 weeks 2+ death week 156 week 169 week 182 week 195 week chemotherapy drugs 208 week 221 radiotherapy £3000 per person week 234 hospital inpatient £1500 per person week 247 hospital outpatient week £0 per person 260
- 109. Total Incremental Costs and QALYs total incremental total incremental 7 QALYs costs week 1 surgery week 1 week 2 weeks 2-6 post-op recovery week 7 weeks 7-12 radiotherapy week week 13+ stable/progressive/death 13 week week 26 26 is graphic shows how week week adopting temozolomide 39 39 treatment would a ect week week quality of life and costs 52 52 incurred over time. week week 65 65 e grey bars show the change week week in costs and QALYs each week, 78 78 and the solid black lines show the cumulative e ects of week week 91 91 adopting the treatment. week week 104 104 It should be noted that the weekly values are presented week week on a di erent scale to the 117 117 cumulative values. is week week is necessary for the two 130 130 to be compared overlaid week week in this manner. 143 143 week week 156 156 week week 169 169 week week 182 182 week week 195 195 week week 208 208 week week incremental 221 221 costs / QALYs week week 234 234 cumulative incremental week week costs / QALYs 247 247 week week 260 260 total QALY gain: total cost: 223.1 £8.1m ICER = £36,171 per QALY
- 110. Overall Conclusions
- 111. TAR review NICE interviews Methods study Design & critique COGS SOC test test
- 112. Research Question How should information graphics be designed, produced and used in health technology assessment?
- 113. Design
- 114. Design RESEARCH OUTPUTS Research Area ECEHH
- 115. Production 1. Using standard visualisation tools in spreadsheet software (current situation – suitable for HTA professionals) 2. Developing new specialist software for use by HTA professionals (such as currently used for Forest plots) 3. Designing graphics on an individual basis (ie. in collaboration with trained information design professionals)
- 116. Use Likely that a combination of all three production methods will continue Depends on: - complexity of information to be presented - available resources - available skills - which (or whether) specialist software tools are developed
- 117. Production 1. Using standard visualisation tools in spreadsheet software (current situation – suitable for HTA professionals) Suitable for simpler reports: - small number of trials in review - few subgroups, sequencial treatments or other complicating factors - simple treatment pathway for model
- 118. Production 2. Developing new specialist software for use by HTA professionals (such as currently used for Forest plots) COGS software would be suitable for giving overview of more complex systematic reviews SOC suitable for models in which time is a key consideration Likely to be other graphics - these would further testing and evaluation
- 119. Production 3. Designing graphics on an individual basis (ie. in collaboration with trained information design professionals) Suitable for the most complex reviews and models, where: - different media become useable / dominant - particular information needs highlighting (area of world, timing of trials, etc)
- 120. Kessler Illg et al. Nikolopoulos MED-EL Staller Manrique Nikolopoulos Harrison et al. et al. et al. et al. et al. et al. 1997 1999 1999 2001 2002 2004 2004 2005 auth date ages 0 5 10 15 20 0 5 10 15 20 N = 49 N = 82 N = 78 N = 82 N = 82 N = 167 N = 126 N = 182 0yr 5 10 15 0yr 5 10 15 Pr Pr Pr Pr Pr Pr Pr Pr Se Se Se Se Se Se Se Se As As As As As As As As At At At At At At At At Po Po Po Po Po Po Po Po design/size follow-up quality Ot Ot Ot Ot Ot Ot Ot Ot 1ST 1ST 2ST 2 ST AVGN AVGN AB AB BKB BKB
- 121. 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 Harrison et al. Nikolopolous et al. Manrique et al. Staller et al. MED-EL Nikolopolous et al. Illg et al. Kessler et al.
- 122. Future research Developed graphics - Use and monitoring Evaluation of new graphics - Other graphics designed for PhD - Different designers’ work - Different media? Different audiences - Public - Medical professionals
- 123. Current work
- 124. European Centre for Environment and Human Health
- 126. UK CARBON EMISSIONS IN 2009, THE UK’S DEPARTMENT FOR ENERGY AND CLIMATE CHANGE 2009 CALCULATED THAT WE EMMITTED 564 TONNES OF CO2 - CARBON DIOXIDE. HERE’S HOW THAT 8t BREAKS DOWN INTO DIFFERENT 195t SECTORS. 10t ENERGY 18t TRANSPORT BUSINESS RESIDENTIAL 123t 50t AGRICULTURE WASTE 86t INDUSTRIAL 79t PUBLIC SECTOR
- 127. ATMOSPHERIC COLUMN ATMOSPHERIC SERVICES SATELLITES SERVICES AT ALL THREE ALTITUDES LOW EARTH ORBIT 160—2000 km PLASMA AND METEORS SOUNDING ROCKETS 50—1500 km DISPERSION OF AIR POLLUTION STRATOSPHERE 10—50 km PROPERTIES WEATHER BALLOONS 0—40 km UPPER AND LOWER TROPOSPHERE UPPER TROPOSPHERE AIRCRAFT 1—10 km CRUISING 6—12 km LOWER TROPOSPHERE POWER STATIONS 0—1 km 80—350 m COLUMN BASE: WIND TURBINES 80—130 m 1 km2