Bayesian Modeling for Economic Evaluation using Real World Evidence

# Bayesian Modeling for Economic Evaluation using Real World Evidence

## Gianluca Baio

### [Department of Statistical Science](https://www.ucl.ac.uk/statistics/) | University College London    
<br>

.title-small[
<svg viewBox="0 0 512 512" style="position:relative;display:inline-block;top:.1em;fill:#00acee;height:0.8em;" xmlns="http://www.w3.org/2000/svg">  <path d="M502.3 190.8c3.9-3.1 9.7-.2 9.7 4.7V400c0 26.5-21.5 48-48 48H48c-26.5 0-48-21.5-48-48V195.6c0-5 5.7-7.8 9.7-4.7 22.4 17.4 52.1 39.5 154.1 113.6 21.1 15.4 56.7 47.8 92.2 47.6 35.7.3 72-32.8 92.3-47.6 102-74.1 131.6-96.3 154-113.7zM256 320c23.2.4 56.6-29.2 73.4-41.4 132.7-96.3 142.8-104.7 173.4-128.7 5.8-4.5 9.2-11.5 9.2-18.9v-19c0-26.5-21.5-48-48-48H48C21.5 64 0 85.5 0 112v19c0 7.4 3.4 14.3 9.2 18.9 30.6 23.9 40.7 32.4 173.4 128.7 16.8 12.2 50.2 41.8 73.4 41.4z"></path></svg>  [g.baio@ucl.ac.uk](mailto:g.baio@ucl.ac.uk)
<svg viewBox="0 0 512 512" style="position:relative;display:inline-block;top:.1em;fill:#EA7600;height:0.8em;" xmlns="http://www.w3.org/2000/svg">  <path d="M503.52,241.48c-.12-1.56-.24-3.12-.24-4.68v-.12l-.36-4.68v-.12a245.86,245.86,0,0,0-7.32-41.15c0-.12,0-.12-.12-.24l-1.08-4c-.12-.24-.12-.48-.24-.6-.36-1.2-.72-2.52-1.08-3.72-.12-.24-.12-.6-.24-.84-.36-1.2-.72-2.4-1.08-3.48-.12-.36-.24-.6-.36-1-.36-1.2-.72-2.28-1.2-3.48l-.36-1.08c-.36-1.08-.84-2.28-1.2-3.36a8.27,8.27,0,0,0-.36-1c-.48-1.08-.84-2.28-1.32-3.36-.12-.24-.24-.6-.36-.84-.48-1.2-1-2.28-1.44-3.48,0-.12-.12-.24-.12-.36-1.56-3.84-3.24-7.68-5-11.4l-.36-.72c-.48-1-.84-1.8-1.32-2.64-.24-.48-.48-1.08-.72-1.56-.36-.84-.84-1.56-1.2-2.4-.36-.6-.6-1.2-1-1.8s-.84-1.44-1.2-2.28c-.36-.6-.72-1.32-1.08-1.92s-.84-1.44-1.2-2.16a18.07,18.07,0,0,0-1.2-2c-.36-.72-.84-1.32-1.2-2s-.84-1.32-1.2-2-.84-1.32-1.2-1.92-.84-1.44-1.32-2.16a15.63,15.63,0,0,0-1.2-1.8L463.2,119a15.63,15.63,0,0,0-1.2-1.8c-.48-.72-1.08-1.56-1.56-2.28-.36-.48-.72-1.08-1.08-1.56l-1.8-2.52c-.36-.48-.6-.84-1-1.32-1-1.32-1.8-2.52-2.76-3.72a248.76,248.76,0,0,0-23.51-26.64A186.82,186.82,0,0,0,412,62.46c-4-3.48-8.16-6.72-12.48-9.84a162.49,162.49,0,0,0-24.6-15.12c-2.4-1.32-4.8-2.52-7.2-3.72a254,254,0,0,0-55.43-19.56c-1.92-.36-3.84-.84-5.64-1.2h-.12c-1-.12-1.8-.36-2.76-.48a236.35,236.35,0,0,0-38-4H255.14a234.62,234.62,0,0,0-45.48,5c-33.59,7.08-63.23,21.24-82.91,39-1.08,1-1.92,1.68-2.4,2.16l-.48.48H124l-.12.12.12-.12a.12.12,0,0,0,.12-.12l-.12.12a.42.42,0,0,1,.24-.12c14.64-8.76,34.92-16,49.44-19.56l5.88-1.44c.36-.12.84-.12,1.2-.24,1.68-.36,3.36-.72,5.16-1.08.24,0,.6-.12.84-.12C250.94,20.94,319.34,40.14,367,85.61a171.49,171.49,0,0,1,26.88,32.76c30.36,49.2,27.48,111.11,3.84,147.59-34.44,53-111.35,71.27-159,24.84a84.19,84.19,0,0,1-25.56-59,74.05,74.05,0,0,1,6.24-31c1.68-3.84,13.08-25.67,18.24-24.59-13.08-2.76-37.55,2.64-54.71,28.19-15.36,22.92-14.52,58.2-5,83.28a132.85,132.85,0,0,1-12.12-39.24c-12.24-82.55,43.31-153,94.31-170.51-27.48-24-96.47-22.31-147.71,15.36-29.88,22-51.23,53.16-62.51,90.36,1.68-20.88,9.6-52.08,25.8-83.88-17.16,8.88-39,37-49.8,62.88-15.6,37.43-21,82.19-16.08,124.79.36,3.24.72,6.36,1.08,9.6,19.92,117.11,122,206.38,244.78,206.38C392.77,503.42,504,392.19,504,255,503.88,250.48,503.76,245.92,503.52,241.48Z"></path></svg>  [https://gianluca.statistica.it/](https://gianluca.statistica.it/)
<svg viewBox="0 0 512 512" style="position:relative;display:inline-block;top:.1em;fill:#EA7600;height:0.8em;" xmlns="http://www.w3.org/2000/svg">  <path d="M503.52,241.48c-.12-1.56-.24-3.12-.24-4.68v-.12l-.36-4.68v-.12a245.86,245.86,0,0,0-7.32-41.15c0-.12,0-.12-.12-.24l-1.08-4c-.12-.24-.12-.48-.24-.6-.36-1.2-.72-2.52-1.08-3.72-.12-.24-.12-.6-.24-.84-.36-1.2-.72-2.4-1.08-3.48-.12-.36-.24-.6-.36-1-.36-1.2-.72-2.28-1.2-3.48l-.36-1.08c-.36-1.08-.84-2.28-1.2-3.36a8.27,8.27,0,0,0-.36-1c-.48-1.08-.84-2.28-1.32-3.36-.12-.24-.24-.6-.36-.84-.48-1.2-1-2.28-1.44-3.48,0-.12-.12-.24-.12-.36-1.56-3.84-3.24-7.68-5-11.4l-.36-.72c-.48-1-.84-1.8-1.32-2.64-.24-.48-.48-1.08-.72-1.56-.36-.84-.84-1.56-1.2-2.4-.36-.6-.6-1.2-1-1.8s-.84-1.44-1.2-2.28c-.36-.6-.72-1.32-1.08-1.92s-.84-1.44-1.2-2.16a18.07,18.07,0,0,0-1.2-2c-.36-.72-.84-1.32-1.2-2s-.84-1.32-1.2-2-.84-1.32-1.2-1.92-.84-1.44-1.32-2.16a15.63,15.63,0,0,0-1.2-1.8L463.2,119a15.63,15.63,0,0,0-1.2-1.8c-.48-.72-1.08-1.56-1.56-2.28-.36-.48-.72-1.08-1.08-1.56l-1.8-2.52c-.36-.48-.6-.84-1-1.32-1-1.32-1.8-2.52-2.76-3.72a248.76,248.76,0,0,0-23.51-26.64A186.82,186.82,0,0,0,412,62.46c-4-3.48-8.16-6.72-12.48-9.84a162.49,162.49,0,0,0-24.6-15.12c-2.4-1.32-4.8-2.52-7.2-3.72a254,254,0,0,0-55.43-19.56c-1.92-.36-3.84-.84-5.64-1.2h-.12c-1-.12-1.8-.36-2.76-.48a236.35,236.35,0,0,0-38-4H255.14a234.62,234.62,0,0,0-45.48,5c-33.59,7.08-63.23,21.24-82.91,39-1.08,1-1.92,1.68-2.4,2.16l-.48.48H124l-.12.12.12-.12a.12.12,0,0,0,.12-.12l-.12.12a.42.42,0,0,1,.24-.12c14.64-8.76,34.92-16,49.44-19.56l5.88-1.44c.36-.12.84-.12,1.2-.24,1.68-.36,3.36-.72,5.16-1.08.24,0,.6-.12.84-.12C250.94,20.94,319.34,40.14,367,85.61a171.49,171.49,0,0,1,26.88,32.76c30.36,49.2,27.48,111.11,3.84,147.59-34.44,53-111.35,71.27-159,24.84a84.19,84.19,0,0,1-25.56-59,74.05,74.05,0,0,1,6.24-31c1.68-3.84,13.08-25.67,18.24-24.59-13.08-2.76-37.55,2.64-54.71,28.19-15.36,22.92-14.52,58.2-5,83.28a132.85,132.85,0,0,1-12.12-39.24c-12.24-82.55,43.31-153,94.31-170.51-27.48-24-96.47-22.31-147.71,15.36-29.88,22-51.23,53.16-62.51,90.36,1.68-20.88,9.6-52.08,25.8-83.88-17.16,8.88-39,37-49.8,62.88-15.6,37.43-21,82.19-16.08,124.79.36,3.24.72,6.36,1.08,9.6,19.92,117.11,122,206.38,244.78,206.38C392.77,503.42,504,392.19,504,255,503.88,250.48,503.76,245.92,503.52,241.48Z"></path></svg>  [https://egon.stats.ucl.ac.uk/research/statistics-health-economics/](https://egon.stats.ucl.ac.uk/research/statistics-health-economics/)
<svg viewBox="0 0 496 512" style="position:relative;display:inline-block;top:.1em;fill:black;height:0.8em;" xmlns="http://www.w3.org/2000/svg">  <path d="M165.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6zm-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3zm44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9zM244.8 8C106.1 8 0 113.3 0 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C428.2 457.8 496 362.9 496 252 496 113.3 383.5 8 244.8 8zM97.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1zm-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7zm32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1zm-11.4-14.7c-1.6 1-1.6 3.6 0 5.9 1.6 2.3 4.3 3.3 5.6 2.3 1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2z"></path></svg>  [https://github.com/giabaio](https://github.com/giabaio)
<svg viewBox="0 0 496 512" style="position:relative;display:inline-block;top:.1em;fill:black;height:0.8em;" xmlns="http://www.w3.org/2000/svg">  <path d="M165.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6zm-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3zm44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9zM244.8 8C106.1 8 0 113.3 0 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C428.2 457.8 496 362.9 496 252 496 113.3 383.5 8 244.8 8zM97.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1zm-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7zm32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1zm-11.4-14.7c-1.6 1-1.6 3.6 0 5.9 1.6 2.3 4.3 3.3 5.6 2.3 1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2z"></path></svg>  [https://github.com/StatisticsHealthEconomics](https://github.com/StatisticsHealthEconomics)
<svg viewBox="0 0 512 512" style="position:relative;display:inline-block;top:.1em;fill:#00acee;height:0.8em;" xmlns="http://www.w3.org/2000/svg">  <path d="M459.37 151.716c.325 4.548.325 9.097.325 13.645 0 138.72-105.583 298.558-298.558 298.558-59.452 0-114.68-17.219-161.137-47.106 8.447.974 16.568 1.299 25.34 1.299 49.055 0 94.213-16.568 130.274-44.832-46.132-.975-84.792-31.188-98.112-72.772 6.498.974 12.995 1.624 19.818 1.624 9.421 0 18.843-1.3 27.614-3.573-48.081-9.747-84.143-51.98-84.143-102.985v-1.299c13.969 7.797 30.214 12.67 47.431 13.319-28.264-18.843-46.781-51.005-46.781-87.391 0-19.492 5.197-37.36 14.294-52.954 51.655 63.675 129.3 105.258 216.365 109.807-1.624-7.797-2.599-15.918-2.599-24.04 0-57.828 46.782-104.934 104.934-104.934 30.213 0 57.502 12.67 76.67 33.137 23.715-4.548 46.456-13.32 66.599-25.34-7.798 24.366-24.366 44.833-46.132 57.827 21.117-2.273 41.584-8.122 60.426-16.243-14.292 20.791-32.161 39.308-52.628 54.253z"></path></svg>  [@gianlubaio](https://twitter.com/gianlubaio) &nbsp;&nbsp; <svg viewBox="0 0 448 512" style="position:relative;display:inline-block;top:.1em;fill:#563acc;height:0.8em;" xmlns="http://www.w3.org/2000/svg">  <path d="M433 179.11c0-97.2-63.71-125.7-63.71-125.7-62.52-28.7-228.56-28.4-290.48 0 0 0-63.72 28.5-63.72 125.7 0 115.7-6.6 259.4 105.63 289.1 40.51 10.7 75.32 13 103.33 11.4 50.81-2.8 79.32-18.1 79.32-18.1l-1.7-36.9s-36.31 11.4-77.12 10.1c-40.41-1.4-83-4.4-89.63-54a102.54 102.54 0 0 1-.9-13.9c85.63 20.9 158.65 9.1 178.75 6.7 56.12-6.7 105-41.3 111.23-72.9 9.8-49.8 9-121.5 9-121.5zm-75.12 125.2h-46.63v-114.2c0-49.7-64-51.6-64 6.9v62.5h-46.33V197c0-58.5-64-56.6-64-6.9v114.2H90.19c0-122.1-5.2-147.9 18.41-175 25.9-28.9 79.82-30.8 103.83 6.1l11.6 19.5 11.6-19.5c24.11-37.1 78.12-34.8 103.83-6.1 23.71 27.3 18.4 53 18.4 175z"></path></svg>  [@gianlubaio@mas.to](https://mas.to/@gianlubaio)      
]

<br>

### Pharmalex Webinar Series, Internetville

16 November 2022

.logo-stats[]

<p style="position: absolute; top:75%; left:2.5%; font-family: Nanum Pen Script; font-size:85%; text-decoration: none; color: #000; background: #ffc; display: block; height:6.3em; width:6.3em; padding: .5em; box-shadow: 5px 5px 7px rgba(33,33,33,.7); transform: rotate(-8deg); border-bottom-right-radius: 60px 5px;">Check out our departmental podcast "Random Talks" on Soundcloud!&nbsp;<a style=";" href="https://soundcloud.com/uclsound/sets/sample-space" title="Random Talks"><svg viewBox="0 0 640 512" style="position:relative;display:inline-block;top:.1em;fill:#FE5000;height:0.8em;bottom:1em;" xmlns="http://www.w3.org/2000/svg">
  <path d="M111.4 256.3l5.8 65-5.8 68.3c-.3 2.5-2.2 4.4-4.4 4.4s-4.2-1.9-4.2-4.4l-5.6-68.3 5.6-65c0-2.2 1.9-4.2 4.2-4.2 2.2 0 4.1 2 4.4 4.2zm21.4-45.6c-2.8 0-4.7 2.2-5 5l-5 105.6 5 68.3c.3 2.8 2.2 5 5 5 2.5 0 4.7-2.2 4.7-5l5.8-68.3-5.8-105.6c0-2.8-2.2-5-4.7-5zm25.5-24.1c-3.1 0-5.3 2.2-5.6 5.3l-4.4 130 4.4 67.8c.3 3.1 2.5 5.3 5.6 5.3 2.8 0 5.3-2.2 5.3-5.3l5.3-67.8-5.3-130c0-3.1-2.5-5.3-5.3-5.3zM7.2 283.2c-1.4 0-2.2 1.1-2.5 2.5L0 321.3l4.7 35c.3 1.4 1.1 2.5 2.5 2.5s2.2-1.1 2.5-2.5l5.6-35-5.6-35.6c-.3-1.4-1.1-2.5-2.5-2.5zm23.6-21.9c-1.4 0-2.5 1.1-2.5 2.5l-6.4 57.5 6.4 56.1c0 1.7 1.1 2.8 2.5 2.8s2.5-1.1 2.8-2.5l7.2-56.4-7.2-57.5c-.3-1.4-1.4-2.5-2.8-2.5zm25.3-11.4c-1.7 0-3.1 1.4-3.3 3.3L47 321.3l5.8 65.8c.3 1.7 1.7 3.1 3.3 3.1 1.7 0 3.1-1.4 3.1-3.1l6.9-65.8-6.9-68.1c0-1.9-1.4-3.3-3.1-3.3zm25.3-2.2c-1.9 0-3.6 1.4-3.6 3.6l-5.8 70 5.8 67.8c0 2.2 1.7 3.6 3.6 3.6s3.6-1.4 3.9-3.6l6.4-67.8-6.4-70c-.3-2.2-2-3.6-3.9-3.6zm241.4-110.9c-1.1-.8-2.8-1.4-4.2-1.4-2.2 0-4.2.8-5.6 1.9-1.9 1.7-3.1 4.2-3.3 6.7v.8l-3.3 176.7 1.7 32.5 1.7 31.7c.3 4.7 4.2 8.6 8.9 8.6s8.6-3.9 8.6-8.6l3.9-64.2-3.9-177.5c-.4-3-2-5.8-4.5-7.2zm-26.7 15.3c-1.4-.8-2.8-1.4-4.4-1.4s-3.1.6-4.4 1.4c-2.2 1.4-3.6 3.9-3.6 6.7l-.3 1.7-2.8 160.8s0 .3 3.1 65.6v.3c0 1.7.6 3.3 1.7 4.7 1.7 1.9 3.9 3.1 6.4 3.1 2.2 0 4.2-1.1 5.6-2.5 1.7-1.4 2.5-3.3 2.5-5.6l.3-6.7 3.1-58.6-3.3-162.8c-.3-2.8-1.7-5.3-3.9-6.7zm-111.4 22.5c-3.1 0-5.8 2.8-5.8 6.1l-4.4 140.6 4.4 67.2c.3 3.3 2.8 5.8 5.8 5.8 3.3 0 5.8-2.5 6.1-5.8l5-67.2-5-140.6c-.2-3.3-2.7-6.1-6.1-6.1zm376.7 62.8c-10.8 0-21.1 2.2-30.6 6.1-6.4-70.8-65.8-126.4-138.3-126.4-17.8 0-35 3.3-50.3 9.4-6.1 2.2-7.8 4.4-7.8 9.2v249.7c0 5 3.9 8.6 8.6 9.2h218.3c43.3 0 78.6-35 78.6-78.3.1-43.6-35.2-78.9-78.5-78.9zm-296.7-60.3c-4.2 0-7.5 3.3-7.8 7.8l-3.3 136.7 3.3 65.6c.3 4.2 3.6 7.5 7.8 7.5 4.2 0 7.5-3.3 7.5-7.5l3.9-65.6-3.9-136.7c-.3-4.5-3.3-7.8-7.5-7.8zm-53.6-7.8c-3.3 0-6.4 3.1-6.4 6.7l-3.9 145.3 3.9 66.9c.3 3.6 3.1 6.4 6.4 6.4 3.6 0 6.4-2.8 6.7-6.4l4.4-66.9-4.4-145.3c-.3-3.6-3.1-6.7-6.7-6.7zm26.7 3.4c-3.9 0-6.9 3.1-6.9 6.9L227 321.3l3.9 66.4c.3 3.9 3.1 6.9 6.9 6.9s6.9-3.1 6.9-6.9l4.2-66.4-4.2-141.7c0-3.9-3-6.9-6.9-6.9z"></path>
</svg></a>&nbsp;</p>

<p style="position: absolute; top:53%; left:6.5%; font-family: Nanum Pen Script; font-size:85%; text-decoration: none; color: #000; background: #ffc; display: block; height:6.3em; width:6.3em; padding: .5em; box-shadow: 5px 5px 7px rgba(33,33,33,.7); transform: rotate(6deg); border-bottom-right-radius: 60px 5px;">Follow our departmental social media accounts&nbsp;<a style=";" href="https://twitter.com/stats_ucl" title="@stats_UCL"><svg viewBox="0 0 512 512" style="position:relative;display:inline-block;top:.1em;fill:#00acee;height:0.8em;bottom:1em;" xmlns="http://www.w3.org/2000/svg">
  <path d="M459.37 151.716c.325 4.548.325 9.097.325 13.645 0 138.72-105.583 298.558-298.558 298.558-59.452 0-114.68-17.219-161.137-47.106 8.447.974 16.568 1.299 25.34 1.299 49.055 0 94.213-16.568 130.274-44.832-46.132-.975-84.792-31.188-98.112-72.772 6.498.974 12.995 1.624 19.818 1.624 9.421 0 18.843-1.3 27.614-3.573-48.081-9.747-84.143-51.98-84.143-102.985v-1.299c13.969 7.797 30.214 12.67 47.431 13.319-28.264-18.843-46.781-51.005-46.781-87.391 0-19.492 5.197-37.36 14.294-52.954 51.655 63.675 129.3 105.258 216.365 109.807-1.624-7.797-2.599-15.918-2.599-24.04 0-57.828 46.782-104.934 104.934-104.934 30.213 0 57.502 12.67 76.67 33.137 23.715-4.548 46.456-13.32 66.599-25.34-7.798 24.366-24.366 44.833-46.132 57.827 21.117-2.273 41.584-8.122 60.426-16.243-14.292 20.791-32.161 39.308-52.628 54.253z"></path>
</svg></a>&nbsp;&nbsp;<a style=";" href="https://www.linkedin.com/in/statistical-science-ucl-906b9a201" title="LinkedIn"><svg viewBox="0 0 448 512" style="position:relative;display:inline-block;top:.1em;fill:#2867b2;height:0.8em;bottom:1em;" xmlns="http://www.w3.org/2000/svg">
  <path d="M416 32H31.9C14.3 32 0 46.5 0 64.3v383.4C0 465.5 14.3 480 31.9 480H416c17.6 0 32-14.5 32-32.3V64.3c0-17.8-14.4-32.3-32-32.3zM135.4 416H69V202.2h66.5V416zm-33.2-243c-21.3 0-38.5-17.3-38.5-38.5S80.9 96 102.2 96c21.2 0 38.5 17.3 38.5 38.5 0 21.3-17.2 38.5-38.5 38.5zm282.1 243h-66.4V312c0-24.8-.5-56.7-34.5-56.7-34.6 0-39.9 27-39.9 54.9V416h-66.4V202.2h63.7v29.2h.9c8.9-16.8 30.6-34.5 62.9-34.5 67.2 0 79.7 44.3 79.7 101.9V416z"></path>
</svg></a>&nbsp;&nbsp;<a style=";" href="https://www.instagram.com/ucl.stats/" title="Instagram"><svg viewBox="0 0 448 512" style="position:relative;display:inline-block;top:.1em;fill:#2867b2;height:0.8em;bottom:1em;" xmlns="http://www.w3.org/2000/svg">
  <path d="M224.1 141c-63.6 0-114.9 51.3-114.9 114.9s51.3 114.9 114.9 114.9S339 319.5 339 255.9 287.7 141 224.1 141zm0 189.6c-41.1 0-74.7-33.5-74.7-74.7s33.5-74.7 74.7-74.7 74.7 33.5 74.7 74.7-33.6 74.7-74.7 74.7zm146.4-194.3c0 14.9-12 26.8-26.8 26.8-14.9 0-26.8-12-26.8-26.8s12-26.8 26.8-26.8 26.8 12 26.8 26.8zm76.1 27.2c-1.7-35.9-9.9-67.7-36.2-93.9-26.2-26.2-58-34.4-93.9-36.2-37-2.1-147.9-2.1-184.9 0-35.8 1.7-67.6 9.9-93.9 36.1s-34.4 58-36.2 93.9c-2.1 37-2.1 147.9 0 184.9 1.7 35.9 9.9 67.7 36.2 93.9s58 34.4 93.9 36.2c37 2.1 147.9 2.1 184.9 0 35.9-1.7 67.7-9.9 93.9-36.2 26.2-26.2 34.4-58 36.2-93.9 2.1-37 2.1-147.8 0-184.8zM398.8 388c-7.8 19.6-22.9 34.7-42.6 42.6-29.5 11.7-99.5 9-132.1 9s-102.7 2.6-132.1-9c-19.6-7.8-34.7-22.9-42.6-42.6-11.7-29.5-9-99.5-9-132.1s-2.6-102.7 9-132.1c7.8-19.6 22.9-34.7 42.6-42.6 29.5-11.7 99.5-9 132.1-9s102.7-2.6 132.1 9c19.6 7.8 34.7 22.9 42.6 42.6 11.7 29.5 9 99.5 9 132.1s2.7 102.7-9 132.1z"></path>
</svg></a>&nbsp;</p>

---

.my-footer[ 
&copy; Gianluca Baio (UCL) &nbsp; | &nbsp; &nbsp;<a style=";" href="https://twitter.com/giabaio" title="Follow me on Twitter"><svg viewBox="0 0 512 512" style="position:relative;display:inline-block;top:.1em;fill:#bcc0c4;height:0.8em;bottom:1em;" xmlns="http://www.w3.org/2000/svg">
  <path d="M459.37 151.716c.325 4.548.325 9.097.325 13.645 0 138.72-105.583 298.558-298.558 298.558-59.452 0-114.68-17.219-161.137-47.106 8.447.974 16.568 1.299 25.34 1.299 49.055 0 94.213-16.568 130.274-44.832-46.132-.975-84.792-31.188-98.112-72.772 6.498.974 12.995 1.624 19.818 1.624 9.421 0 18.843-1.3 27.614-3.573-48.081-9.747-84.143-51.98-84.143-102.985v-1.299c13.969 7.797 30.214 12.67 47.431 13.319-28.264-18.843-46.781-51.005-46.781-87.391 0-19.492 5.197-37.36 14.294-52.954 51.655 63.675 129.3 105.258 216.365 109.807-1.624-7.797-2.599-15.918-2.599-24.04 0-57.828 46.782-104.934 104.934-104.934 30.213 0 57.502 12.67 76.67 33.137 23.715-4.548 46.456-13.32 66.599-25.34-7.798 24.366-24.366 44.833-46.132 57.827 21.117-2.273 41.584-8.122 60.426-16.243-14.292 20.791-32.161 39.308-52.628 54.253z"></path>
</svg></a>&nbsp; &nbsp;<a style=";" href="https://mas.to/@gianlubaio" title="Follow me on Mastodon"><svg viewBox="0 0 448 512" style="position:relative;display:inline-block;top:.1em;fill:#bcc0c4;height:0.8em;bottom:1em;" xmlns="http://www.w3.org/2000/svg">
  <path d="M433 179.11c0-97.2-63.71-125.7-63.71-125.7-62.52-28.7-228.56-28.4-290.48 0 0 0-63.72 28.5-63.72 125.7 0 115.7-6.6 259.4 105.63 289.1 40.51 10.7 75.32 13 103.33 11.4 50.81-2.8 79.32-18.1 79.32-18.1l-1.7-36.9s-36.31 11.4-77.12 10.1c-40.41-1.4-83-4.4-89.63-54a102.54 102.54 0 0 1-.9-13.9c85.63 20.9 158.65 9.1 178.75 6.7 56.12-6.7 105-41.3 111.23-72.9 9.8-49.8 9-121.5 9-121.5zm-75.12 125.2h-46.63v-114.2c0-49.7-64-51.6-64 6.9v62.5h-46.33V197c0-58.5-64-56.6-64-6.9v114.2H90.19c0-122.1-5.2-147.9 18.41-175 25.9-28.9 79.82-30.8 103.83 6.1l11.6 19.5 11.6-19.5c24.11-37.1 78.12-34.8 103.83-6.1 23.71 27.3 18.4 53 18.4 175z"></path>
</svg></a>&nbsp; &nbsp;<a style=";" href="https://github.com/giabaio" title="Check out my repos"><svg viewBox="0 0 496 512" style="position:relative;display:inline-block;top:.1em;fill:#bcc0c4;height:0.8em;" xmlns="http://www.w3.org/2000/svg">
  <path d="M165.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6zm-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3zm44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9zM244.8 8C106.1 8 0 113.3 0 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C428.2 457.8 496 362.9 496 252 496 113.3 383.5 8 244.8 8zM97.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1zm-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7zm32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1zm-11.4-14.7c-1.6 1-1.6 3.6 0 5.9 1.6 2.3 4.3 3.3 5.6 2.3 1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2z"></path>
</svg></a>&nbsp; &nbsp;<a style=";" href="mailto:g.baio@ucl.ac.uk" title="Email me"><svg viewBox="0 0 512 512" style="position:relative;display:inline-block;top:.1em;fill:#bcc0c4;height:0.8em;" xmlns="http://www.w3.org/2000/svg">
  <path d="M502.3 190.8c3.9-3.1 9.7-.2 9.7 4.7V400c0 26.5-21.5 48-48 48H48c-26.5 0-48-21.5-48-48V195.6c0-5 5.7-7.8 9.7-4.7 22.4 17.4 52.1 39.5 154.1 113.6 21.1 15.4 56.7 47.8 92.2 47.6 35.7.3 72-32.8 92.3-47.6 102-74.1 131.6-96.3 154-113.7zM256 320c23.2.4 56.6-29.2 73.4-41.4 132.7-96.3 142.8-104.7 173.4-128.7 5.8-4.5 9.2-11.5 9.2-18.9v-19c0-26.5-21.5-48-48-48H48C21.5 64 0 85.5 0 112v19c0 7.4 3.4 14.3 9.2 18.9 30.6 23.9 40.7 32.4 173.4 128.7 16.8 12.2 50.2 41.8 73.4 41.4z"></path>
</svg></a>&nbsp; &nbsp;<a style=";" href="https://gianluca.statistica.it" title="Visit my website"><svg viewBox="0 0 512 512" style="position:relative;display:inline-block;top:.1em;fill:#bcc0c4;height:0.8em;" xmlns="http://www.w3.org/2000/svg">
  <path d="M503.52,241.48c-.12-1.56-.24-3.12-.24-4.68v-.12l-.36-4.68v-.12a245.86,245.86,0,0,0-7.32-41.15c0-.12,0-.12-.12-.24l-1.08-4c-.12-.24-.12-.48-.24-.6-.36-1.2-.72-2.52-1.08-3.72-.12-.24-.12-.6-.24-.84-.36-1.2-.72-2.4-1.08-3.48-.12-.36-.24-.6-.36-1-.36-1.2-.72-2.28-1.2-3.48l-.36-1.08c-.36-1.08-.84-2.28-1.2-3.36a8.27,8.27,0,0,0-.36-1c-.48-1.08-.84-2.28-1.32-3.36-.12-.24-.24-.6-.36-.84-.48-1.2-1-2.28-1.44-3.48,0-.12-.12-.24-.12-.36-1.56-3.84-3.24-7.68-5-11.4l-.36-.72c-.48-1-.84-1.8-1.32-2.64-.24-.48-.48-1.08-.72-1.56-.36-.84-.84-1.56-1.2-2.4-.36-.6-.6-1.2-1-1.8s-.84-1.44-1.2-2.28c-.36-.6-.72-1.32-1.08-1.92s-.84-1.44-1.2-2.16a18.07,18.07,0,0,0-1.2-2c-.36-.72-.84-1.32-1.2-2s-.84-1.32-1.2-2-.84-1.32-1.2-1.92-.84-1.44-1.32-2.16a15.63,15.63,0,0,0-1.2-1.8L463.2,119a15.63,15.63,0,0,0-1.2-1.8c-.48-.72-1.08-1.56-1.56-2.28-.36-.48-.72-1.08-1.08-1.56l-1.8-2.52c-.36-.48-.6-.84-1-1.32-1-1.32-1.8-2.52-2.76-3.72a248.76,248.76,0,0,0-23.51-26.64A186.82,186.82,0,0,0,412,62.46c-4-3.48-8.16-6.72-12.48-9.84a162.49,162.49,0,0,0-24.6-15.12c-2.4-1.32-4.8-2.52-7.2-3.72a254,254,0,0,0-55.43-19.56c-1.92-.36-3.84-.84-5.64-1.2h-.12c-1-.12-1.8-.36-2.76-.48a236.35,236.35,0,0,0-38-4H255.14a234.62,234.62,0,0,0-45.48,5c-33.59,7.08-63.23,21.24-82.91,39-1.08,1-1.92,1.68-2.4,2.16l-.48.48H124l-.12.12.12-.12a.12.12,0,0,0,.12-.12l-.12.12a.42.42,0,0,1,.24-.12c14.64-8.76,34.92-16,49.44-19.56l5.88-1.44c.36-.12.84-.12,1.2-.24,1.68-.36,3.36-.72,5.16-1.08.24,0,.6-.12.84-.12C250.94,20.94,319.34,40.14,367,85.61a171.49,171.49,0,0,1,26.88,32.76c30.36,49.2,27.48,111.11,3.84,147.59-34.44,53-111.35,71.27-159,24.84a84.19,84.19,0,0,1-25.56-59,74.05,74.05,0,0,1,6.24-31c1.68-3.84,13.08-25.67,18.24-24.59-13.08-2.76-37.55,2.64-54.71,28.19-15.36,22.92-14.52,58.2-5,83.28a132.85,132.85,0,0,1-12.12-39.24c-12.24-82.55,43.31-153,94.31-170.51-27.48-24-96.47-22.31-147.71,15.36-29.88,22-51.23,53.16-62.51,90.36,1.68-20.88,9.6-52.08,25.8-83.88-17.16,8.88-39,37-49.8,62.88-15.6,37.43-21,82.19-16.08,124.79.36,3.24.72,6.36,1.08,9.6,19.92,117.11,122,206.38,244.78,206.38C392.77,503.42,504,392.19,504,255,503.88,250.48,503.76,245.92,503.52,241.48Z"></path>
</svg></a>&nbsp;  &nbsp; | &nbsp; 
Bayesian HTA with RWEs  &nbsp; | &nbsp; Pharmalex Webinar Series  &nbsp; | &nbsp; 16 Nov 2022
]

---

# Health technology assessment (HTA)

## Objective 
- Combine .red[costs] and .blue[benefits] of a given intervention into a rational scheme for allocating resources 
> *Health technology assessment (HTA) is a method of evidence synthesis that
considers evidence regarding clinical effectiveness, safety, cost-effectiveness and,
when broadly applied, includes social, ethical, and legal aspects of the use of health
technologies. The precise balance of these inputs depends on the purpose of each
individual HTA. A major use of HTAs is in informing reimbursement and coverage
decisions, in which case HTAs should include benefit-harm assessment and economic
evaluation.* .alignright[<svg viewBox="0 0 384 512" xmlns="http://www.w3.org/2000/svg" style="height:1em;fill:currentColor;position:relative;display:inline-block;top:.1em;">  <g label="icon" id="layer6" groupmode="layer">    <path id="path2" d="M 120.19265,177.73779 C 123.18778,77.35076 64.277527,63.999998 64.277527,63.999998 v 31.26245 C 40.834519,83.611374 18.32863,81.929634 18.32863,81.929634 V 337.10903 c 0,0 98.10414,-11.41744 98.10414,84.40952 0,0 36.58424,-153.37442 248.86103,26.48145 0,-61.59342 0.37757,-216.93925 0.37757,-268.28471 C 169.9561,37.131382 120.1931,177.73779 120.1931,177.73779 Z m 187.20631,173.82056 -12.37599,-97.65441 h -0.448 l -40.72819,97.65441 h -17.55994 l -38.9362,-97.65441 h -0.448 l -14.17589,97.65441 h -43.87514 l 28.8015,-169.61925 h 43.42716 l 34.43518,90.6496 36.46566,-90.6496 h 43.87513 l 25.6817,169.61925 h -44.13938 z" style="stroke-width:0.0675239"></path>  </g></svg> [Luce et al, 2010](https://pubmed.ncbi.nlm.nih.gov/20579285/)]
   
   .small[(Quote stolen from a brilliant presentation by [Cynthia Iglesias](https://www.york.ac.uk/healthsciences/our-staff/cynthia-iglesias/))]

##  A relatively new discipline
- Basically becomes "a thing" in the 1970s
- Arguably, a **historical accident**... 
   - Economists take the lead in developing the main theory `$\Rightarrow$` *Health Economics*
   - But there's so much more to it (more on this later...)

## (Truly...) World-beating Britain
- Since its establishment, the **[National Institute for Health and Care Excellence](https://www.nice.org.uk/)** (originally: National Institute for Clinical Excellence, **NICE**), has gained prominence as the global powerhouse for HTA

---

# Health technology assessment (HTA)

**Objective**: Combine .red[costs] and .blue[benefits] of a given intervention into a rational scheme for allocating resources

---

**Objective**: Combine .red[costs] and .blue[benefits] of a given intervention into a rational scheme for allocating resources

---

**Objective**: Combine .red[costs] and .blue[benefits] of a given intervention into a rational scheme for allocating resources

---

**Objective**: Combine .red[costs] and .blue[benefits] of a given intervention into a rational scheme for allocating resources

---

# *To be or not to be (a Bayesian)?...**

.center[
.pull-left[
### Frequentist ("standard")
]
.pull-right[
### Bayesian 
]
]
.center[
<center><img src=./img/freq_bayes.png width='75%' title=''></center>
]

- A Bayesian only speaks one language: probability distributions to describe    
   - Sampling variability (relevant for observ.blue[***ed***] data)
   - Epistemic uncertainty (relevant for .orange[***un***]observ.orange[***able***] parameters + yet .magenta[***un***]observ.magenta[***ed***] future data)
--

- Contextual (="prior") information to be formally included in the construction of the model    
   - Almost irrelevant when evidence is "definitive" (large and consistent data)    
   - Crucial when data are sparse! (... But this isn't preposterous, is it?...)

---

.center[
.pull-left[
### Frequentist ("standard")
<center><img src=./img/two-stage.png width='610px' title=''></center>
]
]

---

.center[
.pull-left[
### Frequentist ("standard")
<center><img src=./img/two-stage.png width='610px' title=''></center>
]

.pull-right[
### Bayesian     
<center><img src=./img/integrated.png width='610px' title=''></center>
]
]

---

# Example

## Economic modelling for cancer drug: 3-state cancer "Markov model"

- **Very** prevalent in HTA
- Use "multistate" model to simulate progression of patients over a set of health states
- Often based on **time-to-event data** to estimate the "transition probabilities" `$\color{red}{\boldsymbol\lambda}$`

---

exclude: true
# Uncertainty analysis .small[(**P**robabilistic **S**ensitivity **A**nalysis)]

.three-column[
### Statistical model
<img src="./img/unnamed-chunk-3-1.png" style="display: block; margin: auto;" width="50%">
]
.three-column[
### Economic model
<span style="display:block; margin-top: 50px ;"></span>
.center[
Status quo 
]

.content-box-gray[
<center><img src=./img/3state-2.png width='100%' title='INCLUDE TEXT HERE'></center>
]

<span style="display:block; margin-top: 80px ;"></span>
.center[
New drug
]

.content-box-gray[
<center><img src=./img/3state-2.png width='100%' title='INCLUDE TEXT HERE'></center>
]
]
.three-column[
### Decision analysis

<table class=" lightable-classic table" style='font-family: "Arial Narrow", "Source Sans Pro", sans-serif; margin-left: auto; margin-right: auto; font-size: 12px; width: auto !important; margin-left: auto; margin-right: auto;'>
 <thead>
<tr><th style="padding-bottom:0; padding-left:3px;padding-right:3px;text-align: center; font-weight: bold; padding-right: 4px; padding-left: 4px; background-color: white !important;" colspan="2"><div style="border-bottom: 1px solid #111111; margin-bottom: -1px; ">Status quo</div></th></tr>
  <tr>
   <th style="text-align:left;background-color: white !important;text-align: center;"> Benefits </th>
   <th style="text-align:left;background-color: white !important;text-align: center;"> Costs </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;width: 35%; "> 741 </td>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;width: 35%; "> 670382.1 </td>
  </tr>
  <tr>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;width: 35%; "> 699 </td>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;width: 35%; "> 871273.3 </td>
  </tr>
  <tr>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;width: 35%; "> ... </td>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;width: 35%; "> ... </td>
  </tr>
  <tr>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;border-bottom: 1px solid;width: 35%; "> 726 </td>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;border-bottom: 1px solid;width: 35%; "> 425822.2 </td>
  </tr>
  <tr>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;font-weight: bold;width: 35%; "> 716.2 </td>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;font-weight: bold;width: 35%; "> 790381.2 </td>
  </tr>
</tbody>
</table>

<table class=" lightable-classic table" style='font-family: "Arial Narrow", "Source Sans Pro", sans-serif; margin-left: auto; margin-right: auto; font-size: 12px; width: auto !important; margin-left: auto; margin-right: auto;'>
 <thead>
<tr><th style="padding-bottom:0; padding-left:3px;padding-right:3px;text-align: center; font-weight: bold; padding-right: 4px; padding-left: 4px; background-color: white !important;" colspan="2"><div style="border-bottom: 1px solid #111111; margin-bottom: -1px; ">New drug</div></th></tr>
  <tr>
   <th style="text-align:left;background-color: white !important;text-align: center;"> Benefits </th>
   <th style="text-align:left;background-color: white !important;text-align: center;"> Costs </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;width: 35%; "> 732 </td>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;width: 35%; "> 1131978 </td>
  </tr>
  <tr>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;width: 35%; "> 664 </td>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;width: 35%; "> 1325654 </td>
  </tr>
  <tr>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;width: 35%; "> ... </td>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;width: 35%; "> ... </td>
  </tr>
  <tr>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;border-bottom: 1px solid;width: 35%; "> 811 </td>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;border-bottom: 1px solid;width: 35%; "> 766411.4 </td>
  </tr>
  <tr>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;font-weight: bold;width: 35%; "> 774.5 </td>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;font-weight: bold;width: 35%; "> 1066849.8 </td>
  </tr>
</tbody>
</table>

]

---

exclude: true
count: false
# Uncertainty analysis .small[(**P**robabilistic **S**ensitivity **A**nalysis)]

.three-column[
### Statistical model
<img src="./img/unnamed-chunk-7-1.png" style="display: block; margin: auto;" width="50%">
]
.three-column[
### Economic model
<span style="display:block; margin-top: 50px ;"></span>
.center[
Status quo 
]

.content-box-gray[
<center><img src=./img/3state-2.png width='100%' title='INCLUDE TEXT HERE'></center>
]

<span style="display:block; margin-top: 80px ;"></span>
.center[
New drug
]

.content-box-gray[
<center><img src=./img/3state-2.png width='100%' title='INCLUDE TEXT HERE'></center>
]
]
.three-column[
### Decision analysis
<table class=" lightable-classic table" style='font-family: "Arial Narrow", "Source Sans Pro", sans-serif; margin-left: auto; margin-right: auto; font-size: 12px; width: auto !important; margin-left: auto; margin-right: auto;'>
 <thead>
<tr><th style="padding-bottom:0; padding-left:3px;padding-right:3px;text-align: center; font-weight: bold; padding-right: 4px; padding-left: 4px; background-color: white !important;" colspan="2"><div style="border-bottom: 1px solid #111111; margin-bottom: -1px; ">Status quo</div></th></tr>
  <tr>
   <th style="text-align:left;background-color: white !important;text-align: center;"> Benefits </th>
   <th style="text-align:left;background-color: white !important;text-align: center;"> Costs </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;background-color: white !important;text-align: center;width: 35%; "> 741 </td>
   <td style="text-align:left;background-color: white !important;text-align: center;width: 35%; "> 670382.1 </td>
  </tr>
  <tr>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;width: 35%; "> 699 </td>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;width: 35%; "> 871273.3 </td>
  </tr>
  <tr>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;width: 35%; "> ... </td>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;width: 35%; "> ... </td>
  </tr>
  <tr>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;border-bottom: 1px solid;width: 35%; "> 726 </td>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;border-bottom: 1px solid;width: 35%; "> 425822.2 </td>
  </tr>
  <tr>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;font-weight: bold;width: 35%; "> 716.2 </td>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;font-weight: bold;width: 35%; "> 790381.2 </td>
  </tr>
</tbody>
</table>

<table class=" lightable-classic table" style='font-family: "Arial Narrow", "Source Sans Pro", sans-serif; margin-left: auto; margin-right: auto; font-size: 12px; width: auto !important; margin-left: auto; margin-right: auto;'>
 <thead>
<tr><th style="padding-bottom:0; padding-left:3px;padding-right:3px;text-align: center; font-weight: bold; padding-right: 4px; padding-left: 4px; background-color: white !important;" colspan="2"><div style="border-bottom: 1px solid #111111; margin-bottom: -1px; ">New drug</div></th></tr>
  <tr>
   <th style="text-align:left;background-color: white !important;text-align: center;"> Benefits </th>
   <th style="text-align:left;background-color: white !important;text-align: center;"> Costs </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;background-color: white !important;text-align: center;width: 35%; "> 732 </td>
   <td style="text-align:left;background-color: white !important;text-align: center;width: 35%; "> 1131978 </td>
  </tr>
  <tr>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;width: 35%; "> 664 </td>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;width: 35%; "> 1325654 </td>
  </tr>
  <tr>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;width: 35%; "> ... </td>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;width: 35%; "> ... </td>
  </tr>
  <tr>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;border-bottom: 1px solid;width: 35%; "> 811 </td>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;border-bottom: 1px solid;width: 35%; "> 766411.4 </td>
  </tr>
  <tr>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;font-weight: bold;width: 35%; "> 774.5 </td>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;font-weight: bold;width: 35%; "> 1066849.8 </td>
  </tr>
</tbody>
</table>

]

---

exclude: true
count: false
# Uncertainty analysis .small[(**P**robabilistic **S**ensitivity **A**nalysis)]

.three-column[
### Statistical model
<img src="./img/unnamed-chunk-10-1.png" style="display: block; margin: auto;" width="50%">
]
.three-column[
### Economic model
<span style="display:block; margin-top: 50px ;"></span>
.center[
Status quo 
]

.content-box-gray[
<center><img src=./img/3state-2.png width='100%' title='INCLUDE TEXT HERE'></center>
]

<span style="display:block; margin-top: 80px ;"></span>
.center[
New drug
]

.content-box-gray[
<center><img src=./img/3state-2.png width='100%' title='INCLUDE TEXT HERE'></center>
]
]
.three-column[
### Decision analysis
<table class=" lightable-classic table" style='font-family: "Arial Narrow", "Source Sans Pro", sans-serif; margin-left: auto; margin-right: auto; font-size: 12px; width: auto !important; margin-left: auto; margin-right: auto;'>
 <thead>
<tr><th style="padding-bottom:0; padding-left:3px;padding-right:3px;text-align: center; font-weight: bold; padding-right: 4px; padding-left: 4px; background-color: white !important;" colspan="2"><div style="border-bottom: 1px solid #111111; margin-bottom: -1px; ">Status quo</div></th></tr>
  <tr>
   <th style="text-align:left;background-color: white !important;text-align: center;"> Benefits </th>
   <th style="text-align:left;background-color: white !important;text-align: center;"> Costs </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;background-color: white !important;text-align: center;width: 35%; "> 741 </td>
   <td style="text-align:left;background-color: white !important;text-align: center;width: 35%; "> 670382.1 </td>
  </tr>
  <tr>
   <td style="text-align:left;background-color: white !important;text-align: center;width: 35%; "> 699 </td>
   <td style="text-align:left;background-color: white !important;text-align: center;width: 35%; "> 871273.3 </td>
  </tr>
  <tr>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;width: 35%; "> ... </td>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;width: 35%; "> ... </td>
  </tr>
  <tr>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;border-bottom: 1px solid;width: 35%; "> 726 </td>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;border-bottom: 1px solid;width: 35%; "> 425822.2 </td>
  </tr>
  <tr>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;font-weight: bold;width: 35%; "> 716.2 </td>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;font-weight: bold;width: 35%; "> 790381.2 </td>
  </tr>
</tbody>
</table>

<table class=" lightable-classic table" style='font-family: "Arial Narrow", "Source Sans Pro", sans-serif; margin-left: auto; margin-right: auto; font-size: 12px; width: auto !important; margin-left: auto; margin-right: auto;'>
 <thead>
<tr><th style="padding-bottom:0; padding-left:3px;padding-right:3px;text-align: center; font-weight: bold; padding-right: 4px; padding-left: 4px; background-color: white !important;" colspan="2"><div style="border-bottom: 1px solid #111111; margin-bottom: -1px; ">New drug</div></th></tr>
  <tr>
   <th style="text-align:left;background-color: white !important;text-align: center;"> Benefits </th>
   <th style="text-align:left;background-color: white !important;text-align: center;"> Costs </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;background-color: white !important;text-align: center;width: 35%; "> 732 </td>
   <td style="text-align:left;background-color: white !important;text-align: center;width: 35%; "> 1131978 </td>
  </tr>
  <tr>
   <td style="text-align:left;background-color: white !important;text-align: center;width: 35%; "> 664 </td>
   <td style="text-align:left;background-color: white !important;text-align: center;width: 35%; "> 1325654 </td>
  </tr>
  <tr>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;width: 35%; "> ... </td>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;width: 35%; "> ... </td>
  </tr>
  <tr>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;border-bottom: 1px solid;width: 35%; "> 811 </td>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;border-bottom: 1px solid;width: 35%; "> 766411.4 </td>
  </tr>
  <tr>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;font-weight: bold;width: 35%; "> 774.5 </td>
   <td style="text-align:left;background-color: white !important;text-align: center;color: white !important;font-weight: bold;width: 35%; "> 1066849.8 </td>
  </tr>
</tbody>
</table>

]

---

exclude: true
count: false
# Uncertainty analysis .small[(**P**robabilistic **S**ensitivity **A**nalysis)]

.three-column[
### Statistical model
<img src="./img/unnamed-chunk-13-1.png" style="display: block; margin: auto;" width="50%">
]
.three-column[
### Economic model
<span style="display:block; margin-top: 50px ;"></span>
.center[
Status quo 
]

.content-box-gray[
<center><img src=./img/3state-2.png width='100%' title='INCLUDE TEXT HERE'></center>
]

<span style="display:block; margin-top: 80px ;"></span>
.center[
New drug
]

.content-box-gray[
<center><img src=./img/3state-2.png width='100%' title='INCLUDE TEXT HERE'></center>
]
]
.three-column[
### Decision analysis
<table class=" lightable-classic table" style='font-family: "Arial Narrow", "Source Sans Pro", sans-serif; margin-left: auto; margin-right: auto; font-size: 12px; width: auto !important; margin-left: auto; margin-right: auto;'>
 <thead>
<tr><th style="padding-bottom:0; padding-left:3px;padding-right:3px;text-align: center; font-weight: bold; padding-right: 4px; padding-left: 4px; background-color: white !important;" colspan="2"><div style="border-bottom: 1px solid #111111; margin-bottom: -1px; ">Status quo</div></th></tr>
  <tr>
   <th style="text-align:left;background-color: white !important;text-align: center;"> Benefits </th>
   <th style="text-align:left;background-color: white !important;text-align: center;"> Costs </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;background-color: white !important;text-align: center;width: 35%; "> 741 </td>
   <td style="text-align:left;background-color: white !important;text-align: center;width: 35%; "> 670382.1 </td>
  </tr>
  <tr>
   <td style="text-align:left;background-color: white !important;text-align: center;width: 35%; "> 699 </td>
   <td style="text-align:left;background-color: white !important;text-align: center;width: 35%; "> 871273.3 </td>
  </tr>
  <tr>
   <td style="text-align:left;background-color: white !important;text-align: center;width: 35%; "> ... </td>
   <td style="text-align:left;background-color: white !important;text-align: center;width: 35%; "> ... </td>
  </tr>
  <tr>
   <td style="text-align:left;background-color: white !important;text-align: center;border-bottom: 1px solid;width: 35%; "> 726 </td>
   <td style="text-align:left;background-color: white !important;text-align: center;border-bottom: 1px solid;width: 35%; "> 425822.2 </td>
  </tr>
  <tr>
   <td style="text-align:left;background-color: white !important;text-align: center;font-weight: bold;width: 35%; "> 716.2 </td>
   <td style="text-align:left;background-color: white !important;text-align: center;font-weight: bold;width: 35%; "> 790381.2 </td>
  </tr>
</tbody>
</table>

<table class=" lightable-classic table" style='font-family: "Arial Narrow", "Source Sans Pro", sans-serif; margin-left: auto; margin-right: auto; font-size: 12px; width: auto !important; margin-left: auto; margin-right: auto;'>
 <thead>
<tr><th style="padding-bottom:0; padding-left:3px;padding-right:3px;text-align: center; font-weight: bold; padding-right: 4px; padding-left: 4px; background-color: white !important;" colspan="2"><div style="border-bottom: 1px solid #111111; margin-bottom: -1px; ">New drug</div></th></tr>
  <tr>
   <th style="text-align:left;background-color: white !important;text-align: center;"> Benefits </th>
   <th style="text-align:left;background-color: white !important;text-align: center;"> Costs </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;background-color: white !important;text-align: center;width: 35%; "> 732 </td>
   <td style="text-align:left;background-color: white !important;text-align: center;width: 35%; "> 1131978 </td>
  </tr>
  <tr>
   <td style="text-align:left;background-color: white !important;text-align: center;width: 35%; "> 664 </td>
   <td style="text-align:left;background-color: white !important;text-align: center;width: 35%; "> 1325654 </td>
  </tr>
  <tr>
   <td style="text-align:left;background-color: white !important;text-align: center;width: 35%; "> ... </td>
   <td style="text-align:left;background-color: white !important;text-align: center;width: 35%; "> ... </td>
  </tr>
  <tr>
   <td style="text-align:left;background-color: white !important;text-align: center;border-bottom: 1px solid;width: 35%; "> 811 </td>
   <td style="text-align:left;background-color: white !important;text-align: center;border-bottom: 1px solid;width: 35%; "> 766411.4 </td>
  </tr>
  <tr>
   <td style="text-align:left;background-color: white !important;text-align: center;font-weight: bold;width: 35%; "> 774.5 </td>
   <td style="text-align:left;background-color: white !important;text-align: center;font-weight: bold;width: 35%; "> 1066849.8 </td>
  </tr>
</tbody>
</table>

<span style="display:block; margin-top: -15px ;"></span>
`\begin{align}
\class{myblue}{\style{font-family:inherit;}{\text{ICER}}} & \class{myblue}{=} \frac{\class{myblue}{\style{font-family:inherit;}{\text{276468.6}}}}{\class{myblue}{\style{font-family:inherit;}{\text{58.3}}}}\\
& \class{myblue}{= \style{font-family:inherit;}{\text{6497.1}}}
\end{align}`
]

---

# The problem with survival analysis in HTA

Time-to-event data constitute the main outcome in a large number of HTAs (e.g. for cancer drugs &ndash)

<center><img src=./img/cake.gif width='100%' title=''></center>
]

## Data

1. We may (or may not!) access **individual level data** for "our" trial, but not for the competitors'    
2. The trial data have a very limited follow up, which implies large amount of censoring
   - This is often OK(-ish!) for "medical stats" analysis. But **HORRIBLE** for economic evaluation! `$\Rightarrow$` .blue[**Extrapolation**] (more on this later...)   
3. Often the data are manipulated by the stats team within the sponsor and the economic modellers only get summaries/estimates    
   - It is **ALWAYS** good to leave things to statisticians. But the modellers can (should?!) be statisticians too, so they could handle the data!...

]

---

count: false
<style type="text/css">
.pull-left-30 {
  float: left;
  width: 25%;
  margin-top: 10px;
}
.pull-right-60 {
  float: right;
  width: 65%;
  margin-top: 10px;
}
.pull-right-60 ~ * {
  content: "";
  display: table;
  clear: both;
}
</style>

# Survival analysis in HTA

<center><img src=./img/survival_hta1.png width='85%' title='The Kaplan-Meier curves are non-parametric statistics used to estimate the survival function from lifetime data. They resemble closely the observed data'></center>
]

<span style="display:block; margin-top: -50px ;"></span>
.small[<svg viewBox="0 0 581 512" xmlns="http://www.w3.org/2000/svg" style="height:1em;fill:currentColor;position:relative;display:inline-block;top:.1em;">  [ comment ]  <path d="M581 226.6C581 119.1 450.9 32 290.5 32S0 119.1 0 226.6C0 322.4 103.3 402 239.4 418.1V480h99.1v-61.5c24.3-2.7 47.6-7.4 69.4-13.9L448 480h112l-67.4-113.7c54.5-35.4 88.4-84.9 88.4-139.7zm-466.8 14.5c0-73.5 98.9-133 220.8-133s211.9 40.7 211.9 133c0 50.1-26.5 85-70.3 106.4-2.4-1.6-4.7-2.9-6.4-3.7-10.2-5.2-27.8-10.5-27.8-10.5s86.6-6.4 86.6-92.7-90.6-87.9-90.6-87.9h-199V361c-74.1-21.5-125.2-67.1-125.2-119.9zm225.1 38.3v-55.6c57.8 0 87.8-6.8 87.8 27.3 0 36.5-38.2 28.3-87.8 28.3zm-.9 72.5H365c10.8 0 18.9 11.7 24 19.2-16.1 1.9-33 2.8-50.6 2.9v-22.1z"></path></svg> [survHE](http://www.statistica.it/gianluca/software/survhe/)]     
.small[<svg viewBox="0 0 496 512" xmlns="http://www.w3.org/2000/svg" style="height:1em;fill:currentColor;position:relative;display:inline-block;top:.1em;">  [ comment ]  <path d="M165.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6zm-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3zm44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9zM244.8 8C106.1 8 0 113.3 0 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C428.2 457.8 496 362.9 496 252 496 113.3 383.5 8 244.8 8zM97.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1zm-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7zm32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1zm-11.4-14.7c-1.6 1-1.6 3.6 0 5.9 1.6 2.3 4.3 3.3 5.6 2.3 1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2z"></path></svg> [https://github.com/giabaio/survHE](https://github.com/giabaio/survHE)]
---

<center><img src=./img/survival_hta2.png width='85%' title='The median survival time is the time (on the x-axis) in correspondence of which the estimated survival curve is equal to 0.5. That is the point in the follow up at which 50% of the population have experienced the event'></center>
]

---

.pull-left-30[
.ubuntublue[**Mean** time:] `$\class{ubuntublue}{\displaystyle\int_0^\infty S(t)dt}$`
]

<center><img src=./img/survival_hta3.png width='85%' title='Conversely, the mean survival time gives the point on the x-axis that balances the distribution of the times. Because the underlying time distributions is generally skewed, mean and median times tend to be different'></center>
]

---

# Extrapolation

## A recipe for disaster?...

.pull-left[
<center><img src=./img/ristorante.png width='74%' title=''></center>
]
.pull-right[
<center><img src=./img/pizza.png width='185%' title=''></center>
]

---

## A recipe for disaster?...

---

## A recipe for disaster?...

---

## A recipe for disaster?...

---

Time-to-event data constitute the main outcome in a large number of HTAs (e.g. for cancer drugs)

<center><img src=./img/destinyschild.gif width='100%' title=''></center>
]

1. Which model is the "best fit" &ndash; how to judge that?

2. Is modelling even enough? (How to make the most of "external data")

3. Should you be Bayesians about this? 
   - (Spoiler alert: the answer is *always* Yes!... 😉)

]

---

## A recipe for disaster?...

.pull-left[
<img src="./img/unnamed-chunk-20-1.png" style="display: block; margin: auto;" width="100%" title="">
]

.pull-right[
<img src="./img/unnamed-chunk-21-1.png" style="display: block; margin: auto;" width="100%" title="">
]

- **NB**: Any \*IC can only tell us about model fit **for the observed data**!     
- Extrapolation (like missing data) is based on (virtually) untestable assumptions

---

## Why does this matter?...

.pull-left[
- Intrinsic/pathological uncertainty in the output of the (time-to-event) statistical modelling does carry through the entire process, all the way to the decision-making

<span style="display:block; margin-top: 30px ;"></span>
- It is not impossible (especially in cases involving new, innovative *immuno-oncology drugs*) that the observed data be extremely sparse and subject to high censoring

]

---

## Why does this matter?...

.pull-left[
- Intrinsic/pathological uncertainty in the output of the (time-to-event) statistical modelling does carry through the entire process, all the way to the decision-making

- In the case depicted here, the **best fitting** model responds by extrapolating a survival curve that implies .blue[Pr(alive at 15 years) > 0.5]
   
   - This may be obviously wrong/against expert or clinical opinion!
   
<svg viewBox="0 0 576 512" style="position:relative;display:inline-block;fill:red;height:1.7em;top:.45em;" xmlns="http://www.w3.org/2000/svg">  <path d="M569.517 440.013C587.975 472.007 564.806 512 527.94 512H48.054c-36.937 0-59.999-40.055-41.577-71.987L246.423 23.985c18.467-32.009 64.72-31.951 83.154 0l239.94 416.028zM288 354c-25.405 0-46 20.595-46 46s20.595 46 46 46 46-20.595 46-46-20.595-46-46-46zm-43.673-165.346l7.418 136c.347 6.364 5.609 11.346 11.982 11.346h48.546c6.373 0 11.635-4.982 11.982-11.346l7.418-136c.375-6.874-5.098-12.654-11.982-12.654h-63.383c-6.884 0-12.356 5.78-11.981 12.654z"></path></svg>   We need to **formally** and **quantitatively** consider what the implications of this uncertainty are on the decision-making process!

]

<center><img src=./img/surv2-1.png width='100%' title=''></center>
]

---

# Combining composite data sources

## Increasingly popular
- 13 Technology Assessments (TAs) in immuno-oncology in the period 2019-2021

- 7 formally included external data, of various form

- Sources used to support treatment effect waning (or lack of it) included:   
    - Other non-pivotal clinical trials and published sources with specific % of patients alive at a time point   
    - Flatiron (or other registries such as SEER)   
    - Clinical expert opinion ("soft" vs "hard" data... `$\Rightarrow$` more on this later)   
       - On % of patients surviving at a specific time `$t$`   
       - On clinical implausibility of hazards crossing and becoming higher for intervention vs comparator

## Challenges
- Heterogeneity/representativeness 
   - "Exchangeability"
   
- Afterthought vs plan ahead...

- KOL/Expert opinion/soft evidence: elicitation, formal modelling?...

---

# So how do we do this?...

## A couple of examples

1. ICD & Cardiac death
   
   - Relatively old-ish work
   - Combination of RCT and observational/registry data

<ol style="counter-reset: my-counter 1; color: lightgrey">
<li> "Blending" 
</li>
<ul> 
<li> Newer method </li>
<li> <b><i>Similar</i></b> to mixture cure models &ndash; but not quite the same...</li>
</ul>
</ol>

---

# Example: ICD & Cardiac death

.alignright[<svg viewBox="0 0 384 512" xmlns="http://www.w3.org/2000/svg" style="height:1em;fill:currentColor;position:relative;display:inline-block;top:.1em;">  <g label="icon" id="layer6" groupmode="layer">    <path id="path2" d="M 120.19265,177.73779 C 123.18778,77.35076 64.277527,63.999998 64.277527,63.999998 v 31.26245 C 40.834519,83.611374 18.32863,81.929634 18.32863,81.929634 V 337.10903 c 0,0 98.10414,-11.41744 98.10414,84.40952 0,0 36.58424,-153.37442 248.86103,26.48145 0,-61.59342 0.37757,-216.93925 0.37757,-268.28471 C 169.9561,37.131382 120.1931,177.73779 120.1931,177.73779 Z m 187.20631,173.82056 -12.37599,-97.65441 h -0.448 l -40.72819,97.65441 h -17.55994 l -38.9362,-97.65441 h -0.448 l -14.17589,97.65441 h -43.87514 l 28.8015,-169.61925 h 43.42716 l 34.43518,90.6496 36.46566,-90.6496 h 43.87513 l 25.6817,169.61925 h -44.13938 z" style="stroke-width:0.0675239"></path>  </g></svg> [Benaglia et al (2015)](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4847642/)]

## Set up/interventions

- ICD (Implantable Cardioverter Defibrillators) compared to anti-arrhythmic drugs (AAD) for prevention of sudden cardiac death in people with cardiac arrhythmia

## Data

- Individual data from cohort of 535 UK cardiac arrhythmia patients implanted with ICDs between 1991 and 2002

-  Meta-analysis of three (non-UK) RCTs providing published HRs
   – Relatively short-term follow-up: approximately 75% people, followed for less than 5 years, maximum 10 years
   
- UK population mortality statistics by age, sex, cause of death

## Objective

- Estimate the survival curve over the lifetime of ICD and AAD patients in UK

- Extrapolate the output to inform the wider economic model

---

## Basic idea

Use UK population data (matched by age/sex) to "**anchor**" the ICD population at risk

---

## Basic idea

Use UK population data (matched by age/sex) to "**anchor**" the ICD population at risk

- Perhaps the easiest way to do this is to relate the hazard between the two populations &ndash; eg **proportional hazard** (PH) model
<span style="display:block; margin-top: -20px ;"></span>
   `$$\class{myblue}{h_{\rm{ICD}}(t) = e^{\beta}h_{\rm{UK}}(t) \qquad \Leftrightarrow \qquad \HR = \frac{h_{\rm{ICD}}(t)}{h_{\rm{UK}}(t)} = e^{\beta} = \style{font-family:inherit;}{\text{Constant}}}$$`
   
<span style="display:block; margin-top: -20px ;"></span>
   
- Relatively easy to model &ndash; but probably very unrealistic!

- ICD patients are at (much?) greater risk of arrhythmia death 
   - If the proportion of deaths caused by arrythmia changes over time, we would induce bias, because we would be extrapolate a constant HR for all causes mortality
   
--

- Formally account for multiple mortality causes (.blue[**Poly-Weibull**] model <svg viewBox="0 0 384 512" xmlns="http://www.w3.org/2000/svg" style="height:1em;fill:currentColor;position:relative;display:inline-block;top:.1em;">  <g label="icon" id="layer6" groupmode="layer">    <path id="path2" d="M 120.19265,177.73779 C 123.18778,77.35076 64.277527,63.999998 64.277527,63.999998 v 31.26245 C 40.834519,83.611374 18.32863,81.929634 18.32863,81.929634 V 337.10903 c 0,0 98.10414,-11.41744 98.10414,84.40952 0,0 36.58424,-153.37442 248.86103,26.48145 0,-61.59342 0.37757,-216.93925 0.37757,-268.28471 C 169.9561,37.131382 120.1931,177.73779 120.1931,177.73779 Z m 187.20631,173.82056 -12.37599,-97.65441 h -0.448 l -40.72819,97.65441 h -17.55994 l -38.9362,-97.65441 h -0.448 l -14.17589,97.65441 h -43.87514 l 28.8015,-169.61925 h 43.42716 l 34.43518,90.6496 36.46566,-90.6496 h 43.87513 l 25.6817,169.61925 h -44.13938 z" style="stroke-width:0.0675239"></path>  </g></svg> [Demiris et al, 2015](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4456429/)):
`\begin{align}
\class{myblue}{h_{\rm{ICD}}(t)} &\class{myblue}{= h}_{\rm{\class{red}{ICD}}}^{\rm{\class{myblue}{arr}}}\class{myblue}{(t) + h}_{\rm{\class{red}{ICD}}}^{\rm{\class{myblue}{oth}}}\class{myblue}{(t)} \\
&\class{myblue}{=} \class{orange}{e^\beta} \class{myblue}{h^{\rm{arr}}}_{\rm{\class{blue}{UK}}}\class{myblue}{(t)} + \class{myblue}{h^{\rm{oth}}}_{\rm{\class{blue}{UK}}}\class{myblue}{(t)} \\
&\class{myblue}{=} \class{orange}{e^\beta}\class{myblue}{\alpha_1 \mu_1 t^{\alpha_1-1} + \alpha_2 \mu_2 t^{\alpha_2-1}}
\end{align}`

<span style="display:block; margin-top: -20px ;"></span>
- This assumes that
   -  Arrhythmia hazard is .orange[**proportional**] to matched UK population
   - Other causes hazard is **identical** to matched UK population

---

count: false
<style type="text/css">
.pull-left-nospace {
  float: left;
  width: 30%;
  margin-left: 40px;
  margin-top: 10px;
}
.pull-right-nospace {
  float: right;
  width: 60%;
  margin-top: 10px;
  margin-left: -80px;
}
.pull-right-nospace ~ * {
  content: "";
  display: table;
  clear: both;
}
</style>

# Example &ndash; ICD & Cardiac death

## Turning prior *information* into a prior *distribution*

- In the ICD case, age at entry is around 60 &ndash; we **know** that people won't survive more than 60 more years
   - Setting a prior for the scale `$\mu_i \sim \dunif(0,100)$` implies that the prior mean survival of the resulting Weibull distribution is 
   `$$\class{myblue}{\style{font-family:inherit;}{\text{expected survival time}}=\mu_i\Gamma\left(1+\frac{1}{\alpha}\right) < 60}$$`
   
- Can also include some knowledge on the shape `$\alpha$` and the coefficient `$\beta$` to limit their variations in reasonable ranges...

.pull-left-nospace[
<center><img src=./img/friends-gif.gif width='100%' title=''></center>
]
.pull-right-nospace[
- This isn't necessarily easy! 
   - You need to be friends with a statistician...

- Don't be lost in translation...
   - *Elicit* the actual .blue[**information**] and then map it onto a possible and reasonable .red[**distribution**]
   - Mapping changes with the mathematical properties of the underlying sampling distribution selected...
]

---

- Ignoring cause-specific mortality (simple .red[Weibull model]) results in larger bias, especially for females, mostly because the arrhythmia proportion of deaths does vary over time in that subgroup

---

# So how do we do this?...

## A couple of examples

<ol style="counter-reset: my-counter 0; color: lightgrey">
<li> ICD & Cardiac death
</li>   
<ul>
<li> Relatively old-ish work</li>
<li>Combination of RCT and observational/registry data</li>
</ul>
</ol>

<ol style="counter-reset: my-counter 1">
<li> "Blending" 
</li>
<ul> 
<li> Newer method </li>
<li> <b><i>Similar</i></b> to mixture cure models &ndash; but not quite the same...</li>
</ul>
</ol>

---

# Example

### Observed data ([NICE TA 174](https://www.nice.org.uk/guidance/ta174))
<center><img src=./img/surv1-1.png width='72%' title=''></center>

---

### Parametric fitting/extrapolation
<center><img src=./img/surv2-1.png width='72%' title=''></center>

---

## "Blended" survival curves

<span style="display:block; margin-top: -50px ;"></span>
### .alignright[<svg viewBox="0 0 512 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg">  <g groupmode="layer" id="layer6" label="icon">    <path style="stroke-width:0.07717" d="m 115.59247,222.50738 c -9.72601,0 -18.734334,7.07397 -24.986084,14.41694 -5.061076,5.95394 -7.591049,15.13238 -7.591049,22.77486 0,7.6402 2.870733,16.13879 7.931808,22.09274 6.152292,7.34297 12.778135,12.37522 22.603045,12.37522 8.83314,0 19.33786,-3.70766 25.389,-9.76105 6.1523,-6.05338 11.26931,-16.21737 11.26931,-25.04826 0,-8.73142 -2.39548,-19.18696 -8.54776,-25.24039 -6.15229,-6.15455 -17.33741,-11.61061 -26.06883,-11.61061 z m 174.6438,2.39944 c -10.12046,0 -18.35798,3.98058 -24.70919,11.93724 -5.15827,6.38628 -7.73967,13.66255 -7.73967,21.82831 0,8.27088 2.5814,15.59914 7.73967,21.98538 6.35064,7.95667 14.58873,11.93729 24.70919,11.93729 9.03149,0 16.67002,-3.25109 22.9212,-9.74241 6.25399,-6.59703 9.37846,-14.65259 9.37846,-24.18026 0,-9.422 -3.17646,-17.37639 -9.52708,-23.86775 -6.2495,-6.59477 -13.8428,-9.89721 -22.77258,-9.89721 z M 255.99998,7.9999981 C 119.03396,7.9999981 7.9999985,119.03394 7.9999985,255.99998 7.9999985,392.96602 119.03396,504 255.99998,504 392.96606,504 504,392.96602 504,255.99998 504,119.03394 392.96606,7.9999981 255.99998,7.9999981 Z M 197.17431,331.79065 h -45.93563 v -18.69931 c -4.66437,5.85506 -15.63869,11.82369 -20.40025,14.50172 -8.33473,4.66267 -19.65377,7.70067 -30.27265,7.70067 -17.167865,0 -32.44885,-5.425 -45.844653,-17.23173 -15.977193,-14.0903 -23.483472,-35.63322 -23.483472,-58.85447 0,-23.61798 8.186103,-42.47212 24.559999,-56.56242 12.999092,-11.21393 28.896046,-20.18949 45.766106,-20.18949 9.82264,0 21.93961,0.88269 30.57214,5.04865 4.96218,2.38076 13.84449,7.98548 19.10448,13.44381 v -98.50164 h 45.93564 v 229.34421 z m 157.21544,-21.13999 c -15.28099,17.58718 -36.66682,26.38303 -64.15348,26.38303 -27.58613,0 -49.01942,-8.79585 -64.30436,-26.38303 -12.60069,-14.44803 -18.90387,-32.19228 -18.90387,-53.23507 0,-18.94908 6.35065,-35.75132 19.0525,-50.4079 15.37988,-17.69285 37.26185,-26.54011 65.64139,-26.54011 26.09766,0 46.93585,8.84558 62.51694,26.54011 12.70185,14.44806 19.05247,31.66734 19.05247,51.66468 0.003,20.20586 -6.29921,37.53085 -18.90159,51.97886 z m 38.76899,-199.21794 c 5.35888,-5.35888 11.85868,-8.03746 19.49892,-8.03746 7.64024,0 14.14005,2.67858 19.49893,8.03746 5.35888,5.25945 8.03746,11.70955 8.03746,19.35034 0,7.73965 -2.67858,14.28864 -8.03746,19.64753 -5.25944,5.25944 -11.75925,7.8883 -19.49893,7.8883 -7.64247,0 -14.14227,-2.67861 -19.49892,-8.0375 -5.2594,-5.35885 -7.88829,-11.85871 -7.88829,-19.49892 -0.003,-7.6402 2.62889,-14.0903 7.88829,-19.35029 z m 43.91029,220.24265 H 388.24626 V 185.84118 h 48.82277 z" id="path2"></path>  </g></svg> [Che et al (2022)](https://doi.org/10.1177/0272989X221134545)]

### Consider two separate process

1. Driven *exclusively* by the .red[**observed data**]

- Similar to a "standard" HTA analysis &ndash; use this to estimate `$S_{obs}(t\mid\bm\theta_{obs})$`
   - Main objective: produce the **best** fit possible to the *observed* information 
   - **NB**: Unlike in a "standard" modelling exercise where the issue of overfitting is potentially critical, achieving a very close approximation to the observed dynamics has much less important implications in the case of blending

<ol style="counter-reset: my-counter 1;">
<li><p style="color:blue;"><b>"External" process</b></p>
   <ul>
   <li>Used to derive a separate survival curve, ${S_{ext}(t\mid\bm\theta_{ext})}$ to describe the <b><i>long-term</i></b> estimate for the survival probabilities</li>
   <li>Could use "hard" evidence (eg RWE/registries/cohort studies/etc)...</li>
   <li>...Or, purely subjective knwoledge elicited from experts (or both!)</li>
   </ul>
</li>
<b>NB</b>: Most likely need to use suitable statistical methods to "de-bias" the RWE 
<ul> 
<li> Propensity score, g-computation, ... </li>
</ul>
</ol>

.footnote[
<svg viewBox="0 0 496 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg">  <path d="M165.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6zm-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3zm44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9zM244.8 8C106.1 8 0 113.3 0 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C428.2 457.8 496 362.9 496 252 496 113.3 383.5 8 244.8 8zM97.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1zm-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7zm32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1zm-11.4-14.7c-1.6 1-1.6 3.6 0 5.9 1.6 2.3 4.3 3.3 5.6 2.3 1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2z"></path></svg> [`R` Code (for the paper)](https://github.com/StatisticsHealthEconomics/blendR-paper)
.alignright[<svg viewBox="0 0 496 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg">  <path d="M165.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6zm-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3zm44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9zM244.8 8C106.1 8 0 113.3 0 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C428.2 457.8 496 362.9 496 252 496 113.3 383.5 8 244.8 8zM97.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1zm-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7zm32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1zm-11.4-14.7c-1.6 1-1.6 3.6 0 5.9 1.6 2.3 4.3 3.3 5.6 2.3 1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2z"></path></svg> [`R` package `blendR`](https://github.com/StatisticsHealthEconomics/blendR)]
]

---

count: false
# Example &ndash; Blended survival curves
.panelset[
.panel[.panel-name[Stats]
Combine the two processes to obtain
`\begin{align}
\class{myblue}{S_{ble}(t\mid\bm\theta) = S_{obs}(t\mid\bm\theta_{obs})^{1-\pi(t; \alpha, \beta, a, b)}\times S_{ext}(t\mid\bm\theta_{ext})^{\pi(t;\alpha, \beta, a, b)}}
\end{align}`
<span style="display:block; margin-top: -20px ;"></span>
where:
<span style="display:block; margin-top: 30px ;"></span>

- `$\bm \theta = \{\bm \theta_{obs}, \bm \theta_{ext}, \alpha, \beta, a, b\}$` is the vector of .red[**model parameters**] 
<span style="display:block; margin-top: 30px ;"></span>

- `$\displaystyle \class{myblue}{\pi(t;\alpha,\beta,a,b) = \Pr\left(T\leq \frac{t-a}{b-a}\mid \alpha,  \beta\right) = F_{\text{Beta}}\left (\frac{t-a}{b-a}\mid \alpha, \beta \right)}$` is a .blue[**weight** function] controlling the extent to which `$S_{obs}(\cdot)$` and `$S_{ext}(\cdot)$` are blended together 
<span style="display:block; margin-top: 30px ;"></span>

- `$t \in [0,T^*]$`, is the .orange[**interval of times**] over which we want to perform our evaluation

<span style="display:block; margin-top: 50px ;"></span>
.content-box-beamer[
### 
**NB**: This is *not* the same as a "mixture cure model"!

- In MCM, one mixed survival curve (cured vs non cured individuals)
- In BSC, short- vs long-term processed modelled explicitly
]
]

<span style="display:block; margin-top: -10px ;"></span>
<center><img src=./img/blending_process-1.png width='75%' title=''></center>
]

<span style="display:block; margin-top: -10px ;"></span>
<center><img src=./img/betacdf_weight-1.png width='75%' title=''></center>
]

<span style="display:block; margin-top: -10px ;"></span>
<center><img src=./img/blending_weights-1.png width='75%' title=''></center>
]

]

---

---

exclude: true
class: hide-logo
count: false
background-image: url("img/limitations.gif")
background-size: cover

---

# Comments

- The main point of the "blending" procedure is to recognise that, sometimes (often...), the observed data are just not good enough to simultaneously
   
   1. Provide the best fit to the observed data
   2. Provide a reasonable extrapolation for the long-term survival

- Instead, we let the observed data tell us about the short-term survival **and** some external information tell us something about the long-term survival

- When external data/RWE are available, they should be leveraged

- BSCs allow to do this in a relatively straightforward way &ndash; **but** need to make sure the RWE are *exchangeable*/unbiased (as much as we possibly can...)
   
   - The "heavy-lifting" is done by the weight function that determines how the sources are blended together
   
   - This is based on (possibly untestable, but certainly open/upfront!) assumptions

- This combination of difference sources of evidence is naturally Bayesian

- Ultimately, we don't really care about the two components &ndash; rather we want to fully characterise the uncertainty in the blended curve
   
   - ... But to get that is simple algebra to combine the posterior distributions for `$S_{obs}(t\mid\bm\theta_{obs})$` and `$S_{ext}(t\mid\bm\theta_{ext})$`

---

# Conclusions

## Too much, too soon?

- Tension between early introduction in the market and reimbursment decisions on the back of promising, but extremely immature data

- Early plateau that doesn't materialise in later data cuts

- Divorce between "medical" and "economic" analysis

- Lancet papers are OK with estimating median survival time and HRs... Economic evaluations need extrapolation to estimate mean survival time
   
--

## All the help you can get

- Long-term data are ideal &ndash; if they're aligned with the population of interest and heterogeneity is manageable (and managed!)

- Often, even defining a comparator is a very complex operation and the market landscape is tricky...

- Registry data can produce information "in real time". But: at the price of confounding/need for confirmation periods (conditional registration/reimbursment?)

## Know what you know

- Some information is controversial and subjective and could bias the assessment. But: other simply isn't and we shouldn't be afraid to use it!

---

NULL

---