class: title-slide # Making robust decisions in health technology assessment: the value of value of information ## Gianluca Baio ### [Department of Statistical Science](https://www.ucl.ac.uk/statistics/) | University College London <br> .title-small[ <svg viewBox="0 0 512 512" style="position:relative;display:inline-block;top:.1em;fill:#00acee;height:0.8em;" xmlns="http://www.w3.org/2000/svg"> <path d="M502.3 190.8c3.9-3.1 9.7-.2 9.7 4.7V400c0 26.5-21.5 48-48 48H48c-26.5 0-48-21.5-48-48V195.6c0-5 5.7-7.8 9.7-4.7 22.4 17.4 52.1 39.5 154.1 113.6 21.1 15.4 56.7 47.8 92.2 47.6 35.7.3 72-32.8 92.3-47.6 102-74.1 131.6-96.3 154-113.7zM256 320c23.2.4 56.6-29.2 73.4-41.4 132.7-96.3 142.8-104.7 173.4-128.7 5.8-4.5 9.2-11.5 9.2-18.9v-19c0-26.5-21.5-48-48-48H48C21.5 64 0 85.5 0 112v19c0 7.4 3.4 14.3 9.2 18.9 30.6 23.9 40.7 32.4 173.4 128.7 16.8 12.2 50.2 41.8 73.4 41.4z"></path></svg> [g.baio@ucl.ac.uk](mailto:g.baio@ucl.ac.uk) <svg viewBox="0 0 512 512" style="position:relative;display:inline-block;top:.1em;fill:#EA7600;height:0.8em;" xmlns="http://www.w3.org/2000/svg"> <path d="M503.52,241.48c-.12-1.56-.24-3.12-.24-4.68v-.12l-.36-4.68v-.12a245.86,245.86,0,0,0-7.32-41.15c0-.12,0-.12-.12-.24l-1.08-4c-.12-.24-.12-.48-.24-.6-.36-1.2-.72-2.52-1.08-3.72-.12-.24-.12-.6-.24-.84-.36-1.2-.72-2.4-1.08-3.48-.12-.36-.24-.6-.36-1-.36-1.2-.72-2.28-1.2-3.48l-.36-1.08c-.36-1.08-.84-2.28-1.2-3.36a8.27,8.27,0,0,0-.36-1c-.48-1.08-.84-2.28-1.32-3.36-.12-.24-.24-.6-.36-.84-.48-1.2-1-2.28-1.44-3.48,0-.12-.12-.24-.12-.36-1.56-3.84-3.24-7.68-5-11.4l-.36-.72c-.48-1-.84-1.8-1.32-2.64-.24-.48-.48-1.08-.72-1.56-.36-.84-.84-1.56-1.2-2.4-.36-.6-.6-1.2-1-1.8s-.84-1.44-1.2-2.28c-.36-.6-.72-1.32-1.08-1.92s-.84-1.44-1.2-2.16a18.07,18.07,0,0,0-1.2-2c-.36-.72-.84-1.32-1.2-2s-.84-1.32-1.2-2-.84-1.32-1.2-1.92-.84-1.44-1.32-2.16a15.63,15.63,0,0,0-1.2-1.8L463.2,119a15.63,15.63,0,0,0-1.2-1.8c-.48-.72-1.08-1.56-1.56-2.28-.36-.48-.72-1.08-1.08-1.56l-1.8-2.52c-.36-.48-.6-.84-1-1.32-1-1.32-1.8-2.52-2.76-3.72a248.76,248.76,0,0,0-23.51-26.64A186.82,186.82,0,0,0,412,62.46c-4-3.48-8.16-6.72-12.48-9.84a162.49,162.49,0,0,0-24.6-15.12c-2.4-1.32-4.8-2.52-7.2-3.72a254,254,0,0,0-55.43-19.56c-1.92-.36-3.84-.84-5.64-1.2h-.12c-1-.12-1.8-.36-2.76-.48a236.35,236.35,0,0,0-38-4H255.14a234.62,234.62,0,0,0-45.48,5c-33.59,7.08-63.23,21.24-82.91,39-1.08,1-1.92,1.68-2.4,2.16l-.48.48H124l-.12.12.12-.12a.12.12,0,0,0,.12-.12l-.12.12a.42.42,0,0,1,.24-.12c14.64-8.76,34.92-16,49.44-19.56l5.88-1.44c.36-.12.84-.12,1.2-.24,1.68-.36,3.36-.72,5.16-1.08.24,0,.6-.12.84-.12C250.94,20.94,319.34,40.14,367,85.61a171.49,171.49,0,0,1,26.88,32.76c30.36,49.2,27.48,111.11,3.84,147.59-34.44,53-111.35,71.27-159,24.84a84.19,84.19,0,0,1-25.56-59,74.05,74.05,0,0,1,6.24-31c1.68-3.84,13.08-25.67,18.24-24.59-13.08-2.76-37.55,2.64-54.71,28.19-15.36,22.92-14.52,58.2-5,83.28a132.85,132.85,0,0,1-12.12-39.24c-12.24-82.55,43.31-153,94.31-170.51-27.48-24-96.47-22.31-147.71,15.36-29.88,22-51.23,53.16-62.51,90.36,1.68-20.88,9.6-52.08,25.8-83.88-17.16,8.88-39,37-49.8,62.88-15.6,37.43-21,82.19-16.08,124.79.36,3.24.72,6.36,1.08,9.6,19.92,117.11,122,206.38,244.78,206.38C392.77,503.42,504,392.19,504,255,503.88,250.48,503.76,245.92,503.52,241.48Z"></path></svg> [https://gianluca.statistica.it/](https://gianluca.statistica.it/) <svg viewBox="0 0 512 512" style="position:relative;display:inline-block;top:.1em;fill:#EA7600;height:0.8em;" xmlns="http://www.w3.org/2000/svg"> <path d="M503.52,241.48c-.12-1.56-.24-3.12-.24-4.68v-.12l-.36-4.68v-.12a245.86,245.86,0,0,0-7.32-41.15c0-.12,0-.12-.12-.24l-1.08-4c-.12-.24-.12-.48-.24-.6-.36-1.2-.72-2.52-1.08-3.72-.12-.24-.12-.6-.24-.84-.36-1.2-.72-2.4-1.08-3.48-.12-.36-.24-.6-.36-1-.36-1.2-.72-2.28-1.2-3.48l-.36-1.08c-.36-1.08-.84-2.28-1.2-3.36a8.27,8.27,0,0,0-.36-1c-.48-1.08-.84-2.28-1.32-3.36-.12-.24-.24-.6-.36-.84-.48-1.2-1-2.28-1.44-3.48,0-.12-.12-.24-.12-.36-1.56-3.84-3.24-7.68-5-11.4l-.36-.72c-.48-1-.84-1.8-1.32-2.64-.24-.48-.48-1.08-.72-1.56-.36-.84-.84-1.56-1.2-2.4-.36-.6-.6-1.2-1-1.8s-.84-1.44-1.2-2.28c-.36-.6-.72-1.32-1.08-1.92s-.84-1.44-1.2-2.16a18.07,18.07,0,0,0-1.2-2c-.36-.72-.84-1.32-1.2-2s-.84-1.32-1.2-2-.84-1.32-1.2-1.92-.84-1.44-1.32-2.16a15.63,15.63,0,0,0-1.2-1.8L463.2,119a15.63,15.63,0,0,0-1.2-1.8c-.48-.72-1.08-1.56-1.56-2.28-.36-.48-.72-1.08-1.08-1.56l-1.8-2.52c-.36-.48-.6-.84-1-1.32-1-1.32-1.8-2.52-2.76-3.72a248.76,248.76,0,0,0-23.51-26.64A186.82,186.82,0,0,0,412,62.46c-4-3.48-8.16-6.72-12.48-9.84a162.49,162.49,0,0,0-24.6-15.12c-2.4-1.32-4.8-2.52-7.2-3.72a254,254,0,0,0-55.43-19.56c-1.92-.36-3.84-.84-5.64-1.2h-.12c-1-.12-1.8-.36-2.76-.48a236.35,236.35,0,0,0-38-4H255.14a234.62,234.62,0,0,0-45.48,5c-33.59,7.08-63.23,21.24-82.91,39-1.08,1-1.92,1.68-2.4,2.16l-.48.48H124l-.12.12.12-.12a.12.12,0,0,0,.12-.12l-.12.12a.42.42,0,0,1,.24-.12c14.64-8.76,34.92-16,49.44-19.56l5.88-1.44c.36-.12.84-.12,1.2-.24,1.68-.36,3.36-.72,5.16-1.08.24,0,.6-.12.84-.12C250.94,20.94,319.34,40.14,367,85.61a171.49,171.49,0,0,1,26.88,32.76c30.36,49.2,27.48,111.11,3.84,147.59-34.44,53-111.35,71.27-159,24.84a84.19,84.19,0,0,1-25.56-59,74.05,74.05,0,0,1,6.24-31c1.68-3.84,13.08-25.67,18.24-24.59-13.08-2.76-37.55,2.64-54.71,28.19-15.36,22.92-14.52,58.2-5,83.28a132.85,132.85,0,0,1-12.12-39.24c-12.24-82.55,43.31-153,94.31-170.51-27.48-24-96.47-22.31-147.71,15.36-29.88,22-51.23,53.16-62.51,90.36,1.68-20.88,9.6-52.08,25.8-83.88-17.16,8.88-39,37-49.8,62.88-15.6,37.43-21,82.19-16.08,124.79.36,3.24.72,6.36,1.08,9.6,19.92,117.11,122,206.38,244.78,206.38C392.77,503.42,504,392.19,504,255,503.88,250.48,503.76,245.92,503.52,241.48Z"></path></svg> [https://egon.stats.ucl.ac.uk/research/statistics-health-economics/](https://egon.stats.ucl.ac.uk/research/statistics-health-economics/) <svg viewBox="0 0 496 512" style="position:relative;display:inline-block;top:.1em;fill:black;height:0.8em;" xmlns="http://www.w3.org/2000/svg"> <path d="M165.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6zm-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3zm44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9zM244.8 8C106.1 8 0 113.3 0 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C428.2 457.8 496 362.9 496 252 496 113.3 383.5 8 244.8 8zM97.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1zm-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7zm32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1zm-11.4-14.7c-1.6 1-1.6 3.6 0 5.9 1.6 2.3 4.3 3.3 5.6 2.3 1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2z"></path></svg> [https://github.com/giabaio](https://github.com/giabaio) <svg viewBox="0 0 496 512" style="position:relative;display:inline-block;top:.1em;fill:black;height:0.8em;" xmlns="http://www.w3.org/2000/svg"> <path d="M165.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6zm-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3zm44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9zM244.8 8C106.1 8 0 113.3 0 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C428.2 457.8 496 362.9 496 252 496 113.3 383.5 8 244.8 8zM97.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1zm-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7zm32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1zm-11.4-14.7c-1.6 1-1.6 3.6 0 5.9 1.6 2.3 4.3 3.3 5.6 2.3 1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2z"></path></svg> [https://github.com/StatisticsHealthEconomics](https://github.com/StatisticsHealthEconomics) <svg viewBox="0 0 512 512" style="position:relative;display:inline-block;top:.1em;fill:#00acee;height:0.8em;" xmlns="http://www.w3.org/2000/svg"> <path d="M459.37 151.716c.325 4.548.325 9.097.325 13.645 0 138.72-105.583 298.558-298.558 298.558-59.452 0-114.68-17.219-161.137-47.106 8.447.974 16.568 1.299 25.34 1.299 49.055 0 94.213-16.568 130.274-44.832-46.132-.975-84.792-31.188-98.112-72.772 6.498.974 12.995 1.624 19.818 1.624 9.421 0 18.843-1.3 27.614-3.573-48.081-9.747-84.143-51.98-84.143-102.985v-1.299c13.969 7.797 30.214 12.67 47.431 13.319-28.264-18.843-46.781-51.005-46.781-87.391 0-19.492 5.197-37.36 14.294-52.954 51.655 63.675 129.3 105.258 216.365 109.807-1.624-7.797-2.599-15.918-2.599-24.04 0-57.828 46.782-104.934 104.934-104.934 30.213 0 57.502 12.67 76.67 33.137 23.715-4.548 46.456-13.32 66.599-25.34-7.798 24.366-24.366 44.833-46.132 57.827 21.117-2.273 41.584-8.122 60.426-16.243-14.292 20.791-32.161 39.308-52.628 54.253z"></path></svg> [@gianlubaio](https://twitter.com/gianlubaio) &nbsp;&nbsp; <svg viewBox="0 0 448 512" style="position:relative;display:inline-block;top:.1em;fill:#563acc;height:0.8em;" xmlns="http://www.w3.org/2000/svg"> <path d="M433 179.11c0-97.2-63.71-125.7-63.71-125.7-62.52-28.7-228.56-28.4-290.48 0 0 0-63.72 28.5-63.72 125.7 0 115.7-6.6 259.4 105.63 289.1 40.51 10.7 75.32 13 103.33 11.4 50.81-2.8 79.32-18.1 79.32-18.1l-1.7-36.9s-36.31 11.4-77.12 10.1c-40.41-1.4-83-4.4-89.63-54a102.54 102.54 0 0 1-.9-13.9c85.63 20.9 158.65 9.1 178.75 6.7 56.12-6.7 105-41.3 111.23-72.9 9.8-49.8 9-121.5 9-121.5zm-75.12 125.2h-46.63v-114.2c0-49.7-64-51.6-64 6.9v62.5h-46.33V197c0-58.5-64-56.6-64-6.9v114.2H90.19c0-122.1-5.2-147.9 18.41-175 25.9-28.9 79.82-30.8 103.83 6.1l11.6 19.5 11.6-19.5c24.11-37.1 78.12-34.8 103.83-6.1 23.71 27.3 18.4 53 18.4 175z"></path></svg> [@gianlubaio@mas.to](https://mas.to/@gianlubaio) ] <br> ### VVSOR Annual Meeting 2023, Utrecht (Netherlands) <!-- Can also separate the various components of the extra argument 'params', eg as in ### VVSOR Annual Meeting 2023, Utrecht (Netherlands), 23 March 2023, Decision making in HTA with VoI, TRUE, https://gianluca.statistica.it/slides/vvsor-2023 --> 23 March 2023 <!-- Adds a departmental logo on the right-bottom corner (Only with 'ucl-stats') --> .logo-stats[] <!-- Can also add sticky notes: --> <p style="position: absolute; top:75%; left:2.5%; font-family: Nanum Pen Script; font-size:85%; text-decoration: none; color: #000; background: #ffc; display: block; height:6.3em; width:6.3em; padding: .5em; box-shadow: 5px 5px 7px rgba(33,33,33,.7); transform: rotate(-8deg); border-bottom-right-radius: 60px 5px;">Check out our departmental podcast "Random Talks" on Soundcloud!&nbsp;<a style=";" href="https://soundcloud.com/uclsound/sets/sample-space" title="Random Talks"><svg viewBox="0 0 640 512" style="position:relative;display:inline-block;top:.1em;fill:#FE5000;height:0.8em;bottom:1em;" xmlns="http://www.w3.org/2000/svg"> <path d="M111.4 256.3l5.8 65-5.8 68.3c-.3 2.5-2.2 4.4-4.4 4.4s-4.2-1.9-4.2-4.4l-5.6-68.3 5.6-65c0-2.2 1.9-4.2 4.2-4.2 2.2 0 4.1 2 4.4 4.2zm21.4-45.6c-2.8 0-4.7 2.2-5 5l-5 105.6 5 68.3c.3 2.8 2.2 5 5 5 2.5 0 4.7-2.2 4.7-5l5.8-68.3-5.8-105.6c0-2.8-2.2-5-4.7-5zm25.5-24.1c-3.1 0-5.3 2.2-5.6 5.3l-4.4 130 4.4 67.8c.3 3.1 2.5 5.3 5.6 5.3 2.8 0 5.3-2.2 5.3-5.3l5.3-67.8-5.3-130c0-3.1-2.5-5.3-5.3-5.3zM7.2 283.2c-1.4 0-2.2 1.1-2.5 2.5L0 321.3l4.7 35c.3 1.4 1.1 2.5 2.5 2.5s2.2-1.1 2.5-2.5l5.6-35-5.6-35.6c-.3-1.4-1.1-2.5-2.5-2.5zm23.6-21.9c-1.4 0-2.5 1.1-2.5 2.5l-6.4 57.5 6.4 56.1c0 1.7 1.1 2.8 2.5 2.8s2.5-1.1 2.8-2.5l7.2-56.4-7.2-57.5c-.3-1.4-1.4-2.5-2.8-2.5zm25.3-11.4c-1.7 0-3.1 1.4-3.3 3.3L47 321.3l5.8 65.8c.3 1.7 1.7 3.1 3.3 3.1 1.7 0 3.1-1.4 3.1-3.1l6.9-65.8-6.9-68.1c0-1.9-1.4-3.3-3.1-3.3zm25.3-2.2c-1.9 0-3.6 1.4-3.6 3.6l-5.8 70 5.8 67.8c0 2.2 1.7 3.6 3.6 3.6s3.6-1.4 3.9-3.6l6.4-67.8-6.4-70c-.3-2.2-2-3.6-3.9-3.6zm241.4-110.9c-1.1-.8-2.8-1.4-4.2-1.4-2.2 0-4.2.8-5.6 1.9-1.9 1.7-3.1 4.2-3.3 6.7v.8l-3.3 176.7 1.7 32.5 1.7 31.7c.3 4.7 4.2 8.6 8.9 8.6s8.6-3.9 8.6-8.6l3.9-64.2-3.9-177.5c-.4-3-2-5.8-4.5-7.2zm-26.7 15.3c-1.4-.8-2.8-1.4-4.4-1.4s-3.1.6-4.4 1.4c-2.2 1.4-3.6 3.9-3.6 6.7l-.3 1.7-2.8 160.8s0 .3 3.1 65.6v.3c0 1.7.6 3.3 1.7 4.7 1.7 1.9 3.9 3.1 6.4 3.1 2.2 0 4.2-1.1 5.6-2.5 1.7-1.4 2.5-3.3 2.5-5.6l.3-6.7 3.1-58.6-3.3-162.8c-.3-2.8-1.7-5.3-3.9-6.7zm-111.4 22.5c-3.1 0-5.8 2.8-5.8 6.1l-4.4 140.6 4.4 67.2c.3 3.3 2.8 5.8 5.8 5.8 3.3 0 5.8-2.5 6.1-5.8l5-67.2-5-140.6c-.2-3.3-2.7-6.1-6.1-6.1zm376.7 62.8c-10.8 0-21.1 2.2-30.6 6.1-6.4-70.8-65.8-126.4-138.3-126.4-17.8 0-35 3.3-50.3 9.4-6.1 2.2-7.8 4.4-7.8 9.2v249.7c0 5 3.9 8.6 8.6 9.2h218.3c43.3 0 78.6-35 78.6-78.3.1-43.6-35.2-78.9-78.5-78.9zm-296.7-60.3c-4.2 0-7.5 3.3-7.8 7.8l-3.3 136.7 3.3 65.6c.3 4.2 3.6 7.5 7.8 7.5 4.2 0 7.5-3.3 7.5-7.5l3.9-65.6-3.9-136.7c-.3-4.5-3.3-7.8-7.5-7.8zm-53.6-7.8c-3.3 0-6.4 3.1-6.4 6.7l-3.9 145.3 3.9 66.9c.3 3.6 3.1 6.4 6.4 6.4 3.6 0 6.4-2.8 6.7-6.4l4.4-66.9-4.4-145.3c-.3-3.6-3.1-6.7-6.7-6.7zm26.7 3.4c-3.9 0-6.9 3.1-6.9 6.9L227 321.3l3.9 66.4c.3 3.9 3.1 6.9 6.9 6.9s6.9-3.1 6.9-6.9l4.2-66.4-4.2-141.7c0-3.9-3-6.9-6.9-6.9z"></path> </svg></a>&nbsp;</p> <p style="position: absolute; top:53%; left:6.5%; font-family: Nanum Pen Script; font-size:85%; text-decoration: none; color: #000; background: #ffc; display: block; height:6.3em; width:6.3em; padding: .5em; box-shadow: 5px 5px 7px rgba(33,33,33,.7); transform: rotate(6deg); border-bottom-right-radius: 60px 5px;">Follow our departmental social media accounts&nbsp;<a style=";" href="https://twitter.com/stats_ucl" title="@stats_UCL"><svg viewBox="0 0 512 512" style="position:relative;display:inline-block;top:.1em;fill:#00acee;height:0.8em;bottom:1em;" xmlns="http://www.w3.org/2000/svg"> <path d="M459.37 151.716c.325 4.548.325 9.097.325 13.645 0 138.72-105.583 298.558-298.558 298.558-59.452 0-114.68-17.219-161.137-47.106 8.447.974 16.568 1.299 25.34 1.299 49.055 0 94.213-16.568 130.274-44.832-46.132-.975-84.792-31.188-98.112-72.772 6.498.974 12.995 1.624 19.818 1.624 9.421 0 18.843-1.3 27.614-3.573-48.081-9.747-84.143-51.98-84.143-102.985v-1.299c13.969 7.797 30.214 12.67 47.431 13.319-28.264-18.843-46.781-51.005-46.781-87.391 0-19.492 5.197-37.36 14.294-52.954 51.655 63.675 129.3 105.258 216.365 109.807-1.624-7.797-2.599-15.918-2.599-24.04 0-57.828 46.782-104.934 104.934-104.934 30.213 0 57.502 12.67 76.67 33.137 23.715-4.548 46.456-13.32 66.599-25.34-7.798 24.366-24.366 44.833-46.132 57.827 21.117-2.273 41.584-8.122 60.426-16.243-14.292 20.791-32.161 39.308-52.628 54.253z"></path> </svg></a>&nbsp;&nbsp;<a style=";" href="https://www.linkedin.com/in/statistical-science-ucl-906b9a201" title="LinkedIn"><svg viewBox="0 0 448 512" style="position:relative;display:inline-block;top:.1em;fill:#2867b2;height:0.8em;bottom:1em;" xmlns="http://www.w3.org/2000/svg"> <path d="M416 32H31.9C14.3 32 0 46.5 0 64.3v383.4C0 465.5 14.3 480 31.9 480H416c17.6 0 32-14.5 32-32.3V64.3c0-17.8-14.4-32.3-32-32.3zM135.4 416H69V202.2h66.5V416zm-33.2-243c-21.3 0-38.5-17.3-38.5-38.5S80.9 96 102.2 96c21.2 0 38.5 17.3 38.5 38.5 0 21.3-17.2 38.5-38.5 38.5zm282.1 243h-66.4V312c0-24.8-.5-56.7-34.5-56.7-34.6 0-39.9 27-39.9 54.9V416h-66.4V202.2h63.7v29.2h.9c8.9-16.8 30.6-34.5 62.9-34.5 67.2 0 79.7 44.3 79.7 101.9V416z"></path> </svg></a>&nbsp;&nbsp;<a style=";" href="https://www.instagram.com/ucl.stats/" title="Instagram"><svg viewBox="0 0 448 512" style="position:relative;display:inline-block;top:.1em;fill:#2867b2;height:0.8em;bottom:1em;" xmlns="http://www.w3.org/2000/svg"> <path d="M224.1 141c-63.6 0-114.9 51.3-114.9 114.9s51.3 114.9 114.9 114.9S339 319.5 339 255.9 287.7 141 224.1 141zm0 189.6c-41.1 0-74.7-33.5-74.7-74.7s33.5-74.7 74.7-74.7 74.7 33.5 74.7 74.7-33.6 74.7-74.7 74.7zm146.4-194.3c0 14.9-12 26.8-26.8 26.8-14.9 0-26.8-12-26.8-26.8s12-26.8 26.8-26.8 26.8 12 26.8 26.8zm76.1 27.2c-1.7-35.9-9.9-67.7-36.2-93.9-26.2-26.2-58-34.4-93.9-36.2-37-2.1-147.9-2.1-184.9 0-35.8 1.7-67.6 9.9-93.9 36.1s-34.4 58-36.2 93.9c-2.1 37-2.1 147.9 0 184.9 1.7 35.9 9.9 67.7 36.2 93.9s58 34.4 93.9 36.2c37 2.1 147.9 2.1 184.9 0 35.9-1.7 67.7-9.9 93.9-36.2 26.2-26.2 34.4-58 36.2-93.9 2.1-37 2.1-147.8 0-184.8zM398.8 388c-7.8 19.6-22.9 34.7-42.6 42.6-29.5 11.7-99.5 9-132.1 9s-102.7 2.6-132.1-9c-19.6-7.8-34.7-22.9-42.6-42.6-11.7-29.5-9-99.5-9-132.1s-2.6-102.7 9-132.1c7.8-19.6 22.9-34.7 42.6-42.6 29.5-11.7 99.5-9 132.1-9s102.7-2.6 132.1 9c19.6 7.8 34.7 22.9 42.6 42.6 11.7 29.5 9 99.5 9 132.1s2.7 102.7-9 132.1z"></path> </svg></a>&nbsp;</p> <!-- This adds a footer (optional and with other possibilities...) For example, can use <span><a href="https://gianluca.statistica.it/"><img src="assets/logo.png" title="Go home" width="2.0%"></a></span> to add the Samp Tux log, or change the bottom space to align the text, etc .footer-left[ <span style="position: relative; bottom: 0px; color: #D5D5D5;"> &nbsp; &copy; Gianluca Baio (UCL)</span> ] --> --- layout: true <!-- this selects the footer from a template, depending on the value passed in the YAML of the main presentation --> <style type="text/css"> /* Redefines the my-footer class to have the footer centered and smaller */ .my-footer { position: absolute; bottom: 0px; text-align: center; height: 20px; width: 100%; color: #D5D5D5; font-size: 62%; background-image: linear-gradient(to right, rgb(0, 44, 83), rgb(100, 125, 151)); /* vertical alignment (as to play nicely with page numbering!) */ margin-bottom: -2.0px; } /* Redefines the slide numbering (smaller font), to align with the footer */ .remark-slide-number { color: #FFFFFF; opacity: 1; font-size: 62%; /* Customise position of numbering */ margin-bottom: -8.5px; margin-right: -5.0px; } </style> .my-footer[ &copy; Gianluca Baio (UCL) &nbsp; | &nbsp; &nbsp;<a style="border-bottom: 0px;;" href="https://twitter.com/giabaio" title="Follow me on Twitter"><svg viewBox="0 0 512 512" style="position:relative;display:inline-block;top:.1em;fill:#bcc0c4;height:0.8em;bottom:1em;" xmlns="http://www.w3.org/2000/svg"> <path d="M459.37 151.716c.325 4.548.325 9.097.325 13.645 0 138.72-105.583 298.558-298.558 298.558-59.452 0-114.68-17.219-161.137-47.106 8.447.974 16.568 1.299 25.34 1.299 49.055 0 94.213-16.568 130.274-44.832-46.132-.975-84.792-31.188-98.112-72.772 6.498.974 12.995 1.624 19.818 1.624 9.421 0 18.843-1.3 27.614-3.573-48.081-9.747-84.143-51.98-84.143-102.985v-1.299c13.969 7.797 30.214 12.67 47.431 13.319-28.264-18.843-46.781-51.005-46.781-87.391 0-19.492 5.197-37.36 14.294-52.954 51.655 63.675 129.3 105.258 216.365 109.807-1.624-7.797-2.599-15.918-2.599-24.04 0-57.828 46.782-104.934 104.934-104.934 30.213 0 57.502 12.67 76.67 33.137 23.715-4.548 46.456-13.32 66.599-25.34-7.798 24.366-24.366 44.833-46.132 57.827 21.117-2.273 41.584-8.122 60.426-16.243-14.292 20.791-32.161 39.308-52.628 54.253z"></path> </svg></a>&nbsp; &nbsp;<a style="border-bottom: 0px;;" href="https://mas.to/@gianlubaio" title="Follow me on Mastodon"><svg viewBox="0 0 448 512" style="position:relative;display:inline-block;top:.1em;fill:#bcc0c4;height:0.8em;bottom:1em;" xmlns="http://www.w3.org/2000/svg"> <path d="M433 179.11c0-97.2-63.71-125.7-63.71-125.7-62.52-28.7-228.56-28.4-290.48 0 0 0-63.72 28.5-63.72 125.7 0 115.7-6.6 259.4 105.63 289.1 40.51 10.7 75.32 13 103.33 11.4 50.81-2.8 79.32-18.1 79.32-18.1l-1.7-36.9s-36.31 11.4-77.12 10.1c-40.41-1.4-83-4.4-89.63-54a102.54 102.54 0 0 1-.9-13.9c85.63 20.9 158.65 9.1 178.75 6.7 56.12-6.7 105-41.3 111.23-72.9 9.8-49.8 9-121.5 9-121.5zm-75.12 125.2h-46.63v-114.2c0-49.7-64-51.6-64 6.9v62.5h-46.33V197c0-58.5-64-56.6-64-6.9v114.2H90.19c0-122.1-5.2-147.9 18.41-175 25.9-28.9 79.82-30.8 103.83 6.1l11.6 19.5 11.6-19.5c24.11-37.1 78.12-34.8 103.83-6.1 23.71 27.3 18.4 53 18.4 175z"></path> </svg></a>&nbsp; &nbsp;<a style="border-bottom: 0px;;" href="https://github.com/giabaio" title="Check out my repos"><svg viewBox="0 0 496 512" style="position:relative;display:inline-block;top:.1em;fill:#bcc0c4;height:0.8em;" xmlns="http://www.w3.org/2000/svg"> <path d="M165.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6zm-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3zm44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9zM244.8 8C106.1 8 0 113.3 0 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C428.2 457.8 496 362.9 496 252 496 113.3 383.5 8 244.8 8zM97.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1zm-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7zm32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1zm-11.4-14.7c-1.6 1-1.6 3.6 0 5.9 1.6 2.3 4.3 3.3 5.6 2.3 1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2z"></path> </svg></a>&nbsp; &nbsp;<a style="border-bottom: 0px;;" href="mailto:g.baio@ucl.ac.uk" title="Email me"><svg viewBox="0 0 512 512" style="position:relative;display:inline-block;top:.1em;fill:#bcc0c4;height:0.8em;" xmlns="http://www.w3.org/2000/svg"> <path d="M502.3 190.8c3.9-3.1 9.7-.2 9.7 4.7V400c0 26.5-21.5 48-48 48H48c-26.5 0-48-21.5-48-48V195.6c0-5 5.7-7.8 9.7-4.7 22.4 17.4 52.1 39.5 154.1 113.6 21.1 15.4 56.7 47.8 92.2 47.6 35.7.3 72-32.8 92.3-47.6 102-74.1 131.6-96.3 154-113.7zM256 320c23.2.4 56.6-29.2 73.4-41.4 132.7-96.3 142.8-104.7 173.4-128.7 5.8-4.5 9.2-11.5 9.2-18.9v-19c0-26.5-21.5-48-48-48H48C21.5 64 0 85.5 0 112v19c0 7.4 3.4 14.3 9.2 18.9 30.6 23.9 40.7 32.4 173.4 128.7 16.8 12.2 50.2 41.8 73.4 41.4z"></path> </svg></a>&nbsp; &nbsp;<a style="border-bottom: 0px;;" href="https://gianluca.statistica.it" title="Visit my website"><svg viewBox="0 0 512 512" style="position:relative;display:inline-block;top:.1em;fill:#bcc0c4;height:0.8em;" xmlns="http://www.w3.org/2000/svg"> <path d="M503.52,241.48c-.12-1.56-.24-3.12-.24-4.68v-.12l-.36-4.68v-.12a245.86,245.86,0,0,0-7.32-41.15c0-.12,0-.12-.12-.24l-1.08-4c-.12-.24-.12-.48-.24-.6-.36-1.2-.72-2.52-1.08-3.72-.12-.24-.12-.6-.24-.84-.36-1.2-.72-2.4-1.08-3.48-.12-.36-.24-.6-.36-1-.36-1.2-.72-2.28-1.2-3.48l-.36-1.08c-.36-1.08-.84-2.28-1.2-3.36a8.27,8.27,0,0,0-.36-1c-.48-1.08-.84-2.28-1.32-3.36-.12-.24-.24-.6-.36-.84-.48-1.2-1-2.28-1.44-3.48,0-.12-.12-.24-.12-.36-1.56-3.84-3.24-7.68-5-11.4l-.36-.72c-.48-1-.84-1.8-1.32-2.64-.24-.48-.48-1.08-.72-1.56-.36-.84-.84-1.56-1.2-2.4-.36-.6-.6-1.2-1-1.8s-.84-1.44-1.2-2.28c-.36-.6-.72-1.32-1.08-1.92s-.84-1.44-1.2-2.16a18.07,18.07,0,0,0-1.2-2c-.36-.72-.84-1.32-1.2-2s-.84-1.32-1.2-2-.84-1.32-1.2-1.92-.84-1.44-1.32-2.16a15.63,15.63,0,0,0-1.2-1.8L463.2,119a15.63,15.63,0,0,0-1.2-1.8c-.48-.72-1.08-1.56-1.56-2.28-.36-.48-.72-1.08-1.08-1.56l-1.8-2.52c-.36-.48-.6-.84-1-1.32-1-1.32-1.8-2.52-2.76-3.72a248.76,248.76,0,0,0-23.51-26.64A186.82,186.82,0,0,0,412,62.46c-4-3.48-8.16-6.72-12.48-9.84a162.49,162.49,0,0,0-24.6-15.12c-2.4-1.32-4.8-2.52-7.2-3.72a254,254,0,0,0-55.43-19.56c-1.92-.36-3.84-.84-5.64-1.2h-.12c-1-.12-1.8-.36-2.76-.48a236.35,236.35,0,0,0-38-4H255.14a234.62,234.62,0,0,0-45.48,5c-33.59,7.08-63.23,21.24-82.91,39-1.08,1-1.92,1.68-2.4,2.16l-.48.48H124l-.12.12.12-.12a.12.12,0,0,0,.12-.12l-.12.12a.42.42,0,0,1,.24-.12c14.64-8.76,34.92-16,49.44-19.56l5.88-1.44c.36-.12.84-.12,1.2-.24,1.68-.36,3.36-.72,5.16-1.08.24,0,.6-.12.84-.12C250.94,20.94,319.34,40.14,367,85.61a171.49,171.49,0,0,1,26.88,32.76c30.36,49.2,27.48,111.11,3.84,147.59-34.44,53-111.35,71.27-159,24.84a84.19,84.19,0,0,1-25.56-59,74.05,74.05,0,0,1,6.24-31c1.68-3.84,13.08-25.67,18.24-24.59-13.08-2.76-37.55,2.64-54.71,28.19-15.36,22.92-14.52,58.2-5,83.28a132.85,132.85,0,0,1-12.12-39.24c-12.24-82.55,43.31-153,94.31-170.51-27.48-24-96.47-22.31-147.71,15.36-29.88,22-51.23,53.16-62.51,90.36,1.68-20.88,9.6-52.08,25.8-83.88-17.16,8.88-39,37-49.8,62.88-15.6,37.43-21,82.19-16.08,124.79.36,3.24.72,6.36,1.08,9.6,19.92,117.11,122,206.38,244.78,206.38C392.77,503.42,504,392.19,504,255,503.88,250.48,503.76,245.92,503.52,241.48Z"></path> </svg></a>&nbsp; &nbsp; | &nbsp; Decision making in HTA with VoI &nbsp; | &nbsp; VVSOR Annual Meeting 2023 &nbsp; | &nbsp; 23 Mar 2023 ] .slide-url[ https://gianluca.statistica.it/slides/vvsor-2023 ] --- # Disclaimer... <center> <blockquote class="twitter-tweet"><p lang="en" dir="ltr">Best opening sentence <a href="https://twitter.com/hashtag/ISPOREurope?src=hash&amp;ref_src=twsrc%5Etfw">#ISPOREurope</a> from Gianluca Baio: “statisticians should rule the world and Bayesian statisticians should rule all statisticians” <a href="https://t.co/GN2w7liAcR">https://t.co/GN2w7liAcR</a></p>&mdash; Manuela Joore (@ManuelaJoore) <a href="https://twitter.com/ManuelaJoore/status/1191397718930939904?ref_src=twsrc%5Etfw">November 4, 2019</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script> </center> <span style="display:block; margin-top: 10px ;"></span> ...Just so you know what you're about to get into... 😉 --- count: false # Health technology assessment (HTA) **Objective**: Combine .red[costs] and .blue[benefits] of a given intervention into a rational scheme for allocating resources <center><img src=./img/hta-scheme4.png width='80%' title=''></center> --- count: false # Health technology assessment (HTA) **Objective**: Combine .red[costs] and .blue[benefits] of a given intervention into a rational scheme for allocating resources <center><img src=./img/integrated.png width='75%' title=''></center> --- # Knowledge *is* power? ## (A tale of two stupid examples) <style type="text/css"> .left-column30 { width: 30%; height: 92%; float: left; } .left-column30 h2, .left-column h3 { color: #035AA699; } .left-column30 h2:last-of-type, .left-column h3:last-child { color: #035AA6; } .right-column70 { width: 65%; float: right; padding-top: 0em; } </style> <span style="display:block; margin-top: 40px ;"></span> .left-column30[ <center><img src=./img/knowledge_power.jpg width='75%' title='INCLUDE TEXT HERE'></center> ] .right-columnt70[ - **Example 1**: Intervention `\(t=1\)` is more cost-effective, given current evidence - `\(\class{myblue}{\Pr(t=1 \style{font-family:inherit;}{\text{ is cost-effective}}) = \style{font-family:inherit;}{\text{0.51}}}\)` - If we get it wrong: - Increase in population average costs `\(=\)` £3 - Decrease in population average effectiveness `\(=\)` 0.000001 QALYs - .red[Large uncertainty]/.traffic-light-green[negligible consequences] `\(\Rightarrow\)` .traffic-light-green[**can afford uncertainty**!] ] --- count: false # Knowledge *is* power? ## (A tale of two stupid examples) <span style="display:block; margin-top: 40px ;"></span> .left-column30[ <center><img src=./img/knowledge_power.jpg width='75%' title='INCLUDE TEXT HERE'></center> ] .right-columnt70[ - **Example 1**: Intervention `\(t=1\)` is more cost-effective, given current evidence - `\(\class{myblue}{\Pr(t=1 \style{font-family:inherit;}{\text{ is cost-effective}}) = \style{font-family:inherit;}{\text{0.51}}}\)` - If we get it wrong: - Increase in population average costs `\(=\)` £3 - Decrease in population average effectiveness `\(=\)` 0.000001 QALYs - .red[Large uncertainty]/.traffic-light-green[negligible consequences] `\(\Rightarrow\)` .traffic-light-green[**can afford uncertainty**!] <span style="display:block; margin-top: 60px ;"></span> - **Example 2**: Intervention `\(t=1\)` is more cost-effective, given current evidence - `\(\class{myblue}{\Pr(t=1 \style{font-family:inherit;}{\text{ is cost-effective}}) = \style{font-family:inherit;}{\text{0.999}}}\)` - If we get it wrong: - Increase in population average costs `\(=\)` £1000000000 - Decrease in population average effectiveness `\(=\)` 999999 QALYs - .traffic-light-green[Tiny uncertainty]/.red[dire consequences] `\(\Rightarrow\)` .traffic-light-amber[**probably should think about it...**!] ] --- class: hide-logo count: false background-image: url("img/voi-scooter.jpg") background-size: cover --- # VoI: Basic ideas - A new study will provide more data - Reducing (or even eliminating?...) uncertainty in a subset of the model parameters - Update the cost-effectiveness model - If optimal decision changes, gain in monetary **.blue[net benefit]** (NB = utility) from using new optimal treatment - If optimal decision doesn't change, no gain in NB - .red[**Expected**] VoI is the average gain in NB --- count: false # VoI: Basic ideas & relevant measures - A new study will provide more data - Reducing (or even eliminating?...) uncertainty in a subset of the model parameters - Update the cost-effectiveness model - If optimal decision changes, gain in monetary net benefit (NB = utility) from using new optimal treatment - If optimal decision doesn't change, no gain in NB - .red[**Expected**] VoI is the average gain in NB <span style="display:block; margin-top: -10px ;"></span> 1. **Expected value of Perfect Information** (EVPI) - Value of completely resolving uncertainty in all input parameters to decision model - Infinite-sized, long-term follow up trial measuring everything!... - Gives an upper bound on the value of the new study &ndash; low EVPI suggests we can make our decision based on existing information --- count: false # VoI: Basic ideas & relevant measures - A new study will provide more data - Reducing (or even eliminating?...) uncertainty in a subset of the model parameters - Update the cost-effectiveness model - If optimal decision changes, gain in monetary net benefit (NB = utility) from using new optimal treatment - If optimal decision doesn't change, no gain in NB - .red[**Expected**] VoI is the average gain in NB <span style="display:block; margin-top: -10px ;"></span> 1. **Expected value of Perfect Information** (EVPI) - Value of completely resolving uncertainty in all input parameters to decision model - Infinite-sized, long-term follow up trial measuring everything!... - Gives an upper bound on the value of the new study &ndash; low EVPI suggests we can make our decision based on existing information 2. **Expected value of Partial Perfect Information** (EVPPI) - Value of eliminating uncertainty in subset of input parameters to decision model - e.g.: Infinite-sized trial measuring relative effects on 1-year survival - Useful to identify which parameters are responsible for decision uncertainty --- count: false # VoI: Basic ideas & relevant measures - A new study will provide more data - Reducing (or even eliminating?...) uncertainty in a subset of the model parameters - Update the cost-effectiveness model - If optimal decision changes, gain in monetary net benefit (NB = utility) from using new optimal treatment - If optimal decision doesn't change, no gain in NB - .red[**Expected**] VoI is the average gain in NB <span style="display:block; margin-top: -10px ;"></span> 1. **Expected value of Perfect Information** (EVPI) - Value of completely resolving uncertainty in all input parameters to decision model - Infinite-sized, long-term follow up trial measuring everything!... - Gives an upper bound on the value of the new study &ndash; low EVPI suggests we can make our decision based on existing information 2. **Expected value of Partial Perfect Information** (EVPPI) - Value of eliminating uncertainty in subset of input parameters to decision model - e.g.: Infinite-sized trial measuring relative effects on 1-year survival - Useful to identify which parameters are responsible for decision uncertainty 3. **Expected value of Sample Information** (EVSI) - Value of reducing uncertainty by conducting a specific study of a given design - Can compare the benefits and costs of a study with given design - Is the proposed study likely to be a good use of resource? What is the optimal design? --- count: false # VoI: Basic ideas & relevant measures <svg viewBox="0 0 512 512" style="position:relative;display:inline-block;fill:#035AA6;height:1.7em;top:.45em;" xmlns="http://www.w3.org/2000/svg"> <path d="M256 8C119.043 8 8 119.083 8 256c0 136.997 111.043 248 248 248s248-111.003 248-248C504 119.083 392.957 8 256 8zm0 110c23.196 0 42 18.804 42 42s-18.804 42-42 42-42-18.804-42-42 18.804-42 42-42zm56 254c0 6.627-5.373 12-12 12h-88c-6.627 0-12-5.373-12-12v-24c0-6.627 5.373-12 12-12h12v-64h-12c-6.627 0-12-5.373-12-12v-24c0-6.627 5.373-12 12-12h64c6.627 0 12 5.373 12 12v100h12c6.627 0 12 5.373 12 12v24z"></path></svg> In general, VoI measures are always expressed as something like <span style="display:block; margin-top: 40px ;"></span> .center[ .content-box-gray[ .olive[**VoI measure**] `\(=\)` .blue[**Some idealised decision-making process**] `\(-\)` .magenta[**current decision-making process**] ] ] -- <span style="display:block; margin-top: 40px ;"></span> ## Complexity - There's no natural upper bound - Voi measures are positive, but *how low is low?*... - Need to account for other factors - How much would it cost to get to the point when we can make the idealised decision-making process? - Who would that affect? - For how long? - ... - Computational & modelling issues - You need to know what you're doing (again, modelling **fundamentally** Bayesian) - And use suitable tools (basically, never use spreadsheets...) --- exclude: true # Summarising uncertainty analysis (PSA) ## Expected Value of Perfect Information .alignleft[ <table class=" lightable-classic table" style='font-family: "Arial Narrow", "Source Sans Pro", sans-serif; margin-left: auto; margin-right: auto; margin-left: auto; margin-right: auto;'> <thead> <tr> <th style="empty-cells: hide;" colspan="1"></th> <th style="padding-bottom:0; padding-left:3px;padding-right:3px;text-align: center; " colspan="4"><div style="border-bottom: 1px solid #111111; margin-bottom: -1px; ">Parameter simulations</div></th> </tr> <tr> <th style="text-align:center;"> Iteration </th> <th style="text-align:center;"> \(\pi_0\) </th> <th style="text-align:center;"> \(\rho\) </th> <th style="text-align:center;"> \(\ldots\) </th> <th style="text-align:center;"> \(\gamma\) </th> </tr> </thead> <tbody> <tr> <td style="text-align:center;width: 2cm; "> 1 </td> <td style="text-align:center;width: 2cm; "> 0.585 </td> <td style="text-align:center;width: 2cm; "> 0.3814 </td> <td style="text-align:center;width: 2cm; "> \(\ldots\) </td> <td style="text-align:center;width: 2cm; "> 0.4194 </td> </tr> <tr> <td style="text-align:center;width: 2cm; "> 2 </td> <td style="text-align:center;width: 2cm; "> 0.515 </td> <td style="text-align:center;width: 2cm; "> 0.0166 </td> <td style="text-align:center;width: 2cm; "> \(\ldots\) </td> <td style="text-align:center;width: 2cm; "> 0.0768 </td> </tr> <tr> <td style="text-align:center;width: 2cm; "> 3 </td> <td style="text-align:center;width: 2cm; "> 0.611 </td> <td style="text-align:center;width: 2cm; "> 0.1373 </td> <td style="text-align:center;width: 2cm; "> \(\ldots\) </td> <td style="text-align:center;width: 2cm; "> 0.0592 </td> </tr> <tr> <td style="text-align:center;width: 2cm; "> 4 </td> <td style="text-align:center;width: 2cm; "> 0.195 </td> <td style="text-align:center;width: 2cm; "> 0.7282 </td> <td style="text-align:center;width: 2cm; "> \(\ldots\) </td> <td style="text-align:center;width: 2cm; "> 0.7314 </td> </tr> <tr> <td style="text-align:center;width: 2cm; "> \(\ldots\) </td> <td style="text-align:center;width: 2cm; "> \(\ldots\) </td> <td style="text-align:center;width: 2cm; "> \(\ldots\) </td> <td style="text-align:center;width: 2cm; "> \(\ldots\) </td> <td style="text-align:center;width: 2cm; "> \(\ldots\) </td> </tr> <tr> <td style="text-align:center;width: 2cm; border-bottom: 2px solid;"> 1000 </td> <td style="text-align:center;width: 2cm; border-bottom: 2px solid;"> 0.0305 </td> <td style="text-align:center;width: 2cm; border-bottom: 2px solid;"> 0.204 </td> <td style="text-align:center;width: 2cm; border-bottom: 2px solid;"> \(\ldots\) </td> <td style="text-align:center;width: 2cm; border-bottom: 2px solid;"> 0.558 </td> </tr> </tbody> </table> ] <span style="display:block; margin-top: 9.5cm ;"></span> - Characterise uncertainty in the model parameters - In a full Bayesian setting, these are drawings from the posterior distribution of `\(\boldsymbol\theta\)` - In a frequentist setting, these are typically bootstrap draws from a set of univariate ditributions that describe some level of uncertainty around the MLEs --- exclude: true count: false # Summarising uncertainty analysis (PSA) ## Expected Value of Perfect Information .alignleft[ <table class=" lightable-classic table" style='font-family: "Arial Narrow", "Source Sans Pro", sans-serif; margin-left: auto; margin-right: auto; margin-left: auto; margin-right: auto;'> <thead> <tr> <th style="empty-cells: hide;" colspan="1"></th> <th style="padding-bottom:0; padding-left:3px;padding-right:3px;text-align: center; " colspan="4"><div style="border-bottom: 1px solid #111111; margin-bottom: -1px; ">Parameter simulations</div></th> <th style="padding-bottom:0; padding-left:3px;padding-right:3px;text-align: center; " colspan="2"><div style="border-bottom: 1px solid #111111; margin-bottom: -1px; ">Expected utility</div></th> </tr> <tr> <th style="text-align:center;"> Iteration </th> <th style="text-align:center;"> \(\pi_0\) </th> <th style="text-align:center;"> \(\rho\) </th> <th style="text-align:center;"> \(\ldots\) </th> <th style="text-align:center;"> \(\gamma\) </th> <th style="text-align:center;"> \(\nb_0(\boldsymbol\theta)\) </th> <th style="text-align:center;"> \(\nb_1(\boldsymbol\theta)\) </th> </tr> </thead> <tbody> <tr> <td style="text-align:center;width: 2cm; "> 1 </td> <td style="text-align:center;width: 2cm; "> 0.585 </td> <td style="text-align:center;width: 2cm; "> 0.3814 </td> <td style="text-align:center;width: 2cm; "> \(\ldots\) </td> <td style="text-align:center;width: 2cm; "> 0.4194 </td> <td style="text-align:center;width: 2cm; font-style: italic;"> 77480.00 </td> <td style="text-align:center;width: 2cm; color: black !important;"> 67795.00 </td> </tr> <tr> <td style="text-align:center;width: 2cm; "> 2 </td> <td style="text-align:center;width: 2cm; "> 0.515 </td> <td style="text-align:center;width: 2cm; "> 0.0166 </td> <td style="text-align:center;width: 2cm; "> \(\ldots\) </td> <td style="text-align:center;width: 2cm; "> 0.0768 </td> <td style="text-align:center;width: 2cm; "> 87165.00 </td> <td style="text-align:center;width: 2cm; font-style: italic;color: black !important;"> 106535.00 </td> </tr> <tr> <td style="text-align:center;width: 2cm; "> 3 </td> <td style="text-align:center;width: 2cm; "> 0.611 </td> <td style="text-align:center;width: 2cm; "> 0.1373 </td> <td style="text-align:center;width: 2cm; "> \(\ldots\) </td> <td style="text-align:center;width: 2cm; "> 0.0592 </td> <td style="text-align:center;width: 2cm; font-style: italic;"> 58110.00 </td> <td style="text-align:center;width: 2cm; color: black !important;"> 38740.00 </td> </tr> <tr> <td style="text-align:center;width: 2cm; "> 4 </td> <td style="text-align:center;width: 2cm; "> 0.195 </td> <td style="text-align:center;width: 2cm; "> 0.7282 </td> <td style="text-align:center;width: 2cm; "> \(\ldots\) </td> <td style="text-align:center;width: 2cm; "> 0.7314 </td> <td style="text-align:center;width: 2cm; "> 77480.00 </td> <td style="text-align:center;width: 2cm; font-style: italic;color: black !important;"> 87165.00 </td> </tr> <tr> <td style="text-align:center;width: 2cm; "> \(\ldots\) </td> <td style="text-align:center;width: 2cm; "> \(\ldots\) </td> <td style="text-align:center;width: 2cm; "> \(\ldots\) </td> <td style="text-align:center;width: 2cm; "> \(\ldots\) </td> <td style="text-align:center;width: 2cm; "> \(\ldots\) </td> <td style="text-align:center;width: 2cm; "> \(\ldots\) </td> <td style="text-align:center;width: 2cm; color: black !important;"> \(\ldots\) </td> </tr> <tr> <td style="text-align:center;width: 2cm; border-bottom: 2px black solid;"> 1000 </td> <td style="text-align:center;width: 2cm; border-bottom: 2px black solid;"> 0.0305 </td> <td style="text-align:center;width: 2cm; border-bottom: 2px black solid;"> 0.204 </td> <td style="text-align:center;width: 2cm; border-bottom: 2px black solid;"> \(\ldots\) </td> <td style="text-align:center;width: 2cm; border-bottom: 2px black solid;"> 0.558 </td> <td style="text-align:center;border-bottom: 2px black solid;width: 2cm; "> 48425.00 </td> <td style="text-align:center;border-bottom: 2px black solid;width: 2cm; font-style: italic;color: black !important;"> 87165.00 </td> </tr> <tr> <td style="text-align:center;width: 2cm; border-bottom: 2px black solid;"> </td> <td style="text-align:center;width: 2cm; border-bottom: 2px black solid;"> </td> <td style="text-align:center;width: 2cm; border-bottom: 2px black solid;"> </td> <td style="text-align:center;width: 2cm; border-bottom: 2px black solid;"> </td> <td style="text-align:center;width: 2cm; border-bottom: 2px black solid;"> Average </td> <td style="text-align:center;border-bottom: 2px black solid;width: 2cm; "> 72365.35 </td> <td style="text-align:center;border-bottom: 2px black solid;width: 2cm; font-style: italic;color: magenta !important;"> 77403.49 </td> </tr> </tbody> </table> ] <span style="display:block; margin-top: 9.5cm ;"></span> - Uncertainty in the parameters induces a distribution of decisions - Typically based on the **net benefits**: `\(\class{myblue}{\nb_t(\boldsymbol\theta)=k\mu_{et}-\mu_{ct}}\)` - In each parameter configuration can identify the *optimal strategy* - Averaging over the uncertainty in `\(\boldsymbol\theta\)` provides `\(t^*\)`, the overall optimal decision *given current uncertainty* (= choose the intervention associated with .magenta[*highest* **expected utility**]) --- exclude: true count: false # Summarising uncertainty analysis (PSA) ## Expected Value of Perfect Information .alignleft[ <table class=" lightable-classic table" style='font-family: "Arial Narrow", "Source Sans Pro", sans-serif; margin-left: auto; margin-right: auto; margin-left: auto; margin-right: auto;'> <thead> <tr> <th style="empty-cells: hide;" colspan="1"></th> <th style="padding-bottom:0; padding-left:3px;padding-right:3px;text-align: center; " colspan="4"><div style="border-bottom: 1px solid #111111; margin-bottom: -1px; ">Parameter simulations</div></th> <th style="padding-bottom:0; padding-left:3px;padding-right:3px;text-align: center; " colspan="2"><div style="border-bottom: 1px solid #111111; margin-bottom: -1px; ">Expected utility</div></th> <th style="empty-cells: hide;" colspan="2"></th> </tr> <tr> <th style="text-align:center;"> Iteration </th> <th style="text-align:center;"> \(\pi_0\) </th> <th style="text-align:center;"> \(\rho\) </th> <th style="text-align:center;"> \(\ldots\) </th> <th style="text-align:center;"> \(\gamma\) </th> <th style="text-align:center;"> \(\nb_0(\boldsymbol\theta)\) </th> <th style="text-align:center;"> \(\nb_1(\boldsymbol\theta)\) </th> <th style="text-align:center;"> Maximum net benefit </th> <th style="text-align:center;"> Opportunity loss </th> </tr> </thead> <tbody> <tr> <td style="text-align:center;width: 2cm; "> 1 </td> <td style="text-align:center;width: 2cm; "> 0.585 </td> <td style="text-align:center;width: 2cm; "> 0.3814 </td> <td style="text-align:center;width: 2cm; "> \(\ldots\) </td> <td style="text-align:center;width: 2cm; "> 0.4194 </td> <td style="text-align:center;width: 2cm; font-style: italic;"> 77480.00 </td> <td style="text-align:center;width: 2cm; color: black !important;"> 67795.00 </td> <td style="text-align:center;color: black !important;"> 77480.00 </td> <td style="text-align:center;color: black !important;"> 9685.00 </td> </tr> <tr> <td style="text-align:center;width: 2cm; "> 2 </td> <td style="text-align:center;width: 2cm; "> 0.515 </td> <td style="text-align:center;width: 2cm; "> 0.0166 </td> <td style="text-align:center;width: 2cm; "> \(\ldots\) </td> <td style="text-align:center;width: 2cm; "> 0.0768 </td> <td style="text-align:center;width: 2cm; "> 87165.00 </td> <td style="text-align:center;width: 2cm; font-style: italic;color: black !important;"> 106535.00 </td> <td style="text-align:center;color: black !important;"> 106535.00 </td> <td style="text-align:center;color: black !important;"> 0.00 </td> </tr> <tr> <td style="text-align:center;width: 2cm; "> 3 </td> <td style="text-align:center;width: 2cm; "> 0.611 </td> <td style="text-align:center;width: 2cm; "> 0.1373 </td> <td style="text-align:center;width: 2cm; "> \(\ldots\) </td> <td style="text-align:center;width: 2cm; "> 0.0592 </td> <td style="text-align:center;width: 2cm; font-style: italic;"> 58110.00 </td> <td style="text-align:center;width: 2cm; color: black !important;"> 38740.00 </td> <td style="text-align:center;color: black !important;"> 58110.00 </td> <td style="text-align:center;color: black !important;"> 19370.00 </td> </tr> <tr> <td style="text-align:center;width: 2cm; "> 4 </td> <td style="text-align:center;width: 2cm; "> 0.195 </td> <td style="text-align:center;width: 2cm; "> 0.7282 </td> <td style="text-align:center;width: 2cm; "> \(\ldots\) </td> <td style="text-align:center;width: 2cm; "> 0.7314 </td> <td style="text-align:center;width: 2cm; "> 77480.00 </td> <td style="text-align:center;width: 2cm; font-style: italic;color: black !important;"> 87165.00 </td> <td style="text-align:center;color: black !important;"> 87165.00 </td> <td style="text-align:center;color: black !important;"> 0.00 </td> </tr> <tr> <td style="text-align:center;width: 2cm; "> \(\ldots\) </td> <td style="text-align:center;width: 2cm; "> \(\ldots\) </td> <td style="text-align:center;width: 2cm; "> \(\ldots\) </td> <td style="text-align:center;width: 2cm; "> \(\ldots\) </td> <td style="text-align:center;width: 2cm; "> \(\ldots\) </td> <td style="text-align:center;width: 2cm; "> \(\ldots\) </td> <td style="text-align:center;width: 2cm; color: black !important;"> \(\ldots\) </td> <td style="text-align:center;color: black !important;"> \(\ldots\) </td> <td style="text-align:center;color: black !important;"> \(\ldots\) </td> </tr> <tr> <td style="text-align:center;width: 2cm; border-bottom: 2px black solid;"> 1000 </td> <td style="text-align:center;width: 2cm; border-bottom: 2px black solid;"> 0.0305 </td> <td style="text-align:center;width: 2cm; border-bottom: 2px black solid;"> 0.204 </td> <td style="text-align:center;width: 2cm; border-bottom: 2px black solid;"> \(\ldots\) </td> <td style="text-align:center;width: 2cm; border-bottom: 2px black solid;"> 0.558 </td> <td style="text-align:center;border-bottom: 2px black solid;width: 2cm; "> 48425.00 </td> <td style="text-align:center;border-bottom: 2px black solid;width: 2cm; font-style: italic;color: black !important;"> 87165.00 </td> <td style="text-align:center;border-bottom: 2px black solid;color: black !important;"> 87165.00 </td> <td style="text-align:center;border-bottom: 2px black solid;color: black !important;"> 0.00 </td> </tr> <tr> <td style="text-align:center;width: 2cm; border-bottom: 2px black solid;"> </td> <td style="text-align:center;width: 2cm; border-bottom: 2px black solid;"> </td> <td style="text-align:center;width: 2cm; border-bottom: 2px black solid;"> </td> <td style="text-align:center;width: 2cm; border-bottom: 2px black solid;"> </td> <td style="text-align:center;width: 2cm; border-bottom: 2px black solid;"> Average </td> <td style="text-align:center;border-bottom: 2px black solid;width: 2cm; "> 72365.35 </td> <td style="text-align:center;border-bottom: 2px black solid;width: 2cm; font-style: italic;color: magenta !important;"> 77403.49 </td> <td style="text-align:center;border-bottom: 2px black solid;font-weight: bold;color: blue !important;"> 91192.02 </td> <td style="text-align:center;border-bottom: 2px black solid;font-weight: bold;color: olive !important;"> 13788.53 </td> </tr> </tbody> </table> ] <span style="display:block; margin-top: 9.5cm ;"></span> - **Expected value of "Perfect" Information** (EVPI) summarises uncertainty in the decision - Defined as the .olive[**average Opportunity Loss**], or .blue[**average maximum expected utility under "perfect" information**] `\(-\)` .magenta[**maximum expected utility overall**]: <span style="display:block; margin-top: -20px ;"></span> `$$\class{olive}{\evpi} = \class{blue}{\E_\boldsymbol{\theta}\left[\max_t \nb_t(\boldsymbol\theta) \right]} - \class{magenta}{\max_t \E_\boldsymbol\theta \left[\nb_t(\boldsymbol\theta)\right]}$$` --- exclude: true count: false # Summarising PSA <img src="./img/unnamed-chunk-6-1.png" style="display: block; margin: auto;" width="53%"> --- # Summarising PSA + Research priority <style type="text/css"> <!-- https://www.garrickadenbuie.com/blog/better-progressive-xaringan/?panelset=r-markdown --> .shading > li { color: gray; } .shading > li:last-of-type { color: #24568c; font-weight: normal; } </style> ## Expected Value of Partial Perfect Information - `\(\class{blue}{\bm\theta} =\)` all the model parameters; can be split into two subsets - The ".myblue[**parameters of interest**]", `\(\class{myblue}{\bm\phi}\)`, e.g. prevalence of a disease, HRQL measures, length of stay in hospital, ... - The ".olive[**remaining parameters**], `\(\class{olive}{\bm\psi}\)`, e.g. cost of treatment with other established medications, ... - We are interested in quantifying the value of gaining more information on `\(\class{myblue}{\bm\phi}\)`, while leaving current level of uncertainty on `\(\class{olive}{\bm\psi}\)` unchanged -- <ul class="shading"> <li>First, consider the expected utility (EU) <b>if</b> we were able to learn \(\class{myblue}{\bm\phi}\) but not \(\class{myblue}{\bm\psi}\)</li> </ul> <span style="display:block; margin-top: 3cm ;"></span> `$$\class{myblue}{\E_{\bm\psi\mid\bm\phi}[\nb_t(\bm\theta)]}$$` --- count: false # Summarising PSA + Research priority ## Expected Value of Partial Perfect Information - `\(\class{blue}{\bm\theta} =\)` all the model parameters; can be split into two subsets - The ".myblue[**parameters of interest**]", `\(\class{myblue}{\bm\phi}\)`, e.g. prevalence of a disease, HRQL measures, length of stay in hospital, ... - The ".olive[**remaining parameters**], `\(\class{olive}{\bm\psi}\)`, e.g. cost of treatment with other established medications, ... - We are interested in quantifying the value of gaining more information on `\(\class{myblue}{\bm\phi}\)`, while leaving current level of uncertainty on `\(\class{olive}{\bm\psi}\)` unchanged <ul class="shading"> <li>First, consider the expected utility (EU) <b>if</b> we were able to learn \(\class{gray}{\bm\phi}\) but not \(\class{gray}{\bm\psi}\)</li> <li><b>If</b> we knew \(\class{myblue}{\bm\phi}\) perfectly, best decision = the maximum of this EU</li> </ul> <span style="display:block; margin-top: 2.25cm ;"></span> `$$\class{myblue}{\max_t}\class{gray}{\E_{\bm\psi\mid\bm\phi}[\nb_t(\bm\theta)]}$$` --- count: false # Summarising PSA + Research priority ## Expected Value of Partial Perfect Information - `\(\class{blue}{\bm\theta} =\)` all the model parameters; can be split into two subsets - The ".myblue[**parameters of interest**]", `\(\class{myblue}{\bm\phi}\)`, e.g. prevalence of a disease, HRQL measures, length of stay in hospital, ... - The ".olive[**remaining parameters**], `\(\class{olive}{\bm\psi}\)`, e.g. cost of treatment with other established medications, ... - We are interested in quantifying the value of gaining more information on `\(\class{myblue}{\bm\phi}\)`, while leaving current level of uncertainty on `\(\class{olive}{\bm\psi}\)` unchanged <ul class="shading"> <li>First, consider the expected utility (EU) <b>if</b> we were able to learn \(\class{gray}{\bm\phi}\) but not \(\class{gray}{\bm\psi}\)</li> <li><b>If</b> we knew \(\class{gray}{\bm\phi}\) perfectly, best decision = the maximum of this EU</li> <li>Of course, we cannot know \(\class{myblue}{\bm\phi}\) <b>perfectly</b>, so take the expected value</li> </ul> <span style="display:block; margin-top: 1.1cm ;"></span> `$$\class{myblue}{\E_{\bm\phi}}\class{myblue}{\left [\class{gray}{\max_t\E_{\bm\psi\mid\bm\phi}[\nb_t(\bm\theta)]}\right]}$$` --- count: false # Summarising PSA + Research priority ## Expected Value of Partial Perfect Information - `\(\class{blue}{\bm\theta} =\)` all the model parameters; can be split into two subsets - The ".myblue[**parameters of interest**]", `\(\class{myblue}{\bm\phi}\)`, e.g. prevalence of a disease, HRQL measures, length of stay in hospital, ... - The ".olive[**remaining parameters**], `\(\class{olive}{\bm\psi}\)`, e.g. cost of treatment with other established medications, ... - We are interested in quantifying the value of gaining more information on `\(\class{myblue}{\bm\phi}\)`, while leaving current level of uncertainty on `\(\class{olive}{\bm\psi}\)` unchanged <ul class="shading"> <li>First, consider the expected utility (EU) <b>if</b> we were able to learn \(\class{gray}{\bm\phi}\) but not \(\class{gray}{\bm\psi}\)</li> <li><b>If</b> we knew \(\class{gray}{\bm\phi}\) perfectly, best decision = the maximum of this EU</li> <li>Of course, we cannot know \(\class{gray}{\bm\phi}\) <b>perfectly</b>, so take the expected value</li> <li>And compare this with the <b>maximum expected utility overall</b></li> </ul> <span style="display:block; margin-top: .5cm ;"></span> `$$\class{gray}{\E_{\bm\phi}\left[\max_t\E_{\bm\psi\mid\bm\phi}[\nb_t(\bm\theta)]\right]}\class{myblue}{-\max_t \E_{\bm\theta}[\nb_t(\bm\theta)]}$$` --- count: false # Summarising PSA + Research priority ## Expected Value of Partial Perfect Information - `\(\class{blue}{\bm\theta} =\)` all the model parameters; can be split into two subsets - The ".myblue[**parameters of interest**]", `\(\class{myblue}{\bm\phi}\)`, e.g. prevalence of a disease, HRQL measures, length of stay in hospital, ... - The ".olive[**remaining parameters**], `\(\class{olive}{\bm\psi}\)`, e.g. cost of treatment with other established medications, ... - We are interested in quantifying the value of gaining more information on `\(\class{myblue}{\bm\phi}\)`, while leaving current level of uncertainty on `\(\class{olive}{\bm\psi}\)` unchanged <ul class="shading"> <li>First, consider the expected utility (EU) <b>if</b> we were able to learn \(\class{gray}{\bm\phi}\) but not \(\class{gray}{\bm\psi}\)</li> <li><b>If</b> we knew \(\class{gray}{\bm\phi}\) perfectly, best decision = the maximum of this EU</li> <li>Of course, we cannot know \(\class{gray}{\bm\phi}\) <b>perfectly</b>, so take the expected value</li> <li>And compare this with the <b>maximum expected utility overall</b></li> <li>This is the EVPPI</li> </ul> <span style="display:block; margin-top: -.55cm ;"></span> `$$\class{myblue}{\evppi = \E_{\bm\phi}\left[\max_t\E_{\bm\psi\mid\bm\phi}[\nb_t(\bm\theta)]\right]-\max_t \E_{\bm\theta}[\nb_t(\bm\theta)]}$$` --- count: false # Summarising PSA + Research priority ## Expected Value of Partial Perfect Information - `\(\class{blue}{\bm\theta} =\)` all the model parameters; can be split into two subsets - The ".myblue[**parameters of interest**]", `\(\class{myblue}{\bm\phi}\)`, e.g. prevalence of a disease, HRQL measures, length of stay in hospital, ... - The ".olive[**remaining parameters**], `\(\class{olive}{\bm\psi}\)`, e.g. cost of treatment with other established medications, ... - We are interested in quantifying the value of gaining more information on `\(\class{myblue}{\bm\phi}\)`, while leaving current level of uncertainty on `\(\class{olive}{\bm\psi}\)` unchanged <ul class="shading"> <li>First, consider the expected utility (EU) <b>if</b> we were able to learn \(\class{gray}{\bm\phi}\) but not \(\class{gray}{\bm\psi}\)</li> <li><b>If</b> we knew \(\class{gray}{\bm\phi}\) perfectly, best decision = the maximum of this EU</li> <li>Of course, we cannot know \(\class{gray}{\bm\phi}\) <b>perfectly</b>, so take the expected value</li> <li>And compare this with the <b>maximum expected utility overall</b></li> <li>This is the EVPPI</li> </ul> <span style="display:block; margin-top: -.55cm ;"></span> `$$\class{myblue}{\evppi = \E_{\bm\phi}\left[\class{red}{\max_t\E_{\bm\psi\mid\bm\phi}[\nb_t(\bm\theta)]}\right]-\max_t \E_{\bm\theta}[\nb_t(\bm\theta)]}$$` - .red[**That**] is the difficult part! - Can do nested Monte Carlo, but takes for ever to get accurate results - .orange[**Recent methods**] based on **GAMs**/**Gaussian Process regression**/**spatial modelling** very efficient and quick! <span style="display:block; margin-top: -17px ;"></span> .alignright[.small[<svg viewBox="0 0 512 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg"> <g groupmode="layer" id="layer6" label="icon"> <path style="stroke-width:0.07717" d="m 115.59247,222.50738 c -9.72601,0 -18.734334,7.07397 -24.986084,14.41694 -5.061076,5.95394 -7.591049,15.13238 -7.591049,22.77486 0,7.6402 2.870733,16.13879 7.931808,22.09274 6.152292,7.34297 12.778135,12.37522 22.603045,12.37522 8.83314,0 19.33786,-3.70766 25.389,-9.76105 6.1523,-6.05338 11.26931,-16.21737 11.26931,-25.04826 0,-8.73142 -2.39548,-19.18696 -8.54776,-25.24039 -6.15229,-6.15455 -17.33741,-11.61061 -26.06883,-11.61061 z m 174.6438,2.39944 c -10.12046,0 -18.35798,3.98058 -24.70919,11.93724 -5.15827,6.38628 -7.73967,13.66255 -7.73967,21.82831 0,8.27088 2.5814,15.59914 7.73967,21.98538 6.35064,7.95667 14.58873,11.93729 24.70919,11.93729 9.03149,0 16.67002,-3.25109 22.9212,-9.74241 6.25399,-6.59703 9.37846,-14.65259 9.37846,-24.18026 0,-9.422 -3.17646,-17.37639 -9.52708,-23.86775 -6.2495,-6.59477 -13.8428,-9.89721 -22.77258,-9.89721 z M 255.99998,7.9999981 C 119.03396,7.9999981 7.9999985,119.03394 7.9999985,255.99998 7.9999985,392.96602 119.03396,504 255.99998,504 392.96606,504 504,392.96602 504,255.99998 504,119.03394 392.96606,7.9999981 255.99998,7.9999981 Z M 197.17431,331.79065 h -45.93563 v -18.69931 c -4.66437,5.85506 -15.63869,11.82369 -20.40025,14.50172 -8.33473,4.66267 -19.65377,7.70067 -30.27265,7.70067 -17.167865,0 -32.44885,-5.425 -45.844653,-17.23173 -15.977193,-14.0903 -23.483472,-35.63322 -23.483472,-58.85447 0,-23.61798 8.186103,-42.47212 24.559999,-56.56242 12.999092,-11.21393 28.896046,-20.18949 45.766106,-20.18949 9.82264,0 21.93961,0.88269 30.57214,5.04865 4.96218,2.38076 13.84449,7.98548 19.10448,13.44381 v -98.50164 h 45.93564 v 229.34421 z m 157.21544,-21.13999 c -15.28099,17.58718 -36.66682,26.38303 -64.15348,26.38303 -27.58613,0 -49.01942,-8.79585 -64.30436,-26.38303 -12.60069,-14.44803 -18.90387,-32.19228 -18.90387,-53.23507 0,-18.94908 6.35065,-35.75132 19.0525,-50.4079 15.37988,-17.69285 37.26185,-26.54011 65.64139,-26.54011 26.09766,0 46.93585,8.84558 62.51694,26.54011 12.70185,14.44806 19.05247,31.66734 19.05247,51.66468 0.003,20.20586 -6.29921,37.53085 -18.90159,51.97886 z m 38.76899,-199.21794 c 5.35888,-5.35888 11.85868,-8.03746 19.49892,-8.03746 7.64024,0 14.14005,2.67858 19.49893,8.03746 5.35888,5.25945 8.03746,11.70955 8.03746,19.35034 0,7.73965 -2.67858,14.28864 -8.03746,19.64753 -5.25944,5.25944 -11.75925,7.8883 -19.49893,7.8883 -7.64247,0 -14.14227,-2.67861 -19.49892,-8.0375 -5.2594,-5.35885 -7.88829,-11.85871 -7.88829,-19.49892 -0.003,-7.6402 2.62889,-14.0903 7.88829,-19.35029 z m 43.91029,220.24265 H 388.24626 V 185.84118 h 48.82277 z" id="path2"></path> </g></svg> [Strong et al (2014)](https://journals.sagepub.com/doi/full/10.1177/0272989x13505910) and <svg viewBox="0 0 512 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg"> <g groupmode="layer" id="layer6" label="icon"> <path style="stroke-width:0.07717" d="m 115.59247,222.50738 c -9.72601,0 -18.734334,7.07397 -24.986084,14.41694 -5.061076,5.95394 -7.591049,15.13238 -7.591049,22.77486 0,7.6402 2.870733,16.13879 7.931808,22.09274 6.152292,7.34297 12.778135,12.37522 22.603045,12.37522 8.83314,0 19.33786,-3.70766 25.389,-9.76105 6.1523,-6.05338 11.26931,-16.21737 11.26931,-25.04826 0,-8.73142 -2.39548,-19.18696 -8.54776,-25.24039 -6.15229,-6.15455 -17.33741,-11.61061 -26.06883,-11.61061 z m 174.6438,2.39944 c -10.12046,0 -18.35798,3.98058 -24.70919,11.93724 -5.15827,6.38628 -7.73967,13.66255 -7.73967,21.82831 0,8.27088 2.5814,15.59914 7.73967,21.98538 6.35064,7.95667 14.58873,11.93729 24.70919,11.93729 9.03149,0 16.67002,-3.25109 22.9212,-9.74241 6.25399,-6.59703 9.37846,-14.65259 9.37846,-24.18026 0,-9.422 -3.17646,-17.37639 -9.52708,-23.86775 -6.2495,-6.59477 -13.8428,-9.89721 -22.77258,-9.89721 z M 255.99998,7.9999981 C 119.03396,7.9999981 7.9999985,119.03394 7.9999985,255.99998 7.9999985,392.96602 119.03396,504 255.99998,504 392.96606,504 504,392.96602 504,255.99998 504,119.03394 392.96606,7.9999981 255.99998,7.9999981 Z M 197.17431,331.79065 h -45.93563 v -18.69931 c -4.66437,5.85506 -15.63869,11.82369 -20.40025,14.50172 -8.33473,4.66267 -19.65377,7.70067 -30.27265,7.70067 -17.167865,0 -32.44885,-5.425 -45.844653,-17.23173 -15.977193,-14.0903 -23.483472,-35.63322 -23.483472,-58.85447 0,-23.61798 8.186103,-42.47212 24.559999,-56.56242 12.999092,-11.21393 28.896046,-20.18949 45.766106,-20.18949 9.82264,0 21.93961,0.88269 30.57214,5.04865 4.96218,2.38076 13.84449,7.98548 19.10448,13.44381 v -98.50164 h 45.93564 v 229.34421 z m 157.21544,-21.13999 c -15.28099,17.58718 -36.66682,26.38303 -64.15348,26.38303 -27.58613,0 -49.01942,-8.79585 -64.30436,-26.38303 -12.60069,-14.44803 -18.90387,-32.19228 -18.90387,-53.23507 0,-18.94908 6.35065,-35.75132 19.0525,-50.4079 15.37988,-17.69285 37.26185,-26.54011 65.64139,-26.54011 26.09766,0 46.93585,8.84558 62.51694,26.54011 12.70185,14.44806 19.05247,31.66734 19.05247,51.66468 0.003,20.20586 -6.29921,37.53085 -18.90159,51.97886 z m 38.76899,-199.21794 c 5.35888,-5.35888 11.85868,-8.03746 19.49892,-8.03746 7.64024,0 14.14005,2.67858 19.49893,8.03746 5.35888,5.25945 8.03746,11.70955 8.03746,19.35034 0,7.73965 -2.67858,14.28864 -8.03746,19.64753 -5.25944,5.25944 -11.75925,7.8883 -19.49893,7.8883 -7.64247,0 -14.14227,-2.67861 -19.49892,-8.0375 -5.2594,-5.35885 -7.88829,-11.85871 -7.88829,-19.49892 -0.003,-7.6402 2.62889,-14.0903 7.88829,-19.35029 z m 43.91029,220.24265 H 388.24626 V 185.84118 h 48.82277 z" id="path2"></path> </g></svg> [Heath et al (2016)](https://onlinelibrary.wiley.com/doi/full/10.1002/sim.6983)]] --- exclude: false # EVPPI &ndash; Brute force/nested MC Assuming there are only two interventions, can consider `\(\class{myblue}{\ib(\bm\theta)=\nb_1(\bm\theta)-\nb_0(\bm\theta)}\)` <center><img src=./img/brute-force-1_25.png width='44%' title='INCLUDE TEXT HERE'></center> <span style="display:block; margin-top: -5px ;"></span> .small[Slide stolen from [Mark Strong](https://www.sheffield.ac.uk/scharr/people/staff/mark-strong) &ndash; [Summer School *Bayesian methods in health economics*](https://gianluca.statistica.it/teaching/summer-school/)] --- exclude: false count: false # EVPPI &ndash; Brute force/nested MC Assuming there are only two interventions, can consider `\(\class{myblue}{\ib(\bm\theta)=\nb_1(\bm\theta)-\nb_0(\bm\theta)}\)` <center><img src=./img/brute-force-2_26.png width='44%' title='INCLUDE TEXT HERE'></center> --- exclude: false # EVPPI &ndash; model as a regression problem... Can model as a **regression** problem `\begin{align} \nb_t(\bm\theta) =& \E_{\bm\psi\mid\bm\theta}[\nb_t(\bm\theta)] + \varepsilon, \qquad \style{font-family:inherit;}{\text{ with }} \varepsilon \sim \dnorm(0,\sigma^2_\varepsilon) \\ =& \class{olive}{g(\bm\phi)} + \class{myblue}{\varepsilon} \end{align}` <span style="display:block; margin-top: -20px ;"></span> "Data" - **Simulations** for `\(\nb_t(\bm\theta)\)` as ".myblue[response]" - **Simulations** for `\(\bm\phi\)` as ".olive[covariates]" - **NB**: Only need `\(S\)` data points (=PSA simulations), instead of `\(S_\bm\phi \times S_\bm\psi\)`! .center[ <table class=" lightable-classic table" style='font-family: "Arial Narrow", "Source Sans Pro", sans-serif; margin-left: auto; margin-right: auto; font-size: 18px; width: auto !important; margin-left: auto; margin-right: auto;'> <thead> <tr> <th style="empty-cells: hide;" colspan="1"></th> <th style="padding-bottom:0; padding-left:3px;padding-right:3px;text-align: center; " colspan="4"><div style="border-bottom: 1px solid #111111; margin-bottom: -1px; ">Parameter simulations ('covariates')</div></th> <th style="padding-bottom:0; padding-left:3px;padding-right:3px;text-align: center; " colspan="2"><div style="border-bottom: 1px solid #111111; margin-bottom: -1px; ">'Responses'</div></th> </tr> <tr> <th style="text-align:center;"> Iteration </th> <th style="text-align:center;"> \(\pi_0\) </th> <th style="text-align:center;"> \(\rho\) </th> <th style="text-align:center;"> \(\ldots\) </th> <th style="text-align:center;"> \(\gamma\) </th> <th style="text-align:center;"> \(\nb_0(\boldsymbol\theta)\) </th> <th style="text-align:center;"> \(\nb_1(\boldsymbol\theta)\) </th> </tr> </thead> <tbody> <tr> <td style="text-align:center;width: 2.5cm; "> 1 </td> <td style="text-align:center;width: 2.5cm; "> 0.585 </td> <td style="text-align:center;width: 2.5cm; "> 0.3814 </td> <td style="text-align:center;width: 2.5cm; "> \(\ldots\) </td> <td style="text-align:center;width: 2.5cm; "> 0.4194 </td> <td style="text-align:center;width: 2.5cm; "> 77480 </td> <td style="text-align:center;width: 2.5cm; "> 67795 </td> </tr> <tr> <td style="text-align:center;width: 2.5cm; "> 2 </td> <td style="text-align:center;width: 2.5cm; "> 0.515 </td> <td style="text-align:center;width: 2.5cm; "> 0.0166 </td> <td style="text-align:center;width: 2.5cm; "> \(\ldots\) </td> <td style="text-align:center;width: 2.5cm; "> 0.0768 </td> <td style="text-align:center;width: 2.5cm; "> 87165 </td> <td style="text-align:center;width: 2.5cm; "> 106535 </td> </tr> <tr> <td style="text-align:center;width: 2.5cm; "> 3 </td> <td style="text-align:center;width: 2.5cm; "> 0.611 </td> <td style="text-align:center;width: 2.5cm; "> 0.1373 </td> <td style="text-align:center;width: 2.5cm; "> \(\ldots\) </td> <td style="text-align:center;width: 2.5cm; "> 0.0592 </td> <td style="text-align:center;width: 2.5cm; "> 58110 </td> <td style="text-align:center;width: 2.5cm; "> 38740 </td> </tr> <tr> <td style="text-align:center;width: 2.5cm; "> 4 </td> <td style="text-align:center;width: 2.5cm; "> 0.195 </td> <td style="text-align:center;width: 2.5cm; "> 0.7282 </td> <td style="text-align:center;width: 2.5cm; "> \(\ldots\) </td> <td style="text-align:center;width: 2.5cm; "> 0.7314 </td> <td style="text-align:center;width: 2.5cm; "> 77480 </td> <td style="text-align:center;width: 2.5cm; "> 87165 </td> </tr> <tr> <td style="text-align:center;width: 2.5cm; "> \(\ldots\) </td> <td style="text-align:center;width: 2.5cm; "> \(\ldots\) </td> <td style="text-align:center;width: 2.5cm; "> \(\ldots\) </td> <td style="text-align:center;width: 2.5cm; "> \(\ldots\) </td> <td style="text-align:center;width: 2.5cm; "> \(\ldots\) </td> <td style="text-align:center;width: 2.5cm; "> \(\ldots\) </td> <td style="text-align:center;width: 2.5cm; "> \(\ldots\) </td> </tr> <tr> <td style="text-align:center;width: 2.5cm; border-bottom: 2px solid;"> \(S\) </td> <td style="text-align:center;width: 2.5cm; border-bottom: 2px solid;"> 0.0305 </td> <td style="text-align:center;width: 2.5cm; border-bottom: 2px solid;"> 0.204 </td> <td style="text-align:center;width: 2.5cm; border-bottom: 2px solid;"> \(\ldots\) </td> <td style="text-align:center;width: 2.5cm; border-bottom: 2px solid;"> 0.558 </td> <td style="text-align:center;width: 2.5cm; border-bottom: 2px solid;"> 48425 </td> <td style="text-align:center;width: 2.5cm; border-bottom: 2px solid;"> 87165 </td> </tr> </tbody> </table> ] --- exclude: false count: false # EVPPI &ndash; model as a regression problem... <center><img src=./img/brute-force-3_28.png width='48%' title='INCLUDE TEXT HERE'></center> --- exclude: false count: false # EVPPI &ndash; model as a regression problem... <center><img src=./img/brute-force-4_29.png width='48%' title='INCLUDE TEXT HERE'></center> --- exclude: false count: false # EVPPI &ndash; model as a regression problem... <center><img src=./img/brute-force-5_30.png width='51%' title='INCLUDE TEXT HERE'></center> --- exclude: false count: false # EVPPI &ndash; model as a regression problem... - Once the functions `\(\class{olive}{g_t(\bm\phi)}\)` are estimated, can approximate `\begin{align} \evppi&=\E_\bm\phi \left[ \max_t \E_{\bm\psi\mid\bm\phi} [\nb_t(\bm\theta)] \right] - \max_t \E_\bm\theta [\nb_t(\bm\theta)] \\ &\approx\frac{1}{S}\sum_{s=1}^S \max_t \hat{g}_t(\bm\phi_s) - \max_t \frac{1}{S}\sum_{s=1}^S \hat{g}_t(\bm\phi_s) \end{align}` -- exclude: false - **NB**: `\(\class{olive}{g_t(\bm\phi)}\)` can be complex, so need to use .orange[**flexible**] regression methods - **GAMs**: `\(\displaystyle g_t(\bm\phi)=\sum_{q=1}^{Q_\bm\phi} h_t(\phi_{sq})\)` with `\(h_t(\cdot)=\)` smooth functions (cubic polynomials) - very fast, but only if number of important parameters `\(Q_\bm\phi\leq 5\)` (interactions increase model size exponentially) - If `\(Q_\bm\phi >5\)` then use .blue[**Gaussian Process**] regression (GPR) - [Strong et al](https://journals.sagepub.com/doi/full/10.1177/0272989x13505910): original GPR method - [Heath et al](https://onlinelibrary.wiley.com/doi/full/10.1002/sim.6983): based on spatial modelling; can be more computationally efficient - Other methods based on alternative approaches - Most are implemented in the <svg viewBox="0 0 581 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg"> <path d="M581 226.6C581 119.1 450.9 32 290.5 32S0 119.1 0 226.6C0 322.4 103.3 402 239.4 418.1V480h99.1v-61.5c24.3-2.7 47.6-7.4 69.4-13.9L448 480h112l-67.4-113.7c54.5-35.4 88.4-84.9 88.4-139.7zm-466.8 14.5c0-73.5 98.9-133 220.8-133s211.9 40.7 211.9 133c0 50.1-26.5 85-70.3 106.4-2.4-1.6-4.7-2.9-6.4-3.7-10.2-5.2-27.8-10.5-27.8-10.5s86.6-6.4 86.6-92.7-90.6-87.9-90.6-87.9h-199V361c-74.1-21.5-125.2-67.1-125.2-119.9zm225.1 38.3v-55.6c57.8 0 87.8-6.8 87.8 27.3 0 36.5-38.2 28.3-87.8 28.3zm-.9 72.5H365c10.8 0 18.9 11.7 24 19.2-16.1 1.9-33 2.8-50.6 2.9v-22.1z"></path></svg> package [BCEA](https://gianluca.statistica.it/software/bcea/) (see also: <svg viewBox="0 0 496 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg"> <path d="M165.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6zm-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3zm44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9zM244.8 8C106.1 8 0 113.3 0 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C428.2 457.8 496 362.9 496 252 496 113.3 383.5 8 244.8 8zM97.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1zm-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7zm32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1zm-11.4-14.7c-1.6 1-1.6 3.6 0 5.9 1.6 2.3 4.3 3.3 5.6 2.3 1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2z"></path></svg> [https://github.com/giabaio/BCEA](https://github.com/giabaio/BCEA/tree/dev)) --- count: false # Summarising PSA + Research priority .pull-left[ <center><img src=./img/evppi-plot-1.png width='100%' title=''></center> ] .pull-right[ <center><img src=./img/info-rank-1.png width='100%' title=''></center> ] <span style="display:block; margin-top: -20px ;"></span> .small[<svg viewBox="0 0 581 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg"> <path d="M581 226.6C581 119.1 450.9 32 290.5 32S0 119.1 0 226.6C0 322.4 103.3 402 239.4 418.1V480h99.1v-61.5c24.3-2.7 47.6-7.4 69.4-13.9L448 480h112l-67.4-113.7c54.5-35.4 88.4-84.9 88.4-139.7zm-466.8 14.5c0-73.5 98.9-133 220.8-133s211.9 40.7 211.9 133c0 50.1-26.5 85-70.3 106.4-2.4-1.6-4.7-2.9-6.4-3.7-10.2-5.2-27.8-10.5-27.8-10.5s86.6-6.4 86.6-92.7-90.6-87.9-90.6-87.9h-199V361c-74.1-21.5-125.2-67.1-125.2-119.9zm225.1 38.3v-55.6c57.8 0 87.8-6.8 87.8 27.3 0 36.5-38.2 28.3-87.8 28.3zm-.9 72.5H365c10.8 0 18.9 11.7 24 19.2-16.1 1.9-33 2.8-50.6 2.9v-22.1z"></path></svg> package [BCEA](https://gianluca.statistica.it/software/bcea/)] .small[<svg viewBox="0 0 496 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg"> <path d="M165.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6zm-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3zm44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9zM244.8 8C106.1 8 0 113.3 0 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C428.2 457.8 496 362.9 496 252 496 113.3 383.5 8 244.8 8zM97.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1zm-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7zm32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1zm-11.4-14.7c-1.6 1-1.6 3.6 0 5.9 1.6 2.3 4.3 3.3 5.6 2.3 1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2z"></path></svg> [https://github.com/giabaio/BCEA](https://github.com/giabaio/BCEA/tree/dev)] .small[<svg viewBox="0 0 640 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg"> <path d="M624 416H381.54c-.74 19.81-14.71 32-32.74 32H288c-18.69 0-33.02-17.47-32.77-32H16c-8.8 0-16 7.2-16 16v16c0 35.2 28.8 64 64 64h512c35.2 0 64-28.8 64-64v-16c0-8.8-7.2-16-16-16zM576 48c0-26.4-21.6-48-48-48H112C85.6 0 64 21.6 64 48v336h512V48zm-64 272H128V64h384v256z"></path></svg> [https://egon.stats.ucl.ac.uk/projects/BCEAweb](https://egon.stats.ucl.ac.uk/projects/BCEAweb/)] --- exclude: true <style type="text/css"> .right25 { width: 25%; height: 92%; float: right; margin-left: 50px; } </style> # "*Can **we** do it?...*" ### [https://savi.shef.ac.uk/SAVI/](http://savi.shef.ac.uk/SAVI/) <iframe frameborder="no" src="https://savi.shef.ac.uk/SAVI/" style=" position: fixed; top: -15px; bottom: 0px; right: 0px; left: -200px; width: 100%; border: none; margin: 0; padding: 0; overflow: hidden; z-index: 999999; height: 120%; ms-transform: scale(0.45); -moz-transform: scale(0.45); -o-transform: scale(0.45); -webkit-transform: scale(0.45); transform: scale(0.65); "></iframe> .right25[ <span style="display:block; margin-top: 70px ;"></span> <center><img src=./img/bob-the-builder.jpg width='90%' title=''></center> ] --- # Research priority ## Expected value of **sample** information <span style="display:block; margin-top: 0px ;"></span> <center><img src=./img/voi_scheme1.png width='65%' title='INCLUDE TEXT HERE'></center> --- count: false # Research priority ## Expected value of **sample** information <span style="display:block; margin-top: -0px ;"></span> <center><img src=./img/voi_scheme2.png width='65%' title='INCLUDE TEXT HERE'></center> <span style="display:block; margin-top: 10px ;"></span> .small[Stolen from various presentations by [Anna Heath](https://sites.google.com/site/annaheathstats/)] --- count: false <style type="text/css"> /* Some ad-hoc text boxes */ .box-left { position: fixed; /* if I leave the top spacing then Chrome cuts out the bottom-left part of the footer top: 96.6%; */ bottom: 56.8%; left: 43%; text-align: left; width: 12%; font-size: 60%; text-align: center; color: blue; } .box-right { position: fixed; /* if I leave the top spacing then Chrome cuts out the bottom-left part of the footer top: 96.6%; */ bottom: 57.2%; left: 60%; text-align: left; width: 12%; font-size: 60%; text-align: center; color: magenta; } .arrow1 { position: fixed; top: 20.0%; left: 25%; width: 12%; text-align: center; color: black; } .arrow2 { position: fixed; top: 20.0%; left: 60%; width: 12%; text-align: center; color: black; } </style> # Research priority ## Expected value of **sample** information .alignright[.small[<svg viewBox="0 0 512 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg"> <g groupmode="layer" id="layer6" label="icon"> <path style="stroke-width:0.07717" d="m 115.59247,222.50738 c -9.72601,0 -18.734334,7.07397 -24.986084,14.41694 -5.061076,5.95394 -7.591049,15.13238 -7.591049,22.77486 0,7.6402 2.870733,16.13879 7.931808,22.09274 6.152292,7.34297 12.778135,12.37522 22.603045,12.37522 8.83314,0 19.33786,-3.70766 25.389,-9.76105 6.1523,-6.05338 11.26931,-16.21737 11.26931,-25.04826 0,-8.73142 -2.39548,-19.18696 -8.54776,-25.24039 -6.15229,-6.15455 -17.33741,-11.61061 -26.06883,-11.61061 z m 174.6438,2.39944 c -10.12046,0 -18.35798,3.98058 -24.70919,11.93724 -5.15827,6.38628 -7.73967,13.66255 -7.73967,21.82831 0,8.27088 2.5814,15.59914 7.73967,21.98538 6.35064,7.95667 14.58873,11.93729 24.70919,11.93729 9.03149,0 16.67002,-3.25109 22.9212,-9.74241 6.25399,-6.59703 9.37846,-14.65259 9.37846,-24.18026 0,-9.422 -3.17646,-17.37639 -9.52708,-23.86775 -6.2495,-6.59477 -13.8428,-9.89721 -22.77258,-9.89721 z M 255.99998,7.9999981 C 119.03396,7.9999981 7.9999985,119.03394 7.9999985,255.99998 7.9999985,392.96602 119.03396,504 255.99998,504 392.96606,504 504,392.96602 504,255.99998 504,119.03394 392.96606,7.9999981 255.99998,7.9999981 Z M 197.17431,331.79065 h -45.93563 v -18.69931 c -4.66437,5.85506 -15.63869,11.82369 -20.40025,14.50172 -8.33473,4.66267 -19.65377,7.70067 -30.27265,7.70067 -17.167865,0 -32.44885,-5.425 -45.844653,-17.23173 -15.977193,-14.0903 -23.483472,-35.63322 -23.483472,-58.85447 0,-23.61798 8.186103,-42.47212 24.559999,-56.56242 12.999092,-11.21393 28.896046,-20.18949 45.766106,-20.18949 9.82264,0 21.93961,0.88269 30.57214,5.04865 4.96218,2.38076 13.84449,7.98548 19.10448,13.44381 v -98.50164 h 45.93564 v 229.34421 z m 157.21544,-21.13999 c -15.28099,17.58718 -36.66682,26.38303 -64.15348,26.38303 -27.58613,0 -49.01942,-8.79585 -64.30436,-26.38303 -12.60069,-14.44803 -18.90387,-32.19228 -18.90387,-53.23507 0,-18.94908 6.35065,-35.75132 19.0525,-50.4079 15.37988,-17.69285 37.26185,-26.54011 65.64139,-26.54011 26.09766,0 46.93585,8.84558 62.51694,26.54011 12.70185,14.44806 19.05247,31.66734 19.05247,51.66468 0.003,20.20586 -6.29921,37.53085 -18.90159,51.97886 z m 38.76899,-199.21794 c 5.35888,-5.35888 11.85868,-8.03746 19.49892,-8.03746 7.64024,0 14.14005,2.67858 19.49893,8.03746 5.35888,5.25945 8.03746,11.70955 8.03746,19.35034 0,7.73965 -2.67858,14.28864 -8.03746,19.64753 -5.25944,5.25944 -11.75925,7.8883 -19.49893,7.8883 -7.64247,0 -14.14227,-2.67861 -19.49892,-8.0375 -5.2594,-5.35885 -7.88829,-11.85871 -7.88829,-19.49892 -0.003,-7.6402 2.62889,-14.0903 7.88829,-19.35029 z m 43.91029,220.24265 H 388.24626 V 185.84118 h 48.82277 z" id="path2"></path> </g></svg> [Jackson et al (2021)](https://www.annualreviews.org/doi/pdf/10.1146/annurev-statistics-040120-010730)]] - EVSI measures the value of reducing uncertainty by running a study of **a given design** `$$\evsi = \E_{\boldsymbol{X}} \left[ \max_t\ \color{blue}{\E_{\boldsymbol\theta \mid \boldsymbol{X}} \left[ \nb_t(\boldsymbol\theta) \right]} \right] - \max_{t}\color{magenta}{\E_{\boldsymbol\theta}\left[\nb_{t}(\boldsymbol\theta)\right]}$$` .box-left[ ↑ Value of decision based on **sample** information (for a given study design) ] .box-right[ ↑ Value of decision based on **current** information ] <span style="display:block; margin-top: 85px ;"></span> - Can compare the benefits and costs of a study with given design - To see if a proposed study likely to be a good use of resources - To find the optimal study design --- count: false # Research priority ## Expected value of **sample** information .alignright[.small[<svg viewBox="0 0 512 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg"> <g groupmode="layer" id="layer6" label="icon"> <path style="stroke-width:0.07717" d="m 115.59247,222.50738 c -9.72601,0 -18.734334,7.07397 -24.986084,14.41694 -5.061076,5.95394 -7.591049,15.13238 -7.591049,22.77486 0,7.6402 2.870733,16.13879 7.931808,22.09274 6.152292,7.34297 12.778135,12.37522 22.603045,12.37522 8.83314,0 19.33786,-3.70766 25.389,-9.76105 6.1523,-6.05338 11.26931,-16.21737 11.26931,-25.04826 0,-8.73142 -2.39548,-19.18696 -8.54776,-25.24039 -6.15229,-6.15455 -17.33741,-11.61061 -26.06883,-11.61061 z m 174.6438,2.39944 c -10.12046,0 -18.35798,3.98058 -24.70919,11.93724 -5.15827,6.38628 -7.73967,13.66255 -7.73967,21.82831 0,8.27088 2.5814,15.59914 7.73967,21.98538 6.35064,7.95667 14.58873,11.93729 24.70919,11.93729 9.03149,0 16.67002,-3.25109 22.9212,-9.74241 6.25399,-6.59703 9.37846,-14.65259 9.37846,-24.18026 0,-9.422 -3.17646,-17.37639 -9.52708,-23.86775 -6.2495,-6.59477 -13.8428,-9.89721 -22.77258,-9.89721 z M 255.99998,7.9999981 C 119.03396,7.9999981 7.9999985,119.03394 7.9999985,255.99998 7.9999985,392.96602 119.03396,504 255.99998,504 392.96606,504 504,392.96602 504,255.99998 504,119.03394 392.96606,7.9999981 255.99998,7.9999981 Z M 197.17431,331.79065 h -45.93563 v -18.69931 c -4.66437,5.85506 -15.63869,11.82369 -20.40025,14.50172 -8.33473,4.66267 -19.65377,7.70067 -30.27265,7.70067 -17.167865,0 -32.44885,-5.425 -45.844653,-17.23173 -15.977193,-14.0903 -23.483472,-35.63322 -23.483472,-58.85447 0,-23.61798 8.186103,-42.47212 24.559999,-56.56242 12.999092,-11.21393 28.896046,-20.18949 45.766106,-20.18949 9.82264,0 21.93961,0.88269 30.57214,5.04865 4.96218,2.38076 13.84449,7.98548 19.10448,13.44381 v -98.50164 h 45.93564 v 229.34421 z m 157.21544,-21.13999 c -15.28099,17.58718 -36.66682,26.38303 -64.15348,26.38303 -27.58613,0 -49.01942,-8.79585 -64.30436,-26.38303 -12.60069,-14.44803 -18.90387,-32.19228 -18.90387,-53.23507 0,-18.94908 6.35065,-35.75132 19.0525,-50.4079 15.37988,-17.69285 37.26185,-26.54011 65.64139,-26.54011 26.09766,0 46.93585,8.84558 62.51694,26.54011 12.70185,14.44806 19.05247,31.66734 19.05247,51.66468 0.003,20.20586 -6.29921,37.53085 -18.90159,51.97886 z m 38.76899,-199.21794 c 5.35888,-5.35888 11.85868,-8.03746 19.49892,-8.03746 7.64024,0 14.14005,2.67858 19.49893,8.03746 5.35888,5.25945 8.03746,11.70955 8.03746,19.35034 0,7.73965 -2.67858,14.28864 -8.03746,19.64753 -5.25944,5.25944 -11.75925,7.8883 -19.49893,7.8883 -7.64247,0 -14.14227,-2.67861 -19.49892,-8.0375 -5.2594,-5.35885 -7.88829,-11.85871 -7.88829,-19.49892 -0.003,-7.6402 2.62889,-14.0903 7.88829,-19.35029 z m 43.91029,220.24265 H 388.24626 V 185.84118 h 48.82277 z" id="path2"></path> </g></svg> [Jackson et al (2021)](https://www.annualreviews.org/doi/pdf/10.1146/annurev-statistics-040120-010730)]] - EVSI measures the value of reducing uncertainty by running a study of **a given design** `$$\evsi = \E_{\boldsymbol{X}} \left[ \max_t\ \color{blue}{\E_{\boldsymbol\theta \mid \boldsymbol{X}} \left[ \nb_t(\boldsymbol\theta) \right]} \right] - \max_{t}\color{magenta}{\E_{\boldsymbol\theta}\left[\nb_{t}(\boldsymbol\theta)\right]}$$` .box-left[ ↑ Value of decision based on **sample** information (for a given study design) ] .box-right[ ↑ Value of decision based on **current** information ] <span style="display:block; margin-top: 85px ;"></span> - Can compare the benefits and costs of a study with given design - To see if a proposed study likely to be a good use of resources - To find the optimal study design - Computationally complex - Requires specific knowledge of the model for (future/hypothetical) data collection - Again, recent methods have improved efficiency .alignright[.small[<svg viewBox="0 0 512 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg"> <g groupmode="layer" id="layer6" label="icon"> <path style="stroke-width:0.07717" d="m 115.59247,222.50738 c -9.72601,0 -18.734334,7.07397 -24.986084,14.41694 -5.061076,5.95394 -7.591049,15.13238 -7.591049,22.77486 0,7.6402 2.870733,16.13879 7.931808,22.09274 6.152292,7.34297 12.778135,12.37522 22.603045,12.37522 8.83314,0 19.33786,-3.70766 25.389,-9.76105 6.1523,-6.05338 11.26931,-16.21737 11.26931,-25.04826 0,-8.73142 -2.39548,-19.18696 -8.54776,-25.24039 -6.15229,-6.15455 -17.33741,-11.61061 -26.06883,-11.61061 z m 174.6438,2.39944 c -10.12046,0 -18.35798,3.98058 -24.70919,11.93724 -5.15827,6.38628 -7.73967,13.66255 -7.73967,21.82831 0,8.27088 2.5814,15.59914 7.73967,21.98538 6.35064,7.95667 14.58873,11.93729 24.70919,11.93729 9.03149,0 16.67002,-3.25109 22.9212,-9.74241 6.25399,-6.59703 9.37846,-14.65259 9.37846,-24.18026 0,-9.422 -3.17646,-17.37639 -9.52708,-23.86775 -6.2495,-6.59477 -13.8428,-9.89721 -22.77258,-9.89721 z M 255.99998,7.9999981 C 119.03396,7.9999981 7.9999985,119.03394 7.9999985,255.99998 7.9999985,392.96602 119.03396,504 255.99998,504 392.96606,504 504,392.96602 504,255.99998 504,119.03394 392.96606,7.9999981 255.99998,7.9999981 Z M 197.17431,331.79065 h -45.93563 v -18.69931 c -4.66437,5.85506 -15.63869,11.82369 -20.40025,14.50172 -8.33473,4.66267 -19.65377,7.70067 -30.27265,7.70067 -17.167865,0 -32.44885,-5.425 -45.844653,-17.23173 -15.977193,-14.0903 -23.483472,-35.63322 -23.483472,-58.85447 0,-23.61798 8.186103,-42.47212 24.559999,-56.56242 12.999092,-11.21393 28.896046,-20.18949 45.766106,-20.18949 9.82264,0 21.93961,0.88269 30.57214,5.04865 4.96218,2.38076 13.84449,7.98548 19.10448,13.44381 v -98.50164 h 45.93564 v 229.34421 z m 157.21544,-21.13999 c -15.28099,17.58718 -36.66682,26.38303 -64.15348,26.38303 -27.58613,0 -49.01942,-8.79585 -64.30436,-26.38303 -12.60069,-14.44803 -18.90387,-32.19228 -18.90387,-53.23507 0,-18.94908 6.35065,-35.75132 19.0525,-50.4079 15.37988,-17.69285 37.26185,-26.54011 65.64139,-26.54011 26.09766,0 46.93585,8.84558 62.51694,26.54011 12.70185,14.44806 19.05247,31.66734 19.05247,51.66468 0.003,20.20586 -6.29921,37.53085 -18.90159,51.97886 z m 38.76899,-199.21794 c 5.35888,-5.35888 11.85868,-8.03746 19.49892,-8.03746 7.64024,0 14.14005,2.67858 19.49893,8.03746 5.35888,5.25945 8.03746,11.70955 8.03746,19.35034 0,7.73965 -2.67858,14.28864 -8.03746,19.64753 -5.25944,5.25944 -11.75925,7.8883 -19.49893,7.8883 -7.64247,0 -14.14227,-2.67861 -19.49892,-8.0375 -5.2594,-5.35885 -7.88829,-11.85871 -7.88829,-19.49892 -0.003,-7.6402 2.62889,-14.0903 7.88829,-19.35029 z m 43.91029,220.24265 H 388.24626 V 185.84118 h 48.82277 z" id="path2"></path> </g></svg> [Heath et al (2021)](https://doi.org/10.1177/0272989X20912402)]] - Regression-based .small[(<svg viewBox="0 0 384 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg"> <g label="icon" id="layer6" groupmode="layer"> <path id="path2" d="M 120.19265,177.73779 C 123.18778,77.35076 64.277527,63.999998 64.277527,63.999998 v 31.26245 C 40.834519,83.611374 18.32863,81.929634 18.32863,81.929634 V 337.10903 c 0,0 98.10414,-11.41744 98.10414,84.40952 0,0 36.58424,-153.37442 248.86103,26.48145 0,-61.59342 0.37757,-216.93925 0.37757,-268.28471 C 169.9561,37.131382 120.1931,177.73779 120.1931,177.73779 Z m 187.20631,173.82056 -12.37599,-97.65441 h -0.448 l -40.72819,97.65441 h -17.55994 l -38.9362,-97.65441 h -0.448 l -14.17589,97.65441 h -43.87514 l 28.8015,-169.61925 h 43.42716 l 34.43518,90.6496 36.46566,-90.6496 h 43.87513 l 25.6817,169.61925 h -44.13938 z" style="stroke-width:0.0675239"></path> </g></svg> [Strong et al, 2015](https://pubmed.ncbi.nlm.nih.gov/25810269/))] - Importance Sampling .small[(<svg viewBox="0 0 384 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg"> <g label="icon" id="layer6" groupmode="layer"> <path id="path2" d="M 120.19265,177.73779 C 123.18778,77.35076 64.277527,63.999998 64.277527,63.999998 v 31.26245 C 40.834519,83.611374 18.32863,81.929634 18.32863,81.929634 V 337.10903 c 0,0 98.10414,-11.41744 98.10414,84.40952 0,0 36.58424,-153.37442 248.86103,26.48145 0,-61.59342 0.37757,-216.93925 0.37757,-268.28471 C 169.9561,37.131382 120.1931,177.73779 120.1931,177.73779 Z m 187.20631,173.82056 -12.37599,-97.65441 h -0.448 l -40.72819,97.65441 h -17.55994 l -38.9362,-97.65441 h -0.448 l -14.17589,97.65441 h -43.87514 l 28.8015,-169.61925 h 43.42716 l 34.43518,90.6496 36.46566,-90.6496 h 43.87513 l 25.6817,169.61925 h -44.13938 z" style="stroke-width:0.0675239"></path> </g></svg> [Menzies et al, 2016](https://pubmed.ncbi.nlm.nih.gov/25911600/))] - Gaussian approximation .small[(<svg viewBox="0 0 384 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg"> <g label="icon" id="layer6" groupmode="layer"> <path id="path2" d="M 120.19265,177.73779 C 123.18778,77.35076 64.277527,63.999998 64.277527,63.999998 v 31.26245 C 40.834519,83.611374 18.32863,81.929634 18.32863,81.929634 V 337.10903 c 0,0 98.10414,-11.41744 98.10414,84.40952 0,0 36.58424,-153.37442 248.86103,26.48145 0,-61.59342 0.37757,-216.93925 0.37757,-268.28471 C 169.9561,37.131382 120.1931,177.73779 120.1931,177.73779 Z m 187.20631,173.82056 -12.37599,-97.65441 h -0.448 l -40.72819,97.65441 h -17.55994 l -38.9362,-97.65441 h -0.448 l -14.17589,97.65441 h -43.87514 l 28.8015,-169.61925 h 43.42716 l 34.43518,90.6496 36.46566,-90.6496 h 43.87513 l 25.6817,169.61925 h -44.13938 z" style="stroke-width:0.0675239"></path> </g></svg> [Jalal et al, 2015](https://pubmed.ncbi.nlm.nih.gov/25840900/); <svg viewBox="0 0 384 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg"> <g label="icon" id="layer6" groupmode="layer"> <path id="path2" d="M 120.19265,177.73779 C 123.18778,77.35076 64.277527,63.999998 64.277527,63.999998 v 31.26245 C 40.834519,83.611374 18.32863,81.929634 18.32863,81.929634 V 337.10903 c 0,0 98.10414,-11.41744 98.10414,84.40952 0,0 36.58424,-153.37442 248.86103,26.48145 0,-61.59342 0.37757,-216.93925 0.37757,-268.28471 C 169.9561,37.131382 120.1931,177.73779 120.1931,177.73779 Z m 187.20631,173.82056 -12.37599,-97.65441 h -0.448 l -40.72819,97.65441 h -17.55994 l -38.9362,-97.65441 h -0.448 l -14.17589,97.65441 h -43.87514 l 28.8015,-169.61925 h 43.42716 l 34.43518,90.6496 36.46566,-90.6496 h 43.87513 l 25.6817,169.61925 h -44.13938 z" style="stroke-width:0.0675239"></path> </g></svg> [Jalal and Alarid-Escudero, 2018](https://pubmed.ncbi.nlm.nih.gov/28735563/))] - Moment matching .small[(<svg viewBox="0 0 384 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg"> <g label="icon" id="layer6" groupmode="layer"> <path id="path2" d="M 120.19265,177.73779 C 123.18778,77.35076 64.277527,63.999998 64.277527,63.999998 v 31.26245 C 40.834519,83.611374 18.32863,81.929634 18.32863,81.929634 V 337.10903 c 0,0 98.10414,-11.41744 98.10414,84.40952 0,0 36.58424,-153.37442 248.86103,26.48145 0,-61.59342 0.37757,-216.93925 0.37757,-268.28471 C 169.9561,37.131382 120.1931,177.73779 120.1931,177.73779 Z m 187.20631,173.82056 -12.37599,-97.65441 h -0.448 l -40.72819,97.65441 h -17.55994 l -38.9362,-97.65441 h -0.448 l -14.17589,97.65441 h -43.87514 l 28.8015,-169.61925 h 43.42716 l 34.43518,90.6496 36.46566,-90.6496 h 43.87513 l 25.6817,169.61925 h -44.13938 z" style="stroke-width:0.0675239"></path> </g></svg> [Heath et al, 2018](https://pubmed.ncbi.nlm.nih.gov/29126364/))] - Can be used to drive design of new study (eg sample size calculations) --- count: false # Research priority ## Expected value of **sample** information <span style="display:block; margin-top: -35px ;"></span> .pull-left[ <center><img src=./img/voi1.png width='85%' title='INCLUDE TEXT HERE'></center> ] .pull-right[ <center><img src=./img/voi2.png width='85%' title='INCLUDE TEXT HERE'></center> ] <span style="display:block; margin-top: -25px ;"></span> .small[ <svg viewBox="0 0 496 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg"> <path d="M165.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6zm-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3zm44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9zM244.8 8C106.1 8 0 113.3 0 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C428.2 457.8 496 362.9 496 252 496 113.3 383.5 8 244.8 8zM97.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1zm-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7zm32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1zm-11.4-14.7c-1.6 1-1.6 3.6 0 5.9 1.6 2.3 4.3 3.3 5.6 2.3 1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2z"></path></svg> [https://github.com/giabaio/EVSI](https://github.com/giabaio/EVSI) and [https://github.com/chjackson/voi](https://github.com/chjackson/voi) <svg viewBox="0 0 640 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg"> <path d="M624 416H381.54c-.74 19.81-14.71 32-32.74 32H288c-18.69 0-33.02-17.47-32.77-32H16c-8.8 0-16 7.2-16 16v16c0 35.2 28.8 64 64 64h512c35.2 0 64-28.8 64-64v-16c0-8.8-7.2-16-16-16zM576 48c0-26.4-21.6-48-48-48H112C85.6 0 64 21.6 64 48v336h512V48zm-64 272H128V64h384v256z"></path></svg> [https://egon.stats.ucl.ac.uk/projects/EVSI](https://egon.stats.ucl.ac.uk/projects/EVSI) <span style="display:block; margin-top: 10px ;"></span> <svg viewBox="0 0 512 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg"> <g groupmode="layer" id="layer6" label="icon"> <path style="stroke-width:0.07717" d="m 115.59247,222.50738 c -9.72601,0 -18.734334,7.07397 -24.986084,14.41694 -5.061076,5.95394 -7.591049,15.13238 -7.591049,22.77486 0,7.6402 2.870733,16.13879 7.931808,22.09274 6.152292,7.34297 12.778135,12.37522 22.603045,12.37522 8.83314,0 19.33786,-3.70766 25.389,-9.76105 6.1523,-6.05338 11.26931,-16.21737 11.26931,-25.04826 0,-8.73142 -2.39548,-19.18696 -8.54776,-25.24039 -6.15229,-6.15455 -17.33741,-11.61061 -26.06883,-11.61061 z m 174.6438,2.39944 c -10.12046,0 -18.35798,3.98058 -24.70919,11.93724 -5.15827,6.38628 -7.73967,13.66255 -7.73967,21.82831 0,8.27088 2.5814,15.59914 7.73967,21.98538 6.35064,7.95667 14.58873,11.93729 24.70919,11.93729 9.03149,0 16.67002,-3.25109 22.9212,-9.74241 6.25399,-6.59703 9.37846,-14.65259 9.37846,-24.18026 0,-9.422 -3.17646,-17.37639 -9.52708,-23.86775 -6.2495,-6.59477 -13.8428,-9.89721 -22.77258,-9.89721 z M 255.99998,7.9999981 C 119.03396,7.9999981 7.9999985,119.03394 7.9999985,255.99998 7.9999985,392.96602 119.03396,504 255.99998,504 392.96606,504 504,392.96602 504,255.99998 504,119.03394 392.96606,7.9999981 255.99998,7.9999981 Z M 197.17431,331.79065 h -45.93563 v -18.69931 c -4.66437,5.85506 -15.63869,11.82369 -20.40025,14.50172 -8.33473,4.66267 -19.65377,7.70067 -30.27265,7.70067 -17.167865,0 -32.44885,-5.425 -45.844653,-17.23173 -15.977193,-14.0903 -23.483472,-35.63322 -23.483472,-58.85447 0,-23.61798 8.186103,-42.47212 24.559999,-56.56242 12.999092,-11.21393 28.896046,-20.18949 45.766106,-20.18949 9.82264,0 21.93961,0.88269 30.57214,5.04865 4.96218,2.38076 13.84449,7.98548 19.10448,13.44381 v -98.50164 h 45.93564 v 229.34421 z m 157.21544,-21.13999 c -15.28099,17.58718 -36.66682,26.38303 -64.15348,26.38303 -27.58613,0 -49.01942,-8.79585 -64.30436,-26.38303 -12.60069,-14.44803 -18.90387,-32.19228 -18.90387,-53.23507 0,-18.94908 6.35065,-35.75132 19.0525,-50.4079 15.37988,-17.69285 37.26185,-26.54011 65.64139,-26.54011 26.09766,0 46.93585,8.84558 62.51694,26.54011 12.70185,14.44806 19.05247,31.66734 19.05247,51.66468 0.003,20.20586 -6.29921,37.53085 -18.90159,51.97886 z m 38.76899,-199.21794 c 5.35888,-5.35888 11.85868,-8.03746 19.49892,-8.03746 7.64024,0 14.14005,2.67858 19.49893,8.03746 5.35888,5.25945 8.03746,11.70955 8.03746,19.35034 0,7.73965 -2.67858,14.28864 -8.03746,19.64753 -5.25944,5.25944 -11.75925,7.8883 -19.49893,7.8883 -7.64247,0 -14.14227,-2.67861 -19.49892,-8.0375 -5.2594,-5.35885 -7.88829,-11.85871 -7.88829,-19.49892 -0.003,-7.6402 2.62889,-14.0903 7.88829,-19.35029 z m 43.91029,220.24265 H 388.24626 V 185.84118 h 48.82277 z" id="path2"></path> </g></svg> [Heath et al (2019)](https://doi.org/10.1177/0272989X19837983) ] --- count: false # Research priority ## Expected value of **sample** information <center><img src=./img/voi3.png width='63%' title='INCLUDE TEXT HERE'></center> --- # *"Tell me a sad story in just one slide..."* ### [NICE HTA evaluation methods update](https://www.nice.org.uk/about/what-we-do/our-programmes/nice-guidance/nice-technology-appraisal-guidance/changes-to-health-technology-evaluation) (2021) .medium[ > *2.16. The use of Expected Value of Perfect Information (EVPI) will .red[**not**] be adopted into the NICE methods. Stakeholders raised concerns about this proposal and the majority disagreed with it. It was noted that the added value of EVPI and how it would be used in decision-making was unclear as experiences from other countries suggested that its added value to decision making is minimal. There were concerns that it would add complexity to decision making, and the additional burden for analysts and reviewers may not be worth it. On the other hand, .blue[some stakeholders argued that the proposal did not go far enough and should include expected value of partially perfect information (EVPPI) and expected value of sample information (EVSI)].* ] --- count: false <style type="text/css"> .medium { font-size: 90% } </style> # *"Tell me a sad story in just one slide..."* ### [NICE HTA evaluation methods update](https://www.nice.org.uk/about/what-we-do/our-programmes/nice-guidance/nice-technology-appraisal-guidance/changes-to-health-technology-evaluation) (2021) .medium[ > *2.16. The use of Expected Value of Perfect Information (EVPI) will .red[**not**] be adopted into the NICE methods. Stakeholders raised concerns about this proposal and the majority disagreed with it. It was noted that the added value of EVPI and how it would be used in decision-making was unclear as experiences from other countries suggested that its added value to decision making is minimal. There were concerns that it would add complexity to decision making, and the additional burden for analysts and reviewers may not be worth it. On the other hand, .blue[some stakeholders argued that the proposal did not go far enough and should include expected value of partially perfect information (EVPPI) and expected value of sample information (EVSI)].* ] <span style="display:block; margin-top: 30px ;"></span> .pull-left[ <center><img width="580!important;" src="img/careless-whisper.gif" title=""></center> ] .pull-right[ .medium[ - Push from industrial representatives, despite attempts at clarifying/simplyfing concepts/guidelines - CADHT actually say > *When the decision problem includes consideration of further research to inform future decisions, .blue[a value-of-information analysis should be undertaken as part of the reference case]. [...] To identify these critical values and correctly quantify the impact of a parameter taking a specific value (on both the probability of an intervention being cost-effective and the expected net benefit), .blue[recent methodological work suggests that a two-stage expected value of perfect parameter information analysis may be useful]* ] ] --- # Conclusions - HTA is fundamentally based on statistical modelling &ndash; and statisticians should be much more heavily involved in the whole process - More modellers with stronger statistical background &ndash; often (**though not always!**) use of suboptimal tools is a red flag for lack of necessary sophistication in statistical modelling - More statisticians developing methods, sitting on panels and creating critical mass --- count: false # Conclusions - HTA is fundamentally based on statistical modelling &ndash; and statisticians should be much more heavily involved in the whole process - More modellers with stronger statistical background &ndash; often (**though not always!**) use of suboptimal tools is a red flag for lack of necessary sophistication in statistical modelling - More statisticians developing methods, sitting on panels and creating critical mass <span style="display:block; margin-top: 40px ;"></span> .pull-left[ - VoI methods can be very helpful - The can address multiple questions, including research prioritisation - Vital in issues such as managed entry/conditional reimbursment - Still a battle... ] --- count: false # "*Where can I look for more, you ask?...*" [https://www.convoi-group.org/](https://www.convoi-group.org/) <iframe frameborder="no" src="https://www.convoi-group.org/" style=" position: fixed; top: 30px; bottom: 0px; right: 0px; width: 140%; border: none; margin: 0; padding: 0; overflow: hidden; z-index: 999999; height: 100%; ms-transform: scale(0.45); -moz-transform: scale(0.45); -o-transform: scale(0.45); -webkit-transform: scale(0.45); transform: scale(0.70); "> </iframe> --- exclude: true # References NULL --- class: thankyou-michelle