Bayesian computations for Value of Information measures using Gaussian processes, INLA and Moment Matching

   

Authors

Affiliations

Gianluca Baio

UCL

Anna Heath

Sick Kids Hospital (Toronto)

Ioanna Manolopoulou

UCL

Introduction

Understanding the impact of uncertainty on decision is an important aspect of health-economic decision making. While health-economic decision are typically made in a decision theoretic context, which quantifies the “best” decision under current evidence, the approval of a new health-care technology often comes with some risk. Therefore, funding bodies are often concerned with the impact of the parameter uncertainty on the optimal decision.

The analysis of the Value of Information (VoI) is an increasingly popular method for quantifying the decision uncertainty. However, despite being a theoretically valid measure of decision uncertainty, its practical use has been limited by the immense computational power required to calculate typical VoI summaries, such as Expected Value of Partial Perfect Information (EVPPI) and the Expected Value of Sample Information (EVSI).

For the EVPPI, recent developments using Bayesian non-parametric regression have vastly improved the computational time required but when larger subsets are considered the computational time is still prohibitive. To begin, this research focused on reducing the computation time for the EVPPI still further so that VoI measures can be used in standard Health Economic evaluations.

INLA is an algorithm for fast approximate Bayesian inference that quickly approximates the marginal posteriors for a wide class of models. As part of this project, we worked on adapting the method of approximating the EVPPI such that INLA can be used to fit the Bayesian non-parametric regression.

For the EVSI, a method based on moment matching has been developed to reduce the computational time required produce an estimate. This method is based on the Bayesian interpretation of the EVSI measure and the research project will continue by considering way to extend this methodology so the EVSI can be used to find optimal trial designs. Part of this research has been devoted to developing a web-application including graphical tools to visualise and analyse the EVSI.

EVPPI with Gaussian Processes and INLA (Heath, Manolopoulou, and Baio 2016)

The Expected Value of Partial (Perfect) Information (EVPPI) is a measure of the Value of Information associated with gathering evidence that can reduce uncertainty about a set of parameters of interest in a health economic model.

Typically, we can split the vector of model parameters $\mathit{\theta }$ into two components, $\mathit{\varphi }$ (the parameters we are interest in, which we may gather more information about) and $\mathit{\psi }$, the set of “nuisance” parameters. The monetary value (utility) of a given intervention $t$ is quantified by the (monetary) Net Benefit, ${\text{NB}}_t(\mathit{\theta })={\text{NB}}_t(\mathit{\varphi },\mathit{\psi })$. The EVPPI is the difference between the expected maximum utility (NB) if we were able to learn the value of the important parameters $\mathit{\varphi }$ perfectly and the maximum expected utility calculated under current uncertainty: $$\text{EVPPI}={\text{E}}_{\mathit{\varphi }}\left[\underset{t}{max}{\text{E}}_{\mathit{\psi }\mid \mathit{\varphi }}[{\text{NB}}_t(\mathit{\varphi },\mathit{\psi })]\right]-\underset{t}{max}{\text{E}}_{\mathit{\theta }}[{\text{NB}}_t(\mathit{\varphi },\mathit{\psi })].$$ Computationally, the EVPPI is hard because it involves the maximisation of an expectation calculated with respect to a conditional distribution. Thus, unless simplifying assumptions hold (which usually don’t), it can involve a hard and lengthy computation, particularly when the size of the vector $\mathit{\varphi }$ is large.

We’ve built on the existing literature to combine a framework based on Gaussian Process Regression to estimate ${\text{E}}_{\mathit{\psi }\mid \mathit{\varphi }}[{\text{NB}}_t(\mathit{\varphi },\mathit{\psi })]$ and Integrated Nested Laplace Approximation (INLA), a very fast method for Bayesian computation. This generates accurate estimates with substantial computational savings (Heath, Manolopoulou, and Baio 2017, 2016).

EVSI with the Moment Matching method (Heath, Manolopoulou, and Baio 2019)

Generally speaking and assuming that the utility associated with the application of a given intervention is represented by the monetary net benefit $\text{NB}(\mathit{\theta }{)}_t=k\text{E}[e\mid \mathit{\theta };t]-\text{E}[c\mid \mathit{\theta };t]$, the Expected Value of Sample Information is computed as $$\text{EVSI}={\text{E}}_{\mathbf{X}}\left[\underset{t}{max}\underset{\text{Value of decision based on sample information (for a given study design)}}{\underbrace{{\text{E}}_{\mathit{\theta }\mid \mathbf{X}}[{\text{NB}}_t(\mathit{\theta })]}}\right]-\underset{\text{Value of decision based on currentinformation}}{\underbrace{\underset{t}{max}{\text{E}}_{\mathit{\theta }}[{\text{NB}}_t(\mathit{\theta })]}}$$

The EVSI measures the value of reducing uncertainty by running a study of a given design and can be used to compare the benefits and costs of a study with given design, thus helping in both assessment of the impact of parameters uncertainty on the decision-making process and research prioritisation.

Assuming we are only interested in a pairwise comparison (eg the new intervention vs the standard of care), the EVSI can be re-expressed as $$\text{EVSI}={\text{E}}_{\mathbf{X}}[max\{0,{\text{E}}_{\mathit{\theta }\mid \mathbf{X}}[\text{INB}(\mathit{\theta })]\}]-max\{0,{\text{E}}_{\mathit{\theta }}[\text{INB}(\mathit{\theta })]\}.$$

In practical terms, the inner expectation ${\mu }^{\mathbf{X}}={\text{E}}_{\mathit{\theta }\mid \mathbf{X}}[\text{INB}(\mathit{\theta })]$ is the complicated part to estimate. The ``standard’’ option is to use nested Monte Carlo integration, but this requires a very large number of simulations and thus is impractical.

In effect, the objective is to approximate the distribution $p({\mu }^{\mathbf{X}})$ using the $S$ simulations available from the process of probabilistic sensitivity analysis (PSA, which is mandatory in many jurisdictions). The approximation is made by matching the first two moments of the distribution

• ${\text{E}}_{\mathbf{X}}\left[{\mu }^{\mathbf{X}}\right]={\text{E}}_{\mathbf{X}}\left[{\text{E}}_{\mathit{\theta }\mid \mathbf{X}}[\text{INB}(\mathit{\theta })]\right]={\text{E}}_{\mathit{\theta }}[\text{INB}(\mathit{\theta })]$

• ${\text{Var}}_{\mathbf{X}}\left[{\mu }^{\mathbf{X}}\right]={\text{Var}}_{\mathit{\theta }}[\text{INB}(\mathit{\theta })]-{\text{E}}_{\mathbf{X}}\left[{\text{Var}}_{\mathit{\theta }\mid \mathbf{X}}[\text{INB}(\mathit{\theta })]\right]$

In particular, the variance ${\text{Var}}_{\mathbf{X}}\left[{\mu }^{\mathbf{X}}\right]$ can be efficiently estimated using only a limited number $30<Q<50$ of simulations from the posterior distribution of the parameters given the “hypothetical” data $\mathbf{X}$. Subsequently the original PSA samples for the INB can be rescaled and used to approximate the EVSI as $$\text{EVSI}=\frac{1}{S}\sum _{s=1}^{S}max\{0,{\eta }_s^{\mathbf{X}}\}-max\{0,\mu \},$$ where ${\eta }_s^{\mathbf{X}}$ is the $s-$th simulated rescaled value from the PSA samples of INB$(\mathit{\theta })$ and $\mu$ is the mean of these PSA samples.

Computational tools

The moment matching method is implemented in the R package EVSI, which is also available as a web-application here.

Last updated: 6 March 2024

Relevant publications

Filters Active - 0

20153
20163
20175
20183
20192
20203
20212
20241

Book1
Journal article13
Preprint/Working paper7
Uncategorised1

Search:

2024
A Heath, I Manolopoulou, G Baio, N Welton. (2024). "Value of Information for clinical trial design: the importance of considering all relevant comparators". Pharmacoeconomics
2021
A Adamson, L Portas, S Accordini, A Marcon, D Jarvis, G Baio, C Minelli. (2021). "Communication of personalised disease risk by general practitioners to motivate smoking cessation in England: A cost-effectiveness and research prioritisation study". Addiction Publication DOI
C Jackson, G Baio, A Heath, M Strong, N Welton, E Wilson. (2021). "Value of Information Analysis in Models to inform Health Policy". Annual Review of Statistics and Its Application GitHub repo
2020
NR Kunst, E Wilson, D Glynn, F Alarid-Escudero, G Baio, A Brennan, M Fairley, JD Goldhaber-Fiebert, C Jackson, H Jalal, NA Menzies, M Strong, H Thom, A Heath. (2020). "Computing the Expected Value of Sample Information Efficiently: Practical Guidance and Recommendations for Four Model-Based Methods". Value in Health Publication
A Heath, NR Kunst, C Jackson, M Strong, F Alarid-Escudero, JD Goldhaber-Fiebert, G Baio, NA Menzies, H Jalal. (2020). "Calculating the Expected Value of Sample Information in Practice: Considerations from three case studies". Medical Decision Making DOI
NR Kunst, ECF Wilson, F Alarid-Escudero, G Baio, A Brennan, M Fairley, D Glynn, JD Goldhaber-Fiebert, C Jackson, H Jalal, NA Menzies, M Strong, H Thom, A Heath. (2020). "Practical Considerations for the Efficient Computation of the Expected Value of Sample Information to Prioritize Research in Health Care". Medical Decision Making
2019
A Heath, I Manolopoulou, G Baio. (2019). "Estimating the Expected Value of Sample Information Across Different Sample Sizes Using Moment Matching and Nonlinear Regression". Medical Decision Making GitHub repo DOI
A Heath, NR Kunst, C Jackson, M Strong, F Alarid-Escudero, JD Goldhaber-Fiebert, G Baio, NA Menzies, H Jalal. (2019). "Calculating the Expected Value of Sample Information in Practice: Considerations from Three Case Studies". Arxiv
2018
A Heath, G Baio. (2018). "Calculating the Expected Value of Sample Information using Efficient Nested Monte Carlo: A Tutorial". Value in Health GitHub repo DOI
A Heath, G Baio. (2018). "Calculating the Expected Value of Sample Information using Efficient Nested Monte Carlo: A Tutorial". Arxiv
A Heath, I Manolopoulou, G Baio. (2018). "Estimating the Expected Value of Sample Information across Different Sample Sizes using Moment Matching and Non-Linear Regression". Arxiv
2017
A Heath, G Baio. (2017). "An Efficient Calculation Method for the Expected Value of Sample Information: Can we do it? Yes, we can". Arxiv
G Baio, A Berardi, A Heath. (2017). Bayesian Cost-Effectiveness Analysis with the R package BCEA. Book website Book website DOI
A Heath, G Baio. (2017). "Value of sample information as a tool in clinical trial design". Trials Publication
A Heath, I Manolopoulou, G Baio. (2017). "A Review of Methods for the Analysis of the Expected Value of Information". Medical Decision Making Publication DOI
A Heath, I Manolopoulou, G Baio. (2017). "Efficient Monte Carlo Estimation of the Expected Value of Sample Information Using Moment Matching". Medical Decision Making GitHub repo DOI
2016
A Heath, I Manolopoulou, G Baio. (2016). "Efficient Monte Carlo Estimation of the Expected Value of Sample Information using Moment Matching". Arxiv
A Heath, I Manolopoulou, G Baio. (2016). "Estimating the expected value of partial perfect information in health economic evaluations using Integrated Nested Laplace Approximation". Statistics in Medicine Publication DOI
G Baio, A Heath. (2016). "When simple becomes complicated: Why Excel should lose its place at the top table". Global & Regional Health Technology Assessment Publication DOI
2015
C Minelli, G Baio. (2015). "Value of Information: A Tool to Improve Research Prioritization and Reduce Waste". PLoS Medicine Publication DOI
A Heath, I Manolopoulou, G Baio. (2015). "A Review of Methods for the Analysis of the Expected Value of Information". Arxiv
A Heath, I Manolopoulou, G Baio. (2015). "Estimating the Expected Value of Partial Perfect Information in Health Economic Evaluations using Integrated Nested Laplace Approximation". Arxiv