Health technology assessment (HTA)

Objective: Combine costs and benefits of a given intervention into a rational scheme for allocating resources

Health technology assessment (HTA)

Objective: Combine costs and benefits of a given intervention into a rational scheme for allocating resources

Health technology assessment (HTA)

Objective: Combine costs and benefits of a given intervention into a rational scheme for allocating resources

Health technology assessment (HTA)

Objective: Combine costs and benefits of a given intervention into a rational scheme for allocating resources

Health technology assessment (HTA)

For each module, we may need/use different/specific packages! (the "R-HTA-verse"?)

Health technology assessment (HTA)

For each module, we may need/use different/specific packages! (the "R-HTA-verse"?)

R & HTA

What is R?

• R is a very powerful statistical software
  • Specifically designed for statistical analysis
  • Very large community of contributors – basically you can find code/packages to do any statistical analysis you need
  • Open source and free

But...

"Transparency is in the eye of the beholder"

(Andy Briggs at the R-HTA workshop – October 2020)

BCEA

A R package for (Bayesian) cost-effectiveness analysis

BCEA and its use directly in R are designed with these objectives in mind

1. Checking the model assumptions

   • Do we mean what we mean (eg in terms of PSA simulations)?...
   • Simulation error (especially, but not only, for a Bayesian approach)

BCEA

A R package for (Bayesian) cost-effectiveness analysis

BCEA and its use directly in R are designed with these objectives in mind

1. Checking the model assumptions

   • Do we mean what we mean (eg in terms of PSA simulations)?...
   • Simulation error (especially, but not only, for a Bayesian approach)

2. Produce the base-case economic evaluation

   • What’s the most cost-effective intervention, given current evidence?
   • Cost-effectiveness plane, Expected Incremental Benefit (as a function of $k$),...

BCEA

A R package for (Bayesian) cost-effectiveness analysis

BCEA and its use directly in R are designed with these objectives in mind

1. Checking the model assumptions

   • Do we mean what we mean (eg in terms of PSA simulations)?...
   • Simulation error (especially, but not only, for a Bayesian approach)

2. Produce the base-case economic evaluation

   • What’s the most cost-effective intervention, given current evidence?
   • Cost-effectiveness plane, Expected Incremental Benefit (as a function of $k$),...

3. Perform uncertainty analysis

   • Standard PSA (mandatory): Cost-effectiveness Plane, CEAC, ...
   • Fairly easy (but not always used): CEAF
   • More advanced/"too difficult" (rarely used): EVP(P)I/EVSI

BCEA

A R package for (Bayesian) cost-effectiveness analysis

BCEA and its use directly in R are designed with these objectives in mind

1. Checking the model assumptions

   • Do we mean what we mean (eg in terms of PSA simulations)?...
   • Simulation error (especially, but not only, for a Bayesian approach)

2. Produce the base-case economic evaluation

   • What’s the most cost-effective intervention, given current evidence?
   • Cost-effectiveness plane, Expected Incremental Benefit (as a function of $k$),...

3. Perform uncertainty analysis

   • Standard PSA (mandatory): Cost-effectiveness Plane, CEAC, ...
   • Fairly easy (but not always used): CEAF
   • More advanced/"too difficult" (rarely used): EVP(P)I/EVSI

4. Standardised reporting

   • Graphical tools (use excellent R facilities)
   • Embed code in structured reports (docx/pdf)

How does BCEA work?

> # Install BCEA (only required once and needs an internet connection!). 
> 
> # You can either get the "official" version from CRAN
> install.packages("BCEA")
> 
> # Or the development version from the GitHub repository
> remotes::install_github("giabaio/BCEA")               # stable version (2.4-2)
> 
> remotes::install_github("giabaio/BCEA",ref="dev")     # development version (2.4-2)Copy Code

> library(dplyr)  # (Not necessary - helpful for data manipulation!)
> 
> library(BCEA)   # Then loads the package (so you can access its functions)
> data(Vaccine)   # Loads an example datasetCopy Code

> # The object 'Vaccine' contains a matrix 'vaccine_mat', with all the simulated values for the many model 
> # parameters BCEA can create a matrix with the underlying model simulations starting from various formats 
> # (BUGS/R/Excel) and can get rid of "redundant" columns (those that are linear combination of each other...)
> inp = createInputs(vaccine_mat, print_is_linear_comb = FALSE)
> 
> # Visualise the output: "piping" ('%>%') + nicer visualisation
> inp$mat %>% as_tibble()Copy Code

# A tibble: 1,000 × 56
   Adverse.e…¹ Death…² Death…³ Death…⁴ GP.1.1. GP.2.1. GP.2.2. Hospi…⁵ Hospi…⁶ Hospi…⁷ Infec…⁸ Infec…⁹ Infec…˟ Mild.…˟ Mild.…˟ Mild.…˟ Pneum…˟ Pneum…˟
         <dbl>   <dbl>   <dbl>   <dbl>   <dbl>   <dbl>   <dbl>   <dbl>   <dbl>   <dbl>   <dbl>   <dbl>   <dbl>   <dbl>   <dbl>   <dbl>   <dbl>   <dbl>
 1        1466       1       0       0    1664     958     230       0       1       0    5992    3401     876     691     405     102      18      15
 2        5329       1       1       0    1414     748     276       0       0       1    7471    4024    1536     570     308     112      12      15
 3        5203       1       1       0     809     489      80       0       0       0    6718    4300     788     332     214      40      10       2
 4        2351       2       0       0    1761    1157     261       1       0       0    4837    3269     702     739     479     109      19      17
 5        8303       1       2       0    2472     964     432       1       1       0    4749    1894     846    1049     425     177      25      11
 6        3607       1       1       0    2224    1342     260       1       0       0    4938    2976     596     915     560     100      35      21
 7        6304       4       1       1    3478    1107     591       2       1       0   11080    3547    2045    1505     482     247      66      19
 8        4337       1       1       1    1483     799     189       0       0       0    3867    2164     525     622     333      65      22       7
 9        5482       0       0       0    1587     798     279       0       0       0    5163    2532     910     675     332     115      25      11
10        3125       2       2       0    2578    1681     243       0       0       0    7265    4766     700    1063     711     102      41      17
# … with 990 more rows, 38 more variables: Pneumonia.2.2. <dbl>, Trt.1.1.1. <dbl>, Trt.2.1.1. <dbl>, Trt.1.2.1. <dbl>, Trt.2.2.1. <dbl>,
#   Trt.1.2.2. <dbl>, Trt.2.2.2. <dbl>, beta.1. <dbl>, beta.2. <dbl>, beta.3. <dbl>, beta.4. <dbl>, beta.5. <dbl>, beta.6. <dbl>, beta.7. <dbl>,
#   delta <dbl>, eta <dbl>, gamma.1. <dbl>, gamma.2. <dbl>, lambda <dbl>, n.1.2. <dbl>, n.2.2. <dbl>, phi <dbl>, pi.1.2. <dbl>, psi.1. <dbl>,
#   psi.2. <dbl>, psi.3. <dbl>, psi.4. <dbl>, psi.5. <dbl>, psi.6. <dbl>, psi.7. <dbl>, psi.8. <dbl>, q.1. <dbl>, q.4. <dbl>, q.5. <dbl>, q.6. <dbl>,
#   q.7. <dbl>, rho.2. <dbl>, xi <dbl>, and abbreviated variable names ¹​Adverse.events, ²​Death.1.1., ³​Death.2.1., ⁴​Death.2.2., ⁵​Hospital.1.1.,
#   ⁶​Hospital.2.1., ⁷​Hospital.2.2., ⁸​Infected.1.1., ⁹​Infected.2.1., ˟​Infected.2.2., ˟​Mild.Compl.1.1., ˟​Mild.Compl.2.1., ˟​Mild.Compl.2.2.,
#   ˟​Pneumonia.1.1., ˟​Pneumonia.2.1.Copy Code

> # Defines the number of simulations considered
> n.sims=inp$mat %>% nrow() 
> # applies the function 'nrow' (number of rows) to the object 'inp$mat'
> # NB: Since R 4.1.0, can also use 'native' pipe ('|>')
> # (probably a bit quicker, but in most cases, may be immaterial...)
> 
> # Aggregates the model inputs to compute (e,c)
> QALYs.inf = QALYs.pne <- QALYs.hosp <- QALYs.adv <- QALYs.death <- matrix(0,n.sims,2)
> for (t in 1:2) {
+   QALYs.inf[,t] = ((Infected[,t,1] + Infected[,t,2])*omega[,1]/365)/N
+   QALYs.pne[,t] = ((Pneumonia[,t,1] + Pneumonia[,t,2])*omega[,4]/365)/N
+   QALYs.hosp[,t] = ((Hospital[,t,1] + Hospital[,t,2])*omega[,5]/365)/N
+   QALYs.death[,t] = ((Death[,t,1] + Death[,t,2])*omega[,6])/N
+ }
> QALYs.adv[,2] = (Adverse.events*omega[,7]/365)/N
> e = -(QALYs.inf + QALYs.pne + QALYs.adv + QALYs.hosp + QALYs.death) + ...Copy Code

> cbind(eff,cost) %>% as_tibble(.name_repair="universal")       # ensures that the columns are namedCopy Code

# A tibble: 1,000 × 4
   Status.Quo...1 Vaccination...2 Status.Quo...3 Vaccination...4
            <dbl>           <dbl>          <dbl>           <dbl>
 1      -0.00105        -0.000899          10.4            16.3 
 2      -0.000884       -0.000732           5.83            9.37
 3      -0.000890       -0.000698           5.78           15.9 
 4      -0.00164        -0.00114           12.2            18.7 
 5      -0.00135        -0.000957           9.79           16.5 
 6      -0.00143        -0.000936           6.56            9.69
 7      -0.000960       -0.00105            8.45           11.3 
 8      -0.00181        -0.00139            6.76            9.99
 9      -0.000842       -0.000556           3.60           10.1 
10      -0.00168        -0.00105            4.09           11.0 
# … with 990 more rowsCopy Code

How does BCEA work?

• At this point, we are ready to call the function bcea that runs the economic analysis, for example something like

> treats = c("Status quo","Vaccination")
> m = bcea(e=eff,c=cost,ref=2,interventions=treats,Kmax=50000)Copy Code

• The inputs to the function are
  • eff: a matrix containing the simulations for the clinical benefits (that is $n_{\rm{sim}}\times n_{\rm{int}}$ values)
  • cost: a matrix containing the simulations for the costs (that is $n_{\rm{sim}}\times n_{\rm{int}}$ values)
  • ref: an indication of which intervention is to be taken as reference (default: the intervention in the first column of e or c)
  • interventions: a vector of labels for the interventions being compared
  • Kmax: the maximum value of $k$, the parameter of willingness to pay

How does BCEA work?

• At this point, we are ready to call the function bcea that runs the economic analysis, for example something like

> treats = c("Status quo","Vaccination")
> m = bcea(e=eff,c=cost,ref=2,interventions=treats,Kmax=50000)Copy Code

• The inputs to the function are
  • eff: a matrix containing the simulations for the clinical benefits (that is $n_{\rm{sim}}\times n_{\rm{int}}$ values)
  • cost: a matrix containing the simulations for the costs (that is $n_{\rm{sim}}\times n_{\rm{int}}$ values)
  • ref: an indication of which intervention is to be taken as reference (default: the intervention in the first column of e or c)
  • interventions: a vector of labels for the interventions being compared
  • Kmax: the maximum value of $k$, the parameter of willingness to pay

• The output is an object m containing several elements

> names(m)Copy Code

 [1] "n_sim"         "n_comparators" "n_comparisons" "delta_e"       "delta_c"       "ICER"          "Kmax"          "k"             "ceac"         
[10] "ib"            "eib"           "kstar"         "best"          "U"             "vi"            "Ustar"         "ol"            "evi"          
[19] "ref"           "comp"          "step"          "interventions" "e"             "c"Copy Code

How does BCEA work?

Can visualise the output in various formats (tables/graphs)

> # The 'summary' "method" produces a tabular output
> summary(m)Copy Code

Cost-effectiveness analysis summary 
Reference intervention:  Vaccination
Comparator intervention: Status quo
Optimal decision: choose Status quo for k < 20100 and Vaccination for k >= 20100
Analysis for willingness to pay parameter k = 25000
            Expected net benefit
Vaccination              -36.054
Status quo               -34.826
                             EIB  CEAC  ICER
Vaccination vs Status quo 1.2284 0.529 20098
Optimal intervention (max expected net benefit) for k = 25000: Vaccination
EVPI 2.4145Copy Code

How does BCEA work?

Can visualise the output in various formats (tables/graphs)

> # The 'plot' "method" produces a *specific* 
> # version of graphical output
> plot(m)Copy Code

> ceplane.plot(
+   m,wtp=20000,xlim=c(-.002,.002),ylim=c(-10,20)
+ )Copy Code

Specialised plots

• Can generate a contour plot of the cost-effectiveness plane and estimate the proportion of points in each quadrant

> # "Basic" contourplot
> contour(m)Copy Code

Specialised plots

The specialised function contour2 also shows the sustainability area

> contour2(m)Copy Code

> contour2(m,wtp=100,xlim=c(-.0005,0.0015))Copy Code

Multiple treatment comparisons

Probabilistic "depression model"

• Fictional model comparing antidepressants to cognitive behaviour therapy (CBT) and no treatment in people with depression

• Statistical modelling based on evidence synthesis

  • Benefits: based on QALYs
  • Costs: associated with treatments and various resources use

• Economic modelling: two matrices with relevant population summaries

  • effects
  • costs

• NB: The details of the actual modelling are not important for the purposes of demonstrating the example...

Multiple treatment comparisons

Probabilistic "depression model"

> # Intervention labels
> t.names<-c("No treatment","CBT","Antidepressant")
> 
> # "Standard" analysis: pairwise comparisons
> depression.bcea = bcea(effects,costs,
+                        interventions=t.names,ref=3)  
> # the third intervention is the reference
> 
> # Plots the results
> plot(depression.bcea)Copy Code

Multiple treatment comparisons

Probabilistic "depression model"

> # For multiple treatment comparison
> depression.multi.ce = multi.ce(depression.bcea)
> 
> # Specialised plot method
> ceac.plot(
+   depression.multi.ce,pos=c(1,0.8),
+   graph="ggplot2"
+ )Copy Code

Multiple treatment comparisons

Probabilistic "depression model"

> ceac.plot(
+   depression.multi.ce,pos=c(1,1),
+   graph="ggplot2"
+ ) + 
+   ggplot2::stat_summary(
+     fun=max, geom="line",
+     colour="grey25", 
+     alpha=.3, lwd=2.5
+   )Copy Code

BCEAweb

• Inspired by similar projects – eg SAVI

• Create a web interface to use BCEA without even opening R (or even having it installed on your computer!)

BCEAweb

• Inspired by similar projects – eg SAVI

• Create a web interface to use BCEA without even opening R (or even having it installed on your computer!)

• Typical work flow

  1. Design the economic model (eg Markov model, decision tree, ...)
  2. Run the statistical analysis to estimate the quantities of interest (eg survival analysis, evidence synthesis, ...)
  3. Run the economic model and obtain "PSA samples"
  4. Upload "PSA samples", including values for $(e,c)$ to BCEAweb
  5. Use BCEA in the background to do all the economic analysis
  6. Create reports that can be used as the basis for papers, reimbursement files, ...

BCEAweb

> # Creates a matrix with the underlying model simulations
> inp = createInputs(vaccine_mat, print_is_linear_comb=FALSE)
> 
> # Runs BCEAweb
> BCEAweb(e=e,                 # matrix of simulations for the effectiveness
+         c=c,                 # matrix of simulations for the costs
+         parameters=inp$mat   # matrix of simulations for all the model parameters
+        )Copy Code