Workshop 7: HOW TO MODEL THE COST-EFFECTIVENESS OF HISTOLOGY INDEPENDENT THERAPIES WITHIN HEALTH TECHNOLOGY ASSESSMENT (HTAs)

# Workshop 7: HOW TO MODEL THE COST-EFFECTIVENESS OF HISTOLOGY INDEPENDENT THERAPIES WITHIN HEALTH TECHNOLOGY ASSESSMENT (HTAs)
## (Virtual) ISPOR Europe 2020 | Milan (Italy - in spirit…)
### Thursday, 19 November 2020 | 13:00 - 14:00

---

background-image: url("img/banner.jpg")
background-size: 100%
background-position: 0% 0%

</style>

---
class: virtual
background-image: url("assets/virtual2020.png")
background-size: cover
background-position: 90% 10%

.in-line-right{
 display: inline-block;
 float: right;
 width: 20%
}
</style>
 
**Raquel Aguiar-Ibáñez** MSc 
*MSD Ltd Haarlem, NH, Netherlands*

**Dawn Lee** MSc     
*Bresmed, Sheffield, UK*

**Gianluca Baio** PhD      
*University College London, UK*

**Jacoline Bouvy** PhD     
*National Institute for Health and Care Excellence (NICE), London, UK*

.in-line-left[.small[ [www.statistica.it/gianluca/slides/ispor-2020/](www.statistica.it/gianluca/slides/ispor-2020/)]] .in-line-right[.twitter[.small[ #ISPOREurope]]]
---
class: virtual vertical-align: top
background-image: url("assets/virtual2020.png")
background-size: cover
background-position: 90% 10%

# Conflict of interest and disclosures

## No external funding was received for the development of this workshop

<table class="table" style="font-size: 28px; margin-left: auto; margin-right: auto;">
<tbody>
 <tr>
 <td style="text-align:left;width: 25%; "> Raquel Aguiar-Ibáñez </td>
 <td style="text-align:left;width: 65%; "> Employee of MSD B.V., a subsidiary of Merck &amp; Co., Inc., Kenilworth, NJ, USA. </td>
 </tr>
 <tr>
 <td style="text-align:left;width: 25%; "> Dawn Lee </td>
 <td style="text-align:left;width: 65%; "> No conflicts of interest relevant to the content of this workshop. Employed by BresMed, UK. </td>
 </tr>
 <tr>
 <td style="text-align:left;width: 25%; "> Gianluca Baio </td>
 <td style="text-align:left;width: 65%; "> No conflicts of interest relevant to this presentation. Employed by University College London, UK </td>
 </tr>
 <tr>
 <td style="text-align:left;width: 25%; "> Jacoline Bouvy </td>
 <td style="text-align:left;width: 65%; "> No conflicts of interest relevant to the content of this workshop. Employed by the National Institute for Health and Clinical Excellence, London, UK. </td>
 </tr>
</tbody>
</table>

---
class: virtual vertical-align: top
background-image: url("assets/virtual2020.png")
background-size: cover
background-position: 90% 10%

# Outline

<table class="table" style="font-size: 28px; margin-left: auto; margin-right: auto;">
<tbody>
 <tr>
 <td style="text-align:left;width: 25%; "> Raquel Aguiar-Ibáñez </td>
 <td style="text-align:left;width: 65%; "> Landscape and data challenges </td>
 </tr>
 <tr>
 <td style="text-align:left;width: 25%; "> Dawn Lee </td>
 <td style="text-align:left;width: 65%; "> Histology-Independent Oncology Therapies - The NICE HTA Story So Far </td>
 </tr>
 <tr>
 <td style="text-align:left;width: 25%; "> Gianluca Baio </td>
 <td style="text-align:left;width: 65%; "> Methodological key considerations when building a model </td>
 </tr>
 <tr>
 <td style="text-align:left;width: 25%; "> Jacoline Bouvy </td>
 <td style="text-align:left;width: 65%; "> Histology-independent cancer drugs - HTA perspective </td>
 </tr>
</tbody>
</table>

---

class: segue-pics
background-image: url("img/segue-pics.jpeg")
background-size: cover
background-position: 0% 0%

**Raquel Aguiar-Ibáñez**    
Principal Scientist – Oncology    
Center for Observation and Real World Evidence    
Economic and Data Sciences (EDS)    
MSD BV, Harleem, The Netherlands

.small[
.left[
 [raquel.aguiar-ibanez@merck.com](raquel.aguiar-ibanez@merck.com) 
 [https://www.linkedin.com/in/raquel-aguiar-ibáñez-7172591](https://www.linkedin.com/in/raquel-aguiar-ibáñez-7172591)
]
]
]

---

# Disclaimer

- I  am an employee of MSD B.V., a subsidiary of Merck & Co., Inc., Kenilworth, NJ, USA.

- The views and opinions presented here are my own and do not necessarily reflect those of MSD.

???
Before I do that, I'd like first to mention that the views and opinions I'll be presenting here today are my own and do not necessarily reflect those of MSD

---

class: virtual vertical-align: top
background-image: url("assets/virtual2020.png")
background-size: cover
background-position: 90% 10%

# .ubuntublue[Interactive poll 1]

.left[
- Can you please tell us where you work?
   - In industry
   - As part of consultancy
   - At an HTA agency
   - In academia 
   - Other
]

---
class: virtual vertical-align: top
background-image: url("assets/virtual2020.png")
background-size: cover
background-position: 90% 10%

# .ubuntublue[Interactive poll 2]

.left[
- Do you have any experience with histology-independent oncology therapies?
   - Yes, I’ve read about them
   - Yes, I’ve worked on the development of HTAs and/or cost-effectiveness models
   - Yes, I have assessed them as part of an HTA process
   - Yes, other
   - No
]

---

# Paradigm shift in oncology drug development

???

The last few years have seen a paradigm shift in the development and regulatory approval of cancer therapies.

Traditionally, the development of cancer therapies  was based on tumour type, e.g., whereby a type of tumour was identified (say lung or breast cancer) and also the positioning within the treatment pathway (whether patients were treatment naïve or previously treated, in some cases with a specific previous therapy)

In some cases, approvals have also been based on a biomarker identified within a specific tumor type, e.g., HER-2 positive breast or gastric cancer, or RAS wild-type colorectal cancer

Recently, a new group of therapies, known as tumour-agnostic or histology-independent therapies, have taken the main stage in oncology

The assessment of these new therapies rely on the use of biomarkers to identify patients that are likely to benefit from them.

Rather than targeting a specific tumour histology or where in the body the cancer starts, they target a genetic alteration, which aims to reflect a common immunobiology through the use of a biomarker      
     There must be a strong scientific/biological rationale for the use of the treatment in the presence of the prespecified molecular profile, regardless of the tumour histology     
     There must be compelling clinical data and evidence of a safe profile      
     So far, these therapies have been approved for patients that were lacking alternative, satisfactory therapies

This marks a turning point in precision medicine.

In any case, it’s important to note that, although a paradigm shift has been observed, drug development based on histology site is still ongoing.

---

# FDA & EMA regulatory approvals for histology-independent oncology therapies

???

Let me provide an overview of the regulatory approvals so far by the FDA and EMA reg. histology-independent therapies.

In May 2017 the FDA granted accelerated approval to pembrolizumab as the first site and histology-independent oncology medicine for the treatment of adult and pediatric patients with:

- unresectable or metastatic, solid tumors that are microsatellite instability-high (MSI-H) or mismatch repair deficient (dMMR) 
- and have progressed following prior treatment and who have no satisfactory alternative treatment options 
- Or with MSI-H or dMMR colorectal cancer that has progressed following treatment with a fluoropyrimidine, oxaliplatin, and irinotecan.

The second histology-independent approval by the FDA came a year later, for larotrectinib for NTRK tumours with no satisfactory alternative treatments or that progressed following treatment. This indication was also approved by EMA one year later, and this was the first approval by the EMA of a histology-independent treatment.

This was followed by the approvals of entrectinib for NTRK tumours both by the FDA and EMA, and by the approval of pembrolizumab for the treatment of unresectable or metastatic solid tumours showing high mutational burden tumours, as determined by an FDA-approved test, that have progressed following prior treatment and who have no satisfactory alternative treatment options.

These regulatory approvals of histology-independent medicines have been based mainly on evidence related to high RRs and long DoR.

---

# Challenges with the design of clinical trials for histology-independent oncology medicines

???

Traditional oncology medicines for the treatment of advanced cancers have mostly focused on the development of RCTs to demonstrate the clinical efficacy and safety of a new product compared to one or few relevant comparators. Some typical endpoints in advanced cancers have been PFS, OS in many cases, and other endpoints such as RRs, duration of response, etc.

For histology-independent oncology therapies, there are some very particular challenges associated to implementing an RCT to assess their clinical efficacy and safety:

1. Since the target population will relate to any tumour or all tumours as long as they present the genetic alteration, one important challenge will be to be able to identify and recruit a sufficient number of eligible patients
   - The prevalence of the genetic mutation among the most common cancers is usually low, and/or may vary depending on the type of cancer. Therefore, a high number of patients may need to be tested to identify few eligible patients.

2. It is not possible to include patients with the mutation across all types of tumours
   - Therefore, the trial may be able to recruit a number of patients for a range of tumours expressing the mutation, but not all, with the consequent concerns relating to the generalisability of the data.
   - Moreover, there may be very small patient numbers for a specific type of tumour

3. Each type of cancer may be treated differently and, in many cases, at the end of the treatment pathway there may not be a clear SOC. 
   - This means that it is not possible to include all relevant compactors per type of tumor due to variations across tumour sites and lack of SOC.

---

# Basket trials

???

To deal with the trial design challenges associated with histology-independent oncology therapies basket trials have been implemented

- A basket trial in oncology refers to a trial design in which a medicine is assessed on multiple types of cancer that have a common molecular alteration
   - These trials are single arm trials, with no comparative therapies, commonly comprising small patient numbers and considering ORRs as the main endpoint.
 
There are some obvious questions we can raise in relation to basket trials are, for example:

- How many tumours to include 
   - Although the implicit aim is to cover all tumour types with that particular genetic alteration, in practice it is not possible and only a relatively limited number of cancer types, and a handful of patients with the same tumour type may be included in the study

- Whether there is evidence of a treatment effect across all included tumours
  - Since patients with different tumour types are included, who may have different characteristics, may have received different previous treatments (or no treatment) before recruitment, there may be considerable heterogeneity 
  - This is further complicated by the fact that there is not a comparator arm, and therefore there is not clarity reg. the actual treatment benefit

- The main endpoint used has been response rates, 
   - There is a need to demonstrate that response is independent of the tumour type
   - Moreover, it’s important to understand whether response is a surrogate to OS (ultimate endpoint considered by HTA agencies). There may be evidence of surrogacy for some types of cancers, but this will not be possible for all
   - Whether the findings can be extrapolated to the tumours not included is a question mark

---

# HTAs on tumour-agnostic indications so far...

.white[
- Some common challenges:
  - Uncertainty about the size of the eligible population (unclear financial impact)
  - Generalisability of tumour types and observed responses
  - Difficulties to adjust for heterogeneity and potential biases
  - Response as surrogate for OS
  - Size of treatment benefit
  - Implementation and cost of testing
  - Need for further data collection to reduce uncertainty
]

.small[
* Only for the treatment of pediatric patients with NTRK+ refractory/relapsing childhood fibrosarcoma or other soft tissue sarcoma (not for other pediatric indications or adults with a NTRK positive solid tumor)

1. [https://www.nice.org.uk/guidance/ta644](https://www.nice.org.uk/guidance/ta644)     
2. [https://www.cadth.ca/entrectinib-tbd-neurotrophic-tyrosine-receptor-kinase-ntrk-fusion-positive-solid-tumours](https://www.cadth.ca/entrectinib-tbd-neurotrophic-tyrosine-receptor-kinase-ntrk-fusion-positive-solid-tumours)     
3. [https://www.nice.org.uk/guidance/ta630](https://www.nice.org.uk/guidance/ta630)     
4. [https://www.cadth.ca/larotrectinib-neurotrophic-tyrosine-receptor-kinase-ntrk-locally-advanced-or-metastatic-solid](https://www.cadth.ca/larotrectinib-neurotrophic-tyrosine-receptor-kinase-ntrk-locally-advanced-or-metastatic-solid)     
5. [https://www.cadth.ca/larotrectinib](https://www.cadth.ca/larotrectinib)

]

???

So far, only two medicines have been assessed by HTA agencies with more sophisticated processes and requiring the modelling of the CE of therapies as part of their assessment process, such as NICE and CADTH, including entrectinib and larotrectinib:

- In the case of NICE, both therapies were finally recommended through the CDF while further data is collected, either through CTs or registries/RWE
   - The main areas of uncertainty being further investigated include, among others: 
      - Distribution of tumours with genetic mutation in clinical practice, OS, PFS, heterogeneity, QoL, DoR, ToT

- In Canada there is not a similar process to the CDF for approval of therapies with clinical uncertainty and potential for cost-effectiveness.
   - CADTH has assessed so far larotrectinib 
   - Although larotrectinib was initially recommended for only 4 out of 14 tumours (those that were considered to have the strongest evidence in terms of benefit for patients), the final recommendation was negative, since both company and patient groups supported a recommendation on the full population rather than by tumour

- This points out towards limited access by patients to these histology-independent therapies, despite their regulatory approvals, and towards the need to have processes in place .content-box-yellow[that can help to deal with different levels of evidence and uncertainty in HTA and reimbursement processes while further evidence is collected]

---

# HTAs on histology-independent indications so far...

- Some common challenges:
  - Uncertainty about the size of the eligible population (unclear financial impact)
  - Generalisability of tumour types and observed responses
  - Difficulties to adjust for heterogeneity and potential biases
  - Response as surrogate for OS
  - Size of treatment benefit
  - Implementation and cost of testing
  - Need for further data collection to reduce uncertainty

]

???

There are some clear challenges for HTA agencies reg. the assessment of this type of therapies, including, for example:

- There is uncertainty reg. the amount of patients that will be eligible for treatment, and therefore the true financial impact is unknown

- It is unclear whether tumours included in the trials are representative of those that will present the current genetic mutation making patients eligible for treatment in clinical practice

- Whether the observed response rates within the trial can be generalised to tumours not included in the trial

- How heterogeneity can be assessed, which may be present in relation to tumour types and patient’s characteristics

- Whether response rates are actual proxies for OS benefit

- And actually, what’s the size of the treatment benefit, given the challenges in comparing single arm trials with external sources and all the potential biases and heterogeneity associated to these

- How to approach the costs of testing, depending on whether it’s performed with the only purpose of assessing eligibility for the tumour-agnostic therapy or there may be broader use

- And finally, what additional evidence may be required to reduce the uncertainty associated to these therapies, either further FU within clinical trials or conducting RWE studies

With this, I complete my overview of the current landscape related to histology-independent indications and some general challenges with their assessment.

Let me now hand over to Dawn, who will be focusing on identifying the challenges particularly related to the cost-effectiveness modelling of these therapies identified from NICE submissions.

---
class: virtual vertical-align: top
background-image: url("assets/virtual2020.png")
background-size: cover
background-position: 90% 10%

# .ubuntublue[Interactive poll 3]

.left[
-  Do you think that the current methods used by HTA bodies can sufficiently capture the value of histology independent therapies?
   - Yes
   - No
   - I don't know

]

---

class: segue-pics
background-image: url("img/segue-pics.jpeg")
background-size: cover
background-position: 0% 0%

**Dawn Lee**    
Chief Scientific Officer        
Bresmed (UK)

.small[
.left[
 [dlee@bresmed.com](dlee@bresmed.com) 
 [@BresMedOfficial](https://twitter.com/bresmedofficial) 
 [https://www.linkedin.com/company/bresmed/](https://www.linkedin.com/company/bresmed/)
]
]
]

---

???

Back in 2018 the race between larotrectinib and entrectinib for NICE approval began, along with the parallel process of production of recommendations by CHE and ScHARR for the analysis of histology-independent cancer drugs.

Both companies actively positioned for the CDF; Bayer stole the jump on Roche, managing to get published 2 ½ months earlier – despite an additional public ACM being conducted.

Considerable learnings can be gained from these 2 submissions, both in terms of methodology for evaluating histology-independent oncology therapies, and the best use of the NICE process to gain swift access for patients.

The approaches taken by the 2 companies vary substantially in their tone, if not their substance – with Bayer taking an aggressive stance in relation to the ability of current methods to account for these types of therapies from the outset.

I will focus here on issues raised relating specifically to the evaluation of histology-independent oncology therapies.

---

class: with-logo bresmed 
# Prevalence of NTRK gene fusions varies considerably across tumour types

???

A bit of background first: NTRK gene fusion as a primary oncogenic driver and underlying cause of cancer is known to occur across a diverse range of solid tumour sites, affecting both adult and paediatric patients.

Next generation sequencing allows for efficient testing, with the ability to find NTRK gene fusions and other genomic targets simultaneously. Capacity is currently being ramped up – in 2019, NHSE made a commitment that all 7 Genomic Hubs in England will be ready for testing in 2021, and that this testing will become fully embedded in practice by 2022

The prevalence of NTRK gene fusions varies widely across tumour types from less than 1% in common tumours such as lung, colorectal and breast cancers, to more than 90% in some rare tumour types. It is not known if there are tissue-specific mechanisms for bypassing response to drugs. It is not known how or whether NTRK fusions affect response to drugs, or the patient prognosis

---
exclude: true
class: with-logo bresmed
# What is a basket trial?

.small[
**Key**: CNS, central nervous system;    
**Source**: Company evidence submission larotrectinib for treating NTRK fusion-positive advanced solid tumours; page 42.
]

---
class: with-logo bresmed
# Clinical trial data availability

.small[
**Key**: CNS, central nervous system; PFS, progression-free survival; TE, technical engagement. 
**Notes**: `$^+n =$` 68 adults in efficacy evaluable population + 34 paediatric patients including 9 with primary CNS; 159 patients (153 with evaluable response included at TE stage) 
.white[Notes:] `$^*n =$` 54 adults in efficacy evaluable population + 5 adults with primary CNS + 7 paediatric patients 
.white[Notes:] `$^x$` Submission states 14, but 15 are present
]

???

Both the larotrectinib and entrectinib appraisals used pooled data across a number of basket trials. This included a number of tumour sites and NTRK gene fusion partners. Despite this, you can see the extremely low ns we’re talking about, compared with the number of tumour sites included. The low coverage of the patient population becomes even clearer when we consider that the marketing authorization for these therapies covers ~400 tumour types.

Data maturity at the time of submission was low in TA644, and even lower in TA630.

Both submissions suffered from an extremely high level of redacting, which makes it challenging to draw learnings and for those submitting for future indications.

---

.content-box-purple[
**EMA license wording**:    
The treatment of adult and paediatric patients with tumours that display an NTRK gene fusion and who have disease that is locally advanced, metastatic or where surgical resection is likely to result in severe morbidity, and **no satisfactory treatment options**." &nbsp;&nbsp;.tiny[bold added to highlight key restrictions]
]

.small[
**Key**: BHM, Bayesian hierarchical modelling; CrI, credible interval; ERG, evidence review group; ORR, overall response rate;
]

???

The primary trials in both submissions included a large number of patients at either 1st or 2nd line, representing an earlier line than the final EMA indication which specifies that patients should have no satisfactory treatment options remaining. It is not clear from the EPAR that the EMA planned to approve these therapies as histology independent from the beginning which perhaps contributed to this mismatch. No adjustments were made for this.

In both appraisals, the ERG considered that rare high NTRK prevalence tumour sites were over-represented in the companies’ trials – which may lead to an overestimation of the benefit due to a reduced rate of false-positive results and greater inclusion of rare paediatric indications with more potential to benefit. However, estimates of prevalence are highly uncertain as patients were not regularly screened at the time of submission.

Observed response rates varied considerably across tumour types; from 0% to 100%; although patient numbers are extremely limited.

Importantly, both companies assumed the trials were generalizable and used pooled datasets across tumours – thereby assuming homogeneity of response.

This is despite questions on the generalizability of the trial samples.

The justification was that they did not consider subgroup data robust enough at the specific tumour type level for reliable modelling to assess tumour or response heterogeneity.

Both companies also assumed there was no prognostic effect of NTRK gene fusions in their base-case analysis, and conducted scenarios adjusting comparator survival downwards assuming worse prognosis.

---
class: with-logo bresmed 
# Differing predicted response across tumour types

"Consideration of response by tumour location only serves as a distraction and introduces the potential for decision-making to be based on chance findings."
]

.small[
**Key**: CNS, central nervous system; ERG, evidence review group; GIST, gastrointestinal stromal tumour;    
**Source**: ERG report, larotrectinib submission (TA630), page 67; primary CNS excluded in the base case analysis.
]

???

In both cases, the ERG (and Committee) initially focussed on tumours with the highest response rates but finally preferred to characterize the impact of heterogeneity in response using Bayesian hierarchical modelling; a framework developed specifically for basket trials allowing analysis of a pooled overall response rate, adjusting for the observed heterogeneity and borrowing strength across different tumour types which Gianluca will cover in detail later. However, they were unable to translate this methodology to time-to-event outcomes due to data limitations. The method can produce extreme results - with the response rate prone to changing drastically with even small changes in the absolute number of patients who exhibit a response.

This figure illustrates the wide variation in response predicted by tumour types within the larotrectinib submission.

The mean directional impact of application of BHM varied across the 2 submissions with little variation seen from the assumption of homogeneity in TA644 beyond a widening of the credible intervals. The ORR, however, decreased in TA630 (74% vs 64%; 79% vs 72% with the expanded dataset). You can see on the slide Bayer’s response to this.

In the end the Committee timetable this (and many other issues) as areas for future research.

---
class: virtual vertical-align: top
background-image: url("assets/virtual2020.png")
background-size: cover
background-position: 90% 10%

# .ubuntublue[Interactive poll 4]

.left[
- What do you think HTAs of histology-independent oncology therapies should seek?
   - To optimise decisions per indication
   - To assess cost-effectiveness as for the marketing authorisation
   - Other
   - I don’t know

]

---
class: with-logo bresmed
# So should we be looking at one ICER or many?

"The focus of the company’s submission was on a single answer.....

The general view of the ERG is that optimised decisions are preferable."

.small[
**Key**: ERG, evidence review group; ICER, incremental cost-effectiveness analysis;    
**Source**: ERG report entrectinib submission, page 17
]

???

Given the potential for heterogeneity of response to treatment and absolute outcomes for both arms – one of the fundamental questions in these appraisals is whether we should be looking for one ICER across the entire MA to be cost-effective or instead seek to optimize decision making across each individual tumour type.

The ERG and manufacturer differed in their opinions on this. The Committee sided here with the manufacturer, when the opposite decision has been taken in other appraisals (such as pirfenidone in IPF, where this very consideration led to appeal). Jacoline will discuss in more detail later

---
class: with-logo bresmed
# Base case model structure and method to generate the counterfactual

.small[
**Key**: NICE, National Institute for Health and Care Excellence; PSM, parametric survival model; SLR, systematic literature review; TA, technology appraisal.
]

???

Both manufacturers submitted a 3-state PSM based upon a naïve comparison to previous NICE appraisals of last-line treatments. The method of implementation of this comparison varied across manufacturers with Roche using median survival information and Bayer using parametric models for each TA.  Roche’s approach was considered pragmatic – if not ideal.

---
class: with-logo bresmed
# A bit more on the pros & cons of alternative modelling structures

Key limitation of all these approaches:    
- Potential for implausible estimates with immature data

.small[
**Key**: BHM, Bayesian hierarchical modelling; OS, overall survival; PFS, progression-free survival; SoC, standard of care;
]

???

Confirmatory analyses were presented in the larotrectinib submission using 2 approaches (suggested by NICE during scoping inspired by the idelalisib submission which I had the pleasure to work on):

- Firstly, response-based modelling; this analysis takes within-trial PFS and OS from non-responders to populate the comparator arm

- Secondly, previous line of treatment analysis; which takes comparator data from the time to progression (or time to next treatment) for the line of therapy prior to enrolment

The entrectinib appraisal also investigated both these approaches.

Each of the approaches comes with pros and cons. A key one for all approaches being that data immaturity will lead to both uncertainty and the potential for implausible estimates (particularly obvious within Bayer’s PSM).

Use of external comparator is considered the next best alternative in the absence of a H2H trial; the methods for this being covered in the NICE TSDs however, often, as was the case here, only aggregate data is available to make comparison and this is often insufficient to be able to conclude that there are no unmeasured confounders and therefore an unanchored unbiased comparison is often not possible.

The responder and prior line of treatment analyses both overcome the limi

---
class: with-logo bresmed
# The importance of diagnostic accuracy and an ethical conundrum

- For tumour sites with a low NTRK prevalence, diagnostic accuracy needs to be very high in order to avoid false positive results – for these patients, the tumour would not be expected to respond

.small[
**Source**: Larotrectinib for treating NTRK-fusion positive solid tumours [ID1299] Lead Team Presentation AMC1, slide 20.
]

???

Other than the usual considerations for an oncology submission & heterogeneity across tumour types, diagnostics are critical to histology-independent treatments.

On this slide is a worked example using company breast cancer estimates of NTRK gene fusion prevalence, literature values for sensitivity and 99% or 99.9% for specificity.  This demonstrates how in a large population with a low chance of a particular gene fusion, even a diagnostic with a very high sensitivity and specificity can result in a large number of false-positive results.

In this case, the end-of-line positioning was considered by the Committee to reduce the risk of displacing active treatments for patients with false-positive results. However, there were concerns about the ethics of treatment for any patients with false-positive results because of the adverse events associated with treatment.

---
class: virtual vertical-align: top
background-image: url("assets/virtual2020.png")
background-size: cover
background-position: 90% 10%

# .ubuntublue[Interactive poll 5]

.left[
-  What would you consider to be the fairest way to account for the cost of the rollout of genomic testing?
   - If already available in clinical practice, the cost should not be included as part of the HTA
   - Include the cost of genomic testing if genomic testing was not conducted in clinical practice before the approval of the histology-independent oncology therapy
   - Include the cost of genomic testing as a mean cost that reflects how much a health care system will spend in testing to identify one patient eligible for the histology-independent medicine
   - Other
   - I don’t know

]

---
class: with-logo bresmed
# Cost of genetic testing

.content-box-purple[
*“If a diagnostic test to establish the presence or absence of this biomarker is carried out solely to support the treatment decision for the specific technology, the associated costs of the diagnostic test should be incorporated into the assessments of clinical and cost effectiveness."*

> NICE methods guide
]

NHSE proposed diagnostic cost = average £6,800 per patient `$^*$`

.small[
**Key**: ACM, appraisal committee meeting; FAD, final appraisal determination; NHSE, National Health Service England; NICE, National Institute for Health and
Care Excellence.   
**Source**: `$^*$` [https://www.nice.org.uk/guidance/ta630/documents/committee-papers-2](https://www.nice.org.uk/guidance/ta630/documents/committee-papers-2); page 75
]

???

A major overhaul in diagnostic techniques is ongoing – a situation the methods guide was not really set up to address. So it is perhaps unsurprising that the submitting companies, ERG and NHSE disagreed on the approach to including diagnostic costs.

A pragmatic approach was taken to solving this. NHSE within their submission (written by Peter Clarke) provided both estimates of diagnostic costs, plans for national testing and a request to NICE to explore scenario analyses in which various percentages of the costs of multi-gene panel testing are borne by the individual manufacturers. They stated that they would then decide on the appropriate level of contribution before the Committee meeting having seen the scenarios. Later in the submission process, the agreed estimate of £6,800 appears.

The direct proposal of diagnostic costs by NHSE is one of many examples of an extremely active participation in these submissions.

---
class: with-logo bresmed

# Final ICERs and Committee decision

- The wider benefits associated with genomic testing are not captured in the QALY - but these are not specific to treatment 
- The Committee acknowledges the "difficulty of using adjustment techniques for an unknown treatment effect in immature survival data".
]
.right-column-small[
<img src="img/kicking_can.png"; width=200px;>
]

**Key**: EOL, end of life; ERG, evidence review group; ICER, incremental cost-effectiveness benefit; PAS, patient access scheme; PPS, post-progression survival; QALY, quality-adjusted life year. 
**Notes**: `$^*$` Didn’t include Committee preferences around costing post progression, adjusting for implausible PPS, pre-progression utilities or the impact of cure. 
.white[Notes:] `$^+$` Didn’t include Committee preferences around comparator and intervention arm testing costs, prevalence estimates, subsequent therapies 
**Source**: TA630 & TA644 FADs

]

???

For both therapies, end-of-life criteria were determined to be met at the end of the process, despite a range of issues – the predominant one being that not all indications meet the criteria. The comparator median OS ranged from ~6 months to ~3 years

Within both submissions an extremely high level of behind-the-scenes wrangling and additional analyses outside of usual process can be clearly be seen to have taken place.

A difference in strategy can be seen here – with Bayer insisting on assumptions the Committee viewed as implausible and additional ERG analyses then taking place, and Roche starting from a higher ICER with fewer implausibilities and then putting in an additional late commercial arrangement.

The Committee agreed in both appraisals that there were wider benefits from genomic testing not captured in the QALY calculation but not benefits specific to the treatments under appraisal.

The Committee position here is noteworthy in acknowledging the difficulty of using adjustment techniques for an unknown treatment effect in immature survival data

The final decision at the end of price and analyses adjustments was therefore to recommend both therapies for use within the CDF whilst more data are collected.

Both appraisals may be restarting in late 2023, when planned interim analyses become available. Both companies have agreed to collect data from a range of UK sources focusing on additional follow-up from the clinical trials and data collection from UK sources including SACT, the PHE molecular dataset and Genomics England analyses with Roche also promising to investigate the feasibility of identifying a matched NTRK fusion-negative cohort from Flatiron and the ETOP study

---
class: with-logo bresmed
# Some recommendations for the future

- Plan your analyses and data collection early!

- One ICER

- Clinical plausibility is king (but subjective)

- Do not ignore heterogeneity and generalizability

- Test structural uncertainty

- Include diagnostic costs outside of standard practice

- Remember: enthusiasm may wane
]

.small[
**Key**: ICER, incremental cost-effectiveness ratio; NHSE, National Health Service England.
]

???

A few recommendations for the future based on this:

- Engage with NHSE early and frequently – they will drive the course of your submission

- Plan your analyses and data collection early – get your consultancy to help you with this

- One ICER to rule them all has been accepted, thereby setting the precedent

- Clinical plausibility in the eye of the decision maker as always is king

- Heterogeneity in response & generalizability issues between trial and practice should not just be ignored

- Consider how structural scenarios can be best used to account for the uncertainty in your dataset

- Neither can you ignore diagnostic costs where practice is not already in place

- As these were the first submissions, enthusiasm levels will be lower in future appraisals – there are certainly areas where Committees have been more pessimistic regarding assumptions around less ground breaking therapies

Now Gianluca will provide more detail on key considerations when building a model, including the need to explore heterogeneity and uncertainty related to treatment response and effectiveness

---

class: segue-pics
background-image: url("img/segue-pics.jpeg")
background-size: cover
background-position: 0% 0%

**Gianluca Baio**    
Professor of Statistics and Health Economics     
University College London (UK)

.small[.left[
 [g.baio@ucl.ac.uk](mailto:g.baio@ucl.ac.uk) &nbsp;&nbsp;
 [@gianlubaio](https://twitter.com/gianlubaio) 
 [https://www.linkedin.com/in/gianluca-baio-b893879/](https://www.linkedin.com/in/gianluca-baio-b893879/) 
 [http://www.statistica.it/gianluca/](http://www.statistica.it/gianluca/) 
 [https://egon.stats.ucl.ac.uk/research/statistics-health-economics/](https://egon.stats.ucl.ac.uk/research/statistics-health-economics/) 
 [https://r-hta.org/](https://r-hta.org/) 
 [https://github.com/giabaio](https://github.com/giabaio) 
 [https://github.com/StatisticsHealthEconomics](https://github.com/StatisticsHealthEconomics)
]]
]

---
class: with-logo ucl

# (The real) Disclaimer...

<center>
<blockquote class="twitter-tweet">Best opening sentence <a href="https://twitter.com/hashtag/ISPOREurope?src=hash&amp;ref_src=twsrc%5Etfw">#ISPOREurope</a> from Gianluca Baio: “statisticians should rule the world and Bayesian statisticians should rule all statisticians” <a href="https://t.co/GN2w7liAcR">https://t.co/GN2w7liAcR</a>&mdash; Manuela Joore (@ManuelaJoore) <a href="https://twitter.com/ManuelaJoore/status/1191397718930939904?ref_src=twsrc%5Etfw">November 4, 2019</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script> 
</center>

...Just so you know what you're about to get into... 😉
---
exclude: true
class: with-logo ucl

# Single-arm studies .subtitle[Are they *always* that bad?...]

.pull-left[
<center><img src=img/Parachute.png width=400></center>
]
.pull-right[
<center><img src=img/Parachute2.png width=400></center>
]

---
exclude: true
class: with-logo ucl
# Single-arm studies .subtitle[Are they OK, then?...]

- Unfortunately, almost invariably, they may be *problematic*

- The treatment effect is **much** more uncertain than the parachute's

- This matters, because it means that we **need** to consider some kind of *indirect* comparison

- And **crucially**, propagate the underlying extra uncertainty through the economic model!

--
exclude: true

.content-box-purple[
### Population adjustment methods 
- MAIC, STC `$\Rightarrow$` increasingly popular 
- Controversial ( [Philippo et al, SiM 2020](https://onlinelibrary.wiley.com/doi/full/10.1002/sim.8759)) `$\Rightarrow$` ofter underlying assumptions **not** clearly laid out/explicitly recognised

- New methods/extensions 
 - **Multilevel network meta-regression**. : [Philippo et al, JRSS/A, 2020](https://rss.onlinelibrary.wiley.com/doi/full/10.1111/rssa.12579?af=R)
 - **Population Adjusted Indirect Comparisons**. : [Remiro-Azócar et al., `arxiv` 2020](https://arxiv.org/pdf/2008.05951.pdf)
]

---
class: with-logo ucl
# To be or not to be (*a Bayesian*)?...

<center>
<blockquote class="twitter-tweet" data-conversation="none"><a href="https://twitter.com/krstoffr?ref_src=twsrc%5Etfw">@krstoffr</a> <a href="https://twitter.com/lakens?ref_src=twsrc%5Etfw">@lakens</a> Bayesian: one who vaguely expecting a horse and glimpsing a donkey is pretty sure he&#39;s seen a mule :-)&mdash; Stephen John Senn (@stephensenn) <a href="https://twitter.com/stephensenn/status/668515916230041600?ref_src=twsrc%5Etfw">November 22, 2015</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script> 
</center>

- That's a bit funny (like most things that Stephen Senn says...)

- And while I think is also a bit exaggerated (for comic effect?...), it also reveals a real complexity in *doing* a Bayesian analysis
   > "*How can I form a prior? I know **nothing** about this parameter!*"
   >
   > "*I want to let the data speak for **themselves**, otherwise it's subjective!*"

---
class: with-logo ucl
# To be or not to be (*a Bayesian*)?...

Sometimes it is **really** hard to encode prior **information** into a prior **distribution**

- Treatment effect for a super-innovative cancer drug, on an indication for which no other treatment is currently available and for which prognosis is very poor, under standard of care

.narrow-left[
- Imagine I show you this graph with data fitted to a trial of active intervention vs control 
- The survival curves are computed from a parametric model and then extrapolated beyond the trial time-horizon
- Seems like a rather strong treatment effect (though subject to large censoring!)

]
.wide-right[
<img src="index_files/figure-html/unnamed-chunk-11-1.png" width="60%" style="display: block; margin: auto;" />
]

---

# To be or not to be (*a Bayesian*)?...

Sometimes it is **really** hard to encode prior **information** into a prior **distribution**

- Treatment effect for a super-innovative cancer drug, on an indication for which no other treatment is currently available and for which prognosis is very poor, under standard of care

- But what if I told you that the blue curve is the survival for the **general, healthy population**
   - Would you still believe the extrapolation?

]
.wide-right[
<img src="index_files/figure-html/unnamed-chunk-12-1.png" width="60%" style="display: block; margin: auto;" />
]

---

.content-box-purple[
### **The parachute study** 
- Put a bunch of people on a plane, randomise them to either `$t=0$` (no parachute) or `$t=1$` (parachute) and make them jump
- Observe the outcome `$y=0$` (dead) or `$y=1$` (alive)

- You may expect something like this     
   - `$\theta_1=\Pr(Y=1 \mid X=1)=0.9$` and `$\theta_0=\Pr(Y=1 \mid X=0)=0.1$`   
   (the numbers are not important - but the order of magnitude is sensible?...)
- Then you can compute    
`\begin{align}
\mbox{OR} = & \left. \frac{\theta_1}{1-\theta_1}\middle / \frac{\theta_0}{1-\theta_0} \right. = \left. \frac{0.9}{1-0.9} \middle / \frac{0.1}{1-0.1} \right. = \left. \frac{0.9}{0.1}\middle / \frac{0.1}{0.9} \right. = \left. 9 \middle / \frac{1}{9} \right. = 81.
\end{align}`

- Participants who are randomised to get a parachute are **81 times** more likely to survive
]

--
exclude: true
- Do you believe that?... &nbsp;&nbsp;&nbsp;And: would you believe that for a drug trial?...

---
class: with-logo ucl
# To be or not to be (*a Bayesian*)?...

### All the help you can get...
- External information can be crucial to **regularise** treatment estimates

- Arguably, we'll need **a lot** of evidence to believe that drugs have massive effects...

- Prior distributions can be used to *encode* the assumption that effects are most likely constrained within a reasonable range `$\Rightarrow$` this helps avoid inconsistencies in interval estimates    
   - In most cases, interventions do **not** have dramatic effects: ORs or HRs greater than, say, 3 are already **HUGE** and extremely unlikely to be observed in practice    
   - The expected mean survival in a population of patients who enter observation when they're 60 *and* have cancer is very unlikely to exceed `$x$` years...

- It's not magic... Just formally including *more* information...

### **Propagate uncertainty**

- HTA is **not** about inference! We're using the stats modelling to **aid decision-making**!

- This means that we need to not just measure uncertainty in estimates - we need to **propagate** it from the stats to the economic model and all the way to the decision-making process

- A Bayesian approach naturally do that and effectively deals automatically with PSA

---
class: with-logo ucl

# To be or not to be (*a Bayesian*)?...
<style type="text/css">
.pull-right {
 float: right;
 text-align: left;
 width: 47%;
}
.pull-left {
 float: left;
 text-align: left;
 width: 47%;
}
</style>

.pull-right[
.center[Bayesian approach] 
<center><img src=img/integrated.png width=600></center>
]
---
class: with-logo ucl
# Basket case

.right-column[

(Especially in the case of composite designs, such as basket trials...), the evidence from the available study is **invariably** unlikely to be definitive!

- Sample sizes may be small (again - prior information/distributions can be crucial!)

- Large heterogeneity (hierarchical/multilevel models...)

- No direct comparators

### Need for more complex models

- **Bayesian** integration of heterogenous sources

- **Value of information**: assess whether decisions can be made on the back of existing evidence and the value of actually delaying in order to reduce uncertainty through better data

]
]

---

# Bayesian hierarchical modelling

Typically, in oncology modelling we are interested in survival analysis

- Outcome: `$(t_{ik},d_{ik}) \Rightarrow$` observed time to event and censoring indicator (individual `$i$` + indication `$k$`)    
- We need to extrapolate **beyond** the time-horizon of the trial - Guidelines suggest a set of *parametric* survival models

`$$t_{ik} \sim p(t_{ik} \mid \boldsymbol\theta_k) \qquad \boldsymbol\theta_k = (\color{blue}{\boldsymbol\mu_k}, \color{red}{\alpha_k}) \Rightarrow \color{blue}{\textsf{'location'}} + \color{red}{\textsf{'ancillary'}} \textsf{ parameters} \qquad \log(\mu_{ik}) = \textsf{linear predictor}$$`

### Modelling strategies

**1**. We could model each indication separately     
   - Encode the assumption that they are all independent; knowing something about indication 1 does not give us any information on indication 2
   - Often unreasonable!

**2**. Or, we could pull together the data for **all** the indications     
   - Essentially, estimate a single treatment effect, irrespective of the indications
   - Often even more unreasonable!

**3**. .red[**Compromise**]?     
   - Bayesian hierarchical models!

---

# Bayesian hierarchical modelling .subtitle[Doodling]

---
class: with-logo ucl
# The divorce .subtitle[*Clinical* vs *Cost-effectiveness* modelling...]

- One of the biggest problem is the **divorce** that often still exists between the "clinical" and the "cost-effectiveness" side of things
   - Evidence
   - Modelling

- (**Too**) often, more focus on getting data out so that drugs can get approved by regulators
- But that (**too**) often makes the work of HTA modellers impossible
   - Follow up too short (e.g. median time not even reached...)
   - Treatment switching / heterogeneity in background characteristics (more) difficult to track in the HTA model
   
   
- Mixture of individual level and aggregated level data 
    - Limited / spurious information 
    - **But** needed for the modelling!

]

---
exclude: true
class: with-logo ucl
# Single-arm studies .subtitle[(not the most recent) state of the art, but still...]

.small[
 [Hatswell et al BMJ Open, 2020](https://bmjopen.bmj.com/content/bmjopen/6/6/e011666.full.pdf)
]

---
exclude: true
class: with-logo ucl
# Single-arm studies

---
exclude: true
class: with-logo ucl
# Context matters

- Nature of HTA impacts on the underlying conduct of studies/generation of evidence

- This in turns impacts on the availability of suitable data to ground economic modelling    
   - Often based on spurious information from different jurisdictions

---

class: segue-pics
background-image: url("img/segue-pics.jpeg")
background-size: cover
background-position: 0% 0%

**Jacoline Bouvy**    
Senior Scientific Adviser        
NICE (UK)

.small[
.left[
 [Jacoline.Bouvy@nice.org.uk](Jacoline.Bouvy@nice.org.uk) 
 [@JacolineBouvy](https://twitter.com/JacolineBouvy) 
 [www.nice.org.uk/research](www.nice.org.uk/research) 
 [https://www.linkedin.com/in/jacoline-bouvy-11994811/](https://www.linkedin.com/in/jacoline-bouvy-11994811/) 
]
]
]

---

### 2 technology appraisals

- Larotrectinib

- Entrectinib

### For NTRK-fusion positive solid tumours

- Both recommended for use in the Cancer Drugs Fund

---

- Not recommending a technology that is clinically and cost effective

- Recommending a technology that is not clinically or cost effective

what approaches are optimal to reduce and properly explore uncertainty?

---

---

- Point of testing in NHS will likely differ from trial testing strategy and may differ for tumour histologies

- Very low patient numbers in trial/population evaluated for efficacy

- Extrapolation to histologies not included in the trial at all?
]

---
class: with-logo nice
# Heterogeneity of subgroups

- Even when response to treatment is histology-independent, clinical and cost effectiveness across tumour types will still differ
    - Testing costs (NNT) sensitive to biomarker prevalence
    - Comparator differs across histologies
    - HRQoL, prognosis may differ between histologies

- Does single ICER convey meaningful information on whether drug provides value for money across all indications?

---
class: with-logo nice
# Developing the counterfactual

- No comparator arm in basket trials

- Use of historical data for indirect comparisons, but no evidence on prognostic and predictive properties of biomarker

NTRK positive tumours, treatment with histology independent drug
]
]
]
.pull-right[
.content-box-purple[.center[
**Comparator**:

NTRK positive and NTRK negative tumours, treatment with standard of care
]
]
]

---
class: with-logo nice
# Endpoints

---
class: with-logo nice
# Addressing the challenges

### NIHR HTA programme:

- "Modelling approaches for site-agnostic cancer drugs to inform NICE appraisals"
   - PI: Prof Stephen Palmer, University of York
   - Research team York & Sheffield
   - 2018-2020

- Pre-publication report on NICE website

- NICE methods review: case for change
   - **Consultation open until 18 December**
---
class: virtual vertical-align: top
background-image: url("assets/virtual2020.png")
background-size: cover
background-position: 90% 10%

# .ubuntublue[Interactive poll 6]

.left[
- For histology-independent oncology indications that need to be reassessed by an HTA agency, do you think that additional data collected during a longer follow-up time will address the fundamental areas of uncertainty?
   - Yes, if data is collected through the clinical trial
   - Yes, if real world data is collected
   - Yes, if both types of data above are generated
   - No, uncertainties are unlikely to get resolved
   - I don’t know

]

---
class: with-logo nice
# Recommendations for modelling

- Allow for exploration of heterogeneity among subgroups

- Use different approaches for constructing counterfactual

- Explore use of response-based models and Bayesian hierarchical modelling

---

class: virtual vertical-align: top
background-image: url("assets/virtual2020.png")
background-size: cover
background-position: 90% 10%

# Workshop 7: How to model the cost-effectiveness histology-independent oncology therapies within health technology assessments (HTAs)

## Contact details
Raquel Aguiar-Ibáñez MSc .small[ [raquel.aguiar-ibanez@merck.com](raquel.aguiar-ibanez@merck.com)] 
Dawn Lee MMath, MSc .small[ [dlee@bresmed.com](dlee@bresmed.com)] 
Gianluca Baio PhD .small[ [g.baio@ucl.ac.uk](g.baio@ucl.ac.uk)] 
Jacoline Bouvy PhD .small[ [Jacoline.Bouvy@nice.org.uk](Jacoline.Bouvy@nice.org.uk)]