pre0174-bhattacharyya gouri shankar
TRANSCRIPT
25.11.2016
1
Generating evidence-about effectiveness and value
Dr. G. S. Bhattacharyya
MBBS, MD, MRCP, Ph.D, DNBE
Medical Oncologist
In-Charge, Department of Medical Oncology,
Fortis Hospital, 730 Anandapur, EM By-pass Road,
Adarsha Nagar, Kolkata-700107, West Bengal, India
Member GIN. LMIC Committee Member
“Everything should be made as simple as possible but not simpler.”……………...(Albert Einstein)
The Future: Personalized Healthcare
Clinical Certainty& Efficiency
Acquisition Integration
Personalization
Rx
Diagnostic Complexity
… Sophisticated Decision Support
Diagnostics Growth
… More SpecificDiagnostics
Evidence-Based Medicine &
Patient-Centered Choice
A. Good evidence/Important to patient
B. Good evidenceC. Potential for good evidence
D. Important to patient choice/potential for good evidence
E. Important to patient choice/ No potential for evidence
A B
A. L. Cochrane, from T. Hope Evidence-based patient
choice and the doctor patient relationship in But Will
it Work Doctor? Kings Fund, London 1997, 20 – 24
CD
E
IMPORTANT
EVIDENCE
Trends In Cost Of A Twenty-Four-Week Colorectal Cancer Treatment Regimen And Change In Quality-Adjusted Cost
Of Care For Colorectal Cancer, 1998–2005.
Darius Lakdawalla et al. Health Aff 2015;34:555-561
©2015 by Project HOPE - The People-to-People Health Foundation, Inc.
Hierarchy of evidence
STUDY DESIGN
� Randomized Controlled
Trials
� Cohort Studies and Case Control Studies
� Case Reports and Case Series, Non-systematic observations
BIAS
Expert Opinion
Exp
ert O
pin
ion
Expert Opinion
BEST EVIDENCE
25.11.2016
2
The best evidence
Eminence based medicineEloquence based medicineEmotionbased medicineEvidence Based Medicine (old school)Evidence Based Medicine
1. addresses health outcomes and not just intermediate outcomes
2. is from “real” patients3. considers harms and
benefits4. fits the circumstances5. comes from well-designed,
well conducted studies
Some limitations of evidence based medicine
Many questions do not have answers!Evidence from populations - ?Relevance to individualTrials - not ‘real’ usageLack of local ownership of recommendationsClinical effectiveness vs cost effectiveness
Patients with multiple chronic conditions
• Those to whom most guidelines could apply, yet no guidelines ‘made’ for
them
• To enable stratified guidelines
– Need to generate science
– Ensure reporting and packaging off information
– Perhaps prioritize clusters off conditions
– Encourage guideline developers to take steps in this direction
Clinical Practice Guidelines (CPGs) and People with Multimorbidity
“Treating an Illness Is One Thing. What About a
Patient With Many?”
New York Times, March 31, 2009
Image: Brendan Smialowski for the New York Times
Prevalence of co-occurring chronic
conditions is high
CPGs developed for and emphasize single
disease perspective
Evidence-based Practice and Healthcare
Seven StepsStep 0 - Cultivate a spirit of inquiryStep 1 - Ask clinical question in PICOT format Step 2 - Search for the best evidenceStep 3 - Critically appraise the evidenceStep 4 – Integrate the evidence with clinical expertise and patient preferences and valuesStep 5 – Evaluate the outcomes of the practice decisions or changes based on evidenceStep 6 – Disseminate the result
Other models in use
Population,
Interest (area of )
Comparison intervention or group,
Outcome,
Time
Reassessment of clinical practice guidelines
Editorial by Shaneyfelt and Centor (JAMA 2009)-“Too many current guidelines have become marketing and opinion-based pieces…”“AHA CPG: 48% of recommendations are based on level C = expert opinion…”“…clinicians do not use CPG […] greater concern […] some CPG are turned into performance measures…”“Time has come for CPG development to again be centralized, e.g., AHQR…”
25.11.2016
3
Where GRADE fits in
Prioritize problems, establish panel
Systematic review
Searches, selection of studies, data collection and analysis
Assess the relative importance of outcomes
Prepare evidence profile:
Quality of evidence for each outcome and summary of findings
Assess overall quality of evidence
Decide direction and strength of recommendation
Draft guideline
Consult with stakeholders and / or external peer reviewer
Disseminate guideline
Implement the guideline and evaluateG
RA
DE Systematic review
Guideline development
P
I
C
O
Outcome
Outcome
Outcome
Outcome
Critical
Important
Critical
Not
Summary of findings
& estimate of effect for each outcome
Rate
overall quality of evidence across outcomes based on
lowest quality
of critical outcomes
RCT start high,
obs. data start low
1. Risk of bias
2. Inconsistency3. Indirectness
4. Imprecision
5. Publication bias
Gra
de
d
ow
nG
rad
e
up 1. Large effect
2. Dose response
3. Confounders
Very low
Low
Moderate
High
Formulate recommendations:
• For or against (direction)
• Strong or weak (strength)
By considering:
� Quality of evidence
� Balance benefits/harms
� Values and preferences
Revise if necessary by considering:
� Resource use (cost)
• “We recommend using…”
• “We suggest using…”• “We recommend against using…”
• “We suggest against using…”
Practical Clinical Trials
1. Compare clinically relevant interventions2. Enroll a diverse study population3. Recruit from a variety of practice settings4. Measure a broad range of relevant health
outcomes
PCTs vs. ECTs
Practical Clinical Trials Explanatory Clinical Trials
Hypothesis and study design are formulated based on information needed to make a decision
Designed to better understand how and why an intervention works
Addresses risks, benefits, and costs of an intervention as they would occur in routine clinical practice
Maximize the chance that biological effect of a new treatment will be revealed by the study
Models of Research
Traditional Model Patient‐Centered
Model (PCORI)
Audience Other researchers, maybe
policy makers.
End-users (patients, health care
providers,professional
organizations, policy makers)
Products Presentations and
publications in scholarly
journals
New tools, new evidence*
Patient
Involvement
Often none Essential in determining
acceptability to end-users
Stakeholder
Involvement
Often none Essential for partnering on
dissemination (eg, co- branding)
Mechanisms that allow patients to collect outcomes from home before or after an office visit have proved
helpful.
Improving patient engagement continues to be seen as a necessary step for improving the efficiency and
safety of care……. Health Affairs 35, no.4 (2016):575-582
*Reported in a manner understandable to each target audience.
Clinical benefit
a) Survival extensionb) Improvement in functionc) Quality of LifeCost Effectiveness – Never Considered
Post 1960� Evidence of Effectiveness
1. Defined as benefit to patients, not to doctors or
to society at large
� Safety to the patient
Efficacy Standards ‐ IOM
25.11.2016
4
1. Patient reported outcomes are UNEQUIVOCALPatient reported outcomes are difficult to blind, serial assessments are required
2. Soft end pointsObjective response ratesTime to progressionProgression free survivalDisease free survival
3. Clinical significance to changes of quality of life is unclear4. Combined tumor effects and quality of life are more tangible and
credible5. Using predictive biomarkers – for safety, response and
metabolism
What constitutes reasonable evidence of efficacy and effectiveness in cancer?
1. Evidence – that which tends to prove or disprove something, Ground of belief or proof
2. Criteria for evidence in absolute certainty, is difficult3. P-value is statistical4. Clinical trials are not perfect5. Evidence is often unavailable, inconclusive or contradictory
Efficacy – the capacity for producing a desired result or effect
What is the desired result?Full clinical benefit or a surrogate� Can it be measured precisely and reliably� Is the result transferred to community or real life
Effectiveness – how well a treatment works in practice
1. Most often oncology trials are not evaluated in community
2. Clear standards are not available
3. Collecting and reporting clinical effectiveness data are disorganized
4. Off label use is common
Randomized clinical trials are considered as Gold Standards
Not everything can be randomized
Other forms of analysis such as propensity score, statistical modelling may facilitate casual inference
Co-variates do not determine whether the patient do well or bad
Clinical components of Randomized trials
1. Designation of pre-specified hypothesis with primary, and secondary end points
2. Pre specified data cut-off for any continuous measurement to define what constitutes a positive or negative finding
3. Define sample sets with eligibility criteria that are as inconclusive as possible.
1. Power calculation to show that there is a reasonable probability of definitely answering the research question.
2. Un-bias – ascertainment of end points including blinding wherever possible and ethical protocol specified criteria and independent review of end points
3. Complete information through a standard follow-up schedule and a few patients lost to follow-up
25.11.2016
5
1. After being efficacious in RCTs, treatment are often refined to be studied in community where control environment is lost
2. Doses and schedules are changed3. Combination are made4. Drugs are often used off label
1. Refinements are rarely made2. Comparative effective research3. Meta-analysis on large scale4. Observational studies may prove some
validation
Tools are limited, hence no level 1 evidence
HOW TO BRIDGE ?
TWO SOLUTIONS
a) Cluster Randomizationb) Large simple pragmatic trials - QASARc) Evidence can be evaluated on a hierarchy of end
point strength – eg Overall Survivald) Use of biomarkers
−Predictive−Prognostic
Evidence based medicine (1)
Potential questions
• Is the clinical trial population representative of real-life?
• Cost-effectiveness over established (generic) comparator
• What do we know about the risk of polymedication?
• Will frail patients have the same benefit/risk profile?
• Costs and benefits for individual and society?
Challenges
• Balancing risk of involving frail/older patient in clinical trials
• Creating an “orphan” older population? (not authorising/not reimbursing)
• Design appropriate pharmacovigilance for unknowns
• Communicating to reduce inappropriate prescription
eCTD Module Age 65‐74number / total number (all ages)
Age 75‐84number / total number (all ages)
Age 85+number / total number (all ages)
Efficacy and Safety Studies
Human PK Studies
Human PD Studies
Biopharmaceutical Studies
MedDRA TermsAge <65
number (percentage)
Age 65‐74
number (percentage)
Age 75‐84number
(percentage)
Age 85+
number (percentage)
Total ADRs
Serious ADRs – Total
- Fatal
- Hospitalization/prolong existing hospitalization
- Life-threatening
- Disability/incapacity
- Other (medically significant)
AE leading to drop-out
Psychiatric disorders
Nervous system disorders
Accidents and injuries
Cardiac disorders
Vascular disorders
Cerebrovascular disorders
Infections and infestations
Quality of life decreased
Sum of postural hypotension, falls, black outs, syncope,
dizziness, ataxia, fractures
Evidence based medicine (2)
Assessment report‐ geriatric tables
Take home messages:
1) Qualitative not statistical-
focus attention of reviewer on
available data in relation to
epidemiology of disease
2) Adaptations might be
appropriate depending on
product/disease
3)statements made after
consideration of these data
should be meaningfully
reflected in the product
information.
25.11.2016
6
CER Concept and Definitions
Comparative Effectiveness Research Other definitions offer considerable overlap
Organization Definition
American College of PhysiciansEvaluation of the relative (clinical) effectiveness, safety and cost of 2 or more medical services, drugs, devices,
therapies, or procedures used to treat the same condition.3
Institute of Medicine (IOM) – Roundtable
on Evidence-Based Medicine
Comparison of one diagnostic or treatment option to ≥1 others. Primary CER involves the direct generation of
clinical info on the relative merits or outcomes of one intervention in comparison to ≥1 others. Secondary CER
involves the synthesis of primary studies to allow conclusions to be drawn.4
Agency for Healthcare Research and
Quality (AHRQ)
A type of health care research that compares results of one approach for managing a disease to results of other
approaches. CER usually compares ≥2 types of treatment, such as different drugs, for the same disease but it can
also compare medical procedures and tests. The results can be summarized in a systematic review.5
Medicare Payment Advisory Commission
(MedPAC)
Evaluation of the relative value of drugs, devices, diagnostic and surgical procedures, diagnostic tests, and medical
services. By value, it is meant the clinical effectiveness of a service compared with its alternatives.6
Congressional Budget Office (CBO)A rigorous evaluation of the impact of different options that are available for treating a given medical condition
for a particular set of patients. Such research may compare similar treatments, such as competing drugs, or it
may analyze very different approaches, such as surgery and drug therapy.7
Center for Medical Technology Policy
(CMTP)
The direct comparison of existing health care interventions to determine which work best for which patients and
which pose the greatest benefits and harms. The core question of comparative effectiveness research is which
treatment works best, for whom, and under what circumstances.8
CER is the generation and synthesis of evidence that compares the benefits and harms of alternative methods to
prevent, diagnose,treat and monitor a clinical condition or to improve the delivery of care.The purpose of CER is to
assist consumers, clinicians, purchasers and policy makers to make informed decisions that will improve health care
at both the individual and population levels. Primary CER involves the direct generation of clinical info on the
relative merits or outcomes of one intervention in comparison to ≥1 others. Secondary CER involves the synthesis
of primary studies to allow conclusions to be drawn.4
Weighing the strengths and limitations of each type of comparative effectiveness research (CER) study.
Comparing Pragmatic vs. RCT and Observational
Characteristic RCT Pragmatic Observational
FocusEfficacy and safety; assess
mechanistic effect; Can it work
Effectiveness and safety; assess /
inform decision-making; Does it work
under usual care conditions?
Effectiveness and safety; Does it
work in actual practice?
Setting Ideal / artificialReal-world routine care (with
potential minor departures)Real-world routine care
Population Strictly defined; homogenous Typically broad; heterogeneous Broad; heterogeneous
Randomization Yes Typically yes No
Blinding Typically yes No No
Interventions Fully interventional Minimally interventional (e.g., rand.) Non-interventional
Outcomes Clinical surrogates; short term Longer term outcomes; PROs Long term outcomes; PROs
Sample Size Typically small Typically larger Typically large
ValidityHigh internal (↓ bias); low
external (↓ generalizability)
Moderate internal; moderate to high
externalLow internal; high external
Prospective/Retro Prospective Prospective Prospective or retrospective
Comparable cost Higher Moderate Lower
Example sub design Adaptive design LST; adaptive designDatabase studies, cohort, case-
control, cross-sectional
MCDA FOR HEALTHCARE
Multicriteria decision analysis (MCDA) is an application of analytical methods to help decision-makers to explicitly
consider(qualitatively or/and quantitatively)
multiple criteria to support their holistic integration to achieve a predetermined goal
35
MCDA – OPERATIONALIZING THE GOAL OF HEALTHCARE
EVIDEM Collaboration, a not-for-profit organization developing collaboratively an open source multipurpose MCDA-based approach translated in 10 languages and used throughout the world
www.evidem.orgg
Decision Tree
Criteria
Decision evidence MatrixProcess
“Doing
what is
best”
RootsEthical positions
Goal
25.11.2016
7
� Maximize efficacy/effectiveness
� Maximize safety
� Maximize patient perceived health/reported outcomes
� Best therapeutic benefit (e.g, cure)
� Best preventive benefit - Public health (eradication )
Imperative to help (deontology)
MCDA –TRANSFORMING ETHICAL POSITIONS INTO CRITERIA *
� Benefit greatest number (size of population)
� Economic consequences: savings on cost of intervention
� Economic consequences: savings on other medical cost
� Economic consequences: savings on non-medical cost
� Management of opportunity cost and affordability
� Alignment with mandate/scope of healthcare system
� Minimize environmental impact
Prioritizing those who are worst off (fairness, distributive justice)
� Alleviate suffering in patients with severe diseases
� Alleviate suffering in patients with no options (Unmet needs)
� Promote intervention for established priorities (e.g., vulnerable
populations, rare disease)
Greatest good for greatest number
(utilitarianism)
� Highly valid and relevant evidence
� Expert consensus
� Wise use of resources (see economic criteria above)
� Awareness of system capacity and appropriate use of intervention
� Awareness of common goal and specific interests
� Awareness of political, historical and cultural context
Practical wisdom & goodness (Virtue ethics)
*Each objectives can be justified by several ethical positions
“Doing what
is best”
Outcomes of intervention
Need for intervention
Type of benefit
Economic consequences of intervention
Knowledge on intervention
� Comparative efficacy/effectiveness
� Comparative safety
� Comparative patient reported outcomes
� Type of therapeutic benefit (e.g, cure)
� Type of preventive benefit - Public health
(eradication )
MCDA – STRUCTURING CRITERIA FOR OPERATIONALIZATION
“Doing what
is best”
Qualitative criteria (contextual)
� Disease severity
� Size of population (greatest number)
� Unmet needs
Quantifiable criteria (normative universal)
� Mandate/scope of healthcare system
� Established priorities (e.g., rare disease)
� Common goal and specific interests
� Environmental impact
� Opportunity cost and affordability
� System capacity and appropriate use of intervention
� Political, historical & cultural context
� Economic consequences: Cost of intervention
� Economic consequences: other medical cost
� Economic consequences: non-medical cost
� Relevance and validity of study data
� Expert consensus –clinical practice guidelines
Normative contextual
Feasibility
Procedural values• Participatory• Reflective• Transparent• Systematic
Highly synthesized evidence
39
MCDA – INTEGRATING EVIDENCE AND VALUES INTO A PROCESS
DECISION CRITERIA
Quantitative criteria Relative Weights
Disease severity � Low � High
Efficacy/effectivenessEtc
� Low � High
3 -WEIGTHS
Highly synthesized evidence
Turner syndrome: Female specific generic disorder characterized
by reduced life expectancy, cardiovascular defects, increased risk of diabetes, absence of puberty, infertility, defects in visuo-spatial
organization and non-verbal problem solving, and short stature (details)
APPRAISAL
Score
� High
�
�
� Low
4- EVIDENCE5 - PERFORMANCE
SCORES
Qualitative criteria
System capacity
etc
Impact
� negative� neutral� positive
6 - QUALITATIVE IMPACTS
• Collaborative• Adaptable• Holistic • Accountable
2 - CRITERIA
“Doing what
is best”
1 - GOAL
EVIDEM Collaboration, https://www.evidem.org/docs/2015/EVIDEM-v3-0-Decision-criteria-conceptual-background-definitions-and-
instructions-June-2015b.pdf
40
MCDA EVIDENCE MATRIX
INTERVENTION OUTCOMES CRITERIA HIGHLY SYNTHESIZED INFORMATION SCORES
Comparative efficacy / effectiveness
4 placebo controlled RCTs (2-year (toddlers) to 11-year treatments; N=42 to 104,
1 in Canada, 3 in USA): Final height of treated patients = 147 cm to 150 cm; difference with untreated = 7 cm
Observational controlled studies (2-year to 8-year treatments, N=26 to 123, 1 in Germany, 1 in Greece, 1 in Israel, 3 in Italy): Final height of treated patients = 148 cm to 151 cm; difference with controls = 2.1 to 6.8 cm (see details)
5 Major improvement 4 3 2 1 0 No improvement
-1 -2 -3 -4 -5 Lower than
comparators
Comparative safety
Common AEs (from RCTs –frequency at least twice of placebo): Surgeries (50%), ear problems (6 % to 47 %), joint (13.5%) and respiratory (11%) disorders, sinusitis (18.9%)
Serious AEs (from registries, no control data): Intracranial hypertension (0.2%), slipped capital femoral epiphysis (0.2 – 03.%), scoliosis (0.7%), pancreatitis (0.1%), diabetes mellitus (0.2 to 0.3%), cardiac/aortic events (0.3%), malignancies (0.2%)
Warnings: Scoliosis, slipped capital femoral epiphysis, intracranial hypertension, ear disorders, cardiovascular disorders, autoimmune thyroid disease, insulin resistance
5 Major improvement 4 3 2 1 0 No improvement
-1 -2 -3 -4 -5 Lower than
comparators
Comparative patient-perceived health / patient-reported outcomes
Inconclusive data:
1 RCT (2-year treatment data, N=28, Canada): higher rating on questionnaire by GH treated patients versus untreated for some domains but not for others
2 observational studies : no significant differences on SF-36 dimensions in one study (5-year treatment, N=568, France) and significant differences in another (7-year treatment N=29, Holland); other questionnaires, non significant differences Convenience: Subcutaneous injection 3 days a week or daily
5 Major improvement 4 3 2 1 0 No improvement
-1 -2 -3 -4 -5 Lower than
comparators
4- EVIDENCE5 - PERFORMANCE SCORES
INTERPRETATIONOF EVIDENCE BY EVALUATORS
Sir Rawlins,
NICE, HTAi
Seoul 2013:
“Accept that
interpretation of
data that takes
place during
appraisals
requires
judgement”
The Trillion dollar questions
1.How to collect evidence on value and then incorporate this evidence into decisions on coverage, reimbursement, and payment for healthcare services? 2.How to develop value-based, cost effective healthcare that is trusted and not perceived as only cost cutting for profit/balancing federal budget. 3.Lack of evidence is a real impediment to value-based healthcare. Collecting this data will take time. Policy decisions based on incomplete data is subject to serious negative consequences. Half truth sometimes more dangerous than nothing.
Comparative clinical effectiveness
Incremental cost for better clinical outcomes(long-term)
Other benefits ordisadvantages
Contextual considerations
“Care Value”
Public discussionand vote
HIGH INTERMEDIATE
LOW
“Care Value”
Public discussion and vote
Potential health system budget impact
(short-term)
Provisional“Health System Value”
Public discussion
NO VOTE OR FORMAL DESIGNATION
Maximizing Health System Value
Policy Roundtable discussion
HIGH INTERMEDIATE
LOW
25.11.2016
8
• Comparative clinical effectiveness reflects a joint judgment of the magnitude of the comparative net health benefit and the level of certainty in the evidence on net health benefit.
• Patient groups inform what outcomes are important, differencesacross severity, time in disease course, etc.
• Patient groups inform re: opportunities for using or generating real-world evidence
Comparative Clinical
Effectiveness
Incremental cost per
outcomes achieved
Other Benefits or
Disadvantages
Contextual
ConsiderationsCare Value
The cost-effectiveness plane
New treatment more effective but more costly
New treatment more effective
New treatment more costly
Not costly &very effective
Costly & not very effective
Maximum acceptable cost-effectiveness ratio?
• Long-term perspective on clinical outcomes and cost
• Costs from health system (payer) perspective – all health care costs
• Standard measures of health gain– Additional life-years gained– Improvement in quality of life– Cost per quality-adjusted life year gained, aka “cost per
QALY”
Comparative Clinical
Effectiveness
Incremental Cost per
Outcomes Achieved
Other Benefits or
Disadvantages
Contextual
ConsiderationsCare Value
• Societal “willingness to pay”– WHO 1-3x per capita GDP ($50,000-$150,000)
• Individual “willingness to pay”– ~2 times annual salary ($100,000)
• “Opportunity cost” for the health system– ~1x GDP in UK, Latin America– Extrapolated ~$50,000 per QALY in the US
• ICER: $100,000-$150,000 per QALY
Comparative Clinical
Effectiveness
Incremental Cost per
Outcomes Achieved
Other Benefits or
Disadvantages
Contextual
ConsiderationsCare Value
• Patient groups and others asked about benefits or disadvantages offered by the intervention to the individual patient, caregivers, the delivery system, other patients, or the public that would not have been considered as part of the evidence on comparative clinical effectiveness.
– Methods of administration that improve or diminish patient acceptability and adherence
– A public health benefit, e.g. reducing new infections– More rapid return to work or other positive effects on productivity (if not
considered a benefit as part of comparative clinical effectiveness)
• To be judged not by ICER but by one of its independent public appraisal committees
Comparative Clinical
Effectiveness
Incremental Cost per
Outcomes Achieved
Other Benefits or
Disadvantages
Contextual
ConsiderationsCare Value
• Contextual considerations include ethical, legal, or other issues that influence the relative priority of illnesses and interventions.
• Specific issue to be asked of patient groups and others:– Is this a condition of notably high severity for which other
acceptable treatments do not exist?– Are other, equally or potentially more effective treatments nearing
introduction into practice?– Would other societal values accord substantially more or less
priority to providing access to this treatment for this patient population?
• To be judged not by ICER but by one of its independent public appraisal committees.
Comparative Clinical
Effectiveness
Incremental Cost per
Outcomes Achieved
Other Benefits or
Disadvantages
Contextual
ConsiderationsCare Value
25.11.2016
9
Comparative clinical effectiveness
Incremental cost for better clinical outcomes(long-term)
Other benefits ordisadvantages
Contextual considerations
“Care Value”
Public discussionand vote
HIGH INTERMEDIATE
LOW
“Care Value”
Public discussion and vote
Potential health system budget impact
(short-term)
Provisional“Health System Value”
Public discussion
Maximizing Health System Value
Policy Roundtable discussion
HIGH INTERMEDIATE
LOW
Copyright ICER 2016
Comparative clinical effectiveness
Incremental cost for better clinical outcomes(long-term)
Other benefits ordisadvantages
Contextual considerations
“Care Value”
Public discussionand vote
HIGH INTERMEDIATE
LOW
“Care Value”
Public discussion and vote
Potential health system budget impact
(short-term)
Provisional“Health System Value”
Public discussion
Maximizing Health System Value
Policy Roundtable discussion
HIGH INTERMEDIATE
LOW
Copyright ICER 2016
There are Multiple ValueFrameworks
Clinical benefit: OS>PFS>RR
Toxicity: add points if less toxic, subtract if more
Net Health Benefit
Net Health Benefit and Cost: the ASCO Framework Two versions: advanced and curative contexts
Comparison in a trial: test vs standard
Cost (to system and to the patient)
Bonus: Extended survival
Calculating Clinical Benefit (modification)
Step 1: Determine the regimen’s CLINICAL BENEFIT
1.A. Is Hazard Ratio (HR) for deathreported?
YES. Assign an HR Score for death by subtracting the HR from 1, and thenmultiplying the result by 100. Write this number in the box labeled, “HR Score
(death)”. Proceed to 1.F.HR Score (death)
No. Proceed to 1B.
1.B. If HR for death is not reported, ismedian Overall Survival(OS)
reported?
YES. Assign an OS Score by calculating the % difference in median overall survival between the two regimens. Write this number in the box labeled, “OS Score”.
Proceed to 1.F.OS Score
NO. Proceed to 1.C.
1.C. If median OS is not reported, isHazard Ratio (HR) for disease
progression reported?
YES. Assign an HR Score for disease progression by subtracting the HR from 1, multiplying the result by 100, and then multiplying this number by 0.8. Write this
number in the box labeled, “HR Score (progression)”. Proceed to 1.F.
HR Score (progression)
NO. Proceed to 1.D.
1.D. If HR for disease progression is not reported, is median Progression-
Free Survival(PFS) reported?
YES. Assign an PFS Score by calculating the % difference in median progressionsurvival between the two regimens. Multiply this number by 0.8. Write this number
in the box labeled, “PFS Score”. Proceed to 1.F.
PFS Score
NO. Proceed to 1.E.
1.E. If median PFS is not reported, isResponse Rate (RR) reported?
YES. Assign an RR Score by adding the complete response (CR) and partial response (PR) rates, and then multiplying this number by 0.7. Write this number in
the box labeled, “RR Score.” Proceed to 1.F.
RR Score
1.F. Calculate the Clinical BenefitScore
Insert the HR, OS, PFS, or RR Score. Note: You should have a score for only 1 ofthe clinical benefit scales above. Write the total in the box labeled “Clinical Benefit
Score”. The maximum allowable points are 80 . Proceed to Step 2.Clinical Benefit Score
JCO, July 2016
25.11.2016
10
Calculating Toxicity Score(Modification) Grade and frequency matter
Step 2: Determine the regimen’s TOXICITY
Does the new regimen represent animprovement in toxicity over the
standard of care/ comparator?
For each of the regimens being assessed, compare the number and frequency of clinically relevant toxicities, and assign a Toxicity Score (-20 through +20) as shown below. Each clinically meaningful toxicity (i.e.,
exclude laboratory results only) is assigned a score between 0.5-2.0 based on grade and frequency.· For every Grade 1 or 2 toxicity with a frequency <10%, add 0.5 points.
· For every Grade 1 or 2 toxicity with a frequency greater than or equal to 10%, add 1.0 points.· For every Grade 3 or 4 toxicity with a frequency <5%, add 1.5 points.
· For every Grade 3 or 4 toxicity with a frequency greater than or equal to 5%, add 2.0 points.
Calculate the total number of toxicity points for each regimen. Calculate the % difference in total toxicity points between the two regimens, then multiply by 20 to obtain a toxicity score. If the regimen being
evaluated is more toxic than the comparator, subtract the toxicity score of the regimen from the clinical benefit score. If the regimen is less toxic than the comparator, add the toxicity score of the regimen to the
clinical benefit score. If there are unresolved symptomatic treatment-related toxicities at 1 year after completion of treatment, subtract 5 additional points from the clinical benefit score.
Proceed to Step 3.
JCO, July 2016
Calculating Bonus Points modification
JCO, July 2016
Step 3: Determine Bonus Points
3.A. TAIL OF THE CURVE. At a timepoint that is double the median overall
survival (or median PFS if mOS is not reported) of the comparator regimen,
is the fraction of patients surviving on the comparator regimen at least 20%,
AND is there at least a 50% relative improvement in the fraction of patients
surviving?
YES. If yes, award 20 points, and place this number in the box labeled “Tail of the Curve Bonus Points”. Proceed to Step 3.B.
Tail of the Curve Bonus Pts
NO. No bonus points are awarded. Proceed to Step 3.B.
3.B. PALLIATION BONUS. Are data
related to the palliation of symptoms
reported?
YES. If a statistically significant improvement in cancer-related symptoms is
reported, award 10 points, and place this number in the box labeled “Palliation Bonus Points”. Proceed to Step 3.B.
Palliation Bonus Pts
NO. No bonus points are awarded. Proceed to Step 3.C.
3. C. TREATMENT-FREE INTERVALBONUS. Are data related to treatment-
free interval reported?
YES. If a statistically significant improvement in treatment-free interval is reported,
award points based on the table below, and place this in the box labeled “Clinical Benefit Bonus Points”. This is the interval from completion of study treatment to resumption of treatment. Proceed to 3.D.
Treatment-Free Interval Bonus
Bonus Points 0 5 10 15 20
% Change >0-19 20-35 36-49 50-74 >/=75
NO. No bonus points are awarded. Proceed to Step 3.D. Total Bonus Pts
3.D. Calculate Total Bonus Points Add the Palliation Bonus Points (Step 3.A) and the Treatment-Free Interval Bonus
Points (Step 3.B). Write this number in the box labeled “Total Bonus Pts”. The maximum points available for Bonus Points is 50. Proceed to Step 4.
Step 4: Determine the regimen’s NET HEALTH BENEFIT
Calculate the Net Health Benefit Add the Clinical Benefit Score (Step 1), Toxicity Score (Step 2), and Bonus Points
(Step 3). This yields a Net Health Benefit Score. Write this number in the box labeled “Net Health Benefit”. The maximum points available for Net Health Benefit are 150 (100 + 50 bonus points). Proceed to Step 5.
Net Health Benefit
Calculating Net Health Benefit Juxtaposed against Cost per Month
JCO, July 201635http://www.ncbi.nlm.nih.gov/pubmed/26028407
Nivolumab vs docetaxel
ClinicalBenefit Score
HR (death) = 0.59Clinical benefit = (1 – 0.59) x100 x1 = 41
41
Toxicity Score
Bonus Points
Palliation 0
Treatment-Free Interval 0
Total Bonus Points 20
Net Health Benefit 73.2
DAC $11,699
Cost/NHB 160
Nivolumab
5.5
34.5
0
13
Docetaxel
12.2
Toxicity score: 13/33.5 – 1 = -61%-0.61 * (-20) = 12.2
Tail of the Curve 20
7 x 0.5 = 3.59 x 1 = 9
10 x 1.5 = 153 x 2 = 6
Total 33.5
Gr 1-2 <10%Gr 1-2 >10%
Gr 3-4 <5%Gr 3-4 >5%
CheckMate 017: Nivolumab versus docetaxel
in advanced squamous NSCLC
11 x 0.5 =3 x 1 =
3 x 1.5 =0 x 2 =
Total
1
N Engl J Med 2015; 373:1627-1639
NCCN Evidence Blocks™
NCCN Evidence Blocks
© NationalComprehensiveCancerNetwork,Inc. 2016, All rights reserved.The NCCN Guidelines® andthis illustrationmay not be reproduced in
any formwithout the express written permissionof NCCN®.
• Use consistent methodology and display to inform decision-making
• Measures
– Efficacy
– Safety
– Quality of Evidence
– Consistency ofEvidence
– Affordability
• More shading is better
E S Q C A
25.11.2016
11
Efficacy of Regimens Scale
© NationalComprehensiveCancerNetwork,Inc. 2016, All rights reserved.The NCCN Guidelines® andthis illustrationmay not be reproduced in
any formwithout the express written permissionof NCCN®.
Score Summary Definition
5 Highly effective
Often provides long-term survival advantage or curative potential
4 Very effective
Sometimes provides long-term survival advantage or curative potential
3 Moderatelyeffective
Modest, no, or unknown impact on survival but oftenprovides control of disease
2 Minimally effective
Modest, no, or unknown impact on survival and sometimes provides control of disease
1 Palliativeonly
Symptomatic benefit only
Affordability of Regimens Scale
© NationalComprehensiveCancerNetwork,Inc. 2016, All rights reserved.The NCCN Guidelines® andthis illustrationmay not be reproduced in
any formwithout the express written permissionof NCCN®.
Affordability refers to overall cost of an
intervention including drug cost, required
supportive care, infusions, toxicity monitoring,
management of toxicity, probability of care being
delivered in the hospital
Score Summary/Definition
5 Very inexpensive
4 Inexpensive
3 Moderately expensive
2 Expensive
1 Very expensive
ESMO ValueFrameworks
Factors taken into account for ESMO-MCBS
Magnitude of Clinically Benefit
Overall survival,
Progression free survival
Toxicity
Costs
Prognosis of the
condition
Quality of Life
HR,
Long term survival,
RR
Cherny, N et al, Ann Oncol epub 30 May 2015
25.11.2016
12
Considering the Several Frameworks?
• Shall we strive for a uniform approach to determining clinicalbenefit”?
• Cross- trial comparisons are necessary if clinical benefit assessments are to have meaning
• Reasonable cost; after accounting for net health benefit and cost
-threshold for value – how much will we/can we spend forwhat degree of gain?
- is there a role for value-based pricing?
• How to incorporate the value of agents/regimens into clinicalpathways?
• Shared decision making
The Goal
• All patients with cancer will have lifelong access tohigh-quality, effective, affordable andcompassionate care
• The most accurate cancer information will beavailable so that patients and physicians can makeinformed decisions about cancer prevention andtreatment (shared decision-making)
• The most accurate cancer information will beavailable so that policy makers will make informeddecisions based upon the value to be delivered by cancer prevention and treatment
Discussion Good News….
Our future’s so
bright we gotta
wear shades
Evidence as the Basisfor Clinical Policy
Clinical
GuidelinesEvidence
FormularyClinical
Processes & Systems
Appropriateness
Measures
Decision
SupportPerformance
Measures
Clinical
Reminders