chapter 2 what’s the question?

1

Chapter 2Chapter 2

What’s the Question?What’s the Question?

2

NoteNote

• Common painkillers raise heart risk • Popular painkillers such as aspirin,

ibuprofen (易布普洛芬錠 ) and acetaminophen (乙醯氨酚 ) can raise blood pressure and thus the risk of heart disease among men, U.S. researchers reported on Monday

3

Primary vs. Secondary QuestionPrimary vs. Secondary Question

• Primary– most important, central question– ideally, only one (or at least <=2)– stated in advance– basis for design and sample size

• Secondary– related to primary– stated in advance– limited in number

4

Examples (1)Examples (1)

• Physicians Health Study (PHS)– aspirin vs placebo– primary: total mortality– secondary: fatal + nonfatal myocardial

infarction ( 心肌梗塞 , MI)

5


• Eastern Cooperative Oncology Group (ECOG - 1178) – tamoxifen ( 抗雌激素藥物 , 賀爾蒙阻斷劑 )

vs placebo– primary: tumor recurrence/relapse,

disease-free survival– secondary: total mortality

6


• Multicenter Investigation of Limitation of Infarction Size (MILIS)– propranolol ( 心康樂 ) vs. placebo – primary: ultimate size of an acute

myocardial infarction ( 急性心肌梗塞 )– secondary: left ventricular ejection fraction

( 左心室輸出容積比例 )

7


• Chronic Study of Intermittent Positive Pressure Breathing (IPPB)– long-term intermittent positive pressure

breathing ( 間歇性陽壓呼吸 ) vs. nebulizer ( 氣霧式 )

– primary: forced expiratory volume ( 用力呼氣一秒量 )(FEV1)

– secondary: quality of life

8

A-HEFTA-HEFT(The African-American Heart Failure Trial )(The African-American Heart Failure Trial )

• Ref: NEJM, Nov 11, 2004• 1050 African Americans with Class III-IV CHF

(Congestive Heart Failure, 鬱血性心衰竭 ) • Isosorbide Dinitrate ( 二硝酸異山梨酯 )+ Hydrolyzine vs.

Plbo• Composite outcome (death, HF hospitalizations,

change in QoL) (a composite score made up of weighted values ranged from -6 to +2)

• DMC terminated trial early

9

A-HEFTA-HEFT

10

A-HEFTA-HEFT

11

2. Subgroup Questions2. Subgroup Questions

• Questions about effect of therapy in a sub-population of subjects entered into the trial

• Assess internal consistency of results• Confirm previous hypothesis• Generate new hypotheses

12

Subgroup AnalysesSubgroup AnalysesExamples:

Breast Cancer: Does the benefit of treatment depend on: menopausal status, stage of disease, age, etc.

AIDS: Does the benefit of treatment depend on: gender, age, initial CD4 counts, race, etc.

Analyses of a trial by subgroup results in a separate statistical test for each subgroup. As a result the probability of false positive conclusions arising in the analysis of a trial will increase.

13

False Positive RatesFalse Positive Rates

The greater the number of subgroups analyzed separately, the larger the probability of making false positive conclusions.

No. of Subgroups At Least One False Positive

1 .05 2 .097 3 .143 4 .186 5 .226

14

Example - Subgroup ConcernExample - Subgroup Concern

• Second International Study of Infarct Survival (ISIS 2) – 2 x 2 factorial design

(aspirin vs. placebo and streptokinase ( 抗凝血劑 ) vs. placebo)

– vascular and total mortality in patients with an acute myocardial infarction (MI)

– Gemini or Libra astrological birth signs did somewhat worse on aspirin while all other signs and overall results impressive and highly significant benefit from aspirin

15

Subgroup ConsiderationsSubgroup Considerations

• Rules for Subgroups

1. Stated in advance (in protocol)2. Limited in number3. Interpreted cautiously, qualitatively4. Look for consistency of results

• May be used to

1. Confirm or answer specific questions generated in aprevious trial (e.g. Metroprolol <65 vs. >65 age total

mortality

2. Generate new hypothesis to be tested in some future trial

3. Consistency of primary outcomes

16

MERIT-HF Study DesignMERIT-HF Study Design((MeMetoprolol toprolol RRandomized andomized IIntervention ntervention TTrial in rial in

congestive congestive HHeart eart FFailure)ailure)

• Chronic heart failure patients

• Randomized placebo controlled

• Metoprolol (beta-blockers, 心臟 / 降血壓 )vs. placebo

• Two-week placebo run in (compliance)

• Entered 3991 patients

• Terminated early

• Mean follow-up approximately one year

The International Steering Committee on Behalf of the MERIT-HF Study Group,

Am J Cardiol 1997; 80(9B):54J-58J. The MERIT-HF Study Group, ACC, March 1999.

17

MERIT Total MortalityMERIT Total Mortality

18

MERITMERIT

19

MERITMERIT(AHJ, 2001)

20

• Model Choice

– Cox– Logistic

• Test Statistic– Wald (Reg co-efficient)– Score (likelihood)

• Definition of Subgroups– US vs. World– All Countries Separately

Interaction Tests Not Unique

21

Subgroup x TreatmentSubgroup x TreatmentInteractionInteraction

• Qualitative InteractionTreatment effect is different in direction in two subgroups

• Quantitative InteractionTreatment effect is of same direction but of different magnitude

• Statistical tests for interaction not very powerful

• Even if statistically significant, must be cautious in interpretation (PRAISE)

22

PRAISE IPRAISE I(Prospective Randomized Amlodipine Survival Evaluation)(Prospective Randomized Amlodipine Survival Evaluation)

Ref: NEJM, 1996

• Amlodipine ( 脈優錠 , calcium channel blocker, 鈣離子阻斷劑 ) vs. placebo

• NYHA (New York Heart Association) class II-III

( 心臟功能區分為ⅠⅡⅢⅣ四等級 , 第Ⅰ級輕度功能障礙，第Ⅱ級為中度功能障礙，第Ⅲ , Ⅳ 級為嚴重功能障礙 )

• Randomized double-blind• Mortality/hospitalization outcomes• Stratified by etiology (ischemic ( 缺血性 )/non-ischemic)• 1153 patients

23

PRAISE IPRAISE I

24

PRAISE I - InteractionPRAISE I - Interaction

• Overall P = 0.07

• Etiology by Trt InteractionP = 0.004

• Ischemic P = NS

• Non-Ischemic P < 0.001

25

PRAISE I - IschemicPRAISE I - Ischemic

26

PRAISE I – Non- IschemicPRAISE I – Non- Ischemic

27

PRAISE IIPRAISE II

• Repeated non-ischemic strata • Amlodipine vs. placebo• Randomized double-blind• 1653 patients• Mortality outcome• RR 1.0

28

Three Views:Three Views:

• Ignore subgroups and analyze only by treatment groups.

• Plan for subgroup analyses in advance. Do not “mine” data.

• Do subgroup analyses --- However view all results with caution.

29

3. Adverse Effects3. Adverse Effects

• Any intervention should do more benefit than harm

• Not always easy to specify in advance - many variables will be measured (clinical, laboratory)

• Usually not willing or interested in demonstrating an intervention to be harmful

• May be known adverse effects from earlier trials

30

Serious Adverse Events (SAEs)Serious Adverse Events (SAEs)

• Results in death• Life-threatening• Requires inpatient hospitalization or

prolongation (延長 ) of existing hospitalization

• Results in persistent (持續 ) or significant disability/incapacity (傷殘 )

• Congenital anomaly/birth defect• Results in another medically

important condition

31

Serious Adverse Events (SAEs)Serious Adverse Events (SAEs)

Must be reported to regulatory

agencies (DOH) and IRBs

32

Adverse EventsAdverse Events

• Challenges– Short term vs longer term– Longer term follow-up in face of early benefit– Rare AEs may be seen only with very large numbers of

exposed patients and long term follow-up

• Recent Example – COX II (cyclo-oxygenase,還氧化 )– Immediate pain reduction vs longer term increase in

cardiovascular risk– Vioxx ( 偉克適 , Merck) & Celebrex ( 希樂葆 , 關節炎藥 , Pfizer)

33

What’s the Question? What’s the Question? 4. “Natural History”

• Question not related to intervention• Control group, often a “placebo,” may be used to

describe how prognostic factors relate to eventual subject outcome (predictive, not causative)

e.g. Coronary Drug Project: Aided greatly in defining natural history of patients following a heart attack

5. Ancillary• Questions not related at all but still of scientific interest• Usually piggy-backed ( 搭便車 ) onto trial• Must not interfere with trial!

34

What’s the Question?What’s the Question?6. Exploratory

• Most studies conducted to test some hypothesis• Most studies can generate new hypotheses• Multiple analyses often conducted

increased false positive (Type I) error rate• Could demand reduced significance level (or p-value)

for each test•e.g. /K (assuming independent variables)• = .05, K = 10 /K =.005 • But can’t afford this usually

• Could be selective in number of primary hypotheses• Should state key comparisons in advance• Relegate other comparisons to either

• Confirmatory or Exploratory

35

Outcome AssessmentOutcome Assessment

36

What’s the Response Variable? What’s the Response Variable?

• Used to answer primary/secondary questions

• Characteristics for primary/secondary outcomes

1. Well defined & stable

2. Ascertained in all subjects

3. Unbiased

4. Reproducible

5. Specificity to question

37

• Examples

1. MILISInfarct size measurement?- Enzymes (area under curve or peaks)

- Radionuclide (放射性核素 ) imaging - EKG

Issues of definition, ascertainment, reproducible

2. NOTT(Nocturnal Oxygen Therapy Trial ) To compare the efficacy of long-term use of nocturnal ( 夜間 ) oxygen therapy (12 hours) with that of continuous, low-flow oxygen therapy ( 低流量氧療法 ) (24 hours) in patients with chronic hypoxic

lung disease ( 慢性缺氧肺病 )Quality of Life?- POMS (Profile of Mood)- SIP (Sickness Impact Profile)- Pulmonary Function- Survival

Response VariableResponse Variable

38

3. Cardiovascular Disease Trials- Total mortality- CHD mortality- Non-fatal MI- PVC’s

4. Diabetes- Mortality- Blindness- Visual impairment- Retinopathy ( 視網膜病變 )

- Microaneurisms ( 微動脈瘤 )

Response VariableResponse Variable

39

Surrogate Response VariablesSurrogate Response Variables

• Used as alternative to desired or ideal clinical response

• Examples– Suppression of arrhythmia (sudden death)– T4 cell counts (AIDS or ARC)

• Used often - therapeutic exploratory (Phase I, Phase II)

• Use with caution - therapeutic confirmatory (Phase III)

40

Surrogate Response Variables (2)Surrogate Response Variables (2)

• Frequent Criticism of Clinical Trials– Too long– Too large– Too expensive

• Advantages– Perhaps smaller sample size– Detect earlier effect shorter trial– Easier

41

Examples of FDA Approval of Examples of FDA Approval of Drugs Using Surrogates (1)Drugs Using Surrogates (1)

• Lower cholesterol without evidence of survival benefit

• Lower blood pressure without evidence of benefit for stroke, MI, congestive heart failure, or survival

• Increase bone density without evidence of decreased fractures in osteoporosis

42

Examples of FDA Approval of Examples of FDA Approval of Drugs Using Surrogates (2)Drugs Using Surrogates (2)

• Increase cardiac function in congestive heart failure without evidence of survival benefit

• Decrease rate of arrhythmias (VPBs) without evidence of survival benefit

• Lower blood glucose and glycosylated hemoglobin without evidence about diabetic complications or survival benefit

43

Surrogate Response VariablesSurrogate Response Variables• Requirements (Prentice, 1989)

T = True clinical endpoint

S = Surrogate

Z = Treatment

• H0: P(T|Z) = P(T) P(S|Z) = P(S)

• Sufficient Conditions

1. S is informative about T (predictive)

P(T|S) P(T)

2. S fully captures effect of Z on T

P(T|S,Z) = P(T|S)

44

Concerns About SurrogatesConcerns About Surrogates

1. Relationship between surrogate and true endpoint may not be causal, but coincidental to a third factor

2. Other unfavorable effects of the drug

3. Effect on surrogate may correlate with one clinical endpoint, but not others

45

Time

Surrogate

Intervention

Disease End Point

True Clinical Outcome

The setting that provides the greatest potential for the surrogate endpoint to be valid. Reprinted from Ann Intern Med 1996; 125:605-13.

46

Time

True Clinical OutcomeDisease

SurrogateEnd PointA

Surrogate

Intervention

BDisease

End Point


Intervention

CDisease

SurrogateEnd Point



DiseaseSurrogateEnd Point

D

Intervention

Reasons for failure of surrogate end points. A. The surrogate is not in the causal pathway of the disease process. B. Of several causal pathways of disease, the intervention affects only the pathway mediatedthrough the surrogate. C. The surrogate is not in the pathway of the intervention’s effect or is insensitive to its effect. D. The intervention has mechanisms for action independent of the disease process. Dotted lines = mechanisms of action that might exist.

47

Examples Using “Surrogates”Examples Using “Surrogates”

• Chronic Obstructive Pulmonary Disease

• Cardiac Arrhythmias

• Heart Failure

• AIDS

• Osteoporosis

48

Nocturnal Oxygen Nocturnal Oxygen Therapy Trial (NOTT)Therapy Trial (NOTT)

• Hypothesis– Is continuous oxygen therapy better than nocturnal oxygen

therapy in chronic obstructive lung disease patients? • Surrogates• Survival

• Design– 203 patients– Two-sided 0.05 Type I error– Randomized– Multicenter– Sequential data monitoring

49

Possible NOTT SurrogatesPossible NOTT Surrogates• PaO2

• Hematocrit

• FEV1 % Predicted

• FVC % Predicted• Maximum Workload• Heart Rate• Mean Pulmonary Artery Pressure• Cardiac Index• Pulmonary Vascular Resistance• Neuropsychiatric Impairment• Quality of Life

50

The Nocturnal Oxygen Therapy TrialThe Nocturnal Oxygen Therapy Trial

NOTT Survival Experience for 102 Patients on Nocturnal Oxygen (NOT) and 101 Patients on Continuous Oxygen Therapy (COT)

51

Cardiac ArrhythmiasCardiac Arrhythmias

• Cardiac arrhythmias associated with sudden death

• Class of drugs developed to suppress arrhythmias

• FDA approved for high risk patients

• “Off-label” use increased

52

Cardiac Arrhythmia Suppression TrialCardiac Arrhythmia Suppression Trial

Hypothesis

Does suppression of arrhythmia following an MI reduce incidence of:

1. Sudden death

2. Total mortality

53


Design

• Randomized Double Blind

• Three Drug Arms vs. Placebo

• Multicenter Study

• Group Sequential Data Monitoring

• One Sided (0.025 Type I Error) for Benefit

• Advisory One Sided (0.025) for Harm

• Run-in Period (Arrhythmia Suppression)

54


Early Termination in Two Drug Arms

Drugs Placebo

Sudden Death 33 9

Total Mortality 56 22

55

CAST Sequential BoundariesCAST Sequential Boundaries

Early Termination in Two Drug Arms

Drugs Placebo

Sudden Death 33 9

Total Mortality 56 22

56

Chronic Heart Failure (CHF)Chronic Heart Failure (CHF)

• CHF is a serious problem• Patients have reduced cardiac function &

reduced ability to conduct daily activities• Severity stages: NYHA Class I-IV• Mortality rates increase with severity class• Improving cardiac function, exercise

capacity & quality of life desirable• Drugs developed/approved on that basis

57

PROMISEPROMISE(Prospective Randomized Milrinone Survival Evaluation)(Prospective Randomized Milrinone Survival Evaluation)

(Packer et al. NEJM 1991)(Packer et al. NEJM 1991)• Problem

– Patients with advanced (Class IV) congestive heart failure have 40% one year mortality

– Milrinone (a phosphodiesterase inhibitor,心收縮劑 ) enhances cardiac contractility

– Milrinone improved cardiac output, exercise tolerance, and symptoms

• Hypothesis

Does milrinone increase survival in severe (Class III or IV) congestive heart failure patients?

58

PROMISEPROMISEDesign

• Randomized multicenter double-blind, placebo-control trial

• Patients with Class III or IV congestive heart failure for 3 months

• Two-sided 5% significance level, 90% power for 25% reduction in mortality

• 1088 patients entered

• Milrinone (10 mg/4 times per day) vs. matched placebo

• Standard therapy of digoxin, diuretics, and a converting enzyme inhibitor for all patients

59

PROMISE Mortality ResultsPROMISE Mortality Results

60

AIDS Clinical TrialsAIDS Clinical Trials

• Clinical Outcomes– Death– Progression to AIDS– Progression to ARC

• Surrogate Outcome– CD4 Cell Count

61

State-of-the-Art ConferenceState-of-the-Art Conference• Results

– AIDS/Death• *8 trials positive• 7/8 had positive CD4 cell changes• *8 trials negative• 6/8 had positive CD4 cell change

– Death• *4 trials positive• 2/4 CD4 positive• *7 trials negative• 6/7 CD4 cell positive

62

OsteoporosisOsteoporosis(( 骨質疏鬆症骨質疏鬆症 ))

(Riggs et al. NEJM, 1990)

• Bone loss in postmenopausal women leads to increase risk of fracture

• Sodium Fluoride ( 氟化鈉 ) stimulates bone formation and increased bone mass (double)

• Hypothesis– Will Fluoride treatment decrease rate of vertebral fractures ( 椎

骨破裂 )?

• Design– Randomized, double blind, placebo-controlled– 202 postmenopausal women randomized– All received calcium supplementation

63

Osteoporosis Fluoride TrialOsteoporosis Fluoride TrialResults

• Fluoride increased bone density by 35%– 35% (p = 0.0001) in spine– 12% (p = 0.0001) in femoral neck

• Fluoride decrease bone density by 4% in wrist (p = 0.02)

• Vertebral fractures higher on Fluoride (F 163, P 136, p < 0.05)

• Non-vertebral fractures higher on Fluoride (72 vs. 24; p = 0.01)

• Fluoride concluded not effective as a treatment for post-menopausal osteoporosis

64

Concluding Remarks Concluding Remarks on Surrogateson Surrogates

• Surrogates play an important role in the development of Phase I, II, and pilot Phase III studies

• Treatments may affect more than one mechanism

• “Surrogates” do not reliably predict treatment on clinical outcome

• Continued success in a given field is not even guaranteed

• Reliance on “surrogates” should be minimized

65

Measures of Efficacy Measures of Efficacy from Clinical Trialsfrom Clinical Trials

66

Characteristics of a Characteristics of a Good Summary Measure Good Summary Measure

• Easy to compute

• Easily understand by all (non-technical)

• Minimal variance across baseline characteristics

• Statistically sound

67

Purpose and Limitations Purpose and Limitations of Clinical Trialsof Clinical Trials

• Clinical trials are designed to detect differences between treatment groups– relative risk ( or relative risk reduction)

– mean absolute risk reduction (relative to placebo)

• In clinical trials, the method of assessing the primary endpoints is usually pre-specified and stated in terms of RRR or RHR.

• Clinical trials are not designed to directly estimate the incidence in the population at risk.

• The population in a clinical trial may not completely represent the population to be treated

68

Measures Currently UsedMeasures Currently Used

• Relative Risk (RR) and Relative Risk Reduction (RRR)

• Odds ratio (OR)

• Relative Hazard (RH) and Relative Hazard Reduction (RHR)

• Absolute Risk Reduction (ARR)

69

Outcome MeasuresOutcome MeasuresRelative Risk (RR)

RR = P1/P2

Relative Risk Reduction

RRR = 1 - RR

Odds Ratio (OR)

Absolute Risk Reduction (ARR)

ARR = P1 - P2

)P(1P

)P(1POR

12

21

70

0

10

20

30

40

50

Study (first author of paper)

Ann

ual i

ncid

ence

rat

e (%

)

Low

Medium

High

ARR

Placebo incidence rates of vertebral fracture from several studies Placebo incidence rates of vertebral fracture from several studies Efficacy as measured by relative risk reduction was reasonable stable over studiesEfficacy as measured by relative risk reduction was reasonable stable over studies

Absolute risk reduction varied across studiesAbsolute risk reduction varied across studies

71

Study Incidence inplacebo (%)

Incidence inrisedronate (%)

RRR ARR

VERT NA 16.3 11.3 59% 5.0%

VERT MN 29.0 18.1 51% 10.9%

There is a danger in using ARR to compare efficacyThere is a danger in using ARR to compare efficacy

The drug used is the same in both studiesThe drug used is the same in both studies

72

We Need Both MeasuresWe Need Both Measures

• Effectiveness– Related to RR

• Benefits– Related to absolute risk

73

RRRRRR

• Usually– Constant over baseline characteristic– Constant over study time– Easy test of interaction – When not constant it is usually piece-

wise constant– Differences seen among different

studies can be viewed as random– Good statistical models are available

74

Absolute RiskAbsolute Risk

• Unlikely– To be constant over time– To be constant over baseline characteristics– To be able to describe with simple models

• Consequences– Patients characteristics can change with study time– Differences among studies cannot be ignored

75

ConclusionConclusion

• If the RRR is constant and detailed information about the AR is provided both summary measures provide useful information about the effectiveness and benefits of treatment

• ARR is not a simple index of therapeutic effectiveness. It is a function of the incidence rate for the event of interest in the population studied and may not be reflective of the true ARR for the patient sitting before you.

• There is concerns about using the rate in the placebo group from a clinical trial as a surrogate for the true baseline risk for an individual patient.

• Before making a recommendation, one needs to know the risk profile of the patients to be treated

76

SummarySummaryDefining the QuestionDefining the Question

• Defined carefully in advance• Must be clinically relevant• Prioritize into primary, secondary, …• Design built around primary question(s)• Eligibility criteria define population

studied and inferences to be made• Surrogates desirable but risky• Need the relevant measure of efficacy

chapter 2 what’s the question?

Documents