ebm topik a

Critical appraisal

Treatment

Dr. Zen Ahmad, SpPDDepartemen Penyakit Dalam RSMH Palembang

Clinical Trials

Many types of design Generally : the simpler the better

Straightforward result Easy to understand No or few assumptions

The complicated ones Not easily understood Frequently uses assumptions

Gold standard : randomized, double blind, placebo controlled clinical trial (Randomized controlled trial, RCT)

Clasification : Pragmatic trial Explanatory trial

Pragmatic Trial

Attempt to determine cause-effect relationship Assuming the result will be applied in actual clinical

practice Preferably : binomial outcome (Yes/No) Analysis :

Intention to treat analysis= All randomized subjects are accounted for the final calculation according to their original allocation

Pragmatic Trial

R

Exp

Ctrl

Ca

b

Y

Y

N

N

A, b, c are accounted as failure of Exp arm

Explanatory Trial

Attempt to explain cause-effect relationship Usually in laboratory investigations

(pharmacology, pharmacodinamic, etc) Analysis : on treatment analysis (only subjects

completed the trial are accounted in analysis) Only minimal drop out is allowed, or

replacement for drop outs

Validity

Randomization; was the randomization list concealed ? Was follow-up of patients sufficiently long and complete ? Were all patients analyzed in the groups to which they were

randomized ? Were patients and clinicians kept blind to treatment ? Equal treatment between groups Were the groups similar at the start of the trial ? Sample size

Importance

1. What is the magnitude the treatment effect ?

2. How precise is this estimate of treatment effect

Importance

E 40 10 50

30 20 50C

Y N

X2 = ; df = 1 ; p = 0.04

Importance

E 40 10 50

30 20 50C

Y N

CER = 20/50 = 0.4; EER = 10/50 = 0.2

RRR = (CER-EER)/ CER = (0.4-0.2)/ 0.4 = 50%

Importance

USA, 1960’sNewspaper : the risk of suffering from deep vein thrombosis in OC users was 2 times compared to that in non OC users ! (this is RRR)

Closer examination :The risk for DVT in non OC : 1/ 100.000 person yearThe risk for DVT in OC : 2/ 100.000 person-year

Thus : to have additional bad outcome, one has to treat 100.000 women for year.

Importance

E 40 10 50

30 20 50C

Y N

CER = 20/50 = 0.4 ; EER = 10/50 = 0.2

ARR = (CER-EER) = 0.4 - 0.2 = 0.2

NNT = 1/ ARR = 1/ 0.2 = 5

NNT = number needed to treat= number of patients should be treated to avoid 1 bad outcome= number of patients should be treated to have 1 additional goodd outcome

NNH = number needed to harm

CI for NNT

NNT = 1/ ARR

First calculate CI for ARR(ARR = diff between proportion)

Then calculate1/ (upper CL of ARR)and 1/ (lower CL of ARR)

Calculating CI for NNT

CER = 20/50 = 0.4; EER = 10/50 = 0.2ARR = (CER – EER) = 0.4 – 0.2 = 0.2NNT = 1/ ARR = 1/0.2 = 5

95 % CI ARR = ARR + 1.96V (p1q1/n1 + p2q2/n2) = 0.2 + 1.96V (0.4 x 0.6)/ 50 + (0.2 x 0.8/ 50) = 0.2 + 0.17 = 0.03 ; 0.37

95% CI = 1/ 0.37 ; 1/0.03= 3 ; 34

1. Is our patient so different from those in the study that its result cannot apply ?

2. Is the treatment feasible in our setting ?

3. What are our patient’s potential benefits and harm from the therapy ?

4. What are our patient’s values and expectations for both the outcome we are trying to prevent and the treatment we are offering ?

Applicability

Applicability

Your own (s)

(Educated guess) – determine f, I. e. a factor reflecting how much severe are your patient compared to the average of the important prognostic factors of the study subjects)

Your NNT = f x NNT

Applicability

LLH : likelihood of being helped vs harmedStep 1. To elicit our patients

Applicability

Determine PEER = patient expected event rate (event rate of your patient if he/ she is not treated with the drug under consideration)

Then :

Your NNT = PEER / (PEER – EER)

ApplicabilityWhat are our patient’s values and expectations for both the outcome we are trying to prevent and the treatment we are offering ?

patient to make his own treatment decision

LLH : likelihood of being helped vs harmed

LHH : likelihood of being helped vs harmed

Step 1. To elicit our patients preferences

• Description (oral or written), discuss (patient; family)• Judgment (outcome; adverse event)

• ie: Relapse 20 times as severe as the side effect• rating scale

• from 0 (worse/ death) to 1 (full health)• Example: 0.05 (out come) and 0.95 (adverse e)• Patient believe that disease progression is 19 times

worse than the adverse event

LHH : likelihood of being helped vs harmed

Step 2. To generate the LHH• Reference

• LHH = 1/NNT vs 1/NNH = 1/9 vs 1/4 = 0.11 vs 0.25 (condition, relapse and adverse e were the same severity) th/ is twice as likely to harm you as to help you

Out come CER EER RRR/ RRI

ARR/ ARI

NNT/NNH

Disability 50 % 39 % 22 % 11 % 9Adverse event 37 % 64 % 73 % 27 % 4

LHH : likelihood of being helped vs harmed

Step 2. To generate the LHH• Our patient LHH = (1/NNT) x f vs (1/NNH) x f

= (1/9) x 3 vs (1/4) x 1 = 1.3 : 1 • Final adjustment LHH = (1/NNT) x f x s vs (1/NHH) x f = (1/9) x 3 x 19 vs (1/4) x 1

= 6.3 : 0.25 = 25 : 1• Patients is 25 times as likely to be helped vs harmed by treatment

Critical appraisal

Report of systematic reviews

Dr. Zen Ahmad, SpPDDepartemen Penyakit Dalam RSMH Palembang

Are the results of this systematic review valid

1. Is this a systematic review of randomized trials2. Does this systematic review have a methods

section that describes: Finding and including all relevant trials How the validity of the individual studies was

assessed3. Were the results consistent from study to study4. Were individual patient data used in the

analysis (or aggregate data)

Are the valid results of this SR Importance

1. What is the magnitude the treatment effect ?

2. How precise is this estimate of treatment effect

Formula to convert OR and RR to NNT

For RR < 1NNT = 1/(1 – RR) x PEER

For RR > 1NNT = 1/(RR - 1) x PEER

For OR < 1NNT = 1 - [PEER x (1- OR)] / (1- PEER) x (PEER) x (1- OR)

For OR > 1NNT = 1 + [PEER x (OR - 1)] / (1- PEER) x (PEER) x (OR- 1)

PEER

OR < 10.9 0.8 0.7 0.6 0.5

0.05 209 104 69 52 410.10 110 54 36 27 2210.20 61 30 20 14 110.30 46 22 14 10 80.40 40 19 12 9 70.50 38 18 11 8 60.70 44 20 13 9 60.90 101 46 27 18 12

1. Is our patient so different from those in the study that its result cannot apply ?

2. Is the treatment feasible in our setting ?

3. What are our patient’s potential benefits and harm from the therapy ?

4. What are our patient’s values and preferences for both the outcome we are trying to prevent and the side effect we may cause ?

Applicability

Meta analysis

Systematic review

Review article

Review article

Non systematic In gathering relevant studies No sufficient appraisals

Prone for severe bias Authors tend to cite studies that support their

opinion Still valuable in some areas of study

Clinical & lab descriptions of diseases

Systematic review and Meta analysis

Systematic review Systematic in:

Gathering relevant articles Appraising the articles

No formal statistical analysis Meta-analysis

Systematic review with formal statistical analysis