first in man tokyo

Tokyo First in Man 1

Ethical, practical and statistical considerations in designing first-in-man

studies

Stephen Senn

Outline

• I shall discuss the Royal Statistical Society Working party report in some detail– This made a number of recommendations but

also left many matters open• I shall make some very brief comments

regarding subsequent work



RSS Working Party Members

Dr Dipti Amin, Senior Vice-President, Global Medical & Regulatory Affairs , Quintiles LimitedProfessor R.A. Bailey, Professor of Statistics, Queen Mary, University of LondonProfessor Sheila M. Bird FFPH, Principal Scientist/Statistician, MRC Biostatistics Unit, visiting professor at Department of Statistics and Modelling Science, University of StrathclydeDr Barbara Bogacka, Reader in Probability and Statistics, Queen Mary, University of LondonMr Peter Colman+, Senior Consultant Statistician, Pfizer Global R+D, Statistical ApplicationsDr Andrew Garrett, Vice President Biostatistics, Quintiles LimitedProfessor Andrew Grieve, Professor of Medical Statistics, King’s College LondonProfessor Sir Peter Lachmann, FRS, FMedSci, Emeritus Professor of Immunology, University of CambridgeProfessor Stephen Senn*, Professor of Statistics, University of Glasgow

* Chairman+ Representative of Statisticians in the Pharmaceutical Industry (PSI)

Acknowledgement


TGN1412

• A monoclonal antibody• First-in-man study on 13 March 2006 carried

out by Parexel on behalf of TeGenero• In first cohort 8 volunteers• Six allocated TGN1412 and two allocated

placebo• All six given TGN1412 suffered a cytokine

storm


See. Senn SJ. Lessons from TGN1412. Applied Clinical Trials 2007;16(6):18-22.


A Conventional AnalysisFISHER'S EXACT TEST

Statistic based on the observed 2 by 2 table(x) : P(X) = Hypergeometric Prob. of the table = 0.0357 FI(X) = Fisher statistic = 6.095

Asymptotic p-value: (based on Chi-Square distribution with 1 df ) Two-sided:Pr{FI(X) .GE. 6.095} = 0.0136 One-sided:0.5 * Two-sided = 0.0068

Exact p-value and point probabilities : Two-sided:Pr{FI(X) .GE. 6.095}= Pr{P(X) .LE. 0.0357}= 0.0357 Pr{FI(X) .EQ. 6.095}= Pr{P(X) .EQ. 0.0357}= 0.0357 One-sided:Let y be the value in Row 1 and Column 1 y =6 min(Y) =4 max(Y) =6 mean(Y) = 4.500 std(Y) = 0.5669

Pr { Y .GE. 6 } = 0.0357 Pr { Y .EQ. 6 } = 0.0357


A Slightly Less Conventional AnalysisDatafile: C:\Program Files\Numerical\StatXact-4.0.1\Files\Research\TGN1412.cy3

BARNARD'S UNCONDITIONAL TEST FOR DIFFERENCE OF TWO BINOMIAL PROPORTIONS

Statistic based on the observed 2 by 2 table :

Binomial proportion for column <Yes > : pi_1 = 1.000 Binomial proportion for column <No > : pi_2 = 0.0000 Difference of binomial proportions : Delta = pi_2 - pi_1 = -1.000 Standardized difference of binomial proportions : Delta/Stdev = -2.828

Results:-------------------------------------------------------------------------Method P-value(1-sided) P-value( 2-sided)-------------------------------------------------------------------------Asymp 0.0023 (Left Tail) 0.0047Exact 0.0111 (Left Tail) 0.0113


Conclusions

• “If you need statistics to prove it, I don’t believe it”

• Here the problem is the reverse• You can’t prove it with statistics but

everybody believes• So does this mean statistics is irrelevant• Not if you look more closely…


Further information

• Timing of adverse events• Increasing interest in using this feature in

epidemiological studies– Case series methodology

• Farrington and Whitaker (2006)

• Also if we use background knowledge of risk of cytokine storm we come to quite different conclusions


0 0.2 0.4 0.6 0.8 1

0.005

0.01

0.015

P-value or Likelihood Ratio

Unconditional test

Common probability of reaction

P-va

lue

Barnard

6

8


0 0.01 0.02 0.03 0.04 0.051 10 241 10 231 10 221 10 211 10 201 10 191 10 181 10 171 10 161 10 151 10 141 10 131 10 121 10 111 10 101 10 91 10 81 10 7

L ratio with knowledge of prob adv react placebo

Probability side-effect under placebo

Like

lihoo

d ra

tio

LR 0

0


RSS Recommendations:Themes

• Generic issues• Preparatory work before first in man• Content of protocols (including design)• Risk and information sharing for social

good and reporting standards


Generic

• MHRA to make adequate provision for statistical expertise

• Mandatory insurance of participants• Tertiary care hospitals only if cytokine

storm even a remote possibility


So, were the six healthy volunteers compensated without fuss?

From the protocol

Lawyers acting for the six volunteers who suffered multiple organ failure during the trial of a new drug last month fear the men may not get full compensation because the company accepting liability

for the injuries was underinsured, the Guardian has learned. Martyn Day, solicitor for four of the men, said the German pharmaceutical company TeGenero, which created the drug TGN1412, had insurance of only £2m for the trial but the liability for damages and losses suffered by the men may be much bigger.

Guardian, 26 April 2006


From the informed consent


Herald TribuneTeGenero, the small German biotechnology firm that developed TGN1412 - which it touted as a revolutionary new drug against cancer and arthritis - filed for bankruptcy several weeks ago. ….

Its insurance coverage was not adequate to cover a calamitous outcome, said Martyn Day of Leigh Day & Company in London, the lawyer for four of the six ill men. ….

Day said he would argue that Parexel, which contracts with drug makers to test new medicines, should have made sure that its client had adequate insurance. He also questioned the design and conduct of what it should have seen as a delicate trial, he said.

Elizabeth Rosenthal 30 July 2006


Preparatory• Quantitative justification of dose and risk level

with statements of uncertainty• Classification of studies low, medium high risk• Precautionary approach• Separate document covering these for use by

ethics committees, participants, insurers• Human cell line studies to improve inter-species

scaling• A ‘proper interval’ for dosing to be established for

(or justification as to why not needed)


Content of Protocols• Justification of relevant quantities

– proper interval, dose step, safety, expected number of adverse events etc

• Statistical justification of sample size• Justification of design• Detailed description of intended analysis• Design and analysis should reflect realistic PK models• Plan for blood sampling etc to based on pre-clinical studies• True informed consent: ‘open protocol, hidden allocation’


Questions

1. Why 6 + 2

2. Why four doses?

3. Why quintupling?

4. Why simultaneous treatment in cohorts


The Trial of TGN1412 was also a veiled Trial

• Subjects do not know whether they are getting TGN1412 or placebo

• If they were in the first cohort they knew they could not be receiving higher doses

• This would have had implications for analysis had the trial proceeded normally

• Would you pool all placebo results or not?


To Pool or Not?• ‘Yes’, according to the protocol• But then misleading to describe the trial as

double-blind• Bias variance trade-off

– The proposed analysis would not eliminate the biases blinding is designed to eliminate

• Also what about the analysis at the end of each dose step to guide dose-escalation?

• How would this permit pooling of placebo subjects?


Design Number of subjects Variance of differences between doses, and between placebo and each dose, if

a cohort effect is fitted it is known that there is no cohort effect

1

Dose 0 1 2 3 Cohort 1 2 6 0 0 Cohort 2 2 0 6 0 Cohort 3 2 0 0 6

1 2 3 0 0.67 0.67 0.67 1 1.33 1.33 2 1.33

1 2 3 0 0.33 0.33 0.33 1 0.33 0.33 2 0.33

2


1 2 3 0 0.50 0.50 0.50 1 1.00 1.00 2 1.00

1 2 3 0 0.33 0.33 0.33 1 0.50 0.50 2 0.50

3


1 2 3 0 0.29 0.40 0.65 1 0.40 0.65 2 0.58

1 2 3 0 0.29 0.31 0.39 1 0.31 0.39 2 0.42


Risk and Information Sharing• Public debate regarding maximum acceptable risk• Drug regulators to provide mechanism for

sponsors to share data to improve risk assessment• Statistical reporting to be improved to level

provided by International Conference on Harmonisation for phase 3

• Use of quantitative descriptions to be increased• Mock applications currently available are poor and

need to be improved.

Starting doses

• Two standard relevant concepts – MABEL – minimum anticipated biological

effect level– NOAEL – no observable adverse effect level

• Both need to be considered



Who’s Risk?

• Suppose that the acceptable risk to an individual is 1 in 2000

• But we believe it is 1 in 1000 for this drug• By having one placebo for every active

treatment and randomising we can reduce the risk to 1 in 2000

• Does this make it acceptable?


We Think Not

• Such a device reduces the risk to the individual to acceptable levels

• However, it does not reduce the expected number of side-effects per trial– Nor the risk to the insurer

• Hence we suggest the dual perspective• Acceptable to an individual AND to society


The Future of First-in Man Studies

• More care• More science• More forethought• More transparency• More accountability• To be treated as seriously as phase III• But…..

More Recent Work by Rosemary Bailey

Dose 0 1 2 3 4Cohort 1 5 5 0 0 0

Cohort 2 5 0 5 0 0

Cohort 3 5 0 0 5 0

Cohort 4 5 0 0 0 5

Dose 0 1 2 3 4Cohort 1 5 5 0 0 0

Cohort 2 3 2 5 0 0

Cohort 3 1 1 3 5 0

Cohort 4 1 1 1 2 5


‘Senn’ design ‘Catch-up’ design

NB Dose 0 = placebo


Variance of contrast to placebo

3

0.25

2

0.30

0.35

0.40

4

0.45

0.50

1

0.20

Dose

Variance

'Catch-up' design'Senn' design

However,Bailey assumes that the design will reachCompletion. But the whole point is that the design may not reach completion and we have to decide whether to proceed as each cohort reports.


The dashed lines and asterisks give the variances for each dose contrast at the time that each cohort completes. The final value at the end of the design is labelled F.

The red solid line gives variance at the time the decision has to be made about that dose.

The blue diamond is the value for the ‘Senn’ design.

R

R

R

R

F

F

F

F0.2

1

0.3

3

0.4

0.5

0.6

2 4

Cohort

Variance

Conclusion

• Design theory is important but we have to be careful

• These trials involve dynamic decision making

• We have to consider what may happen not just what we hope will happen



The storm has cleared: lessons from the CD28 superagonist TGN1412 trial

Thomas Hünig The life-threatening cytokine-release syndrome suffered by six volunteers in a Phase I clinical trial following administration of the CD28 superagonist antibody TGN1412 (developed by TeGenero) in March 2006 was completely unpredicted by the preclinical studies. Here, Thomas Hünig, main founder of TeGenero, describes the recent investigations into what went wrong and discusses the lessons learnt for future clinical trials.

NATURE REVIEWS | IMMUNOLOGY VOLUME 12 | MAY 2012 | 317

This is an interesting paper but…

• Discussed– Explanation as to why

the dose was incorrectly calculated

– Why MABEL is preferable to NOAEL

• Not discussed– Basic design– Ethical issues– Monitoring– Insurance


In Conclusion

• We still do not know how to get this right• We still have grounds to be uneasy• We need an ethical debate that is not just

limited to the clinical trial community but involves society more widely


first in man tokyo

Healthcare