Frances M. Zipp President

Lachman Consultants

Westbury, NY 11590 f.zipp@lachmanconsultants.com

GPhA June 3, 2014

Summary Messages

▪ ISPE has launched an operational pilot to demonstrate the feasibility and value of standard Quality Metrics; McKinsey & Company will collect and analyze data confidentially and will share individual results with participating companies

▪ The goal of the pilot is inform the FDA’s development of a risk-based inspection program based on Quality Metrics; interim and final reports will be issued publically

▪ Initial metrics to be tested in the pilot are based on “consensus industry” metrics and additional inputs from the FDA on leading metrics, as shared in the Brookings Institute meeting of May 1-2, 2014

▪ The pilot will be driven in 2 waves: initial metrics will be collected from participating companies starting in June; additional metrics and companies will be added for “Wave 2” starting in 2015

▪ 25 companies have expressed interest to-date from all industry segments (small and large molecule, Rx, Gx, OTC). We encourage all companies (and industry associations) to participate!

Pilot approach was informed by FDA-Industry meeting at Brookings Institution

May 1-2, 2014, Washington, DC

Overview of agenda

▪ Theresa Mullin: Overview of OPQ (“An Evolution Towards Quality in Review, Inspection and Surveillance”)

▪ Russ Wesdyk: Quality Metrics Goals and Stakeholder Feedback ▪ Katy George: Learnings on Establishing and Implementing Standard Quality Metrics

from McKinsey & Company’s POBOS Quality benchmarks ▪ Panel: Reflections on Consensus Set of Metrics and Approaches recommended to

FDA by Industry to-date ▪ Panel: Exploring the Use of Metrics in Purchasing Decisions ▪ Panel: Opportunities and Challenges in Implementing a Core Set of Metrics ▪ Russ Wesdyk: Presentation of Metrics Discussion Set (including introduction of

“Industry Consensus” Metrics list) ▪ Group discussion: Exploring and Adapting the Metrics Discussion: – Concerns, Additions and Deletions – Definitions – Issues in Implementation

Pilot approach was informed by FDA-Industry meeting at Brookings Institution

May 1-2, 2014, Washington, DC

Agency and Brookings participants:

FDA ▪ Rick Friedman ▪ Melissa Matles ▪ Theresa Mullins ▪ Daniel Orr ▪ Suzanne Pattee ▪ Faiad Rahaman ▪ Neil Stiber ▪ Robert Tollefsen ▪ Jason Urban ▪ Alex Viehmann ▪ Russell Wesdyk ▪ Marta Wosinska ▪ Lawrence Yu

European Medicines Agency ▪ Patrick Costello

Brookings ▪ Mark McClellan ▪ Elizabeth Richardson ▪ Greg Daniel

Pilot approach was informed by FDA-Industry meeting at Brookings Institution

May 1-2, 2014, Washington, DC

Industry participants:

▪ Barbara Allen, Eli Lilly ▪ Ferdinando Aspesi, Novartis ▪ Deb Autor, Mylan ▪ Vinay Bhatt, Zydus ▪ Vincent Calabrese, PBM, US Dept of Veterans

Affairs ▪ Joachim del Boca, Vetter Pharma-Fertigung

GmbH&Co ▪ Lina Evans O'Connor, Teva ▪ Joseph Famulare, Genentech Roche ▪ Katy George, McKinsey & Company ▪ Donna Gulbinski, BMS

▪ Lars Guldbaek Karlsen, Novo Nordisk ▪ Paula Gurz, Premier Healthcare Alliance ▪ Margaret Reagan, Premier Healthcare Alliance ▪ Will Mitchell, P&G ▪ Mary Oates, Pfizer ▪ Luisa Paulo, Hovione FarmaCiencia SA ▪ GK Raju, Light Pharma ▪ Frances Richmond, University of SoCal ▪ Susan Schniepp, Allergy Laboratories ▪ Martin VanTrieste, Amgen ▪ Louis Yu, Perrigo ▪ Fran Zipp, Lachman

Preliminary metrics list for “Wave 1” Both leading and lagging indicators included

▪ APQR reviews completed on time

▪ Recurring deviations rate

▪ CAPA effectiveness rate

▪ Media fill (for aseptic sites)

▪ Lot acceptance rate

▪ Right first time

▪ Complaints rate (total and critical)

▪ Confirmed OOS rate (total and stability)

▪ US recall events (total and by class)

▪ Process capability

▪ Environmental monitoring and bioburden (for aseptic sites)

▪ Quality culture

▪ “Industry consensus” metrics as provided by the FDA ▪ Additional metrics based on FDA and Industry discussion, as well as prior work of ISPE Metrics Team

Additional quantitative metrics

“Industry consensus” metrics

Additional survey-based metrics

Guiding principles for defining the pilot metrics

▪ Pragmatic and feasible

▪ Both leading and lagging indicators

▪ Leading metrics include for example: Lot acceptance rate, Right first time, APQR and Media Fill

▪ Tailored survey-based approaches will develop insight and evolve metrics approach on additional leading indicators: e.g., process capability and culture

▪ Site-level metrics, with complaints rate at product-level, and process capability survey to understand feasibility of product-specific metric

▪ Companies to provide monthly data for approximately one year (historic) and 3-months “real-time”, but individual flexibility possible to accommodate data availability

▪ Learnings to be incorporated to refine metrics for Wave 2

Metrics details – Industry consensus

▪ Total lots released for shipping out of the total finally dispositioned lots for commercial use in the period

▪ Total lots that have not been through rework or reprocessing out of the total finally dispositioned lots for commercial use in the period

▪ Total complaints received in the reporting period, related to the quality of products manufactured in the site, normalized by the number of packs released

▪ Critical complaints (indicating a potential failure to meet product specifications, impact product safety and/or lead to regulatory actions), normalized by the number of packs released

▪ Total confirmed OOS (test results that fall outside the specifications or acceptance criteria), out of all lots dispositioned by the lab during the period

▪ Total confirmed OOS related to stability testing, out of all stability lots dispositioned by the lab during the period

▪ Total unconfirmed OOS out of all lots dispositioned by the lab during that period

▪ Total US market recall events (all and class I/II only) US Recall events (total and by class)

Lot acceptance rate

Right first time

Complaints rate (total and critical)

Confirmed OOS rate (total and stability)

Metric High level definition

(more detailed definitions in the pilot template)

Unconfirmed (invalidated OOS rate

Metrics details – Additional quantitative metrics

Metric High level definition

(more detailed definitions in the pilot template)

▪ Number of Annual Product Quality Reviews in the period that were completed by the original due date, normalized by all products subject to APQR

▪ Number of deviations that have occurred during the preceding 12 month period with the same root cause within the same process and/or work area, out of all deviations in the reporting period

▪ Number of CAPAs evaluated as effective (the quality issue subject of the CAPA was resolved, and/or has not reoccurred, and there have been no unintended outcomes from the CAPA implementation) out of all CAPAs with effectiveness check in the reporting period

▪ Number of media fills dispositioned as successful out of all media fills to support commercial products dispositioned during the period

CAPA effectiveness rate

Media fill (for aseptic sites)

APQR reviews completed on time

Recurring deviations rate

Metrics details – Additional survey-based metrics

Metric High level definition

(more detailed definitions in the pilot template)

Bioburden (for aseptic sites)

Quality culture

Process capability

Environmental monitoring (for aseptic sites)

▪ Open and yes/no questions designed to understand whether the site measures state of control for processes, which measures are used and on what parameters, what share of the site product portfolio they cover

▪ Yes/no questions designed to understand how the environmental monitoring program samples and monitors critical surfaces

▪ Yes/no questions designed to understand how the bioburden testing is performed on aseptic products

▪ 15 agree/disagree questions administered to site personnel via anonymous online survey (50%+ response rate required for results validity) and testing 5 dimensions: mindset, integrity, leadership, governance and capabilities

Quality Culture survey will create a site-level assessment relative to “best practice”

Leadership Capabilities

Integrity Governance

▪ 15 questions around 5 dimensions ▪ Appropriate for shop-floor ▪ Calibrated with McKinsey’s “Quality

Maturity Model” already used in practice

Detailed pilot objectives and roles

Objectives Parties to involve/ Proposed approach

Refine set of metrics, definition, data submission, process and evaluation. Test mix of lagging and leading indicators at site and product-level

Detail and document findings and path forward to operationalize standard metric reporting and use of metrics in risk management

Pilot with a sample of companies and sites identified by ISPE individual members; refine methodologies over 2 periods of data collection by assessing predictive power

Identify applicability and methodologies to maximize benefits for all the parties involved and create new insights

▪ FDA: to observe and comment

▪ ISPE PQLI metrics team in collaboration with McKinsey and with input from other industry members, associations and FDA public comments: to design the program, its applicability for the industry and to oversee confidential evaluation of company data1

▪ ISPE/McKinsey pilot team to establish 2 waves of data collection with as broad industry participation as possible: Wave 1 to conclude data collection by 9/15 for release of first report by end 2014. Wave 2 to take place in 2015

▪ McKinsey & Company to run the overall data collection and analysis in a separate ISPE database, ensuring consistency and confidentiality among the different parties involved and to leverage insights from its POBOS Quality database (9 years, 140+ sites, 100+ metrics)

1 Summary-level pilot data and analysis will be presented in aggregate to the pilot team and will be reported by technology (e.g., solid vs. sterile). Individual companies in the pilot will also be

able to see their individual site data relative to the total industry sample. No individual company data will be shared with other industry members or with the FDA. Aggregate data will be

blinded and shared.

SOURCE: ISPE Proposals for FDA Quality Metrics Program - Whitepaper

McKinsey & Company will dedicate a “clean team” and create a special, confidential database to support this pilot

McKinsey will manage data collection, cleaning, analysis and confidentiality with a dedicated “clean team”. This analytical team supports only benchmarking services—no other client service. The team contains currently 6 people and is based in McKinsey’s office in Wroclaw, Poland

▪ All collected data are blinded and treated highly confidentially by the team, nobody else has access to the POBOS databases

▪ The team performs all analyses, prepares and debriefs the client reports

▪ The team also performs industry-wide sophisticated data mining and analyses to derive insights from the data that are relevant for all participants

▪ McKinsey subject matter experts in collaboration with PQLI team will guide the team analysis, but not have access to the collected data except in aggregate form

▪ McKinsey will charge fees to cover its analytic work only (investing planning and leadership time): $30,000 for first site of companies with >$1 billion annual revenue, $15,000 each additional site. For limited number of small companies (<$1 billion annual revenue), first site costs subsidized at $5000.

We are actively recruiting companies to participate

Develop pilot scope and McK agreement

April

Finalize metrics and timelines

June 2nd

ISPE-FDA CGMP

Conference Workshop to explain

and launch pilot

approach

May-June

Pilot: Wave 1

June-Dec.

Wave 2

2015

▪ Kick off effort, set-up ISPE and McKinsey project teams and steering committee

▪ Hold ISPE-McKinsey workshop (working teams and steering committee) to review and refine proposed list of metrics, agree on process and recruitment approach

▪ Start participant recruitment : 25 companies have expressed interest to-date

▪ Participate in Brookings meeting to discuss FDA direction

▪ Finalize metrics and definitions

▪ Prepare data collection templates and surveys

▪ Outline analysis methodology

▪ Announce pilot details ▪ Confirm pilot

participants ▪ Launch data collection ▪ Prepare pilot overview

for ISPE-FDA conference

Formal check-in with FDA

▪ By 9/15: Complete data collection (for Wave 1)

▪ Review progress in ISPE-McKinsey workshop

▪ By 9/30: clean/validate data ▪ By 10/30: Report out findings and

correlations ▪ Workshop with ISPE and FDA on

Wave 1 results ▪ Individual report-outs to all pilot

participants in November and December

▪ Issue ISPE report on Wave 1 findings and implications for FDA metrics program

▪ Expand participants in Wave 2

Pilot team’s status

▪ Final Wave 1 pilot design still being finalized—input welcome!

▪ Additional metrics can be included in Wave 2 based on Wave 1 learnings or additional FDA/industry input

▪ ISPE working teams have been developing detailed definitions

▪ McKinsey & Company is translating definitions into data collection templates and will support pilot participants in collecting and checking data

Next Steps

If you would like your company to participate in the pilot, please contact PQLI@ispe.org, or ISPE_pilot@mckinsey.com

Questions?

Thank You

Frances M. Zipp President

Lachman Consultants Westbury, NY 11590

516-222-6222 Email: f.zipp@LachmanConsultants.com

www.LachmanConsultants.com

17