An update on what happened…

Kevin Kane, PHASTARkevink@phastar.co.uk

FDA / PhUSE CSS Initiative

• FDA sponsored collaborative meeting with PhUSE and CDISC partners

• "Update on Standards, Tools, Process Initiatives across Regulatory Review and Collaboration with Key Working Groups to Improve the Product Lifecycle"

• FDA, industry, academia– “…to establish collaborative working groups”– “…current challenges related to access and review of data”– “…pursue possible solutions and practical

implementations“

PhUSE and CDISC

• PhUSE– Independent , not-for-profit organisation– Over 1600 volunteers– platform for creating and sharing tools and standards

around data, statistical and reporting technologies• CDISC– global non-profit charitable organisation– 300 supporting member organisations – develop industry-wide data standards – harmonisation of clinical data and streamlining research

processes from protocol though analysis and reporting

Six Working Groups

• Data Validation and Quality Assessment• Standardising Data within the Inspection Site Selection

Process• Challenges of Integrating and Converting Data across

Studies• Standards Implementation Issues with the CDISC Data

Models• Development of Standard Scripts for Analysis and

Programming• Non-Clinical Road-map and Impacts on Implementation

Development of Standard Scripts for Analysis and Programming

• Support the development of validated scripts for common data transformations and analyses for use in clinical research

• Review environment to facilitate efficient efficacy and safety reviews

• Develop technical requirements for a sharing platform

• Share approaches for analysis of safety profiles.

Objectives

• Develop a platform for sharing scripts– SAS, R, Stata etc– Leverage CDISC standards– Encourage ongoing improvement

• Develop metadata standards• Identify core TFLs for safety signal detection• Analyses, summaries and data transformations• Process for managing, testing, publishing scripts• Define validation standards

Three subgroups

• What scripts do we want/need? • Creation and validation of scripts • Platform for script management

Creation and validation of scripts

• Define the process of creating a script• Define the process of validation• What “management” do we need?• How can we incentivise?

Script Creation

• Once a script is loaded, the original author is stored as metadata but does not have any further rights or responsibilities

• Basic set of programming standards would be useful. If they are too detailed, may conflict with individual organisations

• We should develop standard templates for specifications and user guides etc

• Investigate “V Model” further for development process• For minor changes, this should not be a separate script –

should be added as an option• Encourage backward compatibility but not an absolute

requirements

Metadata

• Program name• Language• Program version (auto?)• Platform• Purpose• SDTM/ADaM version/NA

(dropbox)• Keywords• Original Author (auto)• Usage counts• Ratings/feedback• Validation status

• Assumptions• Inputs• Outputs• Requirements• Comments/notes• Reason for change• Bug flag (DB table?)• Current author• Language version • Validation documentation

Definition of validated script

• Script does what it says in specification• Specifications are required

– Design– Inputs– Outputs

• Test under various scenarios: these scenarios become assumptions

• Code review• Validation by experience is not enough• Website/wiki needs a disclaimer• ISSUE: What documentation is required for unvalidated scripts

Process for scripts to be validated

• Upload all validation documentation• Approval by moderator (committee?)• Meets all requirements on validation checklist

Can we learn from SAS Online help web pages?

Script governance - functions

• Approve scripts• Draft specs• Call for Scripts• Template specs• Guidelines• Validation checklist• Library management• Ratings management• Define metadata• Change management• Incentive management

Script governance – documentation required

• Guidelines for creating specs• Define metadata– Overlap between specs and metadata

• Web based database?• Template for user guide• Basic programming standards• Checklist for approval to validated state• Definition of requirements to consider a script

validated

Issues to pass to platform group

• Need to be able to review and comment on scripts. Ideally with quality rating

• Create and store multiple versions• Need scripts to be able to have different states: e.g. validated;

unvalidated; in development• Metadata e.g. program name; language; parameters; bug flag;

variables; outcomes; version number (need to decide list of metadata variables)

• Check-in check-out (not 100% defined- what happens if one person checks out for long time)

• Ability to have multi-person multi-function teams• Can we have a metadata database on a Wiki

Incentives:Results from brainstorm

• Maybe we don’t need any incentive• Encourage people to get a top rating leading to enhanced reputation• Platform records downloads – “most cited script”• Messages to “market”:-

– Reputation factor– This system can save organisations money– This is the same code that the FDA will use

• Could offer a PhUSE discount or award• FDA recommendation to use scripts• Airmiles/points system – bronze/silver/gold• Academic encouragement : get your methodology adopted• Confirm if we need any money. Ask PHARMA???

My key learnings

• FDA statisticians are pretty normal• They’re happy with analyses in R• New FDA regulations:-– Draft Dec 2012; Final 12 months later– Electronic submissions required 24 mnths later

• SDTM isn’t standard enough

What happens next?

• Depends on volunteers• Get involved?– phusewiki.org– Join one of the subgroups (I can pass on details)– Slides from the meeting are on phuse.eu

• Come to the PhUSE conference• Encourage your organisation to get involved

with CDISC