extent-2016: realisation of a collaborative approach to test automation
TRANSCRIPT
Duncan Brigginshaw
Co-Founder and
Technical DirectorOdin Technology Ltd
Realisation of a
Collaborative
Approach to Test
Automation
Realisation of a Collaborative Approach to Test
Automation
www.odintech.com
A case study in post-trade
settlement and reconciliation at Clearstream
Duncan Brigginshaw
Co-Founder and Technical Director, Odin
www.odintech.com
Agenda
• Introduction
• Background and Business drivers
• Defining a collaborative approach to functional test
automation
• ClearstreamXact Portal and 1CSR Project
• Lessons and Conclusions
Duncan Brigginshaw
Technical Director - Odin
www.odintech.com
Introduction
• Deutsche Börse AG
• Diversified exchange organisation with wide range of products and services
– Listing and Trading, Clearing, Post-Trading, Market-Data and Services
• International Central Securities depository (ICSD) -Luxembourg
• Post-Trade Infrastructure and Securities service provider
• Central Securities Depository (CSD) – Frankfurt
• QA Software Vendor
• Providers of the Axe Enterprise Test Automation Platform
www.odintech.com
Background and Business drivers
Duncan Brigginshaw
Technical Director - Odin
• Functional GUI Automation key to achieving business goals
• History:
• Isolated GUI Automation in Business Units– Primarily HP Products
– Technical in Nature
• Centralised Automation framework developed in-house– Expensive to build and maintain
– In-Flexible – Proprietary to HP
– Not-Business Oriented
• GUI Automation rates not inline with Business goals
A collaborative test automation approach
www.odintech.com
www.odintech.com
Stages
Duncan Brigginshaw
Technical Director - Odin
• High Level Vision – Define a Collaborative Automation Methodology
• “Involve Functional Analysts/Testers in Automation”
• Requirements– Script-less business focussed automation architecture
– Execution Tool Independent (HP + Selenium)
– Future Proof
– Compatible with current HP ALM Infrastructure
• Process– Initial Research and RFP
– Matrix Scoring on Requirements
– Shortlisted Vendor Proof of Concept
– Vendor Selection
– Rollout Methodology Defined with Vendor Support
– Initial project identified to prove and refine the approach
www.odintech.com
Teams and Geography
BusinessUnits
Technical Test and
Tools
ToolVendors
FunctionalTesters
(Offshore)
ApplicationDevelopment
(Offshore)
FunctionalTesters
(Nearshore)
www.odintech.com
• Functional Acceptance Criteria (FAC)• Test Cases
• Test Design
• Technical Tool Support
• Supporting Tool Infrastructure
Skill sets and Asset Focus
www.odintech.com
Clear Matrix of Roles and Responsibilities
www.odintech.com
HP ALM Central to Test Asset Mgmt.
HP ALM
BusinessUnits
Technical Test and
Tools
FunctionalTesters
(Offshore)
FunctionalTesters
(Nearshore)
BusinessUnits
www.odintech.com
Asset Creation
• Functional Acceptance Criteria (FAC) + Test Case / Condition Design
Excel Template
• Import Creates linked Requirements and Tests in HP ALM automatically
HP ALM
Import
Technical Test and
Tools
www.odintech.com
Asset Creation
Technical Test Assets Creation
Axe Automation Platform
• Project Setup• Supporting Custom Code• Added to ALM Test Resources
HP ALM HP UFT
www.odintech.com
Asset Creation
Functional Test Design / Definition
FunctionalTesters
(Offshore)
FunctionalTesters
(Nearshore)
Axe Automation Platform
• Test Design Steps and Automation Code generated automatically
HP ALM
www.odintech.com
Test Execution
• Automated Tests Scheduled and Executed
HP ALM
• Results and Acceptance Criteria Coverage Captured
HP UFT
The Initial Project:
ClearstreamXact
Web Portal
www.odintech.com
www.odintech.com
ClearstreamXact WebPortal
Duncan Brigginshaw
Technical Director - Odin
• ClearstreamXact– Framework for access to Clearstream
ICSD and CSD Services
– Settlement, Cash Management, Asset Servicing, Collateral Management, Securities Lending, Fund Services
• Xact Automated access
• Xact Web Portal (GUI)
• Clearstream Settlement and Reporting– TARGET2 and Global Securities
– Instructions, Positions, Cash and Securities
www.odintech.com
Project Snapshot
Total
75%
Manual
Automation
Functional Acceptance Criteria
Automated
NotAutomated
64.4% of Target
(45% of Total)
U
Completed
35%
NotCompleted
ForecastTarget
www.odintech.com
Project Timeline & Progress
Completed
NotAutomated
Automated64.4%
Month 1 Month 2 Month 3 Month 4 Month 5
www.odintech.com
Duncan Brigginshaw
Technical Director - Odin
• Application Build and Delivery– Daily Updates
– Weekly Delivery
• Testing– Subset of Tests Run Daily
– Full Regression Pack of Tests run Weekly
– 12hr Execution Across 12 Machines
Application Build and Test Regime
www.odintech.com
Lessons and Conclusions
Duncan Brigginshaw
Technical Director - Odin
• Lessons Learned - Refinements:– Requirement for automated test failure triage process
– Defect management for automation (separate to development)
– Asset naming conventions needed early in the project
– New functionality found more challenging for automation than regression
• Collaboration was key in automation success
• Important to define clear roles and responsibilities
• Leveraging relevant skills and expertise in the appropriate Mix
• Collaborative methodology and selected tooling is proving successful over manual testing and previous automation
