extent-2016: realisation of a collaborative approach to test automation

Duncan Brigginshaw

Co-Founder and

Technical DirectorOdin Technology Ltd

Realisation of a

Collaborative

Approach to Test

Automation

Realisation of a Collaborative Approach to Test

Automation

www.odintech.com

A case study in post-trade

settlement and reconciliation at Clearstream

Duncan Brigginshaw

Co-Founder and Technical Director, Odin

www.odintech.com

Agenda

• Introduction

• Background and Business drivers

• Defining a collaborative approach to functional test

automation

• ClearstreamXact Portal and 1CSR Project

• Lessons and Conclusions

Duncan Brigginshaw

Technical Director - Odin

www.odintech.com

Introduction

• Deutsche Börse AG

• Diversified exchange organisation with wide range of products and services

– Listing and Trading, Clearing, Post-Trading, Market-Data and Services

• International Central Securities depository (ICSD) -Luxembourg

• Post-Trade Infrastructure and Securities service provider

• Central Securities Depository (CSD) – Frankfurt

• QA Software Vendor

• Providers of the Axe Enterprise Test Automation Platform

www.odintech.com

Background and Business drivers

Duncan Brigginshaw

Technical Director - Odin

• Functional GUI Automation key to achieving business goals

• History:

• Isolated GUI Automation in Business Units– Primarily HP Products

– Technical in Nature

• Centralised Automation framework developed in-house– Expensive to build and maintain

– In-Flexible – Proprietary to HP

– Not-Business Oriented

• GUI Automation rates not inline with Business goals

A collaborative test automation approach

www.odintech.com

www.odintech.com

Stages

Duncan Brigginshaw

Technical Director - Odin

• High Level Vision – Define a Collaborative Automation Methodology

• “Involve Functional Analysts/Testers in Automation”

• Requirements– Script-less business focussed automation architecture

– Execution Tool Independent (HP + Selenium)

– Future Proof

– Compatible with current HP ALM Infrastructure

• Process– Initial Research and RFP

– Matrix Scoring on Requirements

– Shortlisted Vendor Proof of Concept

– Vendor Selection

– Rollout Methodology Defined with Vendor Support

– Initial project identified to prove and refine the approach

www.odintech.com

Teams and Geography

BusinessUnits

Technical Test and

Tools

ToolVendors

FunctionalTesters

(Offshore)

ApplicationDevelopment

(Offshore)

FunctionalTesters

(Nearshore)

www.odintech.com

• Functional Acceptance Criteria (FAC)• Test Cases

• Test Design

• Technical Tool Support

• Supporting Tool Infrastructure

Skill sets and Asset Focus

www.odintech.com

Clear Matrix of Roles and Responsibilities

www.odintech.com

HP ALM Central to Test Asset Mgmt.

HP ALM

BusinessUnits

Technical Test and

Tools

FunctionalTesters

(Offshore)

FunctionalTesters

(Nearshore)

BusinessUnits

www.odintech.com

Asset Creation

• Functional Acceptance Criteria (FAC) + Test Case / Condition Design

Excel Template

• Import Creates linked Requirements and Tests in HP ALM automatically

HP ALM

Import

Technical Test and

Tools

www.odintech.com

Asset Creation

Technical Test Assets Creation

Axe Automation Platform

• Project Setup• Supporting Custom Code• Added to ALM Test Resources

HP ALM HP UFT

www.odintech.com

Asset Creation

Functional Test Design / Definition

FunctionalTesters

(Offshore)

FunctionalTesters

(Nearshore)

Axe Automation Platform

• Test Design Steps and Automation Code generated automatically

HP ALM

www.odintech.com

Test Execution

• Automated Tests Scheduled and Executed

HP ALM

• Results and Acceptance Criteria Coverage Captured

HP UFT

The Initial Project:

ClearstreamXact

Web Portal

www.odintech.com

www.odintech.com

ClearstreamXact WebPortal

Duncan Brigginshaw

Technical Director - Odin

• ClearstreamXact– Framework for access to Clearstream

ICSD and CSD Services

– Settlement, Cash Management, Asset Servicing, Collateral Management, Securities Lending, Fund Services

• Xact Automated access

• Xact Web Portal (GUI)

• Clearstream Settlement and Reporting– TARGET2 and Global Securities

– Instructions, Positions, Cash and Securities

www.odintech.com

Project Snapshot

Total

75%

Manual

Automation

Functional Acceptance Criteria

Automated

NotAutomated

64.4% of Target

(45% of Total)

U

Completed

35%

NotCompleted

ForecastTarget

www.odintech.com

Project Timeline & Progress

Completed

NotAutomated

Automated64.4%

Month 1 Month 2 Month 3 Month 4 Month 5

www.odintech.com

Duncan Brigginshaw

Technical Director - Odin

• Application Build and Delivery– Daily Updates

– Weekly Delivery

• Testing– Subset of Tests Run Daily

– Full Regression Pack of Tests run Weekly

– 12hr Execution Across 12 Machines

Application Build and Test Regime

www.odintech.com

Lessons and Conclusions

Duncan Brigginshaw

Technical Director - Odin

• Lessons Learned - Refinements:– Requirement for automated test failure triage process

– Defect management for automation (separate to development)

– Asset naming conventions needed early in the project

– New functionality found more challenging for automation than regression

• Collaboration was key in automation success

• Important to define clear roles and responsibilities

• Leveraging relevant skills and expertise in the appropriate Mix

• Collaborative methodology and selected tooling is proving successful over manual testing and previous automation

Thank-you!

odintech.com

info@odintech.com