T16  Mobile  Testing  5/5/16  13:30  

Stay  Ahead  of  the  Mobile  and  Web  Testing  Maturity  Curve  

Presented  by:  

Danny    McKeown  

Paychex  

Brought  to  you  by:    

350  Corporate  Way,  Suite  400,  Orange  Park,  FL  32073    888-­‐-­‐-­‐268-­‐-­‐-­‐8770  ·∙·∙  904-­‐-­‐-­‐278-­‐-­‐-­‐0524  -­‐  info@techwell.com  -­‐  http://www.stareast.techwell.com/  

Danny    McKeown  Paychex  

Danny  McKeown  has  more  than  thirty  years  of  technical  and  management  experience  in  information  technology.  With  Paychex  for  twelve  years,  Danny  has  spent  the  last  six  years  as  the  test  automation  architect.  In  this  role,  he  is  instrumental  in  implementing  a  secure  integration  of  Perfecto  Mobile's  solution  with  the  Paychex  automation  framework  that  leverages  HP  UFT  and  Selenium  technologies.  In  addition,  Danny  is  an  adjunct  lecturer  in  the  software  engineering  department  at  the  Rochester  Institute  of  Technology.    

Unit

API

UI Unit

API

UI

T16 Danny McKeown

Staying Ahead of the Mobile and Web Testing Maturity Curve

Unit

API

UI

About the Speaker

Danny McKeown has more than 30 years of technical

and management experience in information

technology. With Paychex for 12 years , Danny has

spent the last six years as the test automation

architect. In this role, he is instrumental in

implementing a secure integration of Perfecto

Mobile’s solution with the Paychex automation

framework that leverages HP UFT and Selenium

technologies . In addition, Danny is an adjunct

lecturer in the software engineering department at

the Rochester Institute of Technology.

Unit

API

UI

Unit

API

UI

About Paychex

Leader in payroll, human resource, and

benefits outsourcing solutions

Small- to medium-sized businesses

Over half million clients

24 x 7, 365 days per year availability

Mobile application – Paychex Flex

Unit

API

UI

Unit

API

UI

Today’s Focus Automation Strategy

Environment Readiness Check

Continuous Delivery

Automation Adoption

Impact of Agile Process

Impact of Responsive Technology

Our Mobile Timeline

What’s next…

Unit

API

UI

Unit

API

UI

Change does not come easy

Unit

API

UI

Unit

API

UI

Why Test Automation? Continuous improvement of:

Raising the bar on Quality

“Degree of Excellence”

Driving Efficiency

Time to Market

Delighting the Customers

Unit

API

UI

Unit

API

UI

Automation Strategy

Does your organization have a

test automation strategy?

Paychex adopted the

Test Automation Pyramid

Introduced by Mike Cohn

Built an architecture around it

Unit

API

UI

Unit

API

UI

Test Automation Pyramid

Page #8

Unit Tests

API Tests Headless, Non-UI, Service,

Database, & Business Logic Testing

UI Tests

Supports Agile and Continuous Delivery

Unit

API

UI

Unit

API

UI

Test Automation Software Stack Encapsulation, Scalability, Reuse

Page #9

Operating System Windows, IE, Chrome, Firefox, Edge

API Test/Job Execution, Reports

IT Tool Integration

Resource Objects, Scenarios, Scenario Drivers, Tests

Repository

Engine Database, Services, Web (10 Engines)

Core Code

Language/IDE & 3rd Party Plugins HTML 5, C#/.Net, Visual Studio, Perfecto, Applitools, Selenium, Ranorex

Vendor Integration

Persistent Storage SQL, Server, Entity, Framework, NAS

SOX

Application Suite

User

Unit

API

UI

Unit

API

UI

Test Environments Contribute to false negative test results?

99.9% availability?

99% availability?

90% availability?

What is the availability?

Configured correctly?

Patches up to date?

Monitored?

Change Control?

Unit

API

UI

Unit

API

UI

Apply Pre-Condition Testing

Input + Pre-Conditions = Post Conditions + Output Execute readiness check at each pyramid tier

Unit

API

UI

Unit

API

UI

Executing Automated Scripts Execute scripts often to get automation value!

On Demand Scheduled

Any time, for anyone, and any IT tool?

Paychex implemented an Automation Execution Center (AEC) that supports Continuous Delivery (CD)

Unit

API

UI

Unit

API

UI

Automation Execution Center

Page #13

TRANSPOSE JOB

Exposed API Server

Exposed Web Server LDAP

Tool Independence

QUEUE JOB STORE RESULTS

ARCHIVE

Automation DB

Tool Agnostic Test Job

Secure Controlled Physical Access Account

TARGET ENVIRONMENT

Non-Prod & Prod Environments

Cloud Computing

TEST JOB SUBMISSION

No Test Automation

Tools

Your IT platform

Continuous Delivery Integrated, Scalable, Parallel Execution

EXECUTE JOB Tool Dependent

Parallel Processing Load Balancing

VM Farm (also direct log in)

Tools installed

Unit

API

UI

Execute mobile “build/deploy/test” 3+ cycles per day

Release

Web Service & other Non-UI Regression

Majority of Business Logic is within the Web Services

Majority of User Experience is within the Mobile UI

Perfecto Cloud Connectivity/Latest App Version Copied

Sanity

Regression

Jenkins – Orchestrates Build, Deploy, & Test

Unit

API

UI

Unit

API

UI

Automation Adoption At Paychex many implement automation:

Development – Create unit tests

Test Automation – Develop framework & automation apps

Build Automation – Develop build, deploy, & test pipeline

Testers - Utilize automation apps to create & execute test scripts

Automation is a cultural shift…needed some definitions

(and goals) to assist in the transformation

Unit

API

UI

Unit

API

UI

Test Automation Belts Covers from Concepts to Non-UI & UI understanding

to Script Debugging

Unit

API

UI

Unit

API

UI

Agile impact on automation

Page #17

CORE TEAM

Developers Testers

Business Analysts

Scrum Master

Dev Solution Lead Product Owner

Automation Solution Lead assigned to each team

Definition of “Done” expanded to include automation, i.e., more automation

Developers & Testers both implement functional auto scripts

Developers & Testers naturally collaborating more

Unit

API

UI

Unit

API

UI

Responsive Technology Accelerated the need for mobile

web testing: Chrome, Firefox, IE, Edge (future) Safari Chrome (mobile)

Accelerated the need for CD: Create a sustainable build, deployment

and test pipeline Promotion process from early to late

environments (Test to Prod)

Test Automation went from a “want” to a “need”!

Unit

API

UI

Unit

API

UI

Mobile Timeline

Page #19

Automation Solution Manual Testing

2012 2013

Automated Testing

2014

Continuous Delivery

2015

Agile

2016

Cross Browser

Unit

API

UI

Unit

API

UI

What’s next…

Testing desktop browsers in the cloud

Profiling our test users

Advance cross browser testing

Seamless testing from test to prod with same suite of scripts

Create tools to create test data for spun-up environments (CD)

Unit

API

UI

Unit

API

UI

We are all trained but…

Unit

API

UI

Thank You