test automation in business and enterprise digital strategies

Automation in Digital Strategy

19/05/2016 www.bilot.fi

Worksoft customer conference

Helsinki 18th of May 2016

Krista Männistö, Bilot

Ari Varjonen, Qalmari

19/05/2016 www.bilot.fi 1

BILOT PARTNERSHIPS


PREFERRED SOLMAN

DELIVERY AND

DEVELOPMENT PARTNER

CERTIFIED RESELLER,

DEPLOYMENT AND

DEVELOPMENT PARTNER

NORDICS

Agenda


• Automation in Digital Strategy

• Test Automation for Business

• Role of SAP Solution Manager in Test Automation

• Metso business case

Automation in Digital Strategy


What, How and When to Automate?

• WHAT: • Separate the code testing and the E2E process testing. For process

tests use process knowledge first and automation knowledge next. Letthe process owners design what needs to be tested.

• HOW:• Find a correct set of tools to Automate. Make the Manual testing work

together with Automation because full-automation is not achievable.

• WHEN:• Do you have any Standards to follow, are you already doing structured

testing, what is your current maturity of testing? Process understanding from the perspective of change and E2E testing?


What to ask more?• What is our Automation plan and what is the framework behind it?

Who has the ownership of these?

• Do we run our test scenarios in hybrid landscape?

• Cost-savings from time and resources are expected ROIs, but what else is there?

• How much should we focus on re-usability when designing our Test cases?

• Is the E2E scenarios our main goal when automating?

• Are we doing testing continuously, do we need lights-out testing?

• How Agile our development wants to be?

• Does our Change and Test Management work hand-in-hand?


From IT-oriented way of working…


Business IT

Business Requirement

Design – process owners, project/change managers, developers

Build – process owners, project/change managers, developers

Test – key/end users (testers), test managers

Deploy – project/change managers, technical team

Approval

Maintenance – process, owners, key users, support team

…to more collaborative way of work


Business IT

Business Requirement

Design – process owners, project/change managers, developers

Build – process owners, project/change managers, developers

Test – key/end users (testers), test managers

Deploy – project/change managers, technical team

Approval

Maintenance – process, owners, key users, support team

Test Automation for Business


Business Benefits of Test Automation

Testing costs and resources

Downtime of test clients

Development freeze during updates and main releases

Testing cycles

Time-to-market

Technology risk

Amount of defects when systems change

19/05/2016 www.bilot.fi 10

Reliability of testing

Quality of testing

Operational efficiency

Time for other productive work

Understanding of real end-to-end business processes

Documentation of test cases

Documentation of E2E business processes

Impacts of Test Automation to Release schedules

19/05/2016 www.bilot.fi 11

Scope & Design

Build User Acceptance Testing Regression testing

Final fixes and Technical Go-

Live

Scope & Design

BuildUser Acceptance

TestingRegression

testing

Final fixes and

Technical Go-

Live

Manual testing

Test automation scenario 1

Test automation scenario 2

Scope & Design

Build

Intensive support period

Intensive support

User Acceptance Testing

Regression testing

Final fixes and

Technical Go-

Live

Intensive support

Thrive with Reusable Building blocks

19/05/2016 www.bilot.fi 12

E2E process 3

E2E process 2

E2E process 1

Transactional process step 1

...




...




...



Data variables

Data layouts and

recordsets

Benefits beyond process testingThere is a lot more that you can do with Worksoft Certify besides regular testing activities. Tool could be used in addition of automated unit, integration or regression testing also for:

• Master data maintenance and fixing• Mass uploads either to development, test or production clients • Actual system configuration steps• Process documentation and modelling

• Documenting test cases for individual change requests for business users

• Training purposes

• System recovery testing after unplanned service breaks• Process performance testing• Testing of outputs

19/05/2016 www.bilot.fi 13

Role of SAP Solution

Manager in Test

Automation

19/05/2016 www.bilot.fi 14

Full SAP Change Cycle in Control

19/05/2016 www.bilot.fi 15

Hybrid Landscape

Project/Release/

Maintenance Blueprint

Manual/

Auto Test

Case

Req for Change BPCA

RBPD

Manual &

Automated

Test Cycles

Test

OK

Automated/

Manual

Deployment

STMS,CTS+,cCTS…

Daily, weekly, monthly

Business Requirements

Change Doc

Transport

Request(s)

TBOM

Worksoft

Analyze

Worksoft

Certify

Worksoft

Certify

SAP

UPL

Test

Scope

DEV Build

ChangeOK

Automated Change Management with SAP SolMan & Worksoft Certify Integration

19/05/2016 www.bilot.fi 16

SolMan

Test Mgt

Worksoft

Certify

System

Landscape

Initiate

testing from

test plan

Launch and

run test script

Perform

actual

process steps

Return test

results

Return test

logs

Scope and

create test

plan

Test

management

reporting

SolMan

Change

Mgt

SAP

Solution

Manager

Build change

Validate

Business

Requirement

Approve &

deploy

change to

PRD

SAP SolMan – Worksoft Certify

19/05/2016 www.bilot.fi 17

SAP Solution Manager Worksoft Certify

Solution documentation

Change management

Change analysis

Test scope

Test plan

Test package

Tester assignments

Test configuration

Test data container

Test scheduling

Test defect management

Manual testing

Test reporting

Test scenario generation

Test execution

Test execution logs

Test execution report

Test scenario error correction

Metso Business case

19/05/2016 www.bilot.fi 18

Metso as a corporation –Products and usage of SAP

Metso is the world’s leading industrial company in the mining and aggregates industries and in the flow control business.

- Employs around 14,000 professionals in 50 countries.

- SAP is used as ERP currently in 27 different countries, mainly in Mining and Construction sites.

- Metso has monthly SAP releases and uses Worksoft Certify test automation via SAP Solution Manager for example for monthly regression testing.

- SAP was upgraded to EhP7 05/2015 and just transferred to SAP HANA Enterprise Cloud (HEC).

19/05/2016 www.bilot.fi 19

Test automation project at Metso –From beginning to present

• Test automation project was launched at Metso in February 2014. Target of the project was to automate 80-90% of manual SAP related testing during year 2014. This target was achieved!

• At the moment Metso has documented and automated 135 end-to-end processes and are running automated tests via SAP Solution Manager with lights-out scheduling.

19/05/2016 www.bilot.fi 20

02/14 06/14 08/14 01/15 05/16

Project

Kickoff

First

workshops

on-site

SAP Solman

integration & 45 E2E

automated test cases

135 E2E scenarios created & test

automation was also used for example in

HEC project testing

Automation level of ~90% with 2 virtual

machines and test automation used in

monthly regression testing

Current level of test automation –Amount of test scripts

• Currently Metso has automated 135 E2E test scripts. Also some utilities, transactional unit test scripts and mass upload/data creations have been created in Worksoft Certify side.

These E2E scenarios include:• 570 Transactional processes

• 8600 Process steps

• Test automation used to test processes from:• All used SAP modules (like MM, SD, WM, FI, CO, PP etc...)

• ISA/IPC portals

• WM portal activities

19/05/2016 www.bilot.fi 21

Achieved benefits of test automation –Key learnings of the project

“Key learning of the project has been to understand how important testing truly is in order to deliver changes to production environment in a safely manner. We have also only now came to realize how unreliable and unrepeatable manual testing results have earlier been. This has caused a huge risk in our way of operating with change management and also delayed the end to end lead time of change requests.”

19/05/2016 www.bilot.fi 22

Achieved benefits of test automation –Key learnings of the project

• Test automation has made internal processes more flexible and improved productivity by releasing time and resources for other more productive work than manual testing. Testing could be performed more frequently and whenever needed.

• Cost savings have been achieved from hours saved in testing and quality of testing.

Impact on release schedules as regression testing can be performed now in only 2 days instead of 6 days before including ~300 manual testing hours by offshore team. Testing can be also performed now mainly in night time.

19/05/2016 www.bilot.fi 23

Achieved benefits of test automation –Example how test automation was used in HANA project

• E2E automated test scenarios were run during SAP HANA Enterprise Cloud (HEC) project test phases between March – May 2016

• Sandbox testing phases• 150 test scenarios after sandbox installation• 130 regression test scenarios again after compliance changes were transported to

sandbox

• Quality system testing phases• 100 chosen test scenarios run in quality system before releasing system to business

UVT• In addition 130 regression test scenarios in quality system before actual go-live

• Test automation found for example non working integrations that Metso was then able to fix before business UVT phase

• Automated testing took all together under 60 hours compared to possible thousands of hours of pure manual testing

19/05/2016 www.bilot.fi 24

19/05/2016 www.bilot.fi 25

test automation in business and enterprise digital strategies

Software