dmt automation ui-testing practice7

8/8/2019 DMT Automation UI-Testing Practice7

1/25

DMT UI-testing automationDMT UI-testing automationpracticespractices


2/25

DMT problemsDMT problems

Neither manual nor automatedtest cases cover 100% of code

No instrument to measure testcases coverage


3/25

Deeper view on problemDeeper view on problem

System tests are not flexible Testing whole functionality build to

built is inaccessible and

unnecessary Time spent for manual testing of regression tests could be aimedonly on testing new functionality

There is no information what exactlycould be affected by particular codechange

There is no guaranties we test all

required peaces of code


4/25

System TestsSystem Tests

Manual test cases that aimed to coverwhole functionality and itsdependencies

Problem

Coverage: is unknown and scattered overwhole system Maintainability: very low due to linkage

with several requirements Execution: hard to execute because tests

executed in one way and pauses couldcause lost of this test way

Dependency: every step is dependentfrom data used in case before or fromdump data

Automation: not automatable (support of those cases will be unacceptable


5/25

System Test CaseSystem Test Case


6/25

Functional TestsFunctional Tests

Atomic test cases that aimed to verifyparticular functionality

Benefits

Coverage: measurable! Maintainability: extremely high maintainability!

Case covers atomic peace of requirement Execution: easy to execute! Every case could be

executed separately even in different days by

different test engineers Dependency: test cases are independent! Every

case prepares preconditions (application, data) forits run

Automation: extremely easy to automate! Just walk

through cases and convert it to automationframework API language


7/25

Document for automationDocument for automation


8/25

TraceabilityTraceability

Traceability matrix allows linking Issue for new functionality with

specification document Specification document with

functional test case Functional test case with automated

test script Black box coverage assuming bymanual / automated tests


9/25

Traceability matrixTraceability matrix


10/25

ChangeabilityChangeability

New improved test cases designallows be flexible withcontinuously improvedrequirements:

Easily find exact place in testcase/test script that should beupdated

Create new test case/test scriptfor new functionality withoutaffecting existing test cases

Measure not only code coveragebut also requirements coverage


11/25

MaintainabilityMaintainability


12/25

Summary 1Summary 1

Atomic test cases Traceability matrix binds all

artifacts Easy to support test cases Easy to handle test cases execution

results Test cases are ready to be

automated


13/25

How to automate?How to automate?

DMT automated testingframework

WebUI interaction API Page object pattern

Pre-requisites creation API

u to mat ed Te st Ca se

-Pre requisitescreation

Test Case body

Verifications

Activities

DBUnit

WebDriver

TestNG

use

use

use

being ran by

DMT entities creation API

DMT specific,orms widgets

wrappers


14/25

Why DBUnit?Why DBUnit?

Test cases pre-requisitescreation is significant part of either manual or automatedapproach (time consuming)

Solutions XML based DB fixtures

Data that should be already in databaseto pass test case

XML based GUI fixtures Data that should be entered into form to

pass test case


15/25

Why Selenium/WebDriver?Why Selenium/WebDriver?

Open source Java support

Developers can help to automate testcases

Developers can write functional testson defects

Cross platform/browser


16/25

Build process integrationBuild process integration

You need only maven + browserto run test cases

WebDriver supports: IExplorer Chrome Firefox


17/25

PerformancePerformance

Individual test case takes about90 sec.

About 480 test cases per night


18/25

Summary 2Summary 2

Atomic test cases are easy toautomate and maintain

Pre-requisites creation takesmilliseconds

Test cases can be run againstany browser without changes


19/25

How to measure?How to measure?

Problems mentioned in the verybeginning (just to remind): No guaranties that code in production

is covered by test cases No instruments to measure functional

test case coverage over sourcecode


20/25

Rich UI based apps test coverageRich UI based apps test coveragetooltool

Automated test cases (using WebUIinteraction API)

or QA m anually interact w ith application withinCoverage tool through browser

Coverage tool tracks the source code lines affect edby those activities

overagetool

Browser

- -

Auto test cases

Manual TCexecution App DB


21/25

Coverage tool collecting coverageCoverage tool collecting coverage

Step Action

1.1:1.2 User/WebDriver activities

2.1:2.2 Communication with server

3.1 Cobertura tracks the code execution

4. 1:4.2 Dev Plugin controls the code execution onCode Server

5.1 Emma tracks the code execution

6.1 The stop-tracking command request7.1:7.2 Get client side coverage

8.1:8.2 Get Emma runtime coverage

9.1:9.2 Get Cobertura runtime coverage

10.1:10.2 Persist combined coverage for an activity toDB


22/25

Code Coverage by Test CcasesCode Coverage by Test Ccases

Test Cases

toExecute

Not tested code


23/25

Summary 3Summary 3

We can measure the codecovered by test cases being raneither manually or byautomated tool

We can determine the code notcovered by test cases

We can reduce the testing timeby executing the only testcases against affected source

code


24/25


25/25

Issues & problemsIssues & problems

How to build pre-requisites(database?)

Test execution performanceoptimization (initialloging/loading)

Traceability matrix has to beintegrated with RTH or otherdocument tracking system

dmt automation ui-testing practice7

Documents