agiles testen in großprojekten · &. scrum of scrums -+ -+ -+ (to undifferent) -- (individual...

corporate quality consulting | Phone: +49 (0) 2241 25021 0 | Mail: info@corporatequality.com

Kunde

Projekt

Referent

Datum

Karlsruher Entwicklertage

Stephan Salmann

06. Juni 2013

Agiles Testen in Großprojekten

Agenda

Problem

Big Picture of the Solution

Unternehmenspräsentation

Some Detail Views

2

Das Unternehmen

Mission Statement Wir bauen die Brücke zwischen Business und IT

Mitarbeiter

Rolle

Mediator: für ein gemeinsames Verständnis zwischen Business und IT sowie Kunde und Lieferant

Advisor: strategisch und operativ für Analytik und Konzeption

Man of Action: pragmatische und zielorientierte Lösungsumsetzung

0

20

40

60

80

100

120

2005(Gründung)

2007 2009 2011 2013

Hannover

Siegburg

Frankfurt

München

Hamburg

Wien

Zug

3

Fundierte IT-Management Beratung

4

Spezialisten

Freelancer

commodity Leistungen

operativ

strategisch

spezifische Leistungen

Outsourcer

Integratoren

Strategie- berater Wirtschaftsprüfer

corporate quality

Services & Trainings der CQ

IT-Linienmanagement

Enterprise Architecture

Excellence

IT-Portfolio- Management

IT-Produkt- management

IT-Controlling

Geschäftsprozessberatung in ausgewählten Branchen

Programm- & Projekt-

management IT-Architektur

Requirements Management

Test- management

Projektumsetzung im Managed Service Ansatz

Projekt

IT-S

trat

egi

e

Sou

rcin

g &

In

tern

atio

nal

isie

run

g

Go

vern

ance

, Ris

kman

agem

ent

& C

om

plia

nce

Qu

alit

ätsm

anag

emen

t

& P

roze

sse

Trusted Business Communications

Social Media Business Integration

Mobility Solutions Banking Open Enterprise

5

Agenda

Problem

Big Picture of the Solution

Unternehmenspräsentation

Some Detail Views

6

Conway´s Gesetz

Bedeutung für 3 Teams:

1. Drei Teams implizieren 3

Haupt-Software-Systeme

2. Es sind 3 Team-

Kommunikationen notwendig

7

Bedeutung für 17 Teams:

1. Zehn Teams implizieren 10

Haupt-Software-Systeme

2. Es sind 136 Team-

Kommunikationen notwendig

Die Kommunikation zwischen den

Hauptsystemen ist, nur so gut wie die

Kommunikation zwischen den Teams.

(𝑛 − 1)𝑛

2

Options for large scale agile Projects

8

Comuni-

cation

Steering Meeting

structure

Integration of

results

Scale effects

Option 1: one big

Agile Team

-- (many to many)

-- (missing roles

and hierarchy)

-- (to many

participants)

++ ++

Option 2: many

small agile Teams

&. scrum of

scrums

-+ -+ -+ (to undifferent)

-- (individual

procedures and

tools)

-- (in allocation,

architecture,

provisioning)

Option 3: many

small agile teams

with an adequate

program steering

++ ++ ++ (focussed)

++ ++ favorite

approach

How to organize big project with i.e. 30 - 400 team members

Build Problems

Growing complexity of build

Large variety of inconsistent

library versions

Different compiler versions

No staffing flexibility between

agile teams

Functionality

Incomplete

Redundant parts

Integration problems

Fingerpoint between teams

Missing technical/functional fit

No scale effects

technical/functional redundancies,

Tools cost

low staffing flexibility

Large Toolsets

High Maintenance Effort

Our Experience in large scale agile projects using scrum of scrums

9

Defined by individual agile Team

Overall incomplete or redundant

Defined by individual agile Team

High Maintenance costs by the variety of tools

No scale effects

Problems with interoperability (cross functions: PM, SCCM, QM, …)

Software of different teams can not be integrated with a proper

technical or functional fit

The integration is not a goal of an agile team

The integration leads to a fingerpointing between the agile teams

More and more desorientation over time

Specialists for tasks are bounded in Teams

Extensive costs for learning curves when staff is reallocated in

other agile teams

The split in teams serves for a missing view on the big picture so

that technical cross functions are neglected

The steering on the program level is indirect via different roles in

different agile Teams

Focus

of agile

Team

Processes

Tools

Integration

problems

Staffing

flexibility

Functional

Silos versus

Horizontal

functions

Big Problems

Fuzzy Evolution of Functionality in parallel Scrum Teams

cummulative

Functionalities

1. Agile Team

2. Agile Team

3. Agile Team

Overlapping Development

Characteristics:

Redundancies

Incompleteness

Every team chooses

Own Architecture

Own Design

Own Implementation

Own Test

Own Success

……….

Incompleteness !

10

Evolution of oriented Scrum Team Developement (adequate steering)

1. Iteration 2. Iteration 3. Iteration

Cummulative

Functionalities

1. Mandate

2. Mandate

3. Mandate

Characteristics:

No redundancies

Completeness

Every team has

One Major Architecture

Disjoint task set

Integrated Tests

One Success

measurement

1. Agile Team

2. Agile Team

3. Agile Team

11

Agenda

Problem

Big Picture of the Solution

Unternehmenspräsentation

Some Detail Views

12

Sequence of Tests

13

Program-

management

Agile

Team 1

Business

Archi-

tecture

Technical

Archi-

tecture

Programm Governance Advisory

Config.

& Release

Manage-

ment

Integration

Test

Quality

Manage-

ment

Integration Team

Iteration 1

Agile

Team 2 Iteration 1

Agile

Team 3 Iteration 1

Inte

gra

tio

n T

est

Iteration 2

Iteration 2

Iteration 2 In

teg

rati

on

Test

RF

T

RF

T

RF

T

Iteration 3

Iteration 3

Iteration 3

Inte

gra

tio

n T

est

RF

T

RF

T

RF

T

Man

date

M

an

date

M

an

date

copyright by CQ

Ma

cro

La

ye

r

Pro

gra

m/D

om

ian

Mic

ro L

aye

r

Pro

ject/A

pp

lica

tion

ST

Daily Build

RST

Daily Build

RST

Daily Build

RI RI

Accep

tan

ce T

est

ST ST DT

FT

DT

DT

DT

DT FT

FT

FT

FT

DT FT

DT FT

DT FT

DT FT

DT-Developer Test // FT- Functional Test // ST – Smoke Tests // IT – Integration Test // R<X> - Regressiontest

Operating Phase

Delegate

Ressources

Define

Technical

Domain

Architect.

Define

Business

Achtitect.

Setup the Organization

14

Define Major

Business

Capabilities Program Governance Advisory

Integration Team

AgileTeams

Agile Team 1

Agile Team 2

Agile Team 3

Ma

cro

La

ye

r

Pro

gra

m/D

om

ian

Mic

ro L

aye

r

Pro

ject/A

pp

lica

tion

M

M

M In

teg

rati

on

Test

Setup Phase

Man

date

s:

iterative

Path of the agile Solution Development for one Agile Team

Start

Mandates ~

Cones of Freedom

1. Iteration 2. Iteration 3. Iteration

Fu

nctio

nalit

y

Explanation:

Agile Team receives a

Mandate per Iteration

from the PGA

A Mandate consists of

the Application(s) to be

build and the

capabilities of be

realized

As long as the agile

stays in the cone of the

mandate it moves from

sprint to sprint

If it moves outside the

cone, it has to ask for

approval of the PGA

S1

S2

S3 S4

S1 S2

S3

S4

S1 S2

S3

S4

final

Application

S - Sprints Round Tables Mandates (Application/Capabilities) from the PGA

Time

15

Typical (functional) Testprocesses in large scale agile projects

16

Test Task Quality Criteria Test Object Test Basis Layer Team/Role Regression/

Capture

Regression/

Replay

Remarks

Developer Test

DT

functionality Class capability, user

story

Agile Team Developer always, while

testing

dailiy within daily

build & test

procedure

see J Unit;

special

frameworks

Fuctional Test

FT

functionality application in the

definied

integration

degree (per

component)

capability for the

definied

integration set

Agile Team Team

Member/Tester

when sufficient

stability is

reached, mostly

late in sprint

in the same sprint

and selectively in

the forthcoming

sprints (to proof

the stablity of

already

implemented

functionality) -

RFT

reuse of mock

ups and stubes;

utilization of

common

testing tools

Smoke Test

ST

executability Daily Build over

the components

in their released

status; integrity

of added

interfaces

compiler;

selected streams

through the

business process

model

all Agile Teams,

integrated

Testcase:

Developer;

Testexecution:

C&RM-Team

potentially daily,

but selective

potentially daily -

RST

end 2 end tests;

no proof of

correctiveness

Integration Test

IT

interface

integrity; Sematic

Integrity;

correctiveness of

state transitions

Technical Domain

(all applications

of theprogram) in

their defined

integration

degree

business process

model or

business domain

model (including

business objects

& functionality)

Program Layer Integration Test

Team

selective executed in

Integration + 1 -

RIT

end 2 end tests

incl. proof of

correctness

Acceptance Test

AT

functionality (of

the complete

System)

Technical Domain

(all applications

of the program) in

their final

integration

degree

business process

model or

business domain

model (including

business objects

& functionality)

Program Layer Business Units in

their common

organization

no no end 2 end,

incl.proof of

correctness

non functional tests based on the program risk assessment on all integration levels possible (L&P, usability, security, operations tests, …)

Agenda

Problem

Big Picture of the Solution

Unternehmenspräsentation

Some Detailed Views

17

General Operating Model

18

Enterprise Architecture

Integration Manager

… Project 1 Scrum Master

Project 2 Scrum Master

System Integration Team

Program Manager

M M

M

M

Project Config Mgr.

Business-,Information-, Application-, Integration- Architecture; Release Planning

Release Planning (Components & Capabilities)

Package delivery (Application)

Program CM System

all artefacts checked in & labeled

Baselining Building Packaging

Components of the environment

Run & maintain the environment

Archive (Package, Sripts, Data,…)

Single point of control

Project 3 Scrum Master

Product Owner

Customer

Team End-user

Test Environment Responsible

Integration Team

Orgchart Integration Team

19

Integration

Testmanager

Problem Resolution

Integration

Test Team

Integration Test

Envir. - Mgmt

Config.

& Release

Management

Config

Administration

Build Run the

Environment

Test Data

Deployment

• Test case determination

• Test data definition

• Test execution

• Test evaluation

• Regressions tests (Integration tests)

• Test case sellection

PL

Execution

ST, RST, RFT

• Scheduled Builds

• Build Automation

• CMDB

• Test Automation

• Scheduled Tests

• System

Administration

• System

Deployments

• Preprocess ing

Tickets

• Retrieving

Testdata

• Uploading

Testdata

Quality

Management

(not in scope)

correct

End 2 End

processing

accross all

results of

all

agile teams

all function receive the data they require

i.e. are the underlying securities sufficient for a loan offering?

….Security data are available and approved

all functions transform data only for defined states of the business objects

i.e. does a terminated customer still receive services?

… The customer status is „terminated“; all functions, except „renew membership“ are

not valid for this status

All data are interpreted the same way throughout the whole system/all applications

i.e. does credit rating „A“ mean highest soundness in all applications?

… the value „A“ provides the same meaning for all functions, which interpretes this

data

Which Quality Criteria do we Test with Integration Testing?

20

Interfaces

State Transition

Semantic Integrity

Remark: The correct implementation of functionality in applications is tested in functional testing. This

quality is given and has not to be tested again. Therefore Integration Testing focuses on….

Functional Testing versus Integration Testing

21

Data Object 1

Function 1 Function 2 Function 3

X

X

X

X

X

X

X

X

Integration Testing

• End 2 End workflow per data object

• Low coverage ove all functions

• High coverage for state transitions and

interfaces

• Often done with (migrated) productive data

• Labor intensive for automation

Functional Testing

• all features/user stories of this function

• in deep (high coverage)

• synthetic test data

• suitable for automation

Fct

Fct

Fct

Fct

Fct

Fct D

D

D

D

D D

D

Fct Fct

IT-Landscape

D

Fct D

Fct

D

Appl. 1 Appl. 2

Data Object 2

Data Object 3

Data Object 4

22

Environments, integration grade and activities on the envirnoments, push/pull mechanisms

Development System Test

Environment

Weekly Build & Test

Pre-Production Production

Component System System of Systems

(Technical Domain) Coupled

Systems

Coupled

Systems

Versioning

(CMDB)

Component Test

Comp. Integrat. Test System Test (ST)

RST

SMT, RSMT, RSIT

Acceptance Tests

(OAT, BAT)

Functional SIT

Environmanet

SIT (FIT, UT)

Non Functional

SIT Environment

SIT (SLT, SPST, FOT)

pull

push

Build

Engine

pull pull

pull

pull

pull pull

push

Maintenance

Environment

Maintenance Tests

(analog ST)

pull

► ST: System Test

► SMT: Smoke Test

► SIT: System Integration Test;

five Tasks

► AT: Acceptance Test (BAT,OAT)

► (R) Regression Tests

Consequences of the Daily Build&Test Process

► Build Overall Team Development

► Test Overall Team Development

Agile

Team 1 Iteration

Agile

Team 2

Agile

Team 3

Iteration

Iteration

► The Outcome is delivered to the

Agile Teams

► Interface Problems are discussed by

the Hotfix Team and Chief Architect

► Targets

compilable software

Running basic functionalities

Awareness of Team interaction

Architects awareness on software interaction

Early error indicates of design

Build automation

Deployment automation

Test automation

► Daily &Test as early as possible

ST RST RIT

Inte

gra

tio

nte

st

in Daily Build

23

Entry Exit Criteria

24

Iteration 2

For Application 1

Developer

Test/I Functional

Test/I

Functional Test Environment Developer

Environment

Integration Environment

Application 1

Application 2

Application 3

Smoke

Test/I

Regression

Smoke Test/I-1

Regression Inte-

gration Test/ I-1

Part of the Daily Build

Inte

gra

tio

n T

es

t

Regression

Functional

Test/I-1

Entry

Criteria

Exit

Criteria

Entry

Criteria

Test Complete

ness

Max.

Failures

DT approval

FT all user

stories

(0/A);

(15%/B)

RFT 20% of

all RFTs

(0/A);

(15%/B)

ST/RST 10% of

No. User

Stories

100%

execut-

able

RIT 30%

auto-

mated

(0/A);

(5%/B)

(Example)

Exit

Criteria

Integr.at.

Test

Completeness

Coverage All Business

Objects & States

Coverage All capabilities

Coverage All Interfaces

Max.

Failures

(0/A)

(15%/B)

(Example)

Remark:

SW is deployed in FTE and ITE in parallel

ST, RST, RIT are executed dialy after build

RST, RIT are reduced Test Sets

Wilhelmstraße 63

D-53721 Siegburg

Phone: +49 (0) 2241 25021 0 Fax: +49 (0) 2241 25021 29

Mail: info@corporatequality.de

Web: www.corporatequality.de

Rooseveltplatz 13/5

A-1090 Wien

Phone: +43 (0) 1 89 03 448 Fax: +43 (0) 1 89 03 448 15

Mail: info@corporatequality.at

Web: www.corporatequality.at

An der Lorze 11

CH-6300 Zug

Phone: +41 (0) 41 740 39 19 Fax: +41 (0) 41 740 39 20

Mail: info@corporatequality.ch

Web: www.corporatequality.ch

© 2005-2013 corporate quality | Verwendung nur mit Zustimmung der corporate quality

Stephan Salmann, CEO

corporate quality consulting GmbH

Phone: +49 (0) 2241 25021 10

Mobile: +49 (0) 163 327 27 02

Fax: +49 (0) 2241 25021 29

Mail: stephan.salmann@corporatequality.de

Web: www.corporatequality.de

Thank you for your Attention….

Branch, Merge and Packaging Strategy

System of Systems,

Release and ST

approved Applications

System of Systems

In Weekly

Integration

Grade &

CT, CIT tested

Application

Under

Development

Released,

Application

(Release

Candidates)

Component

R1 R 1.1 R 2

Package R 1.1

Development, CT & CIT

Branch Merge &

Label

Package R1

System Integration Test

Package R1 Package R2

Package R2

Remark: Only Happy Path,

Foward Look, without

patching

System Test for Capabilities R1 of Application 1

Weekly Build &Test