funkcinis testavimas

©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 1

Funkcinis testavimas

Programų testavimas remiantis įėjimo, išėjimo ir vidinių būsenų kintamaisiais.


Defektų testavimo procesas

Testavimo atvejai

Testavimo atvejai

Testavimo duomenys

Testavimo duomenys

Testavimo rezultatai

Testavimo rezultatai

Testavimo ataskaitos

Testavimo ataskaitos

Suprojektuoti testavimo atvejus

Suprojektuoti testavimo atvejus

Paruošti testavimo duomenis

Paruošti testavimo duomenis

Paleisti programą su test. duom.

Paleisti programą su test. duom.

Palyginti rezultatus su testavimo atvejais

Palyginti rezultatus su testavimo atvejais


“Juodos dėžės “ testavimas

Testavimo metodas, kai programa yra įsivaizduojama kaip “juoda dėžė”.

Programos testavimo atvejai yra paremti sistemos specifikacija.

Testo planavimas gali prasidėti labai anksti programinės įrangos kūrimo procese.


Funkcinė specifikacija Sąsajos ekranas ir vykdoma užduotis Paprastai pagal funkcinę specifikaciją testuoja ne

programos realizuotojas Geriausiai tinka nepriklausomiems fragmentams Susietiems fragmentams turi įtaką vykdymo

tvarka


Vidinės būsenos Kiekvienas automatas remiasi perėjimais tarp

būsenų Testavimo metu turi būti nagrinėjami visi

perėjimai tarp būsenų Būsenų gali būti labai daug


B01 architecture BEHAV of b01 is constant a:integer:=0; constant b:integer:=1; constant c:integer:=2; constant e:integer:=3; constant f:integer:=4; constant g:integer:=5; constant wf0:integer:=6; constant wf1:integer:=7;


B01 when a => if line1='1' and

line2='1' then stato:=f; else stato:=b ; end if; outp <= line1 xor

line2; overflw <= '0' ;


Perėjimų tarp būsenų grafas

A

F

B

G

C

1

0

E


“Juodos dėžės “ testavimas

e

e

I

O

SistemaSistema

Įvesti testavimo duomenys

Įšvesti testavimo rezultatai

Įvedimas, sukeliantis nenormalų elgesį.

Išvedimai, kurie parodo defektų

buvimą.


Ekvivalentinis dalinimas Įvedami duomenys ir išvedami rezultatai dažnai

paskirstomi į atskiras klases, kur visi klasių nariai yra susieti.

Kiekviena iš šių klasių yra lygiaverčio suskirstymo rezultatas, kur programos elgesys su kiekvienu klasės nariu yra toks pats (ekvivalentiškas).

Testiniai atvejai turi būti pasirinkti iš kiekvienos dalies.


Ekvivalentinis dalinimas

SistemaSistema

Teisingos įvestys Neteisingos įvestys

Išvestys


Padalinti sistemos įėjimų ir išėjimų reikšmes į “ekvivalentiškas aibes”.

Jei įvedamas -5 skaitmenų sveikas skaičius tarp 10000 ir 99999, ekvivalentinis padalinimas yra <10000, 10000-99999 ir > 99999.

Išrinkti testinius atvejus ribose šių aibių 0000, 09999, 10000, 99999, 100000.

Ekvivalentinis dalinimas


Ekvivalentinis padalinimas

999910000 50000

10000099999

34 7

1110

Mažiau nei 4 Tarp 4 ir 10 Daugiau nei 10Mažiau nei 4 Tarp 4 ir 10 Daugiau nei 10

Įvedimų reikšmių skaičius

Mažiau nei 10000 Tarp 10000 ir 99999 Daugiau nei 99999Mažiau nei 10000 Tarp 10000 ir 99999 Daugiau nei 99999

Įvedamos reikšmės


Paieškos procedūros specifikacija

procedure Search (Key : ELEM ; T: ELEM_ARRAY; Found : in out BOOLEAN; L: in out ELEM_INDEX) ;

Pre-condition-- the array has at least one elementT’FIRST <= T’LAST

Post-condition-- the element is found and is referenced by L( Found and T (L) = Key)

or -- the element is not in the array( not Found and

not (exists i, T’FIRST >= i <= T’LAST, T (i) = Key ))


Išskiriami įėjimo poveikiai (įvestis), kurie atitinka išankstines sąlygas.

Išskiriami įėjimo poveikiai, kur išankstinės sąlygos nėra palaikomos.

Įėjimo poveikis, kur nurodytas elementas yra masyvo narys.

Įėjimo poveikis, kur pagrindinis elementas nėra masyvo narys.

Įėjimo poveikių padalinimas paieškos procedūrai


Testavimo nuorodos masyvams Testuoti programinę įrangą kai masyvas turi tik

vieną elementą. Skirtingiems testams naudoti skirtingo dydžio

masyvus. Parinkti testus taip, kad būtų nagrinėjamas

pirmas, vidurinis ir paskutinis masyvo elementas. Testuoti nulinio ilgio masyvus.


Testai paieškos procedūraiArray ElementSingle value In sequenceSingle value Not in sequenceMore than 1 value First element in sequenceMore than 1 value Last element in sequenceMore than 1 value Middle element in sequenceMore than 1 value Not in sequence

Input sequence (T) Key (Key) Output (Found, L)17 17 true, 117 0 false, ??17, 29, 21, 23 17 true, 141, 18, 9, 31, 30, 16, 45 45 true, 717, 18, 21, 23, 29, 41, 38 23 true, 421, 23, 29, 33, 38 25 false, ??


Binary search (Java)

class BinSearch {

// This is an encapsulation of a binary search function that takes an array of// ordered objects and a key and returns an object with 2 attributes namely// index - the value of the array index// found - a boolean indicating whether or not the key is in the array// An object is returned because it is not possible in Java to pass basic types by// reference to a function and so return two values// the key is -1 if the element is not found

public static void search ( int key, int [] elemArray, Result r ){

int bottom = 0 ;int top = elemArray.length - 1 ;int mid ;r.found = false ; r.index = -1 ;while ( bottom <= top ){

mid = (top + bottom) / 2 ;if (elemArray [mid] == key){

r.index = mid ;r.found = true ;return ;

} // if partelse{

if (elemArray [mid] < key)bottom = mid + 1 ;

elsetop = mid - 1 ;

}} //while loop

} // search} //BinSearch


Patenkintos išankstinės sąlygos, pagrindinis elementas masyve.

Patenkintos išankstinės sąlygos, pagrindinio elemento nėra masyve.

Nepatenkintos išankstinės sąlygos, pagrindinis elementas masyve.

Nepatenkintos išankstinės sąlygos, pagrindinio elemento nėra masyve.

Įėjimo masyvas turi vienintelę reikšmę. Įėjimo masyvas turi lyginį reikšmių skaičių. Įėjimo masyvas turi nelyginį reikšmių skaičių.

Ekvivalentinis sudalinimas dvejetainei paieškai


Ekvivalentinių klasių ribiniai atvejai

Mid-point

Elements < Mid Elements > Mid

Equivalence class boundaries


Dvejetainės paieškos testiniai atvejai

Functional TestingSpecification Based Testing

(Black Box testing)

22


Why do it? Increase our confidence that the software works

and works correctly


Devise a test plan

A program reads 3 integer values. The 3 values areinterpreted as representing the lengths of the sidesof a triangle. The program prints a message thatstates whether the triangle is scalene ( nelygiašonis), isosceles (lygiašonis), or equilateral (lygiakraštis).

Write test cases that would adequately test this program.


Myers Test Cases 1. Valid scalene (5, 3, 4) => scalene 2. Valid isoscleles (3, 3, 4) => isosceles 3. Valid equilateral (3, 3, 3,) => equilateral 4. First permutation of 2 sides (50, 50, 25) =>

isosceles 5. Second perm of 2 sides (25, 50, 50) => isosceles 6. Third perm of 2 sides (50, 25, 50) => isosceles 7. One side zero (1000, 1000, 0) => invalid


More test cases 8. One side has negative length (3, 3, -4) => invalid 9. first perm of two equal sides (5, 5, 10) => invalid 10. Second perm of 2 equal sides (10, 5, 5) => invalid 11. Third perm of 2 equal sides (5, 10, 5) => invalid 12. Three sides >0, sum of 2 smallest < largest (8,2,5) => invalid 13. Perm 2 of line lengths in test 12 (2, 5, 8) => invalid 14. Perm 3 of line lengths in test 12 (2, 8, 5) => invalid


More test cases 15. Perm 4 of line lengths in test 12 (8, 5, 2) => inv 16. Perm 5 of line lengths in test 12 (5, 8, 2) => inv 17. Perm 6 of line lengths in test 12 (5, 2, 8) => inv 18. All sides zero (0, 0, 0) => inv 19. Non-integer input, side a (@, 4, 5) => inv 20. Non-integer input, side b (3, $, 5) => inv 21. Non-integer input, side c (3, 4, %) => inv


More test cases 22. Missing input a (, 4, 5) => invalid 23. Missing input b (3, , 5) => invalid 24. Missing input c (3, 4, ) => invalid 25. Three sides > 0, one side equals the sum of the other

two (12, 5, 7) => inv 26. Perm 2 of line lengths in test 25 (12, 7, 5) => inv 27. Perm 3 of line lengths in test 25 (7, 5, 12) => inv 28. Perm 4 of line lengths in test 25 (7, 12, 5) => inv 29. Perm 5 of line lengths in test 25 (5, 12, 7) => inv 30. Perm 6 of line lengths in test 25 (5, 7, 12) => inv


More test cases 31. Three sides at max values (32767, 32767, 323767) => inv 32. Two sides at max values (32767, 32767, 1) => inv 33. One side at max values (32767, 1, 1) => inv


Testing principles

1. Complete testing is not possible

2. Testing work is creative & difficult• need to understand the system

• most systems are complex

• need business knowledge, testing experience, creativity, insight and testing methodology


Testing principles

4. Testing is risk based• safety-critical systems need high level

• use risk as basis for allocating the test time available and selecting what to test

5. Testing must be planned • plan overall approach

» what to test and when to stop

• design tests

• establish expected results for each selected test case


Testing principles

6. Testing requires independence• unbiased measurement e.g. test team

• goal is to measure software quality accurately

• often conflict with the requirement to understand the system being tested (principle 2)


Black-box methods equivalence partitioning boundary-value analysis error guessing cause-effect graphing


Equivalence partitioning based on the specification; no need to see the code divide up the input domain into equivalence

partitions or classes of data each class is treated identically any datum chosen from a class is as valid as any

other advantage: reduces the input domain to a

manageable size


Equivalence partitioning (cont) to assist in identifying partitions, look in specs

for terms such as “range”, “set” and other similar words

equivalence classes should not overlap in addition to choosing one datum from each

class, invalid data may also be chosen


Equivalence partitioning (cont)example #1

consider the exam processing program where we categorise the grade as A, B, C, D, F

assume A>=90, B>=80, C>=70, D>=60 we can partition the grades, g, into the following

partitions: 0<=g<60, 60<=g<70, 70<=g<80, 80<=g<90, g>=90 and g>100, g<0



input Output Expected output50 F

65 D75 C85 B95 A-5 Invalid input

105 Invalid input



string searching program: the program prompts the user for a positive integer in the range 1 to 20 and then for a string of characters of that length. The program then prompts for a character and returns the position in the string at which the character was first found or a message indicating that the character was not present in the string. The user has the option to search for more characters.



There are 3 equivalence classes to consider• one class of integers in the stated range

• two classes of integers above and below the range

output domain consists of two classes• the position at which the character is found in the string

• a message stating that it was not found



x input string search char response expectedoutput

34 wrong input0 wrong input3 abc c pos = 3

yk not in string

n


strengths and weakness of equivalence partitioning

strength: it does reduce size of input domain well suited to data processing applications where

input variables may be easily identified and take on distinct values

not easily applied to applications where input domain is simple yet the processing is complex


strengths and weakness of equivalence partitioning

problem: although the specification may suggest that a class of data is processed identically, this may not be the case

example: y2k problem also, the technique does not provide an algorithm

for finding the partitions


Boundary value analysis used in conjunction with equivalence partitioning this technique focuses on likely sources of faults:

boundaries of equivalence classes the technique relies on having created the

equivalence classes


Boundary value analysis (cont)input Output Expected output

50 F

65 D75 C85 B95 A-5 Invalid input

105 Invalid input60 D70 C80 B90 A


Boundary value analysis (cont)string processing program

integer values of 0, 1, 20 and 21 are obvious choices,

as well as finding the character in the first and last position


Boundary value analysis (cont)

x input string search char response expectedoutput

21 out of range0 out of range1 a a pos = 1

yX not in string

n20 abcdefghijkl

mnopqrsta pos = 1

yt pos = 20

n


Error guessing an ad hoc approach, based on intuition and

experience identify tests that are likely to expose errors make a list of possible errors or error prone

situations and then develop tests based on the list


Error guessing (cont)some items to try:

empty or null lists/strings zero instances/occurrences blanks or null characters in strings negative numbers


Error guessing (cont)strengths/weaknesses

intuition frequently accurate the technique is efficient technique relies on experience; not always

available the ad hoc nature leaves doubt about quality of

the test: “did I forget any typical error situations?”


Cause-effect graphing functional or spec-based technique systematic approach to selecting a set of high-

yield test cases that explore combinations of input conditions

rigorous method for transforming a natural-language spec into a formal-language spec

exposes incompleteness and ambiguities in the spec


Cause-effect graphing (cont) the spec is analyzed, and all possible causes are identified: inputs, stimuli,

anything that will elicit (išaiškinti) a response from the system

all possible effects are identified: outputs, changes in the system state

causes and effects must be stated so that they can be evaluated as either true or false


Cause-effect graphing (cont) causes and effects are combined into a boolean

graph that describes their relationship each cause and effect is allocated a unique

number for reference create a boolean graph that shows the links

between cause and effect construct test cases that cover all possible

combinations of cause/effect


Cause-effect graphing (cont) graphs are combined using operators:

• not

• and

• or

• nor


Cause-effect graphing (cont)exam processing program

causes:• (1) integer in range 1-20

• (2) search char is in string

• (3) search for another char

effects• (20) integer out of range

• (21) report position of char in string

• (22) char not found in string

• (23) program terminates


Cause-effect graphing (cont)


Cause-effect graphing (cont)strengths/weaknesses

exercises combinations of test data expected results are part of test creation process major drawback is boolean graph: large number

of causes and effects produce highly complex graph

soln to weakness: identify sub-problems


Black-Box Test Reduction Using Input-Output Analysis

ISSTA ‘00

Patrick J. Schroeder, Bogdan Korel

Department of Computer Science

Illinois Institute of Technology

Chicago, IL USA


1) Motivation

2) Black-Box Testing

3) Input-Output Analysis

4) Determining Input-Output Relationships

5) Conclusion

Presentation Overview


Experience in Industry as a black-box tester of large, complex, telecommunications systems

Root cause analysis of customer-found faults often discovered ineffective or redundant black-box tests

Additional information from the software implementation often improves black-box testing (after the initial release!)

Motivation for Research


Black-Box Testing

Black-box testing is testing from a functional or behavioral perspective to ensure a program meets its specification

Testing usually conducted without knowledge of software implementation - system treated as a “black box”

Black-box test design techniques include: equivalence partitioning, boundary value analysis, cause-effect graphing, random testing


Observations on Black-Box Testing

Testing from the black-box perspective generates a large number of tests (e.g. AT&T Definity PBX > 3 million tests)

All testers are faced with resource limitations; the number of tests must be reduced

Test reduction done randomly or based on “engineering judgement” may lead to ineffective or redundant tests


Combinatorial Testing

Combinatorial testing: For programs with multiple inputs, one must decide how to test different combinations of selected test data values

The most thorough approach is to test all combinations - this is quite often impossible due to the large number of tests

However, not to consider testing different input combinations could lead to an unacceptable number of undetected software faults


Combinatorial Explosion

If n = 10 and 10 test data values are selected for each variable, the total number of combinatorial test cases is 1010

X1 X2 X3 . . . Xn

Program P


Current Approaches

Testers may try to identify “important” combinations to test using specifications, historical fault data, and engineering judgement

Orthogonal arrays, or other combinatorial design techniques, may be used to tests pair-wise or n-way combinations of inputs

Difficult to determine impact on fault-detection capability of the test set using these techniques


Definition of the Problem

In combinatorial testing, black-box testers face the situation where the number of possible test cases far exceeds the time and resources available to execute them

How can we reduce the number of combinatorial tests while maintaining the fault-detection capability of combinatorial testing?


Proposed Technique

Observations:Current techniques do not consider program output variables

For a class of programs, not all program inputs affect all program outputs

We propose the use of Input-Output (IO) Analysis to determine which program inputs affect program outputs

This information can be used to reduce the number of combinatorial tests while “maintaining” the fault-detection capability of the test set


Simple Example using IO Analysis

Input data selected using equivalence partitioning:

A={1, 3} B={N,S,E,W} C={TDC, BDM}

W ZTotal Number of Combinatorial Tests: 2 * 4 * 2 = 16


IO Analysis Applied

A={1, 3} B={N,S,E,W} C={TDC, BDM}

W Z


IO Analysis Applied

T(W)(1, N, TDC)(1, N, BDM)(3, N, TDC)(3, N, BDM)

T(Z)(1, N, TDC)(1, S, TDC)(1, E, TDC)(1, W, TDC)

A={1, 3} B={N,S,E,W} C={TDC, BDM}

W Z T(W) T(Z) (1, N, TDC)(1, N, BDM)(3, N, TDC)(3, N, BDM)(1, S, TDC)(1, E, TDC)(1, W, TDC)


Optimization Problem

• Create the smallest possible test set which covers all data combinations in the individual output test sets

• Test set optimized using “Don’t Care” (DC) conditions

Toptimized(1, N, TDC)(1, S, BDM)(3, E, TDC)(3, W, BDM)

T(W)(1, DC, TDC)(1, DC, BDM)(3, DC, TDC)(3, DC, BDM)

T(Z)(DC, N, DC)(DC, S, DC)(DC, E, DC)(DC, W, DC)


Determining Input-Output Relationships

Information may be gathered in a variety of ways (e.g. from specifications, code)

Manual collection of information difficult and time consuming

Automated collection possible through:

Static Analysis

Dynamic Analysis

Execution-oriented Analysis


Automated Analysis Techniques

Static techniques analyze a program’s source code - static program dependencies used to determine IO relationships

Dynamic techniques analyze a program’s execution trace across a set of test data - dynamic dependencies used to determine IO relationships

Execution-oriented techniques manipulate individual inputs across multiple program executions in a test harness to determine IO relationships


Technique Advantages/Disadvantages

Static techniques reliably determine the absence of relationships between inputs and outputs; however, they may identify relationships that are not actually present

Dynamic analysis and execution-oriented analysis reliably determine the presence of input-output relationships; however, they cannot guarantee that all relationships will be detected

Dynamic analysis and execution-oriented analysis may be used when static analysis, perhaps in over-estimation, indicates that all program inputs affect all program outputs


Experimental Study Results

1) Warehouse Management System

Combinatorial Tests: 3,125

Using IO Analysis 40

2) DACS Configuration System

Combinatorial Tests 22,500


3) Liquidity Ratio Spreadsheet

Combinatorial Tests 1.95 Million



Conclusion

For the class of program and type of problem defined, input-output analysis can reduce the number of combinatorial tests while “maintaining” the fault-detection capability of the test set

Research continues on identifying program types and testing situations where our approach is applicable

Additional experimentation with execution-oriented analysis to provide program dependencies, and other information useful in black-box testing, is underway


Example 1

Identify subdomains using cases in the specification Procedure is a black box; intervals are hidden

Int max(int a, int b)// effects: a > b => returns a// a < b => returns b// a = b => returns a

3 obvious tests:(4,3) => 4 (NOTE: any input in the

subdomain of a > b)(3,4) => 4 (3,3) => 3


Example 2

Int find(int[] a, int value)// effects: returns the smallest i// such that a[I]==value,// unless value not in a // Missing

2 obvious tests:({4,5,6},5) => 1({4,5,6},7) => missing

1 more subtle test:({4,5,5},5) => 1

Approach: look for any statement in the effects (or requires) clause that suggests multiple cases


Example 3

Test a function that uses the quadratic formula to determine the two roots of a second-degree polynomial ax2+bx+c.

For simplicity, assume that we are going to work only with real numbers, and print an error message if it turns out that the two roots are complex numbers (numbers involving the square root of a negative number).

• What will be the test data for each of the four cases, based on values of the polynomial's discriminant (b2-4ac)?


Exceptions to be Considered:


Test Design Tips

A command to a program may have immediate effects but also enable, disable, or modify later commands. It's important to both check the immediate effect and also try later commands.

Question: Which commands? And how? Answer: The ones that a user would likely

perform, in the way a user would perform them.


Example 4: Consider a product with password protection. You could change the password:


Example 5: Here's a bug in prerelease

version 0.7.6 of the AbiWord word processor:


Equivalence Partitioning

useruserqueriesqueries mousemouse

pickspicks

outputoutputformatsformats

promptspromptsdatadata


Boundary Value Analysis

userqueries mouse

picks

outputformats

prompts data

outputdomaininput domain


As with white-box testing, we should test our code with each set of test data.

If the answers match, then our code passes the black-box test.


Unit Testing • Informal:

• • Incremental coding

• Static Analysis:• • Hand execution: Reading the source code• • Walk-Through (informal presentation to others)• • Code Inspection (formal presentation to others)• • Automated Tools checking for• • syntactic and semantic errors• • departure from coding standards

• Dynamic Analysis:• • Black-box testing (Test the input/output behavior)• • White-box testing (Test the internal logic of the subsystem or• object)• • Data-structure based testing (Data types determine test cases)


Unit Testing

modulemoduleto beto betestedtested

test casestest cases

resultsresults

softwaresoftwareengineerengineer


Unit Testing

interface

local data structures

boundary conditions

independent pathserror handling paths

modulemoduleto beto betestedtested

test casestest cases


What Should be in a Test Case?


How to Design Unit Test Spec & Cases ?

A 7-step general process for developing a unit test specification as a set of individual unit test cases.

For each step of the process, suitable test-case design techniques are suggested.


Step 1 - Make it run: Execute the unit under test in the simplest way possible

–Specification derived tests –Equivalence partitioning

Step 2 - Positive testing: Walk through the relevant specifications; each test case should test one or more statements of design specification for the unit


Step 3 - Negative testing: Show that the software does not do anything that it is not specified to do -- error guessing

–Error guessing –Boundary value analysis –Internal boundary value testing


Step 4 - Special considerations: Check performance, safety requirements and security requirements.

Specification derived tests Step 5 - Coverage tests

Branch testing Condition testing Data definition-use testing

Step 6 - Test execution: Yield a measure of test coverage, indicating whether coverage objectives have been achieved.

Step 7 - Coverage completion• There may be complex decision conditions, loops and branches within the code for which coverage targets may

not have been met when tests were executed• Infeasible paths or conditions • Unreachable or redundant code • Ideally, the coverage completion step should be conducted without looking at the actual code



Test Case Design Techniques


Example: Specification derived tests

Consider the specification for a function to calculate the square root of a real number: • Input - real number • Output - real number

1. When given an input of zero or greater, the positive square root of the input shall be returned.

2. When given an input of less than 0, the error message "Square root error - negative input" shall be displayed and a value of 0 returned.

3. The library routine Print_Line shall be used to display the error message.

Consider the specification for a function to calculate the square root of a real number: • Input - real number • Output - real number

1. When given an input of zero or greater, the positive square root of the input shall be returned.

2. When given an input of less than 0, the error message "Square root error - negative input" shall be displayed and a value of 0 returned.

3. The library routine Print_Line shall be used to display the error message.


There are three statements in this specification, which can be addressed by two test cases

Print_Line conveys structural information in the specification • Test Case 1: Input 4, Return 2

» Exercises the first statement in the specification ("When given an input of 0 or greater, the positive square root of the input shall be returned.").

• Test Case 2: Input -10, Return 0, Output "Square root error - illegal negative input" using Print_Line

» Exercises the second and third statements in the specification ("When given an input of less than 0, the error message "Square root error - illegal negative input" shall be displayed and a value of 0 returned. The library routine Print_Line shall be used to display the error message.").


Example: Equivalence partitioning

Equivalence partitioning assumes that all values within any individual partition are equivalent for test purposes

Test cases should therefore be designed to test one value in each partition

• The square root function used in the previous example. The square root function has two input partitions and two output partitions


Four partitions can be tested with two test cases: • Test Case 1: Input 4, Return 2

» Exercises the >=0 input partition (ii) » Exercises the >=0 output partition (a)

• Test Case 2: Input -10, Return 0, Output "Square root error - illegal negative input" using Print_Line» Exercises the <0 input partition (i) » Exercises the "error" output partition (b)

ii

i

Partitions for Square

Root


Example: Boundary value analysis

Test cases are designed to exercise the software on and at either side of boundary values

Input partition boundaries in

Square Root: • The zero or greater partition has a

boundary at 0 and a boundary at the most positive real number.

• The less than zero partition shares the boundary at 0 and has another boundary at the most negative real number.

• The output has a boundary at 0, below, which it cannot go.


Test Case 1: Input {the most negative real number}, Return 0, Output "Square root error - illegal negative input" using Print_Line

» Exercises the lower boundary of partition (i).

Test Case 2: Input {just less than 0}, Return 0, Output "Square root error - illegal negative input" using Print_Line

» Exercises the upper boundary of partition (i).

Test Case 3: Input 0, Return 0 » Exercises just outside the upper boundary of partition (i), the lower boundary

of partition (ii) and the lower boundary of partition (a).

Test Case 4: Input {just greater than 0}, Return {the positive square root of the input}

» Exercises just inside the lower boundary of partition (ii). Test Case 5: Input {the most positive real number}, Return {the

positive square root of the input} » Exercises the upper boundary of partition (ii) and the upper boundary of

partition (a).


Example: Branch testing

Given a structural specification for a unit, specifying the control flow within the unit, test cases can be designed to exercise branches.

Such a structural unit specification will typically include a flowchart or flow graph. • For the square root example, a test designer could assume that

there would be a branch between the processing of valid and invalid inputs, leading to the following test cases:

• Test Case 1: Input 4, Return 2 Exercises the valid input processing branch

• Test Case 2: Input -10, Return 0, Output "Square root error - illegal negative input" using Print_Line.

Exercises the invalid input processing branch


However, there could be many different structural implementations of the square root function, which may not be all covered by the two test cases

However, there could be many different structural implementations of the square root function, which may not be all covered by the two test cases


Example: Condition testing

To test the individual elements of logical expressions, both within branch conditions and within other expressions in a unit -- "white box" test design from a structural specification for a unit.

• Consider the specification for the square root function, which uses successive approximation.

• We decide to limit the algorithm to a maximum of 10 iterations, because after 10 iterations the answer would be as close as it would ever get. The PDL specification for the unit could specify an exit condition:

:EXIT_LOOP WHEN(error<desired

accuracy)or(iterations=10) :


Test cases have to prove that both error<desired accuracy and

iterations=10 can independently affect the outcome of the decision. Test Case 1: 10 iterations, error>desired accuracy for all

iterations. • Both parts of the condition are false for the first nine iterations. On the tenth

iteration, the first part of the condition is false and the second part becomes true, showing that the iterations=10 part of the condition can independently affect its outcome.

Test Case 2: 2 iterations, error>=desired accuracy for the first iteration, and error<desired accuracy for the second iteration. • Both parts of the condition are false for the first iteration. On the second

iteration, the first part of the condition becomes true and the second part remains false, showing that the error<desired accuracy part of the condition can independently affect its outcome.

Other Test Cases: Both conditions are true or false.


Step 1 - Make it run: Execute the unit under test in the simplest way possible


Step 2 - Positive testing: Walk through the relevant specifications; each test case should test one or more statements of design specification for the unit


Step 3 - Negative testing: Show that the software does not do anything that it is not specified to do -- error guessing

–Error guessing –Boundary value analysis –Internal boundary value testing


Step 4 - Special considerations: Check performance, safety requirements and security requirements.

Specification derived tests Step 5 - Coverage tests


Step 6 - Test execution: Yield a measure of test coverage, indicating whether coverage objectives have been achieved.

Step 7 - Coverage completion• There may be complex decision conditions, loops and branches within the code for which coverage targets may

not have been met when tests were executed• Infeasible paths or conditions • Unreachable or redundant code • Ideally, the coverage completion step should be conducted without looking at the actual code



Example: Data definition-use testing

To test pairs of data definitions and uses. A data definition is anywhere that the value of a

data item is set, and a data use is anywhere that a data item is read or used. • Consider one PDL specification for the square

root function, which sends every input to the maths co-processor and uses the co-processor status to determine the validity of the result.

• The first step is to list the pairs of definitions and uses. In this specification, there are a number of definition-use pairs.


These pairs of definitions and uses can then be used to design test cases. Two test cases are required to test all six of these definition-use pairs:

• Test Case 1: Input 4, Return 2 Tests definition-use pairs 1, 2, 5, 6

• Test Case 2: Input -10, Return 0, Output "Square root error - illegal negative input" using Print_Line.

Tests definition-use pairs 1, 2, 3, 4


Example: Internal boundary value testing

Consider a fragment of the successive approximation version of the square root unit specification:

:

Calculate first approximation

LOOP

Calculate error

EXIT_LOOP WHEN error<desired accuracy

Adjust approximationEND_LOOP

RETURN the answer :


The calculated error can be in one of two partitions about the desired accuracy, which is not apparent from a purely functional specification.

An analysis of internal boundary values yields three conditions for which test cases need to be designed:

• Test Case 1: Error just greater than the desired accuracy

• Test Case 2: Error equal to the desired accuracy • Test Case 3: Error just less than the desired

accuracy

funkcinis testavimas

Documents