cocomo ii calibration

COCOMO II Calibration

Brad ClarkSoftware Metrics Inc.

Don ReiferReifer Consultants Inc.

22nd International Forum on COCOMO and Systems / Software Cost Modeling

USC Campus, Los Angeles, CA, 31 Oct to 2 Nov 2007

smi Software Metrics, Inc.

smi Software Metrics, Inc. 2

Topics

I. Globbing study results– Glob: (informal noun) a lump of a semi-liquid substance– COCOMO Globbing: lumping projects together based on a

common attributes

II. COCOMO Model calibration status


Globbing Motivation

Create pre-set cost driver ratings for different application domains

– Select an application domain and then create a set of driver ratings for it based on common characteristics

– Allows estimators to generate estimates quickly– Permits estimators to create a knowledge base on the basis

of application domain characteristics

Create calibration groups within the data– Hypothesis: projects in the database are so diverse that

“local” calibration within a group should improve model accuracy and precision

smi Software Metrics, Inc.

Globbing data into application domains is based on productivities and size

What productivity do we use: raw or adjusted?

Why would we consider adjusting actual effort?– Because there are business decisions that impact the project and

are independent of an application domain

Which COCOMO II drivers are business-driven?

How do we create size ranges?

44

€

Raw Productivity =SLOC

Actual Effort

Adjusted Productivity =SLOC

Actual Effort adjusted for project characteristics

Approach -1


Approach -2Business-driven drivers (these are independent of the application)

– Analyst capability

– Programmer capability

– Personnel continuity

– Application experience

– Platform experience

– Language and tool experience

– Use of software tools

– Multi-site development

– Required development schedule

– All Scale Drivers

Application-driven drivers

– Required software reliability

– Database size

– Product complexity

– Developed for Reuse (maybe not)

– Documentation match to life-cycle needs

– Execution time constraint

– Main storage constraint

– Platform volatility


Size BucketsSize versus Productivity

317

247

221230

215196

183171176

100

125

150

175

200

225

250

275

300

325

350

0 100 200 300 400 500 600 700 800 900 1000

KSLOC

COCOMO II Productivities (Nominal Projects)

When Size and Productivity are compared, there are three areas where productivity changes at different rates:

– Small: 2 to 25 KSLOC– Medium: 25 to 100 KSLOC– Large:100 to 1,000 KSLOC

When Size and Productivity are compared, there are three areas where productivity changes at different rates:

– Small: 2 to 25 KSLOC– Medium: 25 to 100 KSLOC– Large:100 to 1,000 KSLOC


Analysis Approach -1Data from COCOMO II 2000 calibration (161projects) were used to conduct the study

The effort was adjusted to remove the business-driven drivers: PM’

Data points were divided into four different groups based on productivities observed in the data

– Glob-1: Defense-like applications (real-time; complex)

– Glob-2: Telecom-like applications (high reliability)

– Glob-3: Scientific-like applications (compute-intensive)

– Glob-4: Business-like applications (data-intensive)

Each Glob was segregated into three size buckets based on COCOMO II model productivity rates

– Small: 2 to 25 KSLOC

– Medium: 25 to 100 KSLOC

– Large: 100+ KSLOC


Analysis Approach -2

Glob-1 Glob-2 Glob-3 Glob-4

REL Y VH H H N

DATA N H L N

CPLX H N H L

RUSE N N N N

DOCU N N N N

TIME VH H N N

STOR VH H N N

PVOL H N N N

Globbing around application-type and size has been successfully demonstrated in other cost models.We defined application-types as follows:

We use such definitions because we can compare results against the norms reported in the Crosstalk Mar 2002 article “Let the Numbers Do The Talking”


COCOMO II 2000 Effort Prediction Accuracy

Prediction Accuracy1

Before Stratification by

OrganizationAfter Stratification

by Organization

PRED(20) 63% 70%

PRED(25) 68% 76%

PRED(30) 75% 80%

As a comparison to the Globbing results, the COCOMO II model accuracy reported in 2000 was:

Note:

1. PRED(X) = Y% means that Y% of the predicted values fall within X% of the actual values


Results -1

Size Bucket Glob-1 Glob-2 Glob-3 Glob-4

2-25 KSLOC

n 6 10 6 10

A 3.13 2.82 2.99 2.82

PRED(25) 83% 80% 100% 70%

PRED(20) 67% 70% 100% 50%

25-100 KSLOC

n 13 15 15 14

A 3.21 3.12 3.09 2.67

PRED(25) 69% 87% 73% 93%

PRED(20) 69% 73% 73% 86%

100+ KSLOC

n 18 8 6 9

A 3.61 3.18 2.40 2.75

PRED(25) 72% 75% 50% 89%

PRED(20) 56% 25% 33% 89%

Projects in each Glob and Size Bucket were used to create a new COCOMO II constant, A, for each group (B set to 0.91).


Results -2Size Bucket Glob-1 Glob-2 Glob-3 Glob-4

2-25 KSLOC

n 6 10 6 10

A 3.53 4.27 2.88 1.90

B 0.87 0.75 0.92 1.07

PRED(25) 83% 80% 100% 60%

PRED(20) 67% 70% 100% 50%

25-100 KSLOC

n 13 15 15 14

A 7.14 5.42 6.44 2.05

B 0.69 0.76 0.72 0.98

PRED(25) 69% 80% 73% 93%

PRED(20) 69% 80% 73% 86%

100+ KSLOC

n 18 8 6 9

A 4.58 9.52 1.20 8.76

B 0.87 0.73 1.04 0.68

PRED(25) 72% 75% 50% 100%

PRED(20) 56% 75% 50% 100%

COCOMO II constants, A and B, were created for each group.


Usage ExampleExample Project:

– Estimated 56,000 SLOC– Application-type: Telecom-like (Glob-2, constant A=3.12)

Results:– Preset drivers:

• RELY: High

• DATA: High

• TIME: High

• STOR: High

– All other drivers: Nominal– 382 PM, 24 Months, 16 Average Staffing Level

As more is known (but not until then), COCOMO II drivers can be adjusted to reflect that informationAs more is known (but not until then), COCOMO II drivers can be adjusted to reflect that information


Globbing Conclusions

Purpose was to create pre-set cost driver ratings for different application domains and a local calibration based on the domain

An application domain is not characterized by all COCOMO II drivers, i.e. only a subset of the drivers describe an application domain (the question is which subset?)

Results for a new COCOMO II calibrated constant, A, for each group makes the most sense and are reasonably accurate.

– Need more data to calibrate A and B (10-12 projects for each group)

– Size buckets account for possible changes in B


Globbing Next Steps

Get feedback on this idea from you, the conference attendees

– Please share your thoughts with Don or Brad

The key to this approach is identifying the correct Globs and the cost-driver setting for each Glob.

– Use a consensus approach to setting the cost drivers for the different Globs

– Name and describe the Globs– Re-run analysis

Re-apply this technique to new project data in the repository


Topics

I. Globbing study results– Glob: (informal noun) a lump of a semi-liquid substance– COCOMO Globbing: lumping projects together based on a

common attribute

II. COCOMO Model calibration status


COCOMO II Calibration Status

Please check the CSSE website under “Past Events” for the results of this work.

http://csse.usc.edu/csse/event/past.html


For More Information

Brad ClarkSoftware Metrics Inc.(703) [email protected]

Don ReiferReifer Consultants Inc.(310) [email protected]

cocomo ii calibration

Documents