Download - デブサミ関西2013【A4】コード品質は曖昧なままか(安竹由起夫氏)
Coverity-DevSumiKansai2013.pptx
Summit Developers
Developers Summit 2013 Kansai Action !
A4 #kansumiA4
Summit Developers
Developers Summit 2013 Kansai Action !
2
Summit Developers
Developers Summit 2013 Kansai Action !
COTS
COTS
h(p://scan.coverity.com
Summit Developers
Developers Summit 2013 Kansai Action !
US SEC. 933. IMPROVEMENTS IN ASSURANCE OF COMPUTER SOFTWARE PROCURED BY THE DEPARTMENT OF DEFENSE.
(a) Baseline SoMware Assurance Policy- The Under Secretary of Defense for AcquisiTon, Technology, and LogisTcs, in coordinaTon with the Chief InformaTon Ocer of the Department of Defense, shall develop and implement a baseline soMware assurance policy for the enTre lifecycle of covered systems. Such policy shall be included as part of the strategy for trusted defense systems of the Department of Defense.
(b) Policy Elements- The baseline soMware assurance policy under subsecTon (a) shall--
(1) require use of appropriate automated vulnerability analysis tools in computer so3ware code during the en7re lifecycle of a covered system, including during development, operaTonal tesTng, operaTons and sustainment phases, and reTrement;
(2) require covered systems to idenTfy and prioriTze security vulnerabiliTes and, based on risk, determine appropriate remediaTon strategies for such security vulnerabiliTes;
(3) ensure such remedia7on strategies are translated into contract requirements and evaluated during source selecTon;
NaTonal Defense AuthorizaTon Act 2013
Summit Developers
Developers Summit 2013 Kansai Action !
5
Summit Developers
Developers Summit 2013 Kansai Action !
SAT &
if x=0
... ...
If x != 0
NULL
X!=0 X=0
X!=0 X=0 void foo(int *p) { *p = 42; }
void bar() { foo(p); if(p != 0) { ... } }
int *p = malloc(sizeof(int)); if(p != 0) *p = 42; ... int *p = malloc(sizeof(int)); if(p != 0) *p = 42;
int *p = malloc(sizeof(int));
*p = 42;
htmlEncode()
< > & " '
a b c d < > &
Summit Developers
Developers Summit 2013 Kansai Action !
7
Address
Summit Developers
Developers Summit 2013 Kansai Action !
8
Summit Developers
Developers Summit 2013 Kansai Action !
(source)
Summit Developers
Developers Summit 2013 Kansai Action !
10
Summit Developers
Developers Summit 2013 Kansai Action !
Python
Summit Developers
Developers Summit 2013 Kansai Action !
Samba
Summit Developers
Developers Summit 2013 Kansai Action !
ANTLR
13
Summit Developers
Developers Summit 2013 Kansai Action !
14
Summit Developers
Developers Summit 2013 Kansai Action !
15
1,000
1,000
1,000
5.9 /K 4.85/K 0.69/K 15/
0.05/K 0/K 0.01/K 1/
1.47/K 0.69/K 0.16/K 9/
20 17900 (179M step)
Summit Developers
Developers Summit 2013 Kansai Action !
24
Summit Developers
Developers Summit 2013 Kansai Action !
25
Summit Developers
Developers Summit 2013 Kansai Action !
26
0"
50"
100"
150"
200"
250"
300"
350"
400"
Alameda" Berkeley" Carmel" Davis" Eureka*"
Num
ber'o
f'Def
ects
'
Defects'Addressed'by'Coverity'Quality/Security'Advisor''
High"Impact" Medium"Impact" Low"Impact"
Summit Developers
Developers Summit 2013 Kansai Action !
:
6 3 6
2 2 2 GA
Alameda, Berkeley, Davis, Eureka, Fresno,
:
2
Jira Pivotal Bugzilla
Summit Developers
Developers Summit 2013 Kansai Action !
Front EndCompilation
AnalysisCore Analysis
Coverity Connect (CC)Defect Management
1 Gc_rc gc_pbkdf2_sha1 (const char *P, size_t Plen, 2 const char *S, size_t Slen, 3 unsigned int c, 4 char *DK, size_t dkLen) 5 { 6 char U[20] T[20]; 7 unsigned int hlen = 20, u, l, r, i, k; 8 int rc; char *tmp; size_t tmplen 9 10 if (c == 0) 11 return GC_PKCS5_INVALID_ITERATION_COUNT; 12 r = dkLen - (l - 1) * hLen; 13 14 memcpy (tmp, S, Slen);
:
Tests automated and run during build
Tests automated and run during build
Mostly manuallytested
End-to-end (E2E) testsperformed manually
: Coverity Connect End-to-End
CondenTal: For Coverity and Partner use only. Copyright Coverity, Inc., 2013 28
Summit Developers
Developers Summit 2013 Kansai Action !
Coverity Connect
Confidential: For Coverity and Partner use only. Copyright Coverity, Inc., 2013
29
Summit Developers
Developers Summit 2013 Kansai Action !
30
% C
ode
Teste
d
Effort to develop tests
100%
Diminishing return forincreased test effort1
Not all code is testable - unreachable statements - dead code, ...
2
Not all tested code adds equal value to the test - non-critical code - debug code, legacy code - exception handling, ...
3
- - - -
- - - ...
Summit Developers
Developers Summit 2013 Kansai Action !
31
SCM: Git, CVS, Mercurial, Subversion, Perforce, ClearCase, AccuRev, MS TFS
gcov (C/C++) BullseyeC/C++) IBM PureCovC/C++) Corbertura (Java)
.
Summit Developers
Developers Summit 2013 Kansai Action !
3 : 100% ( ,
, )
0"
2"
4"
6"
8"
10"
12"
14"
16"
18"
20"
0"
5"
10"
15"
20"
25"
30"
35"
28*Ap
r*12"
5*May*12
"
12*M
ay*12
"
19*M
ay*12
"
26*M
ay*12
"
2*Jun
*12"
9*Jun
*12"
16*Ju
n*12"
23*Ju
n*12"
30*Ju
n*12"
7*Jul*
12"
14*Ju
l*12"
21*Ju
l*12"
Num
ber'o
f'Bug
s'Fou
nd'
Num
ber'o
f'Tes
ts'fr
om'T
A'
Date'
Test'Advisor'Applica:on'in'Frontend'Project'
Tests"added"through"TA" Bugs"found"by"TA"tests"
29 19
Keil
32
Summit Developers
Developers Summit 2013 Kansai Action !
20112013 Coverity Connect End-to-End
0"
20"
40"
60"
80"
100"
120"
Alameda" Berkeley" Carmel" Davis" Today"
Pers
on'D
ays'
CC'Manual'Tes0ng'Eort'
0"
4"
8"
12"
16"
20"
Alameda" Berkeley" Carmel" Davis" Today"
Pers
on'D
ays'
E2E'Manual'Tes1ng'Eort'
Alameda' Berkeley' Carmel' Davis' Today'Manual'CC'GUI'Tests' 347' 661' 1006' 931' 1194'Automated'CC'GUI''Tests' 0' 2' 44' 220' 403'
0'
250'
500'
750'
1000'
1250'
1500'
1750'
CC"Test"Automa,on"Progress"
Alameda' Berkeley' Carmel' Davis' Today'Manual'E2E'Tests' 300' 370' 175' 143' 60'Automated'E2E'Tests' 0' 120' 1468' 1761' 3466'
0'
500'
1000'
1500'
2000'
2500'
3000'
3500'
4000'
E2E#Test#Automa-on#Progress#
33
Summit Developers
Developers Summit 2013 Kansai Action !
34
0"
0.2"
0.4"
0.6"
0.8"
1"
1.2"
1.4"
1.6"
Alameda" Berkeley" Carmel" Davis"
Nor
mal
ized
+Num
ber+
of+D
efec
ts+
Customer4found+Defects+
Summit Developers
Developers Summit 2013 Kansai Action !
() Coverity Test Advisor
35
Summit Developers
Developers Summit 2013 Kansai Action !
I suggest your Next AcTon!
Summit Developers
Developers Summit 2013 Kansai Action !
M Y R E C O M M E N D N E X T A C T I O N !
37
!
Summit Developers
Developers Summit 2013 Kansai Action !
CoverityQA/SA
Jenkins-CI
Coverity QA/SA/TA
CoverityPolicy
Manager QAVIP
ALM
Coverity (Coverity Connect)
QA
QNX