1

Model Checking One Million Lines of C Code

Hao Chen, UC BerkeleyDrew Dean, SRI InternationalDavid Wagner, UC Berkeley

2

MOPS (MOdel checking Programs for Security properties)

• A static analysis tool that checks source programs for temporal safety properties.e.g. a setuid-root program must drop privilege before making risky system calls.

• Analysis– Pushdown model checking– Inter-procedural– Control flow centric

3

The MOPS process

Parser ModelChecker

C Program

SafetyProperty

CFG

FSA Program satisifessafety property

Error TracesFSA: finite state automatonCFG: control flow graph

Treat the model checker as a black box for this talk

4

Is software model checking readyfor prime time?

• Can model checking be used by open source developers to find security vulnerabilities?

• Criteria for a successful tool– It is useful

• Can check many properties• Can check diverse, widely-deployed programs• Requires moderate computational resources

– It is usable• Can be used easily by non-tool developers• Can generate comprehensible error reports

5

Outline

• Experiment– Programs: 8 widely-deployed programs, with over

1 million LOC– Properties: 5 security-related properties

• Findings– More than a dozen vulnerabilities and weaknesses

• Usability improvements• Conclusion

6

Programs

Program Lines of Code (LOC)Apache HTTPD 2.0.40-21 229K

At 3.1.8-33 6K

BIND 9.2.1-16 279K

OpenSSH 3.5p1-6 59K

Postfix 1.1.11-11 94K

Samba 2.2.7a-7.9.0 254K

Sendmail 8.12.8-4 222K

VixieCron 3.0.1-74 4K

Total 1147K

7

Security properties

• Drop privilege completely when needed• Avoid stderr vulnerability • Avoid race condition (TOCTTOU)• Create chroot-jail safely

– chdir(“/”) must follow chroot() immediately• Create temporary files safely

– Use only the safe function mkstemp()– Never reuse filename in mkstemp(filename)

8

Property: drop privilege completely

• Setuid-root programs should drop root privilege completely– before executing an untrusted program via

system(), popen(), execvp() and friends, or– when the program intends to do so

• Otherwise, the remaining privilege may be exploited by– the untrusted program that is executed– malicious code injected via buffer overrun attacks

9

Vulnerability: fail to drop privilege completely

seteuid(getuid());setuid(getuid());…execlp(askpass, askpass, msg, (char *) 0);…

OpenSSH client(in readpass.c)

10

What is wrong?

R≠0, E=S=0

OpenSSH 3.5 on Linux

R=E≠0, S=0

R=E≠0, S=0

seteuid(getuid())

setuid(getuid())

R≠0, E=S=0

OpenSSH 3.5 on OpenBSD

R=E≠0, S=0

R=E=S≠0

seteuid(getuid())

setuid(getuid())

R≠0, E=S=0

OpenSSH 2.5.2 on Linux

R=E=S≠0

setuid(getuid())

11

Potential Vulnerability

• Weaknesses– ssh: fails to drop privilege before executing a user

program– ssh-keysign: fails to drop privilege before doing

complex cryptographic operations• A buffer overrun would allow the attacker to

regain root privilege in euid.

12

Property: drop privilege completelyPackage LOC Running

Time# Error Traces

Real Bugs Total

Sendmail 222K 0:12 0 0Postfix 94K 0:17 0 2OpenSSH 59K 0:23 2 8Apache 229K 0:45 1 4BIND 279K 0:53 0 1At 6K 0:05 0 0Cron 4K 0:05 0 0Samba 254K 1:53 0 5

13

Vulnerability: stderr exploits in at

attack.c

at.c

Code Standard File Descriptorsstdin stdout stderr

close(1); close(2);

execl(“at”, …);

open(LFILE, O_WRONLY);

fd = open(atfile, O_CREAT);

tty tty tty

tty <closed> <closed>

tty <closed> <closed>

tty LFILE <closed>

tty LFILE atfile

Rule: No setuid-root program may open a file for writing to stderr

14

Property: stderr vulnerabilityPackage LOC Running

Time# Error Traces

Real Bugs Total

Sendmail 222K 14:12 0 3Postfix 94K 0:46 0 1OpenSSH 59K 0:58 1 2Apache 229K 0:14 1 1BIND 279K 0:00 0 0At 6K 0:04 1 1Cron 4K 0:05 2 2Samba 254K 0:58 1 1

15

Summary of Findings

Program Errors (All Properties)Real Total

Apache HTTPD 2 6

At 1 7

BIND 0 4

OpenSSH 5 24

Postfix 0 6

Samba 2 8

Sendmail 0 11

VixieCron 3 4

Total 13 70

16

Outline

• Experiment– Programs: 8 widely-deployed programs, with over

1 million LOC– Properties: 5 security-related properties

• Findings– More than a dozen vulnerabilities and weaknesses

• Usability improvements• Conclusion

17

Usability improvement 1:Make it really easy to run!

• Problems– Packages have different build processes– Tool has to be manually configured for each

package• Solution

– Provide a script that integrates model checking into the build processes of packages automatically

– Result: allow the user to run the tool as simple asmops –m setuid.fsa openssh-3.5p1-6.src.rpm

18

Integrating MOPS intoSoftware Build Processes

• 1st attempt: manually edit Makefiles– Too complicated; does not survive autoconf

• 2nd attempt: setenv GCC_EXEC_PREFIX to run MOPS instead of gcc– Build processes generate & run code

• 3rd attempt: build CFG & machine code– Dangling CFGs; links to object files broken

• 4th attempt: Put CFGs into ELF files– Solves all identified problems!

19

Usability improvement 2:report comprehensible error messages

• Problem– One bug may trigger many error traces– The user has to review all the traces manually

• Criteria for good error trace reporting– Reporting one error trace per bug– Reporting shortest error traces

20

Algorithm

1. Find the shortest error trace t and output it2. Find the crucial statement s on t, i.e.

the first statement that causes an error on t3. Prune s from the program4. If the program still has error traces, go to

step 1

21

Criteria for good tools: revisited

• It is useful– Can check many properties– Can check diverse, widely-deployed programs– Requires moderate computational resources

• It is usable– Can be used easily by non-tool developers– Can generate comprehensible error reports

22

Conclusion

• Model checking is ready for prime time use by open source developers to find security vulnerabilities!

• We believe that our experience would transfer to other similar tools as well.

• Work in progress: check all 839 RPM packages in RedHat Linux 9