KAIST Internet Security Lab.

CS710

Behavioral Detection of Malwareon Mobile Handsets

MobiSys 2008, Abhijit Bose et al.

2008.12.11.

이 승 민

“Behavioral Detection of Malware…”-2/18-

CS710

IS Lab

Contents

IntroductionIntroduction11

System OverviewSystem Overview22

Malicious Behavior SignaturesMalicious Behavior Signatures33

Run-time ConstructionRun-time Construction44

EvaluationEvaluation55

ConclusionConclusion66

“Behavioral Detection of Malware…”-3/18-

CS710

IS Lab

1. Introduction

Behavior ?

“Behavioral Detection of Malware…”-4/18-

CS710

IS Lab

1. Introduction

Malware on mobile handsetsThe first mobile worm Cabir appeared in June 2004

By the end of 2006, the known number of mobile malware families and variants increased by 59% and 75% from year 2005

Differences in mobileLimited resources such as CPU, memory and battery

Difficulty of constructing network signature

Spreading via non-traditional vectors (SMS, Bluetooth)

Difference in OS (file permission, modification)

“Behavioral Detection of Malware…”-5/18-

CS710

IS Lab

1. Introduction

Related workNetwork based anomaly detection

Host based anomaly detection• Using consecutive system calls from normal app.• Rule learning, finite-state automata, Hidden Markov Model• But, it could be evaded by simple obfuscation

This paperMonitoring a program run-time behavior at a higher level

Run-time analysis

Using both normal and malware behaviors

“Behavioral Detection of Malware…”-6/18-

CS710

IS Lab

2. System Overview

SystemMonitor agent collects the application behavior in the form of system events/API calls

Aggregated behavior signatures are reported to the detection agent

“Behavioral Detection of Malware…”-7/18-

CS710

IS Lab

3. Malicious Behavior Signatures

Temporal patternsA logical ordering of the steps over time often clearly reveals the malicious intent

Example• Bluetooth OBEX system call (CObexClient::Put()) Harmless• Received file is of type .SIS & that file is later executed & the in-

staller process seeks to overwrite files in the system directory Mabir, Commwarrior

Behavior signatures are best specified using temporal logic in-stead of classical propositional logic

• TLCK (temporal logic of causal knowledge) language

“Behavioral Detection of Malware…”-8/18-

CS710

IS Lab

3. Malicious Behavior Signatures

Temporal logicSpecify malicious behavior in terms of system events, by tem-poral and logical operators

• : true at time t• : true at some instant before t• : true at all instants before t• : true at some instant in the interval [t-k, t]

“Behavioral Detection of Malware…”-9/18-

CS710

IS Lab

3. Malicious Behavior Signatures

Example: Commwarrior WormSymbian OS

Atomic variables are combined into seven higher-level signa-tures

“Behavioral Detection of Malware…”-10/18-

CS710

IS Lab

4. Run-Time Signature Construction

Monitoring API Calls using Proxy DLLProxy DLL intercepts and records details about the API call events from the application (with Symbian OS emulator)

“Behavioral Detection of Malware…”-11/18-

CS710

IS Lab

4. Run-Time Signature Construction

Stage I: Generation of dependency graph Dependency graph is constructed from logged API calls

“Behavioral Detection of Malware…”-12/18-

CS710

IS Lab

4. Run-Time Signature Construction

Stage II: Graph pruning and aggregationDependency graph grows over time

Pruning• The process did not have inter-process dependency relationships

with any other process• Its graph does not partially match with any malicious behavioral

signatures• It did not create or modify any file or directory• It is a helper process that takes input from a process and returns

data to the main process

Aggregation• Each API call is aggregated to reduce the size of the overall stor-

age

Construction of a behavior signature (TLCK)

“Behavioral Detection of Malware…”-13/18-

CS710

IS Lab

5. Evaluation

SVM classificationWhich of the separators is optimal ?

“Behavioral Detection of Malware…”-14/18-

CS710

IS Lab

5. Evaluation

Margin of the separator is the width of separation between classes

Maximizing the margin is good according to intuition

Examples closest to the hyperplane are support vectors

“Behavioral Detection of Malware…”-15/18-

CS710

IS Lab

5. Evaluation

MethodologyMonitoring agent is implemented in the Symbian OS Emulator

• OS dependent

8 applications• 5 worms: Cabir, Mabir, Lasco, Commwarrior, generic worm• 3 legitimate: OBEX file transfer, MMS client, MakeSIS

Detection agent uses SVM classifier• OS independent

“Behavioral Detection of Malware…”-16/18-

CS710

IS Lab

5. Evaluation

Accuracy of SVMDetection for known worms

SVM almost never falsely classifies a legitimate application signature to malicious

“Behavioral Detection of Malware…”-17/18-

CS710

IS Lab

5. Evaluation

Detection for unknown worms

When the training set contains 3 malware, detection is rela-tively high

“Behavioral Detection of Malware…”-18/18-

CS710

IS Lab

6. Conclusion

ContributionFirst attempt to construct a behavioral detection model for mobile environments

Define malicious behaviors with TLCK (temporal logic)

DiscussionWhat is the difference compared to wired network?

How about using HMM (Hidden Markov Model) in behavior de-tection?

Suitable for future research topic?