1

A System Call Analysis Method with MapReduce for Malware Detection

2011 IEEE 17th International Conference on Parallel and Distributed Systems

Shun-Te Liu *, Hui-ching Huang*

Information & Communication Security Lab TL, Chunghwa Telecom Co., Ltd.

Yi-Ming Chen

Department of Information ManagementNational Central University

102062602

黃建忠

/22

2

outline

Introduction Detect malware behavior Evaluation Conclution

/22

3

Malware by categories

/22

4

How to detect malware

Signature-based approach

Behavior-based approach

/22

/225

Behavior-based approach

Detect malware by real-time monitoring mechanisms

Ex: system call monitoring (procMon)

6

Malicious behavior patterns

Privacy invasion

Self-replication

Persistent behavior

/22

7

Mordern malware

Discrete behavior download malicious module

Module-base malware driver or DLL

/22

/228

requirements

the collected and analyzed data is much richer (system calls)

module dependency

/229

Client–server model

10

MapReduce

A programming model for processing large data sets with a parallel, distributed algorithm on a cluster

Apache Hadoop

/22

11

Persistent behavior

Malware

ASEP ( auto-start extensibility point)

Remain alive after system reboot

/22

12

ASEP(1)

Can be a file or registry keys

Ex: autorun.ini

/22

/2213

ASEP(2) HKLM\SOFTWARE\Microsoft\ Windows\CurrentVersion\Run

HKLM\SOFTWARE\Microsoft\ Windows NT\ CurrentVersion\

Winlogon\Notify (dll)

HKLM\System\CurrentControlSet\ (driver)

/2214

Persistent behavior module(1)

15

Persistent behavior module(2)

/22

16

Dependency Relationship(1)

ASEP is seen as a part of module white list filter

/22

17

Dependency Relationship(2)

Mi Mj

/22

18

Dependency structure matrix

Check diagonal cells

A B , B C , C A

/22

19

Accuracy

/22

20

Performance

/22

21

contribution

Propose a relation-based method to correlate the discrete behavior of malware.

Implement a prototype of Maltrix on the Hadoop platform.

/22

22

challenges

Some malwares don’t require ASEP

The cost of data transmission hasn't been measured.

Anti-api hooking

Without using system calls

/22