a spam mail-based solution for botnet detection and network bandwidth protection
DESCRIPTION
A Spam Mail-based Solution for Botnet Detection and Network Bandwidth Protection. 許富皓 資訊工程學系 中央大學. 1. Outline. Introduction Background System Design Work Flow Evaluation Related Work Conclusion. 2. Outline. Introduction Background System Design Work Flow Evaluation Related Work - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: A Spam Mail-based Solution for Botnet Detection and Network Bandwidth Protection](https://reader036.vdocuments.pub/reader036/viewer/2022062305/56816397550346895dd48f95/html5/thumbnails/1.jpg)
1
A Spam Mail-based Solution for Botnet Detection and Network Bandwidth Protection
許富皓資訊工程學系中央大學1
![Page 2: A Spam Mail-based Solution for Botnet Detection and Network Bandwidth Protection](https://reader036.vdocuments.pub/reader036/viewer/2022062305/56816397550346895dd48f95/html5/thumbnails/2.jpg)
2
Outline Introduction Background System Design Work Flow Evaluation Related Work Conclusion
2
![Page 3: A Spam Mail-based Solution for Botnet Detection and Network Bandwidth Protection](https://reader036.vdocuments.pub/reader036/viewer/2022062305/56816397550346895dd48f95/html5/thumbnails/3.jpg)
3
Outline Introduction Background System Design Work Flow Evaluation Related Work Conclusion
3
![Page 4: A Spam Mail-based Solution for Botnet Detection and Network Bandwidth Protection](https://reader036.vdocuments.pub/reader036/viewer/2022062305/56816397550346895dd48f95/html5/thumbnails/4.jpg)
4
Spam Mails and Bots At 2009, research shows that more
than 80% spam mails are sent by the bots, called spam bots hereafter, of botnets.
Spam mails take up more than 50% of network bandwidth.
4
![Page 5: A Spam Mail-based Solution for Botnet Detection and Network Bandwidth Protection](https://reader036.vdocuments.pub/reader036/viewer/2022062305/56816397550346895dd48f95/html5/thumbnails/5.jpg)
5
Objectives Detect members of botnet Filter spam mails Save network bandwidth
5
![Page 6: A Spam Mail-based Solution for Botnet Detection and Network Bandwidth Protection](https://reader036.vdocuments.pub/reader036/viewer/2022062305/56816397550346895dd48f95/html5/thumbnails/6.jpg)
6
Observation As our observation, the majority of
spam bots are not e-mail servers, spam bots usually only send mails but do not receive mails.
6
![Page 7: A Spam Mail-based Solution for Botnet Detection and Network Bandwidth Protection](https://reader036.vdocuments.pub/reader036/viewer/2022062305/56816397550346895dd48f95/html5/thumbnails/7.jpg)
7
Outline Introduction Background System Design Work Flow Evaluation Related Work Conclusion
7
![Page 8: A Spam Mail-based Solution for Botnet Detection and Network Bandwidth Protection](https://reader036.vdocuments.pub/reader036/viewer/2022062305/56816397550346895dd48f95/html5/thumbnails/8.jpg)
8
E-mail Architecture
8
![Page 9: A Spam Mail-based Solution for Botnet Detection and Network Bandwidth Protection](https://reader036.vdocuments.pub/reader036/viewer/2022062305/56816397550346895dd48f95/html5/thumbnails/9.jpg)
9
Botnets & Spam Mails
9
![Page 10: A Spam Mail-based Solution for Botnet Detection and Network Bandwidth Protection](https://reader036.vdocuments.pub/reader036/viewer/2022062305/56816397550346895dd48f95/html5/thumbnails/10.jpg)
10
Outline Introduction Background System Design Work Flow Evaluation Related Work Conclusion
10
![Page 11: A Spam Mail-based Solution for Botnet Detection and Network Bandwidth Protection](https://reader036.vdocuments.pub/reader036/viewer/2022062305/56816397550346895dd48f95/html5/thumbnails/11.jpg)
11
System Layout
confirmation hostconfirm
redirect/blockHoneypot
11
SMTP, POP3, IMAP SMTP, POP3, IMAP
Packet Analyzer
Mail ServerEnd users
![Page 12: A Spam Mail-based Solution for Botnet Detection and Network Bandwidth Protection](https://reader036.vdocuments.pub/reader036/viewer/2022062305/56816397550346895dd48f95/html5/thumbnails/12.jpg)
12
System Component – Packet Analyzer (PA) (1)
Located at a router Detect spam bots based on the IP ack-
packets which use SMTP(S), POP3(S), or IMAP(S)
protocol and whose sizes are less than 200 bytes (describe
later) Use credit number to record mail
transmission status of an IP address.12
![Page 13: A Spam Mail-based Solution for Botnet Detection and Network Bandwidth Protection](https://reader036.vdocuments.pub/reader036/viewer/2022062305/56816397550346895dd48f95/html5/thumbnails/13.jpg)
13
IP Threat Level Table (IPTLT)
IP Credit Action
![Page 14: A Spam Mail-based Solution for Botnet Detection and Network Bandwidth Protection](https://reader036.vdocuments.pub/reader036/viewer/2022062305/56816397550346895dd48f95/html5/thumbnails/14.jpg)
14
System Component – Packet Analyzer (PA) (2)
Clean IPTLT periodically to solve the problem of dynamic allocated IPs (such as, DHCP).
Add an NAT detection mechanism to avoid harming innocent hosts behind an NAT.
14
![Page 15: A Spam Mail-based Solution for Botnet Detection and Network Bandwidth Protection](https://reader036.vdocuments.pub/reader036/viewer/2022062305/56816397550346895dd48f95/html5/thumbnails/15.jpg)
15
Credit Number Credit Number is a property of an IP
address. PA assigns a credit number to every
IP address which has appeared in a mail packet (SMTP/POP/IMAP) as the source IP address.
15
![Page 16: A Spam Mail-based Solution for Botnet Detection and Network Bandwidth Protection](https://reader036.vdocuments.pub/reader036/viewer/2022062305/56816397550346895dd48f95/html5/thumbnails/16.jpg)
16
Operations of Credit Number
Increasing operation When PA detects a SMTP mail packet,
the credit number of the source IP of the mail packet will be increased by 1.
Decreasing operation When PA detects a POP/IMAP mail
packet, the credit number of the source IP of the mail packet will be decreased by a higher value, 3.
P.S.: By analyzing real world traffic, in a network the ratio of sending mails to receiving mails is 1:3.
16
![Page 17: A Spam Mail-based Solution for Botnet Detection and Network Bandwidth Protection](https://reader036.vdocuments.pub/reader036/viewer/2022062305/56816397550346895dd48f95/html5/thumbnails/17.jpg)
17
Approach to Reduce Packet Analyzer Performance Overhead
Sampling A router solution should avoid high
performance overhead; hence, Packet Analyzer can use sampling to reduce the performance overhead of packet analyzer.
The sample rate is an adjustable parameter.
17
![Page 18: A Spam Mail-based Solution for Botnet Detection and Network Bandwidth Protection](https://reader036.vdocuments.pub/reader036/viewer/2022062305/56816397550346895dd48f95/html5/thumbnails/18.jpg)
18
Avoid Noise Created by Large Size Mails
No matter what size a mail has, the number of protocol related packets exchanged between the sender and receiver is similar to each other.
The sizes of protocol related packets are usually smaller than 200 bytes.
To avoid counting large size mails sending by normal users more times, we filter out e-mail related packets with size larger than 200 bytes. 18
![Page 19: A Spam Mail-based Solution for Botnet Detection and Network Bandwidth Protection](https://reader036.vdocuments.pub/reader036/viewer/2022062305/56816397550346895dd48f95/html5/thumbnails/19.jpg)
19
System Component – Confirmer (1)
Located at a confirmation host. Check if a host is a mail server
because a mail server may have the same behavior as a bot.
By connecting to SMTP port of a host to check whether it is a mail server.
19
![Page 20: A Spam Mail-based Solution for Botnet Detection and Network Bandwidth Protection](https://reader036.vdocuments.pub/reader036/viewer/2022062305/56816397550346895dd48f95/html5/thumbnails/20.jpg)
20
System Component – Confirmer (2)
The record that an IP is used by a confirmed host is kept in the IPTLT until the IPTLT is cleaned up; hence, the IP is only needed to be confirmed once before the IPTLT is cleaned up.
20
![Page 21: A Spam Mail-based Solution for Botnet Detection and Network Bandwidth Protection](https://reader036.vdocuments.pub/reader036/viewer/2022062305/56816397550346895dd48f95/html5/thumbnails/21.jpg)
21
Outline Introduction Background System Design Work Flow Evaluation Related Work Conclusion
21
![Page 22: A Spam Mail-based Solution for Botnet Detection and Network Bandwidth Protection](https://reader036.vdocuments.pub/reader036/viewer/2022062305/56816397550346895dd48f95/html5/thumbnails/22.jpg)
22
Work Flow
IP Credit
Action
22
Packet Analyzer
NetFilterPREROUTIN
G
Kernel SpaceIP threat level
table
Kernel thread
Packets
e-mail relate
d traffic
Suspect IP
Clean up periodical
ly
Confirmer
Suspect IP
Confirmation Host
Check result
Check SMTP
Fetch action
Accept / Drop
Suspect Host
Fill action field
Linux Router
![Page 23: A Spam Mail-based Solution for Botnet Detection and Network Bandwidth Protection](https://reader036.vdocuments.pub/reader036/viewer/2022062305/56816397550346895dd48f95/html5/thumbnails/23.jpg)
23
Outline Introduction Background System Design Work Flow Evaluation Related Work Conclusion
23
![Page 24: A Spam Mail-based Solution for Botnet Detection and Network Bandwidth Protection](https://reader036.vdocuments.pub/reader036/viewer/2022062305/56816397550346895dd48f95/html5/thumbnails/24.jpg)
24
Performance Evaluation Scenario
Send 10000 mails. Mail size: 3 KB. Transmitting mails through the router with or without
SpamFinder.
24
SMTP serverSpamFinder
End user (E-mail client)
Host: ASUS Desktop AS-D672CPU: Intel Pentium 4 Dual Core 3.2 GHzRAM: 4GLAN: Gigabit Ethernet NICOS: Windows 7
Host: ASUS Desktop AS-D360CPU: Intel Pentium 4 3.0 GHzRAM: 512 MBLAN1: 10 Mb/100 Mb Ethernet ControllerLAN2: 10 Mb/100 Mb Ethernet ControllerOS: Fedora 10, kernel 2.6.27
![Page 25: A Spam Mail-based Solution for Botnet Detection and Network Bandwidth Protection](https://reader036.vdocuments.pub/reader036/viewer/2022062305/56816397550346895dd48f95/html5/thumbnails/25.jpg)
25
Performance Evaluation Evaluation Result
Overhead, O(n%) n: sample rate
O(100%) = 4.13 %
O(0.2%) = 3.8 %
25
avg time to send mail0.1
0.105
0.11
0.115
0.12
0.125
0.13
0.135
Performance Evaluation
![Page 26: A Spam Mail-based Solution for Botnet Detection and Network Bandwidth Protection](https://reader036.vdocuments.pub/reader036/viewer/2022062305/56816397550346895dd48f95/html5/thumbnails/26.jpg)
26
Effectiveness Evaluation Scenario
Analyze the real world traffic (about 1300 computers of NCTU dorm network) offered by NBL (Network Benchmarking Lab)@NCTU
Analyze the whole day traffic of 6/13/2010 (about 2TB)
Replay traffic (250 ~ 350 Mb/s)
26
Real world traffic replay SpamFinde
rTraffic logs
Host: HP CQ-45 NotebookCPU: Intel Core 2 Duo P7450 / 2.13 GHzRAM: 4GLAN: 10/100/1000 Gigabit Ethernet LANOS: Fedora 12 kernel 2.6.32
![Page 27: A Spam Mail-based Solution for Botnet Detection and Network Bandwidth Protection](https://reader036.vdocuments.pub/reader036/viewer/2022062305/56816397550346895dd48f95/html5/thumbnails/27.jpg)
27
Effective Evaluation According to the result of analyze,
we get the follows information: The rate of sending and receiving data
is 1:3 With credit threshold = 150,
SpamFinder can save 25% e-mail related traffic
Average packet dropped ratio : 0.31 % NBL uses CISCO 7609 router to collect
packet traces. We use a notebook to make our analysis.
27
![Page 28: A Spam Mail-based Solution for Botnet Detection and Network Bandwidth Protection](https://reader036.vdocuments.pub/reader036/viewer/2022062305/56816397550346895dd48f95/html5/thumbnails/28.jpg)
28
Effective Evaluation According to the result of analses,
we get the follows information: SpamFinder detect 2 spam bots after
analyzing 1 day traffic of NCTU dorm network
P.S.: that according to tyc.edu.tw reports: in average in the NCU campus there are 4.1 hosts per day be reported as spam hosts.
28
![Page 29: A Spam Mail-based Solution for Botnet Detection and Network Bandwidth Protection](https://reader036.vdocuments.pub/reader036/viewer/2022062305/56816397550346895dd48f95/html5/thumbnails/29.jpg)
29
Effective Evaluation
29
150 200 300 450 65023.6%
23.8%
24.0%
24.2%
24.4%
24.6%
24.8%
25.0%
25.2%
save e-mail related traffic
credit number threshold
Save
rat
io o
f e-
mai
l rel
ated
tra
ffic
![Page 30: A Spam Mail-based Solution for Botnet Detection and Network Bandwidth Protection](https://reader036.vdocuments.pub/reader036/viewer/2022062305/56816397550346895dd48f95/html5/thumbnails/30.jpg)
30
Effective Evaluation
30
[5068] ip: 140.#.#.135, credit: 18147, nat: 0, mail server: 0, action: 0
。。。Jun 17 14:11:51 [SEND] 140.#.#.135:4552 -> 74.125.157.27:25, total_len:1470Jun 17 14:11:51 [SEND] 140.#.#.135:4552 -> 74.125.157.27:25, total_len:1470Jun 17 14:11:51 [SEND] 140.#.#.135:4552 -> 74.125.157.27:25, total_len:1326Jun 17 14:11:52 [SEND] 140.#.#.135:4839->165.131.174.40:25, total_len:1500Jun 17 14:11:52 [SEND] 140.#.#.135:4839->165.131.174.40:25, total_len:1500Jun 17 14:11:52 [SEND] 140.#.#.135:4839->165.131.174.40:25, total_len:896Jun 17 14:11:53 [SEND] 140.#.#.135:4832 -> 74.86.7.196:25, total_len:1500Jun 17 14:11:53 [SEND] 140.#.#.135:4832 -> 74.86.7.196:25, total_len:1500Jun 17 14:11:53 [SEND] 140.#.#.135:4832 -> 74.86.7.196:25, total_len:811 。。。 repeat this action 3628 times
![Page 31: A Spam Mail-based Solution for Botnet Detection and Network Bandwidth Protection](https://reader036.vdocuments.pub/reader036/viewer/2022062305/56816397550346895dd48f95/html5/thumbnails/31.jpg)
31
Effective Evaluation
31
[3370] ip: 140.#.#.148, credit: 8203, nat: 0, mail server: 0, action: 0
。。。Jun 14 16:24:14 [SEND] 140.#.#.148:6508 -> 148.123.15.75:25, total_len:1500Jun 14 16:24:14 [SEND] 140.#.#.148:6508 -> 148.123.15.75:25, total_len:1500Jun 14 16:24:14 [SEND] 140.#.#.148:6508 -> 148.123.15.75:25, total_len:1142Jun 14 16:24:17 [SEND] 140.#.#.148:6534->75.126.136.141:25, total_len:1500Jun 14 16:24:17 [SEND] 140.#.#.148:6534->75.126.136.141:25, total_len:1500Jun 14 16:24:17 [SEND] 140.#.#.148:6534->75.126.136.141:25, total_len:1500Jun 14 16:24:17 [SEND] 140.#.#.148:6526 -> 74.125.43.27:25, total_len:1470Jun 14 16:24:17 [SEND] 140.#.#.148:6526 -> 74.125.43.27:25, total_len:1470Jun 14 16:24:17 [SEND] 140.#.#.148:6526 -> 74.125.43.27:25, total_len:1470 。。。 repeat this action 2050 times
![Page 32: A Spam Mail-based Solution for Botnet Detection and Network Bandwidth Protection](https://reader036.vdocuments.pub/reader036/viewer/2022062305/56816397550346895dd48f95/html5/thumbnails/32.jpg)
32
Outline Introduction Background System Design Work Flow Evaluation Related Work Conclusion
32
![Page 33: A Spam Mail-based Solution for Botnet Detection and Network Bandwidth Protection](https://reader036.vdocuments.pub/reader036/viewer/2022062305/56816397550346895dd48f95/html5/thumbnails/33.jpg)
33
Related Work BotGraph, Large Scale Spamming
Botnet Detection, USENIX’09 Webmail botnet account detection
BotMiner, Clustering Analysis of Network Traffic for Protocol- and Structure-Independent Botnet Detection, USENIX’08 Network behavior based detection
Wide-scale botnet detection and characterization, HotBots’07, USENIX
33
![Page 34: A Spam Mail-based Solution for Botnet Detection and Network Bandwidth Protection](https://reader036.vdocuments.pub/reader036/viewer/2022062305/56816397550346895dd48f95/html5/thumbnails/34.jpg)
34
Outline Introduction Background System Design Work Flow Evaluation Related Work Conclusion
34
![Page 35: A Spam Mail-based Solution for Botnet Detection and Network Bandwidth Protection](https://reader036.vdocuments.pub/reader036/viewer/2022062305/56816397550346895dd48f95/html5/thumbnails/35.jpg)
35
Limitation If the e-mail sending traffic passes
through the router, but the e-mail receiving traffic doesn’t, then the host would be considered as a spam bot.
SpamFinder cannot detect e-mails that are sent and received through a Webmail, but popular web mail services have their effective anti-spam mechanism to filter spam mails.35
![Page 36: A Spam Mail-based Solution for Botnet Detection and Network Bandwidth Protection](https://reader036.vdocuments.pub/reader036/viewer/2022062305/56816397550346895dd48f95/html5/thumbnails/36.jpg)
36
Attack Analysis and Future Work
Attackers might send fake IP packets to defame some target hosts, we could check the existence of related connections to detect these behavior.
In the future, the spam mails (or IP packets) sent from bots will be redirected to a honeypot for further analysis. 36
![Page 37: A Spam Mail-based Solution for Botnet Detection and Network Bandwidth Protection](https://reader036.vdocuments.pub/reader036/viewer/2022062305/56816397550346895dd48f95/html5/thumbnails/37.jpg)
37
Conclusion We propose a network level spam bot
detection mechanism, SpamFinder Implement it on a Linux router and make
evaluations using real world traffic that offered by NBL(Network Benchmarking Lab)@NCTU
The evaluation result show that SpamFinder has low performance overhead and could detect spam bots and protect network bandwidth effectively
37
![Page 38: A Spam Mail-based Solution for Botnet Detection and Network Bandwidth Protection](https://reader036.vdocuments.pub/reader036/viewer/2022062305/56816397550346895dd48f95/html5/thumbnails/38.jpg)
38
End Q&A
38