an anti-spam method with smtp session abort nariyoshi yamai 1 kiyohiko okayama 1 takumi seike 1...

An Anti-Spam Method with SMTP Session Abort

Nariyoshi YAMAI1 Kiyohiko OKAYAMA1 Takumi SEIKE1

Keita KAWANO1 Motonori NAKAMURA2 Shin MARUYAMA3

1 Okayama University, Japan 　2 National Institute of Informatics, Japan

3 CO-CONV Corporation, Japan

Existing anti-spam methods

2008/3/27 MIT Spam Conference 20083


Tempfailing (1) Utilizes difference of MTA behavior

after temporary error– Legitimate MTAs

Retry to send the temporarily failed messages

– Spam sending MTAs Prefer throughput Give up resending the temporarily failed

messages


FirstDeliverySecond delivery

Tempfailing (2)

Spam sendingMTA

LegitimateMTA

temporary error MTA

temporary error

Recipients

retry

Saves triplet( Sender IP, SMTP From, SMTP To)

Sender IPSender IPSMTP FromSMTP From

SMTP ToSMTP To

Sender IPSender IPSMTP FromSMTP From

SMTP ToSMTP To


Tempfailing (3)

Problems– RFC2821:

4.5.4.1 Sending Strategy (excerpt)

The sender MUST delay retrying a particular

destination after one attempt has failed. In

general, the retry interval SHOULD be at least 30

minutes.Causes large delay

for legitimate mail delivery


Tempfailing (4)

Problems (cont.)– Utilizes the following triplet for

retransmission judgment: Sender IP SMTP From SMTP To

Rejects retries from a different MTA


Tempfailing (5)

Problems (cont.)– Rejects before receiving header/body

– Logs only the triplet

(Sender IP, SMTP From, SMTP To)

Difficult to recover false positives


Distributed collaborative filter

MTASpam sendingMTA

Recipients

Spamdatabase

checknot

foundspamregisterfound

Only messages already read by existent recipients can be filtered out

Anti-spam method withSMTP session abort



Summary of known problems (Tempfailing) Large delay

(Tempfailing) Retries from a different MTA

(Tempfailing) Recovery from false positives

(Distributed collaborative filter)

only messages read by recipients into DB


Features of the proposed method (Tempfailing) Large delay

(Tempfailing) Retries from a different MTA

(Tempfailing) Recovery from false positives

(Distributed collaborative filter)

only messages read by recipients into DB

Introducing two mail gateways (MGs)Immediate fallback to the secondary MG

SMTP session abort functionPreserving header/body on first attemptRetransmission judgment with Message-ID or checksum instead of IP

Automatic registration of

unresent/undeliverable messagesEarly registration of many spam mails


System layout and behavior (1)

Organization

InsideMTA

Recipients

Spamdatabase

Primarymail

gateway

Secondarymail

gateway

Mailgateway

×TCP segment

(RST)SMTP session abort

After SMTP session to the primary MG is aborted, a legitimate MTA usually sends the message to the secondary MG immediately.

Retry

Reducing delay of legitimate mail delivery

headerheaderbodybody PreservingPreserving

header/bodyheader/body

Check tripletCheck triplet(MsgID/checksum,(MsgID/checksum,SMTP From,SMTP From,SMTP To)SMTP To)

Retransmission judgment based on header(MsgID) or body(checksum)

headerheaderbodybody

Sender MTA

Preserving header/body in case of false positive


System layout and behavior (2-1)

Organization

InsideMTA

Recipients

Spamdatabase

Primarymail

gateway

Secondarymail

gateway

Spam sendingMTA

undeliverable

RCPT TO

recipient check

Unknown recipient

register

headerheaderbodybody×

SMTP session abort



System layout and behavior (2-2)

Organization

InsideMTA

Recipients

Spamdatabase

Primarymail

gateway

Secondarymail

gateway

formerlydeliverable

RCPT TO

Recipient check

Unknown recipient

register

headerheaderbodybody×

SMTP session abort


Recipient checkheaderheaderbodybody

cancel

RCPT TO

Automatic registration of unresent/undeliverable messages

Sender MTA

User preference of abort timing (1) Affects network traffic and delay Possible options

– Accept No session abort

– Header Abort after End of Header Low traffic/delay

– Body Abort after End of Message Easy recovery on false positives


16


User preference of abort timing (2)

Organization

InsideMTA

A

Spamdatabase

Primarymail

gateway

Secondarymail

gateway

RCPT TO: ARCPT TO: BRCPT TO: C

RCPT TO: A×SMTP session abortat end of message

RCPT TO: BRCPT TO: C

RCPT TO: ARCPT TO: BRCPT TO: C

Sender MTA

accept

B C

header body


Implementation and evaluation of prototype system



Prototype system implementation Platform

– FreeBSD with sendmail & DCC SMTP session abort function

– An external program using “ipfw” Retransmission judgment

– (Message-ID, SMTP From, SMTP To)


First operation test (1) Objectives

– Performance evaluation of blocking/filtering Test domains

– Some sub-domains in okayama-u.ac.jp– Already obsolete five years before– To be removed in one month– Some legitimate mails were possibly sent to

these domains Test period

– Seven days from Jan. 29 to Feb. 5th, 2006


First operation test (2) Result

Number of mails processed 54,719

Number of mails blocked 44,303

Number of mails received 10,416

Number of mails filtered out by DCC 2,180

81% (44303/54719) of mails processed were blocked by SMTP session abort

20% (2180/10416) of mails received were filtered out by DCC

NB: we counted both legitimate mails and spam mails.


Second operation test (1) Objectives

– Comparison with conventional tempfailing as for processing of legitimate mails

Test domain– New sub-domain dedicated for this test– Only 1 IP address available

Two MGs have the same IP address Usual in small companies in Japan


Second operation test (2) Result

Domain (service) MTA Resend Different MTA Min. interval

cc.okayama-u.ac.jp (Univ.) sendmail YES NO 0 (sec)

nifty.com (ISP) sendmail YES NO 1

listbox.com (ML) postfix YES NO 1

yahoo.com (free mail) ? YES NO 10

gmail.com (free mail) ? YES YES 385

aol.com (free mail) ? YES NO 6

hotmail.com (free mail) SMTPSVC YES NO 6

yahoogroups.jp (free ML) ? YES NO 1

freeml.com (free ML) qmail YES NO 399

mag2.com (mail magazine) qmail YES NO 3264

trashmail.net (anonymous mail) postfix YES NO 6

All messages even from gmail.com were accepted without whitelist

Small delays of mail delivery from many domains

Some domains using qmail still had large delays


Possible false positives Messages without Message-ID

– Use Date: field (mandatory), or– Use the checksum of the body

MTAs without retransmission– Can recover lost headers/bodies easily– Find such MTAs and register them into

whitelist MTAs changing SMTP From address

– Use (Message-ID, SMTP To) without SMTP From for retransmission judgment

Conclusions



Conclusions Combination of three functions

– Tempfailing– Distributed Collaborative filter

– SMPT session abort

Reduces the drawbacks of existing two methods

Future works– Long term actual performance evaluation

– Combination with on-the-fly filters

Questions ?Please speak slowly and clearly


an anti-spam method with smtp session abort nariyoshi yamai 1 kiyohiko okayama 1 takumi seike 1...

Documents