Post on 14-Jan-2016
transcript
1
Information Systems 2/26/03Tom CoppetoMark Silis
MIT Mail System
Update
26 February 2003
2
Information Systems 2/26/03Tom CoppetoMark Silis
Agenda
• Introduction to the mail system
• New Features
• Ongoing Work
3
Information Systems 2/26/03Tom CoppetoMark Silis
The Mail System
Mailhub
Internet
MIT Users
DMZ(MX mit.edu)
Outgoing
Post Office
Other MITMailers
4
Information Systems 2/26/03Tom CoppetoMark Silis
The Outgoing Mailers
• The first hop in all originating mail – A relay dedicated to application
response– Enforces @mit.edu sender addressing– Catches some mail anomalies– Will be the place where future
tightening occurs
Outgoing
Internet
MIT Mail System
MIT Users
5
Information Systems 2/26/03Tom CoppetoMark Silis
The mit.edu (DMZ) Mailers
• Provides queuing for external deliveries to keep the internal queues small
• The place outside mailers beat up• Contains an alias database (users & lists)
of 132,496 entries
DMZ
Internet
MIT Mail System
MIT Users
6
Information Systems 2/26/03Tom CoppetoMark Silis
The Mail Hub
• The core of the mail system• Creates a 3-tier system that optimizes for
local traffic and better insulates the system from Internet anomalies
• Also contains an aliases database
7
Information Systems 2/26/03Tom CoppetoMark Silis
The Post Offices
• Provides email storage• Accepts deliveries from the mailhub and
dmz • Supports pop/krb4, imap/krb4 and imap/ssl
• This has been the focal point of the recent email upgrades
8
Information Systems 2/26/03Tom CoppetoMark Silis
Post Office Upgrade
• quotas 250MB• max message size 25MB• new software
– Spam management– Auto response
9
Information Systems 2/26/03Tom CoppetoMark Silis
About Spam
• No one likes it• There’s no way to exactly know if a message is
or is not wanted• We can offer some heuristics that allow you to
decide what to do• We will not intentionally delete messages on
your behalf
• This requires processing the content of messages which is new and very expensive to do centrally
• With the software and hardware available today, we believe we can pull this off
10
Information Systems 2/26/03Tom CoppetoMark Silis
Spam Management
• New functionality added to post offices• Comprised of:
– spam scoring– spam filtering
• client side• server side
– automatic expunging of server side spam folders
11
Information Systems 2/26/03Tom CoppetoMark Silis
Spam Scoring
• header & body analysis– http://spamassassin.org/tests.html
• scores may be positive or negative– the higher the score, the more likely it is spam– more specifically, the more tests it failed– this is inexact
• score of 7.5 required to flag the message as spam– user configurable
12
Information Systems 2/26/03Tom CoppetoMark Silis
Spam Scoring
• header Subject includes "viagra" 2.896
• header Subject contains "Your Bills" or similar 0.7
• header Missing Date: header 1.37
• body Yahoo! Groups message -5.801
• header From: address is in the user's white-list -100
• body HTML has a low ratio of text to image area 1.101
13
Information Systems 2/26/03Tom CoppetoMark Silis
To: mark@mit.edu, tom@mit.edu From: "James D. Bruce" <jdb@MIT.EDU> Subject: important: please read X-Spam-Score: 9.5 X-Spam-Level: ********* (9.5) X-Spam-Flag: YES X-Scanned-By: MIMEDefang 2.28
Spam Scoring (2)
14
Information Systems 2/26/03Tom CoppetoMark Silis
Spam Scoring (3)
• http://nic.mit.edu/mail/spam– set scoring threshold– allow lists
• specify list of From: addresses that you never want to be flagged as spam
– deny lists• specify list of From: addresses that
you always want to be flagged as spam
15
Information Systems 2/26/03Tom CoppetoMark Silis
Spam Filtering - Client Side
• client side filtering– set up your client to filter on the
X-Spam-Flag field– set up your client to filter on the
X-Spam-Level field (requires no threshold configuration)
• see http://mit.edu/is/help/nospam
16
Information Systems 2/26/03Tom CoppetoMark Silis
Spam Filtering - Server Side
• You may elect to have messages flagged as spam filed to a separate IMAP folder
• Must be using IMAP to see the messages sent to this folder
• This folder counts as quota• You can also elect to have this folder
expunged after 14 days
17
Information Systems 2/26/03Tom CoppetoMark Silis
Spamscreen Web Page
18
Information Systems 2/26/03Tom CoppetoMark Silis
Spamscreen Web Page (2)
19
Information Systems 2/26/03Tom CoppetoMark Silis
Spamscreen Web Page (3)
20
Information Systems 2/26/03Tom CoppetoMark Silis
Automatic Response
• Delivery automatic replies upon delivery• Challenges are technical
– Don’t melt down mail system (will it happen? probably)
• and cultural– Not everyone thinks this is a feature
(spam filtering to the rescue!)
21
Information Systems 2/26/03Tom CoppetoMark Silis
Automatic Response
22
Information Systems 2/26/03Tom CoppetoMark Silis
SMTP Authentication
• MIT mail relays abused by spammers• Outgoing is a quasi-open relay• Need to further tighten outgoing to stop
this• The answer is SMTP authentication
• Only authorized users should be allowed to originate mail sent through the mail system
23
Information Systems 2/26/03Tom CoppetoMark Silis
Problems Under Full Load
• Deliveries outbound from spam filtering too inefficient under load (fixed)
• The new cyrus software dependent on a single process (some workarounds)
• The database cache size too small (fixed)
• Some messages may bypass scoring in the event of a process failure or the need to handle a backlog (we just need to get better at managing this)
24
Information Systems 2/26/03Tom CoppetoMark Silis
Statistics
MIT Mailhub Traffic
0
200,000
400,000
600,000
800,000
1,000,000
1,200,000
1,400,000
1,600,000
1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002
fiscal quarter
messages
incoming messagestotal recipients
25
Information Systems 2/26/03Tom CoppetoMark Silis
Statistics (2)
0
0.5
1
1.5
2
2.5
minutes
2000 2001 2002 2003
year (2nd week in February)
mailhub delivery times
26
Information Systems 2/26/03Tom CoppetoMark Silis
Summary of Changes
• Post office components upgraded• Quotas increased to 250MB• Max message size increased to 25MB• Spam scoring• Optional spam re-filing• Auto responder• Web based controls of spam and auto
response
27
Information Systems 2/26/03Tom CoppetoMark Silis
• Stabilize the system• Rollout spam and responder functionality• Solidify recommended email clients• Upgrade the hardware/software on the outgoing, mailhub and
dmz mailers• Enable SMTP authentication on outgoing mailers• Simplify the system by centralizing the spam scoring• Look at new features
Next Steps