Date post: | 12-Jan-2016 |
Category: |
Documents |
Upload: | barbara-pearson |
View: | 218 times |
Download: | 0 times |
Mail-Filters Technical Presentation
How it works, Why it’s Better
Mail-Filter Technology Overview
• Why Mail-Filters• Bullet Signature Creation• Star Engine Process Overview• Implementation Options• SDK Contents• Getting Started• The API Commands• Testing Options• OEM Implementation Examples• FAQs
Why Mail-Filters
• It’s Fast – 100s of messages per second (or higher)
• It’s Accurate – over 95% of spam caught, less than 1 in 1,000,000 false positive rate
• Many implementation options – the right solution for any environment
• It’s Proprietary – it’s not fooled by spammer tricks - gives time to market and competitive differentiation
• It catches Foreign Language Spam – in over 30 languages – a worldwide solution
• Easy Implementation – usually less than a day• Full Support – Integration, technical support and training,
marketing materials, sales training and lead generation
How Mail-Filters Works
2. Human Editors Craft Bullet Signatures
1. Spam Collection occurs from many sources
3. Bullet Signatures Are Updated Every 1-15 Minutes
4. Mail-Filters Technology Integrated into OEM Solutions - Catches Spam, without False Positives
5. Tuning Users and Administrators provide feedback to help identify spam and those that send them.
Bullet Signature Creation
Mail-Filters’ Process Overview To Capture Spam & Create Bullet Signatures
wwwwww
Phish Trolling
QualityCheck
Translation Tools
Customer submissionsBullet
Signatures
Aristotle(Signature
Auto-Suggest)
Customer
Mail-FiltersTechnology on
Customer Device
SpamDB
PrioritizationProcess
International SpamHarvester
Partner Collections
Phish Traps
www
Traffic and ConnectionHeuristics
Spam Pre-Qualification
Partner Pre-QualificationExpert
Auto-NominateProcess
Bullet Signature Updater
Scam Sensors
Traffic Analysis
Pre-Qualified &Auto-Nominated
ReputationAnalysis
Human Editors
LanguageAssignment
Spammer Profile Creation
Data QualityManager
CullingEngine
Mail-Filters Data Centers
Message Profile Creation
Traffic Profiles
BulletSignatureUpdates
STAR Engine Server
ST
AR
Eng
ine
Man
agem
ent
Mod
ule
MessageNormalizer
SnowFlake Buster
Language Analyzer
Malformed MessageProcessor
Message Analysis
Traffic Analysis
Reputation Analysis
Spammer Profile Check
False PositiveRationalizer
BulletSignatures
Bullet SignatureUpdater
Mail-FiltersData Centers
Known Good Mail
Star Engine Process Overview
Star Engine Interface
OEMSoftware
Is Message Spam?
Yes / No
Implementation Options
• Enterprise– Most typical implementation – highest
performance – uses more resources
• Desktop– Small footprint – message is local – scan and
database is remote
• Embedded– Tiny amount of resources required – scanning
is done remotely
Star Engine – Enterprise(Very High Performance)
• Can process 100s or even over 1000 messages per second
• Requests Bullet Signature updates every 1-10 minutes (only changes are downloaded)
• The SEI and SES are typically deployed on the same hardware
• The SEI is linked into the OEM application using C or C++
• The SES runs as a Service or Daemon and it manages it’s own Database Updates
• The Database is usually between 3-10MB – will download a fresh DB upon startup if none present
OEM Application
Server or Appliance Hardware
Star Engine Interface (SEI)
Star Engine Server (SES)(Service or Daemon)
Linked Together by OEM at compile
C o
r C
++
AP
I
TC
P /
IP
Mail-FiltersData Centers
TC
P /
IP
Star Engine - Enterprise
• The Star Engine Server is fully multi-threaded• The Star Engine Server will run as a Service
under Windows or as a Daemon under Linux, FreeBSD, or Solaris
• TCP/IP outbound on Port 80 is required – IP proxies are supported
• Typical requirements are P4, 100MB RAM, Hard Disk optional
• A unique Mail-Filters Customer ID is required to download the Bullet Signature Database
Star Engine – Desktop(Small Footprint)
• Only requires 128kb of RAM• Can process 10s of messages
per second• Secondary server can be
anywhere, including and typically Mail-Filters’ Data Centers
• Database updates are not required on the SEI (just the SES)
• Same exact API as the Enterprise implementation
• Can also be used in a server cluster environment – many SEI’s feeding one SES
OEM Application
PC or Other Device(with limited resources)
Star Engine Interface
Linked Together by OEM at compile
C o
r C
++
AP
I
TC
P /
IP
Mail-FiltersData Centers
TC
P /
IP
Star Engine Server
Separate Server
Star Engine – EmbeddedA Completely New Approach
• Anti-Spam detection for edge devices with almost no resource requirements
• OEM code requires less than 10kb of RAM
• No software need be installed on any user PC – the service is turned on or off at the OEM device
• Works with POP3 & IMAP• OEM device intercepts the message
delivery request and sends it to Mail-Filters
• Mail-Filters receives the messages on behalf of the end user, filters for viruses and spam, then sends the clean messages to the end user
• OEM or customer determines what happens to spam (delete, mark with an X-header, decorate the subject line)
• Since spam can be deleted and the downlink speed is probably slower than the link from Mail-Filters’ data centers to the email servers – good mail will get to the end user faster.
WWW
PC
Mail-FiltersData Centers
Email Server
1. Email Client requests mail
2. OEM device intercepts the request based on port the request is made on (Ex. 110 = POP3) – and redirects the request to Mail-Filters’ data centers.
3, Mail-Filters makes the request on behalf of the user, filters the messages, then sends the good mail to the user. No mail is kept at Mail-Filters – it just passes through.
4. Mail-Filters’ authenticates as the user to the ISP or Corporate email servers - the mail is delivered
Embedded Architecture
OEM Application
OEM Device
Redirect Code
Outbound Listening Code (Port 110 for POP3 or Port 147 for IMAP Requests)
Customer Premise
The Internet
The Email Client requests email from an email server – it makes the request on port 110 or 147 – the OEM device redirects the request to Mail-Filters. A port is opened by the email server via Mail-Filters to the PC. The email is filtered, a policy is applied, then delivered to the Email Client.
Mail-FiltersData Centers
Email Server
Email Server PCs
SDK Contents
• Star Engine Server software executables
• Star Engine Interface libraries in C and C++
• Simple Single-Threaded implementation example application
• Documentation
• Typical integration time is less than a day
Getting Started with the SDK
• Install the Star Engine Server• Run the Star Engine Server• Run the Example Application
– This application will scan the files in the directory of choice and all sub-directories to see if they are spam. The results will display on the screen.
• Begin the Integration to the OEM application
The Star Engine API(The Star Engine Interface)
• The Commands are Straight-Forward– Initialize – This command establishes a connection to
the Star Engine Server– Shutdown – Used to tear down the thread after a
successful Initialize command– Scan SMTP Buffer – Passes the SES the data to be
scanned – will return TRUE if Spam– SCAN Buffer – Passes the SES data to be scanned –
best used for non-SMTP types of content such as IM, SMS, web pages, etc.
– Version – Returns the versions of all the components currently being used, including the database version date.
Testing Options• The Mail-Filters database is culled to eliminate old/unused signatures.
– As a result, the catch rate will suffer on old corpuses of email– Best results are obtained with live (or very close to it) email.
• There are several options to test the Mail-Filters technology– To test for catch rate or false positive rate
• Use the Example scan utility to check individual messages in a directory• Send mail to an account Mail-Filters can set up for you at Cleantree.com.
Good mail will go to the Inbox, spam to the Spam folder. Check results using your browser.
• Integrate into the OEM application and run it to check catch rate.– To test throughput:
• Unfortunately, the Example application is only a single-threaded application and will not show what the SES can achieve throughput-wise (it does fine on catch rate)
• The only fair test is to do an integration and run email through it. Most OEMs fine the solution throughput is the same whether Mail-Filters technology is running or not.
– To test Foreign Language:• Do a beta test with a customer or partner in the region of interest• Mail-Filters have several partners in various regions that may assist in a beta
test, if desired.
Implementation Examples
• Enterprise– Most OEMs have implemented the Mail-Filters
technology as the primary anti-spam solution• AV solutions company scans for spam while it has the message
in memory to scan for viruses. Because spam is more prevalent and is a much faster scan, spam is typically scanned for first.
– Some have augmented their own anti-spam technology• Because Mail-Filters technology is both fast and accurate, some
have used it as a pre-processor to their own, more computationally expensive technology, to increase the throughput of the overall solution, and to increase spam catch rates.
Implementation Examples
• Desktop– Some devices don’t have the processing power or
resources available for spam detection. For these, the Mail-Filters technology can provide a smaller footprint
• Firewalls, security gateways, messaging gateways, enterprise PCs may prefer a secondary server to handle the scanning to free up resources on their own hardware.
– An MSP has a cluster environment where there are many SEIs feeding one SES per tower. This is very efficient and allows their overall throughput to increase dramatically.
Implementation Examples
• Embedded– Ideal for DSL routers, Cable Modems, Wireless
gateways, SMB security gateways etc.– Because it requires no end user software
installation or configuration, it is simple to sign-up and have spam and viruses eliminated.
Frequently Asked Questions
• How do I get the SDK?– Sign the Mail-Filters MNDA and we’ll send it to you via email.
• Is the Star Engine Server multi-threaded?– Yes.
• Does it handle messages in double-byte character sets?– Yes, our technology catches spam in over 30 languages,
including multi-byte character sets such as Japanese, Korean, Chinese, Arabic, and Hebrew.
• How is the update interval set – can it be changed?– The update interval is set by the OEM, but can be changed on a
customer by customer basis. The default is an incremental every 10 minutes and a full update written to disk once a week.
• Will this solution work on less than a Pentium IV PC?– Yes, but it works more efficiently on a PIV.
Frequently Asked Questions• What happens if the SES can’t get a database, or quits running, or some
other catastrophe?– The SES or SEI will fail safe. It will return a FALSE ( the message isn’t spam)
and continue to process messages while trying to reconnect. The customer will see more missed spam, but won’t miss any messages.
• What if the SES doesn’t have the rights to write the database to disk, or the disk is full?
– The SES will continue to function properly and will acquire updates to the database in memory. The version command will return the database currently being used in RAM.
• Is the API really just 5 functions?– Yes – it doesn’t get much simpler than that.
• Can the SES return a probability of a message being spam?– No - Because the technology uses human editors to craft profiles and message
signatures, we’re very very confident the message is spam if we identify it. Because our false positive rate is so low, our methodology is proven to be correct. A probability is required by technologies that guess or compute whether a message is spam – we know it, so we tell you. For those solutions that require a probability, they set our TRUE response to the highest probability – 10 or 1 or 100.
Conclusions
• The Mail-Filters technology is easy to implement and provides options for any situation.
• The underlying technology far surpasses what others are doing, giving the Mail-Filters OEM a significant advantage over competitors in catch rate and accuracy, language coverage, and throughput.
• Human review provides the difference -the technology delivers it.