Post on 30-Nov-2014
description
transcript
Lessons Learned From the Evolution of Spam Tips to Staying Ahead of the Email Deliverability Curve
Speakers: Autumn Tyr-Salvia, Director, Standards and Best Practices, Message Systems Desiree Hamaoui, Director Demand Generation, Message Systems
Thursday, August 21, 2014 1:00 EDT, (10:00 PDT)
ü Turn Off Pop-Up Blockers ü Technical difficulties?
v Click on “Help?” link v Use Q+A box
ü Submitting questions to speaker v Q+A session at end of webcast v Use “Ask a Question” box to submit questions v Send questions at any time v #DMDwc
Viewing Tips
Today’s Presenters
Autumn Tyr-Salvia, Director of Standards and Best Practices at Message Systems
Desiree Hamaoui, Director of Demand Generation, Message Systems
Spam/Antispam Is An Arms Race
The Path of Spam Filtering
Spam
New Filters
New Spam
Newer Filters
Understand the Past to Know the Future
• A Brief History of Spam • Co-evolution of Spam & Filtering
• Spam techniques • Filtering techniques
• Change in spamming techniques • Change in filtering techniques
• Ad infinitum….
Spam Spam Filtering Spam Spam
Filtering Spam
A Brief History of Spam
First Spam to Arpanet - 1978 DIGITAL WILL BE GIVING A PRODUCT PRESENTATION OF THE NEWEST MEMBERS OF THE DECSYSTEM-20 FAMILY; THE DECSYSTEM-2020, 2020T, 2060, AND 2060T. THE DECSYSTEM-20 FAMILY OF COMPUTERS HAS EVOLVED FROM THE TENEX OPERATING SYSTEM AND THE DECSYSTEM-10 <PDP-10> COMPUTER ARCHITECTURE. BOTH THE DECSYSTEM-2060T AND 2020T OFFER FULL ARPANET SUPPORT UNDER THE TOPS-20 OPERATING SYSTEM. THE DECSYSTEM-2060 IS AN UPWARD EXTENSION OF THE CURRENT DECSYSTEM 2040 AND 2050 FAMILY. THE DECSYSTEM-2020 IS A NEW LOW END MEMBER OF THE DECSYSTEM-20 FAMILY AND FULLY SOFTWARE COMPATIBLE WITH ALL OF THE OTHER DECSYSTEM-20 MODELS. WE INVITE YOU TO COME SEE THE 2020 AND HEAR ABOUT THE DECSYSTEM-20 FAMILY AT THE TWO PRODUCT PRESENTATIONS WE WILL BE GIVING IN CALIFORNIA THIS MONTH. THE LOCATIONS WILL BE: TUESDAY, MAY 9, 1978 - 2 PM HYATT HOUSE (NEAR THE L.A. AIRPORT) LOS ANGELES, CA THURSDAY, MAY 11, 1978 - 2 PM DUNFEY'S ROYAL COACH SAN MATEO, CA (4 MILES SOUTH OF S.F. AIRPORT AT BAYSHORE, RT 101 AND RT 92) A 2020 WILL BE THERE FOR YOU TO VIEW. ALSO TERMINALS ON-LINE TO OTHER DECSYSTEM-20 SYSTEMS THROUGH THE ARPANET. IF YOU ARE UNABLE TO ATTEND, PLEASE FEEL FREE TO CONTACT THE NEAREST DEC OFFICE FOR MORE INFORMATION ABOUT THE EXCITING DECSYSTEM-20 FAMILY.
Origin of “Spam”
SPAM SPAM SPAM SPAM SPAM SPAM SPAM SPAM
Green Card Lottery – Final One?
• April 12, 1994 • Not the first Usenet spam, but the first major one • Usenet spam posted to 5500+ groups with the help of a Perl
script
Laurence Canter 4/12/94 Green Card Lottery 1994 May Be The Last One! THE DEADLINE HAS BEEN ANNOUNCED. The Green Card Lottery is a completely legal program giving away a certain annual allotment of Green Cards to persons born in certain countries. The lottery program was scheduled to continue on a permanent basis. However, recently, Senator Alan J Simpson introduced a bill into the U. S. Congress which could end any future lotteries. THE 1994 LOTTERY IS SCHEDULED TO TAKE PLACE SOON, BUT IT MAY BE THE VERY LAST ONE. PERSONS BORN IN MOST COUNTRIES QUALIFY, MANY FOR FIRST TIME. ….
First Major Usenet Spam
Rise of the DNSBL
• 1997: Real-time Blackhole List (RBL) • 1998: Spamhaus • 1998: Spamcop • 2001: SORBS • …& many more!
Decline of the Open Relay • Email was designed like DNS – open peering
• Spammers exploited open relays • ISPs closed external relays
• Spammers joined ISPs and exploited open internal relays • ISPs required login for all relaying
Bayesian Filtering for Mail • 1996: ifile uses Bayesian filtering to sort mail into
folders • 1998: first scholarly paper on using Bayesian filtering
for spam • 2002: Bayesian filtering improved by Paul Graham’s
Plan for Spam
“I think it's possible to stop spam, and that content-based filters are the way to do it. The Achilles heel of the spammers is their message. They can circumvent any other barrier you set up. They have so far, at least. But they have to deliver their message, whatever it is. If we can write software that recognizes their messages, there is no way they can get around that.” –Paul Graham
SpamAssassin
• Open source spam filtering • Perl rewrite of an earlier filter.plx application • Added to SourceForge in April, 2001
Anti-spam Legislation Begins
• 2003: US CAN-SPAM law • 2003: EU Directive on Privacy & Electronic
Communications • 2004: spam convictions start
New Spamming Techniques Arise
• Image Spam • Faking headers • Phishing • Snowshoe spam • URL redirection • & more!
Co-Evolution of Spam & Filtering
Basic Architecture
Spam Evolves in Response to Filtering
• Exploring the Spam Arms Race to Characterize Spam Evolution • SpamAssassin rules most effective for spam generated
immediately prior to release • Old versions of SpamAssassin become increasingly
ineffective
Blacklisting
• Spammers originally used their own servers • Easy to blacklist these spam sources!
• Gave rise to: • Botnets • Snowshoe spam • Spamtrap “identification services”
• Gave rise to: • Better spamtraps • Domain blocking
Whitelisting • Whitelist only known good sending servers • Gave rise to:
• Header forging • Incentives to hack
• Malware & phishing
• Gave rise to: • Better whitelisting techniques • Sender authentication such as SPF, DKIM
• Gave rise to: • More incentives to hack
• Gave rise to: • IT teams that won’t whitelist anything
Content Filtering
Content Filtering
• Initially used simple word blocking lists • Easy to avoid with misspellings & punctuation • “Scunthorpe problem”
Content Filtering Evolves
• Simple filtering gave rise to: • Bayesian filtering • Misspellings become immediate spam words • Strange punctuation becomes problematic
• Causes unintentional problems with poorly thought out domains and subdomains
Content Filtering Evolves
• Weighting • Creates problem of “too many
spammy words”
• Gave rise to: • Super short messages
• In response, very short messages • get extra attention
• Word salad messages
Link Blocking • Domain reputation comes to content filtering
• Becomes problematic for domains with mixed content such as URL shorteners
• Gives rise to: • Spammers constantly obtain new domains for spamming
• New domains have reduced deliverability • Domain reputation consolidation good for whitehat senders
Image Spam • 2003 • Text filters now very effective • Gave rise to:
• Spammers turn to pictures-of-text • Hard to filter
• All-image messages become difficult to deliver at all • Image-to-text ratio becomes important for delivery
Early Image Spam
Image Spam Evolves
• Spammers create scripts to modify each image slightly
• Spammers start to use CAPTCHA-like techniques
Operation Spamalot
• 2007: Operation Spamalot • Pump & dump scams advertised by image spam
• Spammers get low conversions with no links for any other product
• SEC suspends 35 companies from trading
Image Spam Retreats
• Operation Spamalot catches main gang using image spam
• Spam filtering for images improves dramatically • But…all image messages still not good for
deliverability!
Every Action Causes a Reaction
• Spammers find a way around every block • Filters block the new spamming techniques • Anyone who wants to deliver wanted bulk email
must stay ahead of the curve!
Don’t Repeat History • Deliverability techniques don’t always make sense in
today’s spam landscape • Some filtering techniques are effectively vestigial • Knowing more about the history of spam can help you
deliver your mail tomorrow
99 % EMAILS DELIVERED
22 % OPT-‐IN EMAILS NOT GETTING THRU
Sending Reputa?on, Spam Filters, ISP Rules, Blacklists, Content Filters, etc.
“Deliverability” is Key
FTP ESP Report 99% Delivery
RP Report 78% Delivery
• FTP Package of User and Content Data in ZIP file • ESP Manages/Sends Campaign – 99% delivery to ISP • ISP Filters Delivery To Inbox • Delivery Data Reported to ESP • Engagement Data Reported ESP • Daily Data Bundle Downloaded
Your Company
Sending through a Shared System
ESP ISP
Engagement Data
Delivery Data 24-72 Hour Delay
99 % EMAILS DELIVERED
22 % OPT-‐IN EMAILS NOT GETTING THRU
17 % ESP DEDICATED
INCREASED DELIVERY
“Deliverability” is Key
• App Injects Emails/Content to Momentum • Momentum Manages Delivery to ISP • ISP Filters To Inbox • Delivery and Engagement Data Feeds to Momentum in Real-Time Speed your engagement lifecycle and control your brand.
Sending with A Dedicated System
ISP
Delivery Data
Engagement Data
Your Company
17% increase
How Does Momentum Help Improve Deliverability?
Adaptive Delivery™ - patented technology that auto-tunes outbound email delivery parameters in real time to optimize delivery and safeguard your own reputation • Mail stream segregation
• Auto-tuned delivery parameters
• Real-time traffic shaping
• Real-time decision making & alerts
• Live rule updates
• Configurable rules
• Full FBL capabilities
Automatically staying ahead of the “spam curve” so you don’t have to
Adaptive Delivery Results 10 point Deliverability Gain
FINANCIAL NEWSLETTER
Del
iver
abili
ty
April May June July August
0.85
0.90
0.95
AD Tuning Period
Source: Leading publisher/distributor of opt-in financial newsletters.
Adaptive Delivery Results
INTERNATIONAL NEWSLETTER
Del
iver
abili
ty
April May June July August
0.85
0.90
0.95
• Deliverability Stabilized
• Manual Effort Eliminated AD Tuning Period
Source: Leading publisher/distributor of opt-in financial newsletters.
Translates into Real Revenue Results
"When we have delivery and reputation issues, we know about them right away, and we can resolve problems proactively. Momentum with Adaptive Delivery has certainly had a positive impact on customer satisfaction…. We’ve been sending nearly 60 million messages a week at 99.15% deliverability. It was as high as 99.3% last week!” JAMES THOMPSON DIRECTOR OF EMAIL OPERATIONS, INFUSIONSOFT
INCREASED REVENUE WITHOUT MS WITH MS CHANGE
43
Message Systems: The Leader in Messaging
20%
Email / Marketing Services Providers
Social Networking
Daily Deals Telco Financial
Services Consumer
Technology Travel &
Hospitality Publishing
& Media Hosting & SaaS
Sampling of Key Clients by Market
Key Takeaways § Spammers adapt ! ISP rules/filters adapt to them § Momentum’s Adaptive Delivery™ automates
processes to “stay ahead of the curve” § Optimize deliverability through technology § Moving from a shared system/ESP to a dedicated
system improves deliverability by 17%
§ 17% deliverability increase ! 30% revenue increase
Thank you!
46
Follow us on Twitter: • @messagesystems
Follow us on Linkedin:
• Message Systems
Visit Us
• www.messagesystems.com
Contact Us
• info@messagesystems.com
http://digitalmarketingdepot.com webcasts@digitalmarketingdepot.com #DMD
Questions?
www.messagesystems.com