Blacklist, Whitelist & spamtrap

Terena EQUAL WorkshopDec 9th 2009 amsterdam

Index

• SMTP Blacklist

• SMTP WhiteList

• Spamtraps

IRISRBL: RedIRIS blacklist system

IRISRBL motivations

• Which/How many Blacklist to use ? SMTP traffic can be slowed with too much

DNS checks But better results (more spam blocked)

• What can we do with the false positives ? How fast can a IP address be removed from

a Blacklist system ?

• How can the NREN provide an additional service to their members ?

IRISRLB: Motivations II

• Commercial Blacklist problems: For the SMTP provider (listed in it):

Sometimes outgoing SMTP servers are listed Bounce messages Infected users sending spam …. Politics issues

How to be removed from the list ? Need to pay money ? 48 hours delay

To the user of the Black list: Messages not received Manual removing of black list / white list No information about why this IP address is listed

Blacklist implementation I

• Based on part of a bigger product, Rks from Sandvine, http://www.sandvine.com

• Service only for own constituency http://www.rediris.es/servicios/irisrbl/

• Integrate different sources: Several blacklist White List & exceptions Events (Spamtraps)

• Only one query to DNS check the blacklist• Small web interface to remove IP in the blacklists• Only postmaster of the Blacklists (not IP owner)

can remove IP addresses // false positives

Blacklist implementation: RKS

• Custom DNS server based with a database backend.

• Incremental feed of informationServer don’t need to restart to add new IP

addresses.

• Flexible policy to define which feeds to add and when a IP is listed.

• Support for different sources.• Different operating system support.

IRISRBL Stats

• More than 60% of RedIRIS constituency is using IRISBL.

• About 350 DNS queries/second

Whitelist

White List

2004/2005.• Lot of black listing problems between

Universities & ISP in Spain.• SPF was not widely implemented

• Most of the mail providers, were using some kind of manual white list .

• No coordination .

Other White listprojects

• Some discussion in the E-COAT meetings, provide the initial jumpstart information.

• Dutch ISP WL. http://noc.bit.nl/dnsbl/nlwhitelist/

• DNSWL.org , http://www.dnswl.org

WhiteList motivations

• Our main motivation is to avoid problems with blacklisting of SMTP server.

• We only tried a minimum quality requirement for being listed in the whitelist.

• It’s more important to receive the legal email from a blacklisted smtp server than don’t receive any email at all You can use other filters (content filters, etc)

after the blacklist to avoid this spam

WhiteList Vision: button up

• Organizations usually exchange emails locally (country wide) SME partners and big local ISP are the main

problem

• Including big ISP in the whitelist provide visibilit.

• Focus locally and exchange information with other similar initiatives.

White List format & usage:

• Two white list zones defined: ESWL: outgoing SMTP server of Abuses

members. MTAWL: White list with big international email

providers, other organizations and similar initiatives.

• White list is provided in different formats: DNS based (like blacklist) Configuration files for different SMTP servers.

• The files can be downloaded from the white list page.

• All the IP listed has a abuse/technical contact public address for troubleshooting

RedIRIS white list: Eswl y MTAwl

RedIRIS

TelefónicaEuskaltel

ESwl

ONO

MTAwl

• Goverment

• Yahoo,Gmail, Hotmail

• Agencias, …

• zone high DNSwl.org• Others

RedIRISwitoutSPF

Telecable

Sarenet

Hostalia

Ya.com

TusProfesionales

• Pymes

Hostalia

RedIRIS White List

WL policy:

Don’t spend too much time thinking how to implement it. Simple policy: you are in the list

Because you asked for this Someone added (mtawl )

People using the WL, want to have you in the WL.

WL , don’t provide any kind of reputation “good SMTP behaviour”, only states that this is the address of an SMTP server that “usually” don’t send too much spam. But also you provide contact information for abuse

reporting. And our spamtrap system allow us to monitor IP

address behaviour

Version 1.

• Simple Perl scripts . Manual processing of the information Ad-hoc scripts to add information from other

White List

• Success: Used by Universities & Spanish ISPs Great interest from other groups:

Bank, local government …

Fix most of the black listing problems between ISP & Universities.

Version 2.

• Web interface • Registry of changes• Most of the task can be done by the

domain owners.• Protocol to import information from other

White List systems.

WhiteList soruces

• Spanish Universities & ISP

• SME

• Big SMTP providers

• Feeds from other sources DNSWL trustedsource

Conclusions

• Use a white list to avoid problems caused by blacklist, not to provide any kind of email assurance.

• Whitelist are useful if people knows and use it, (and usually they want also to be there).

• Having different level of quality promotes postmaster to reach the “high” level , improving the email quality overall.

SPAMTRAP system

Spamtrap

• Fake emails accounts to receive spam.

• Provide information for: Bad IP addresses that are sending

spam(feed blacklist system) WL SMTP servers sending spam

(compromise system, detection of bad usage or compromise)

Early detect system of phising attacks.

Spamtrap features:

• Use domains & subdomains never used before. (ej, usr.rediris.es) Avoid collisions with real domains &

addresses.

• Redirect domains to a central machine to avoid parsing receive headers. Source IP address is always in the first

received line.

• Publish email addresses in web pages for crawlers.

Spamtrap : implementation

• Unix server + SMTP server (postfix)• Subdomains provided by universities.• Simple script to generate fake email

addresses for the domains• Publish the information in a web page

with a warning message.• Parsing of the incoming emails to remove

bounces from smtp servers.

Spamttrap implementation (II)

• Batch system to avoid system overload

• Real time check against different DNSzones Detection of Whitelisted servers sending

spam

• URL & binary extraction Extract malware from the files

• Store evidence for later use

Results of Spamtrap

• Blacklist: IP addresses that sent spam are used to feed the blacklist reputation system in real time (~5 minutes delay)

• WhiteList: IP addresses are verified against whitelist to detect infected machine and SMTP problems in the whitelist member.

• Phising/trend reporting: check some patterns to detect phising trends against some organizations in Spain.

• Provide information for security groups.

Expectations:

• Blacklist: Sharing of blacklist between NRENS Commercial agreement (SCS like) for Terena

members ? Improve the tool

• WhiteList: Sharing of information between different

NRENs

• Spamtrap: Improve the tool More robust sensor network.

28

Edificio Bronce

Plaza Manuel Gómez Moreno s/n

28020 Madrid. España

Tel.: 91 212 76 20 / 25

Fax: 91 212 76 35

www.red.es – www.rediris.es