Fighting SPAM Spamassassin

• Statistical based on factors such as

banned words and acronyms

• None plane text or strange ascii coding in mail header

• HTML body with pictures and links.

• Sending/Recieving User exists

• File attachement, extra inspection by external program for viruses and trojans

• Black DNS, blacklisted domains/IP/hosts

• E-Mails per second, DOS/SPAM

• Email Relaying and hops • Help from external databases like: Pyzor Razor • Spamassassin does not delete mail, it marks mail as SPAM and

classify the severity

Downloading And Installing Spamassassin RPM

• From sources: http://spamassassin.apache.org/

• From rpm:

• Starting Spamassassin at boot

• Startup Spamassassin

• Spamassassin configuration sit in /etc/mail/spamassassin and /usr/share/spamassassin/

local.cf and init.pre

• Spamassassin comes preconfigured

• If you install from sources, dont install from RPM first!

# /etc/init.d/spamd start# /etc/init.d/spamd start

# insserv spamd on# insserv spamd on

# rpm –ivh perl-Digest-HMAC-1.01-495.i586.rpm# rpm –ivh perl-HTML-Tagset-3.04-3.i586.rpm# rpm –ivh perl-HTML-Parser-3.45-3.i586.rpm# rpm –ivh perl-Net-DNS-0.48-3.i586.rpm# rpm –ivh perl-spamassassin-3.0.2-4.i386.rpm# rpm –ivh spamassassin-3.0.2-4.i386.rpm# rpm –ivh spamassassin-3.0.2-4.i386.rpm

# rpm –ivh perl-Digest-HMAC-1.01-495.i586.rpm# rpm –ivh perl-HTML-Tagset-3.04-3.i586.rpm# rpm –ivh perl-HTML-Parser-3.45-3.i586.rpm# rpm –ivh perl-Net-DNS-0.48-3.i586.rpm# rpm –ivh perl-spamassassin-3.0.2-4.i386.rpm# rpm –ivh spamassassin-3.0.2-4.i386.rpm# rpm –ivh spamassassin-3.0.2-4.i386.rpm

Configuring Spamassassin • The spamassassin main configuration file is named

– /etc/mail/spamassassin/local.cf

• A full listing of all the options available

in the local.cf file can be found in the

Linux man pages using the following

command

• The spamassassin plugins file init.pre

• Spamassassin searches /etc/mail/spamassassin and /usr/share/spamassassin for .pre and .cf files to read in

• All users home can contain $HOME/.spamassassin/

• Spamassassin is written in PERL

• Spamassasin is 2 components the server spamd and client spamc

# man Mail::SpamAssassin::Conf # man Mail::SpamAssassin::Conf

required_hits 5.0whitelist_from *home.serewrite_subject 1 subject_tag *****SPAM***** report_safe 1 use_terse_report 0 use_bayes 1 auto_learn 1 skip_rbl_checks 0 use_razor2 1 use_dcc 1 use_pyzor 1 ok_languages en ok_locales en sv fi

required_hits 5.0whitelist_from *home.serewrite_subject 1 subject_tag *****SPAM***** report_safe 1 use_terse_report 0 use_bayes 1 auto_learn 1 skip_rbl_checks 0 use_razor2 1 use_dcc 1 use_pyzor 1 ok_languages en ok_locales en sv fi

loadplugin Mail::SpamAssassin::Plugin::URIDNSBLloadplugin Mail::SpamAssassin::Plugin::Hashcashloadplugin Mail::SpamAssassin::Plugin::SPF

loadplugin Mail::SpamAssassin::Plugin::URIDNSBLloadplugin Mail::SpamAssassin::Plugin::Hashcashloadplugin Mail::SpamAssassin::Plugin::SPF

Testing spamassassin • Test the validity of your local.cf and the other files

• Startup spamassassin – If you installed from ”source” you will need to write a proper start and stop script yourself

• Tuning spamassassin by adjusting the required_hits value in the local.cf file

• Sample mail header tagged by spamassassin, here nigerian scam

# spamassassin -d –lintCreated user preferences file: /root/.spamassassin/user_prefs config: SpamAssassin failed to parse line, skipping: use_terse_report 0 config: SpamAssassin failed to parse line, skipping: auto_learn 1 lint: 2 issues detected. please rerun with debug enabled for more information.

# spamassassin -d –lintCreated user preferences file: /root/.spamassassin/user_prefs config: SpamAssassin failed to parse line, skipping: use_terse_report 0 config: SpamAssassin failed to parse line, skipping: auto_learn 1 lint: 2 issues detected. please rerun with debug enabled for more information.

# /etc/init.d/spamd start# /etc/init.d/spamd start

required_hits 5.0required_hits 5.0

X-Spam-Status: Yes, score=20.1 required=2.1 tests=DEAR_FRIEND, DNS_FROM_RFC_POST,FROM_ENDS_IN_NUMS,MSGID_FROM_MTA_HEADER,NA_DOLLARS, NIGERIAN_BODY1,NIGERIAN_BODY2,NIGERIAN_BODY3,NIGERIAN_BODY4, RCVD_IN_BL_SPAMCOP_NET,RCVD_IN_SBL,RISK_FREE,SARE_FRAUD_X3, SARE_FRAUD_X4,SARE_FRAUD_X5,US_DOLLARS_3 autolearn=failed version=3.0.4

X-Spam-Status: Yes, score=20.1 required=2.1 tests=DEAR_FRIEND, DNS_FROM_RFC_POST,FROM_ENDS_IN_NUMS,MSGID_FROM_MTA_HEADER,NA_DOLLARS, NIGERIAN_BODY1,NIGERIAN_BODY2,NIGERIAN_BODY3,NIGERIAN_BODY4, RCVD_IN_BL_SPAMCOP_NET,RCVD_IN_SBL,RISK_FREE,SARE_FRAUD_X3, SARE_FRAUD_X4,SARE_FRAUD_X5,US_DOLLARS_3 autolearn=failed version=3.0.4

The Rules du Jour Spamassassin Tool • Rules Du Jour is a script who downloads filtering rules for

Spamassassin.

• The script is available here: http://sandgnat.com/rdj/rules_du_jour and it is intended to be run from a cron job on daily basis.

• The /etc/rulesdujour/config Configuration File – SA_DIR path to spamassassin

– MAIL_ADDRESS who recieves status messages

– SA_RESTART howto restart spamassassin after new rules is installed

– TRUSTED_RULESETS space delimited line with filter rules to use

SA_DIR="/etc/mail/spamassassin" MAIL_ADDRESS="rulesdujour_update@my-web-site.org" SA_RESTART="service spamd restart" TRUSTED_RULESETS="TRIPWIRE SARE_ADULT SARE_OBFU SARE_URI0 SARE_URI1 ANTIDRUG SARE_SPOOF SARE_BAYES_POISON_NXM SARE_OEM SARE_RANDOM SARE_FRAUD SARE_HEADER0 SARE_HEADER2 SARE_HTML0 SARE_SPECIFIC SARE_BML SARE_GENLSUBJ0 SARE_GENLSUBJ2 SARE_WHITELIST"

SA_DIR="/etc/mail/spamassassin" MAIL_ADDRESS="rulesdujour_update@my-web-site.org" SA_RESTART="service spamd restart" TRUSTED_RULESETS="TRIPWIRE SARE_ADULT SARE_OBFU SARE_URI0 SARE_URI1 ANTIDRUG SARE_SPOOF SARE_BAYES_POISON_NXM SARE_OEM SARE_RANDOM SARE_FRAUD SARE_HEADER0 SARE_HEADER2 SARE_HTML0 SARE_SPECIFIC SARE_BML SARE_GENLSUBJ0 SARE_GENLSUBJ2 SARE_WHITELIST"

Installing Rules du Jour1) Download the rules_du_jour script with the wget command, make it

executable and place it in the /usr/local/bin directory. The script is available here: http://sandgnat.com/rdj/rules_du_jour and it is intended to be run from a cron job on daily basis.

2) Create and edit your /etc/rulesdujour/config configuration file.

3) Run the rules_du_jour script, and then run spamassassin in lint mode to test for errors. There should be none.

4) The final step is to add /usr/local/bin/rules_du_jour to your cron table. In this case, crontab –e

# wget http://sandgnat.com/rdj/rules_du_jour# wget http://sandgnat.com/rdj/rules_du_jour

# chmod 700 rules_du_jour # mv rules_du_jour /usr/local/bin

# chmod 700 rules_du_jour # mv rules_du_jour /usr/local/bin

# mkdir -p /etc/rulesdujour # vi /etc/rulesdujour/config

# mkdir -p /etc/rulesdujour # vi /etc/rulesdujour/config

# /usr/local/bin/rules_du_jour # /usr/local/bin/rules_du_jour

0 23 * * * root /usr/local/bin/rules_du_jour 0 23 * * * root /usr/local/bin/rules_du_jour

Setting up procmail for spamassassin

• Procmail is a mail processor it can search the mail header and body for patterns, keys and attributes

• Procmail uses regular expressions to find or extract keys

• Procmail can move/trunctate/delete and make calls to external programs based on conditions

• Procmail has a mandatory file used in situations where individual users does not have one, /etc/procmailrc

• The user configurable procmail file is $HOME/.procmail• Procmail ”home” is very helpful tolearnmoreabout the

powerful procmail: http://www.procmail.org/

Getting procmail installed

• Install procmail from RPM

• Download procmail source

• Build procmail source

• Inspect procmail builded appz

• Install procmail sources (all the new/ -files)

# rpm –ivh procmail-3.22-41# rpm –ivh procmail-debuginfo-3.22-41

# rpm –ivh procmail-3.22-41# rpm –ivh procmail-debuginfo-3.22-41

# cd /usr/local/src ; wget http://www.procmail.org/procmail-3.22.tar.gz# cd /usr/local/src ; wget http://www.procmail.org/procmail-3.22.tar.gz

# cd procmail-3.22 ; make. . .# make install

# cd procmail-3.22 ; make. . .# make install

# ls new/# ls new/

# make install-suidOr type# make install

# make install-suidOr type# make install

Procmail configuration for Spamassassin

• Procmail comes unconfigured as RPM and Sources

• If you install procmail from source you have sample configuration to start with in the sourcetree /usr/local/src/procmail-3.22/examples

• You will need to modify the sample config or re do everything from scratch

– Here we first copy one of the examples to the mandatory procmail settings

– Secondly we copy it into user root’s personal settings

• Procmail haves to configuration sets:– Mandatory default procmailrc

– Personal .procmailrc

# cp examples/3procmailrc /etc/procmailrc# cp examples/3procmailrc ~root/.procmailrc

# cp examples/3procmailrc /etc/procmailrc# cp examples/3procmailrc ~root/.procmailrc

Procmail mandatory /etc/procmailrc

• Procmailrc has a number of settings & enviroment vars– DROPPRIVS =YES lower priviledges to recieving user level

– VERBOSE=ON log level details

– MAILDIR=$HOME/Mail User home maildir

– DEFAULT=$MAILDIR/mbox User home mail database file

– LOGFILE=$MAILDIR/from where to log procmail activities

– LOCKFILE=$HOME/lockmail protect procmail processing

– COMPANY=PHW General enviroment variable

• Procmail is driven by regular expressionsThis very first rule starts spamc if email

size is less than 256000 bytes

The last rule will move the Email if X-Spam-Status: Yes

is set in the email header

:0fw* < 256000 | /usr/bin/spamc -f

:0fw* < 256000 | /usr/bin/spamc -f

:0: { EXITCODE=$? }:0: * ^X-Spam-Status: Yes $HOME/IMAP-$COMPANY/SPAM

:0: { EXITCODE=$? }:0: * ^X-Spam-Status: Yes $HOME/IMAP-$COMPANY/SPAM

Procmail userdefine $HOME/.procmailrc1) This rule will move all files with lastnames to directory illegal-attach

2) All email less than 250K isprocessed by spamc

3) All email marked withspamlevel greater than 15

is moved to directory

almost-certenly-spam

– All mail who accumulatedmore than required_hits

is moved to directory probely-spam

1) All mail who has subject*****SPAM***** is moved to

directory subject-spam

:0fw* < 256000 | /usr/bin/spamc -f

:0fw* < 256000 | /usr/bin/spamc -f

:0 B* ^Content-Type:.** ^.*name=.*\.(hta|com|pif|vbs|vbe|js|jse|exe|bat|cmd|vxd|scr|shm|dll|SCR)illegal-attach

:0 B* ^Content-Type:.** ^.*name=.*\.(hta|com|pif|vbs|vbe|js|jse|exe|bat|cmd|vxd|scr|shm|dll|SCR)illegal-attach

:0:* ^X-Spam-Level: \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*almost-certainly-spam

:0:* ^X-Spam-Level: \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*almost-certainly-spam

:0:* ^X-Spam-Status: Yesprobably-spam

:0:* ^X-Spam-Status: Yesprobably-spam

:0:* ^Subject: \*\*\*\*\*SPAM\*\*\*\*\*subject-spam

:0:* ^Subject: \*\*\*\*\*SPAM\*\*\*\*\*subject-spam

Procmail is now ready for action• Now it is left to add procmail support in

/etc/mail/sendmail.mc

• Procmail specified attributes (optional)

-t try later, do not bounce-Y Berkeley mailbox format

-a argument added from sendmail enviroment-d delivery mode, set userid $u (from sendmail)

• Make the sendmail.mc

• Last add the .procmailrc to /etc/skel– So all future users added will have

.procmailrc as default

FEATURE(local_procmail,`',`procmail -t -Y -a $h -d $u')dnlFEATURE(local_procmail,`',`procmail -t -Y -a $h -d $u')dnl

define(`PROCMAIL_MAILER_PATH',`/usr/bin/procmail')dnlFEATURE(local_procmail)dnlMAILER(procmail)dnl

define(`PROCMAIL_MAILER_PATH',`/usr/bin/procmail')dnlFEATURE(local_procmail)dnlMAILER(procmail)dnl

# cd /etc/mail ; m4 sendmail.mc > sendmail.cf

# rcsendmail restart

# cd /etc/mail ; m4 sendmail.mc > sendmail.cf

# rcsendmail restart

# cp ~root/.procmail /etc/skel# cp ~root/.procmail /etc/skel

Using Greylisting

• Spammers try to send email as quickly as possible

• Bouncing mails is removed from their mailing lists

• Mailserver can ask the sender to try again later if mails coming in tofast

• Spam emails that need to be resent are usually abandoned

• With greylisting, sources are just asked to resend and thereby getting rid of spam

• The most popular greylist mail filter (milter) products is the milter-greylist package

• Drawback is mail-flow can become slower

Downloading and Installing milter-greylist

• You will have to first install the sendmail-devel software package– You already have it if you installed sendmail from sources

– You can get it as optional RPM, as we installed in beginning of this chapter

• Download greylist-milter

• Untar milter-greylist

• Configure and make milter-greylist

• More info can be found at: http://hcpnet.free.fr/milter-greylist/

# cd /usr/local/src

# wget ftp://ftp.espci.fr/pub/milter-greylist/milter-greylist-2.0.2.tgz

# cd /usr/local/src

# wget ftp://ftp.espci.fr/pub/milter-greylist/milter-greylist-2.0.2.tgz

# tar -xzvf milter-greylist-2.0.2.tgz # tar -xzvf milter-greylist-2.0.2.tgz

# ./configure && make && make install # ./configure && make && make install

Configuring milter-greylist• Add the milter-greylist statements listed in the README file to your

/etc/mail/sendmail.mc file:

• Copy the correct version to your /etc/init.d and prepare it to start at boot

• Edit the /etc/mail/greylist.conf configuration file, add modify:

Here we set the “try again later” to five minutes Deactivate the timer for trusted networks so that mail is delivered

immediately

• Start the milter:

INPUT_MAIL_FILTER(`greylist',`S=local:/var/milter-greylist/milter-greylist.sock') define(`confMILTER_MACROS_CONNECT', `j, {if_addr}') define(`confMILTER_MACROS_HELO', `{verify}, {cert_subject}') define(`confMILTER_MACROS_ENVFROM', `i, {auth_authen}') define(`confMILTER_MACROS_ENVRCPT', `{greylist}')

INPUT_MAIL_FILTER(`greylist',`S=local:/var/milter-greylist/milter-greylist.sock') define(`confMILTER_MACROS_CONNECT', `j, {if_addr}') define(`confMILTER_MACROS_HELO', `{verify}, {cert_subject}') define(`confMILTER_MACROS_ENVFROM', `i, {auth_authen}') define(`confMILTER_MACROS_ENVRCPT', `{greylist}')

# cp rc-suse.sh /etc/init.d/milter-greylist

# chmod 755 /etc/init.d/milter-greylist

# insserv milter-greylist

# cp rc-suse.sh /etc/init.d/milter-greylist

# chmod 755 /etc/init.d/milter-greylist

# insserv milter-greylist

greylist 5m

acl whitelist addr 192.168.0.0/16

greylist 5m

acl whitelist addr 192.168.0.0/16

# ln –s /etc/init.d/milter-greylist /usr/sbin/rcmilter-greylist

# rcmilter-greylist start ; rcsendmail restart

# ln –s /etc/init.d/milter-greylist /usr/sbin/rcmilter-greylist

# rcmilter-greylist start ; rcsendmail restart

Configuring milter-greylist, contined• The /var/log/mail* files should be used to determine what is

happening to your mail

• A request is sent to the sender to resend the email in five minutes

• Here email from a source is autowhitelisted for 24 hours

• We are now done with milter greylist setup!

Dec 24 00:32:31 mail sendmail[28847]: jBO8WVnG028847: Milter: to=<spamvictim@my-web-site.org>, reject=451 4.7.1 Greylisting in action, please come back in 00:05:00

Dec 24 00:32:31 mail sendmail[28847]: jBO8WVnG028847: Milter: to=<spamvictim@my-web-site.org>, reject=451 4.7.1 Greylisting in action, please come back in 00:05:00

Dec 23 20:40:21 mail milter-greylist: jBO4eF2m027418: addr 211.115.216.225 from <slashdot@slashdot.org> rcpt <spamvictim@my-web-site.org>: autowhitelisted for 24:00:00

Dec 23 20:40:21 mail milter-greylist: jBO4eF2m027418: addr 211.115.216.225 from <slashdot@slashdot.org> rcpt <spamvictim@my-web-site.org>: autowhitelisted for 24:00:00

Installing Your POP/IMAP Server • There are several much more powerful IMAP/POP servers than the

one we install. This is for demonstration only. Usally we install UW-IMAP or similar.

• Install the dovecot IMAP/POP server

• Activate dovecot at boot

• Start dovecot now

• Pop and Imap is purposed to serve users and clients with centralized email in a comfortable way.

• Pop and Imap can be both run as cleartext and cryptated

# rpm –ivh dovecot-debuginfo-0.99.14.rpm

# rpm –ivh mysql-shared-4.1.10a-3.i586.rpm# rpm –ivh postgresql-libs-8.0.1-6.i586.rpm

# rpm –ivh dovecot-0.99.14-3.i586.rpm

# rpm –ivh dovecot-debuginfo-0.99.14.rpm

# rpm –ivh mysql-shared-4.1.10a-3.i586.rpm# rpm –ivh postgresql-libs-8.0.1-6.i586.rpm

# rpm –ivh dovecot-0.99.14-3.i586.rpm

# insserv dovecot# insserv dovecot

# rcdovecot start# rcdovecot start

Configuring Your POP/IMAP Server • Protocol selection in /etc/dovecot/dovecot.conf

• Check that dovecot is listening:

• Going from insecure pop/imap to secure, make the certificate

• Change settings to

secure pop/imap

# Protocols we want to be serving:# imap imaps pop3 pop3sprotocols = imap pop3

# Protocols we want to be serving:# imap imaps pop3 pop3sprotocols = imap pop3

netstat -a | egrep -i 'pop|imap'tcp 0 0 *:pop3 *:* LISTENtcp 0 0 *:imap *:* LISTEN

netstat -a | egrep -i 'pop|imap'tcp 0 0 *:pop3 *:* LISTENtcp 0 0 *:imap *:* LISTEN

# cd /usr/share/doc/packages/dovecot # chmod a+x mkcert.sh ; ./ mkcert.sh

# cd /usr/share/doc/packages/dovecot # chmod a+x mkcert.sh ; ./ mkcert.sh

protocols = pop3s imapsssl_disable = nossl_cert_file = /etc/ssl/certs/dovecot.pemssl_key_file = /etc/ssl/private/dovecot.pemssl_parameters_file = /var/run/dovecot/ssl-parameters.datdisable_plaintext_auth = nologin_chroot = yesauth_mechanisms = plain

protocols = pop3s imapsssl_disable = nossl_cert_file = /etc/ssl/certs/dovecot.pemssl_key_file = /etc/ssl/private/dovecot.pemssl_parameters_file = /var/run/dovecot/ssl-parameters.datdisable_plaintext_auth = nologin_chroot = yesauth_mechanisms = plain

Secure Your POP/IMAP Server • Check that dovecot is listening on the secure ports:

• Troubleshooting POP Mail, this example starts and makes a successful secure POP query from a remote POP client

netstat -a | egrep -i 'pop|imap'tcp 0 0 *:pop3s *:* LISTENtcp 0 0 *:imaps *:* LISTEN

netstat -a | egrep -i 'pop|imap'tcp 0 0 *:pop3s *:* LISTENtcp 0 0 *:imaps *:* LISTEN

Aug 11 23:20:33 bigboy ipop3d[18693]: pop3s SSL service init from 172.16.1.103 Aug 11 23:20:40 bigboy ipop3d[18693]: Login user=labmanager host=172-16-1-103.my-site.com [172.16.1.103] nmsgs=0/0 Aug 11 23:20:40 bigboy ipop3d[18693]: Logout user=labmanager host=172-16-1-103.my-site.com [172.16.1.103] nmsgs=0 ndele=0 Aug 11 23:20:52 bigboy ipop3d[18694]: pop3s SSL service init from 172.16.1.103 Aug 11 23:20:52 bigboy ipop3d[18694]: Login user=labmanager host=172-16-1-103.my-site.com [172.16.1.103] nmsgs=0/0 Aug 11 23:20:52 bigboy ipop3d[18694]: Logout user=labmanager host=172-16-1-103.my-site.com [172.16.1.103] nmsgs=0 ndele=0

Aug 11 23:20:33 bigboy ipop3d[18693]: pop3s SSL service init from 172.16.1.103 Aug 11 23:20:40 bigboy ipop3d[18693]: Login user=labmanager host=172-16-1-103.my-site.com [172.16.1.103] nmsgs=0/0 Aug 11 23:20:40 bigboy ipop3d[18693]: Logout user=labmanager host=172-16-1-103.my-site.com [172.16.1.103] nmsgs=0 ndele=0 Aug 11 23:20:52 bigboy ipop3d[18694]: pop3s SSL service init from 172.16.1.103 Aug 11 23:20:52 bigboy ipop3d[18694]: Login user=labmanager host=172-16-1-103.my-site.com [172.16.1.103] nmsgs=0/0 Aug 11 23:20:52 bigboy ipop3d[18694]: Logout user=labmanager host=172-16-1-103.my-site.com [172.16.1.103] nmsgs=0 ndele=0

How To Configure Your Windows Mail Programs • All your POP e-mail accounts are really only regular Linux user

accounts in which sendmail has deposited mail.

• You can now configure your e-mail client such as Outlook Express to use your use your new POP/SMTP mail server quite easily.

• To configure POP Mail, set your POP mail server to be the IP address of your Linux mail server.

• Use your Linux user username and password when prompted.

• Next, set your SMTP mail server to be the IP address/domain name of your Linux mail server.

• You can use similar setup for IMAP

• For secure IMAP/POP you have to select SSL in advanced settings for incoming e-mail.

Conclusions • Sendmail is the most used mailserver• The macrofile sendmail.mc is used togeather with m4 to make

sendmail.cf• Sendmail configuration lives in /etc/mail• The mailserver keep all users inboxes in /var/spool/mail• To prevent SPAM and unauthorized access RELAY is used for

allowed sites in /etc/access• You have to type make and newaliases after editing sendmail

configuration• Sendmail can use dns blacklists to prevent spam directly• Spamassassin can be used to wash mail from SPAM, but

Spamassassin does only MARK and classify mail.• Rules Du Jour can update Spamassassin filters automatically• Procmail is used to process the mail, like dropping, moving,

trunctating and is driven by regular expressions• Greylisting is a complementing SPAM blocking mechanism based

on email resend due to heavy load messages.• IMAP/POP can be used to server users with centralized e-mail in a

comfortable way.