Spam Filtering for Mail Exchangers

Introduction

Purpose of this Document This document discusses various highly effective and low impact ways to weed out spam and malware during incoming SMTP transactions in a mail exchanger (MX host), with an added emphasis on eliminating so-called . The discussions are conceptual in nature, but a sample implementation is provided using the Exim MTA and other specific software tools. Miscellaneous other bigotry is expressed throughout.

Audience The intended audience is mail system administrators, who are already familiar with such acronyms as SMTP, MTA/MDA/MUA, DNS/rDNS, and MX records. If you are an end user who is looking for a spam filtering solution for your mail reader (such as Evolution, Thunderbird, Mail.app or Outlook Express), this document is not for you; but you may wish to point the mail system administrator for your domain (company, school, ISP...) to its existence.

New versions of this document The newest version of this document can be found at . Please check back periodically for corrections and additions.

Revision History 1.0 2004-09-08 TS First public release. 0.18 2004-09-07 TS Incorporated second language review from Tom Wright. 0.17 2004-09-06 TS Incorporated language review from Tom Wright. 0.16 2004-08-13 TS Incorporated third round of changes from Devdas Bhagat. 0.15 2004-08-04 TS Incorporated second round of changes from technical review by Devdas Bhagat. 0.14 2004-08-01 TS Incorporated technical review comments/corrections from Devdas Bhagat. 0.13 2004-08-01 TS Incorporated technical review from Joost De Cock. 0.12 2004-07-27 TS Replaced "A Note on Controversies" with a more opinionated "The Good, The Bad, the Ugly" section. Also rewrote text on DNS blocklists. Some corrections from Seymour J. Metz. 0.11 2004-07-19 TS Incorporated comments from Rick Stewart on RMX++. Swapped order of "Techniques" and "Considerations". Minor typographic fixes in Exim implementation. 0.10 2004-07-16 TS Added <?dbhtml..?> tags to control generated HTML filenames - should prevent broken links from google etc. Swapped order of "Forwarded Mail" and "User Settings". Correction from Tony Finch on Bayesian filters; commented out check for Subject:, Date:, and Message-ID: headers per Johannes Berg; processing time subtracted from SMTP delays per suggestion from Alan Flavell. 0.09 2004-07-13 TS Elaborated on problems with envelope sender signatures and mailing list servers, and a scheme to make such signatures optional per host/domain for each user. Moved "Considerations" section out as a separate chapter; added subsections "Blocking Access to other SMTP Server", "User Settings" and "Forwarded Mail". Incorporated Matthew Byng-Maddick's comments on the mechanism used to generate these signatures, Chris Edwards' comments on sender callout verification, and Hadmut Danisch's comments on RMX++ and other topics. Changed license terms (GPL instead of GFDL). 0.08 2004-07-09 TS Additional work on Exim implementation: Added section on per-user settings and data for SpamAssassin per suggestion from Tollef Fog Heen. Added SPF checks via Exiscan-ACL. Corrections from Sam Michaels. 0.07 2004-07-08 TS Made corrections to the Exim Envelope Sender Signatures examples, and added support for users to "opt in" to this feature, per suggestion from Christian Balzer. 0.06 2004-07-08 TS Incorporated Exim/MySQL greylisting implementation and various corrections from Johannes Berg. Moved "Sender Authorization Schemes" up two levels to become a top-level section in the Techniques chapter. Added greylisting for NULL empty envelope senders after DATA. Added SpamAssassin configuration to match Exim examples. Incorporated corrections from Dominik Ruff, Mark Valites, "Andrew" at Supernews. 0.05 2004-07-07 TS Eliminated the (empty) Sendmail implementation for now, to move ahead with the final review process. 0.04 2004-07-06 TS Reorganized layout a little: Combined "SMTP-Time Filtering", "Introduction to SMTP", and "Considerations" into a single "Background" chapter. Split the previous "Building ACLs" section in the Exim implementation into top-level sections. Added alternate sender authorization schemes to SPF: Microsoft Caller-ID for E-Mail and RMX++. Incorporated comments from Ken Raeburn. 0.03 2004-07-02 TS Added discussion on Multiple Incoming Mail Exchangers; minor corrections related to Sender Callout Verification. 0.02 2004-06-30 TS Added Exim implementation as an appendix 0.01 2004-06-16 TS Initial draft.

Credits A number of people have provided feedback, corrections, and contributions, as indicated in the . Thank you! The following are some of the people and groups that have provided tools and ideas to this document, in no particular order: Evan Harris eharris (at) puremagic.com, who conceived and wrote a white paper on greylisting. Axel Zinser fifi (at) hiss.org, who apparently conceived of teergrubing. The developers of SPF, RMX++, and other . The creators and maintainers of distributed, collaborative junk mail signature repositories, such as DCC, Razor, and Pyzor. The creators and maintainers of various DNS blocklists and whitelists, such as SpamCop, SpamHaus, SORBS, CBL, and many others. The developers of SpamAssassin, who have taken giant leaps forward in developing and integrating various spam filtering techniques into a sophisticated heuristics-based tool. Tim Jackson tim (at) timj.co.uk collated and maintains a list of bogus virus warnings for use with SpamAssassin. A lot of smart people who developed the excellent Exim MTA, including: Philip Hazel ph10 (at) cus.cam.ac.uk, the maintainer; Tom Kistner tom (at) duncanthrax.net, who wrote the Exiscan-ACL patch for SMTP-time content checks; Andreas Metzler ametzler (at) debian.org, who did a really good job of building the Exim 4 Debian packages. Many, many others who contributed ideas, software, and other techniques to counter the spam epidemic. You, for reading this document and your interest in reclaiming e-mail as a useful communication tool

Feedback I would love to hear of your experiences with the techniques outlined in this document, and of any other comments, questions, suggestions, and/or contributions you may have. Please send me an e-mail at: tor@slett.net. If you are able to provide implementations for other s, such as Sendmail or Postfix, please let me know.

Translations No translations exist yet. If you would like to create one, please let me know.

Copyright information Copyright © 2004 Tor Slettnes. This document is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This document is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. A copy of the license is included in . Read The GNU Manifesto if you want to know why this license was chosen for this book. The logos, trademarks and symbols used in this book are the properties of their respective owners.

What do you need? The techniques described in this document predicate system access to the inbound (s) for the internet domain where you receive e-mail. Essentially, you need to be able to install software and/or modify the configuration files for the on that system. Although the discussions in this document are conceptual in nature and can be incorporated into a number of different MTAs, a sample Exim 4 implementation is provided. This implementation, in turn, incorporates other software tools, such as SpamAssassin. See for details.

Conventions used in this document The following typographic and usage conventions occur in this text: Typographic and usage conventions Text type Meaning Quoted text Quotes from people, quoted computer output. terminal view Literal computer input and output captured from the terminal, usually rendered with a light grey background. command Name of a command that can be entered on the command line. VARIABLE Name of a variable or pointer to content of a variable, as in $VARNAME. Option to a command, as in the option to the ls command. argument Argument to a command, as in read man ls . command arguments Command synopsis or general usage, on a separated line. filename Name of a file or directory, for example Change to the /usr/bin directory. Key Keys to hit on the keyboard, such as type Q to quit. Button Graphical button to click, like the OK button. Menu Choice Choice to select from a graphical menu, for instance: Select HelpAbout Mozilla in your browser. Terminology Important term or concept: The Linux kernel is the heart of the system. See link to related subject within this guide. The author Clickable link to an external web resource.

Organization of this document This document is organized into the following chapters: General introduction to SMTP time filtering, and to SMTP. Various ways to block junk mail in an SMTP transaction. Issues that pertain to transaction time filtering. My attempt at anticipating your questions, and then answering them. A sample Exim implementation is provided in .