5164 lines
293 KiB
Plaintext
5164 lines
293 KiB
Plaintext
Spam Filtering for Mail Exchangers
|
||
|
||
How to reject junk mail in incoming SMTP transactions.
|
||
|
||
Tor Slettnes
|
||
|
||
<tor@slett.net>
|
||
|
||
Edited by
|
||
|
||
Joost De Cock
|
||
|
||
Devdas Bhagat
|
||
|
||
Tom Wright
|
||
|
||
Version 1.0 -- Release Edition
|
||
-----------------------------------------------------------------------------
|
||
|
||
Table of Contents
|
||
Introduction
|
||
1. Purpose of this Document
|
||
2. Audience
|
||
3. New versions of this document
|
||
4. Revision History
|
||
5. Credits
|
||
6. Feedback
|
||
7. Translations
|
||
8. Copyright information
|
||
9. What do you need?
|
||
10. Conventions used in this document
|
||
11. Organization of this document
|
||
|
||
|
||
1. Background
|
||
1.1. Why Filter Mail During the SMTP Transaction?
|
||
1.2. The Good, The Bad, The Ugly
|
||
1.3. The SMTP Transaction
|
||
|
||
|
||
2. Techniques
|
||
2.1. SMTP Transaction Delays
|
||
2.2. DNS Checks
|
||
2.3. SMTP checks
|
||
2.4. Greylisting
|
||
2.5. Sender Authorization Schemes
|
||
2.6. Message data checks
|
||
2.7. Blocking Collateral Spam
|
||
|
||
|
||
3. Considerations
|
||
3.1. Multiple Incoming Mail Exchangers
|
||
3.2. Blocking Access to Other SMTP Servers
|
||
3.3. Forwarded Mail
|
||
3.4. User Settings and Data
|
||
|
||
|
||
4. Questions & Answers
|
||
A. Exim Implementation
|
||
A.1. Prerequisites
|
||
A.2. The Exim Configuration File
|
||
A.3. Options and Settings
|
||
A.4. Building the ACLs - First Pass
|
||
A.5. Adding SMTP transaction delays
|
||
A.6. Adding Greylisting Support
|
||
A.7. Adding SPF Checks
|
||
A.8. Adding MIME and Filetype Checks
|
||
A.9. Adding Anti-Virus Software
|
||
A.10. Adding SpamAssassin
|
||
A.11. Adding Envelope Sender Signatures
|
||
A.12. Accept Bounces Only for Real Users
|
||
A.13. Exempting Forwarded Mail
|
||
A.14. Final ACLs
|
||
|
||
|
||
Glossary
|
||
B. GNU General Public License
|
||
B.1. Preamble
|
||
B.2. TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
|
||
B.3. How to Apply These Terms to Your New Programs
|
||
|
||
|
||
|
||
List of Tables
|
||
1. Typographic and usage conventions
|
||
1-1. Simple SMTP dialogue
|
||
A-1. Use of ACL connection/message variables
|
||
|
||
-----------------------------------------------------------------------------
|
||
Introduction
|
||
|
||
1. Purpose of this Document
|
||
|
||
This document discusses various highly effective and low impact ways to
|
||
weed out spam and malware during incoming SMTP transactions in a mail
|
||
exchanger (MX host), with an added emphasis on eliminating so-called
|
||
Collateral Spam.
|
||
|
||
The discussions are conceptual in nature, but a sample implementation is
|
||
provided using the Exim MTA and other specific software tools. Miscellaneous
|
||
other bigotry is expressed throughout.
|
||
-----------------------------------------------------------------------------
|
||
|
||
2. Audience
|
||
|
||
The intended audience is mail system administrators, who are already
|
||
familiar with such acronyms as SMTP, MTA/MDA/MUA, DNS/rDNS, and MX records.
|
||
If you are an end user who is looking for a spam filtering solution for your
|
||
mail reader (such as Evolution, Thunderbird, Mail.app or Outlook Express),
|
||
this document is not for you; but you may wish to point the mail system
|
||
administrator for your domain (company, school, ISP...) to its existence.
|
||
-----------------------------------------------------------------------------
|
||
|
||
3. New versions of this document
|
||
|
||
The newest version of this document can be found at [http://slett.net/
|
||
spam-filtering-for-mx/] http://slett.net/spam-filtering-for-mx/. Please check
|
||
back periodically for corrections and additions.
|
||
-----------------------------------------------------------------------------
|
||
|
||
4. Revision History
|
||
|
||
|
||
Revision History
|
||
Revision 1.0 2004-09-08 Revised by: TS
|
||
First public release.
|
||
Revision 0.18 2004-09-07 Revised by: TS
|
||
Incorporated second language review from Tom Wright.
|
||
Revision 0.17 2004-09-06 Revised by: TS
|
||
Incorporated language review from Tom Wright.
|
||
Revision 0.16 2004-08-13 Revised by: TS
|
||
Incorporated third round of changes from Devdas Bhagat.
|
||
Revision 0.15 2004-08-04 Revised by: TS
|
||
Incorporated second round of changes from technical review by Devdas Bhagat.
|
||
Revision 0.14 2004-08-01 Revised by: TS
|
||
Incorporated technical review comments/corrections from Devdas Bhagat.
|
||
Revision 0.13 2004-08-01 Revised by: TS
|
||
Incorporated technical review from Joost De Cock.
|
||
Revision 0.12 2004-07-27 Revised by: TS
|
||
Replaced "A Note on Controversies" with a more opinionated "The Good, The
|
||
Bad, the Ugly" section. Also rewrote text on DNS blocklists. Some corrections
|
||
from Seymour J. Metz.
|
||
Revision 0.11 2004-07-19 Revised by: TS
|
||
Incorporated comments from Rick Stewart on RMX++. Swapped order of
|
||
"Techniques" and "Considerations". Minor typographic fixes in Exim
|
||
implementation.
|
||
Revision 0.10 2004-07-16 Revised by: TS
|
||
Added <?dbhtml..?> tags to control generated HTML filenames - should prevent
|
||
broken links from google etc. Swapped order of "Forwarded Mail" and "User
|
||
Settings". Correction from Tony Finch on Bayesian filters; commented out
|
||
check for Subject:, Date:, and Message-ID: headers per Johannes Berg;
|
||
processing time subtracted from SMTP delays per suggestion from Alan Flavell.
|
||
Revision 0.09 2004-07-13 Revised by: TS
|
||
Elaborated on problems with envelope sender signatures and mailing list
|
||
servers, and a scheme to make such signatures optional per host/domain for
|
||
each user. Moved "Considerations" section out as a separate chapter; added
|
||
subsections "Blocking Access to other SMTP Server", "User Settings" and
|
||
"Forwarded Mail". Incorporated Matthew Byng-Maddick's comments on the
|
||
mechanism used to generate these signatures, Chris Edwards' comments on
|
||
sender callout verification, and Hadmut Danisch's comments on RMX++ and other
|
||
topics. Changed license terms (GPL instead of GFDL).
|
||
Revision 0.08 2004-07-09 Revised by: TS
|
||
Additional work on Exim implementation: Added section on per-user settings
|
||
and data for SpamAssassin per suggestion from Tollef Fog Heen. Added SPF
|
||
checks via Exiscan-ACL. Corrections from Sam Michaels.
|
||
Revision 0.07 2004-07-08 Revised by: TS
|
||
Made corrections to the Exim Envelope Sender Signatures examples, and added
|
||
support for users to "opt in" to this feature, per suggestion from Christian
|
||
Balzer.
|
||
Revision 0.06 2004-07-08 Revised by: TS
|
||
Incorporated Exim/MySQL greylisting implementation and various corrections
|
||
from Johannes Berg. Moved "Sender Authorization Schemes" up two levels to
|
||
become a top-level section in the Techniques chapter. Added greylisting for
|
||
NULL empty envelope senders after DATA. Added SpamAssassin configuration to
|
||
match Exim examples. Incorporated corrections from Dominik Ruff, Mark
|
||
Valites, "Andrew" at Supernews.
|
||
Revision 0.05 2004-07-07 Revised by: TS
|
||
Eliminated the (empty) Sendmail implementation for now, to move ahead with
|
||
the final review process.
|
||
Revision 0.04 2004-07-06 Revised by: TS
|
||
Reorganized layout a little: Combined "SMTP-Time Filtering", "Introduction to
|
||
SMTP", and "Considerations" into a single "Background" chapter. Split the
|
||
previous "Building ACLs" section in the Exim implementation into top-level
|
||
sections. Added alternate sender authorization schemes to SPF: Microsoft
|
||
Caller-ID for E-Mail and RMX++. Incorporated comments from Ken Raeburn.
|
||
Revision 0.03 2004-07-02 Revised by: TS
|
||
Added discussion on Multiple Incoming Mail Exchangers; minor corrections
|
||
related to Sender Callout Verification.
|
||
Revision 0.02 2004-06-30 Revised by: TS
|
||
Added Exim implementation as an appendix
|
||
Revision 0.01 2004-06-16 Revised by: TS
|
||
Initial draft.
|
||
-----------------------------------------------------------------------------
|
||
|
||
5. Credits
|
||
|
||
A number of people have provided feedback, corrections, and contributions,
|
||
as indicated in the Revision History. Thank you!
|
||
|
||
The following are some of the people and groups that have provided tools
|
||
and ideas to this document, in no particular order:
|
||
|
||
* Evan Harris <eharris (at) puremagic.com>, who conceived and wrote a
|
||
white paper on greylisting.
|
||
|
||
* Axel Zinser <fifi (at) hiss.org>, who apparently conceived of
|
||
teergrubing.
|
||
|
||
* The developers of [http://spf.pobox.com/] SPF, [http://www.danisch.de/
|
||
work/security/antispam.html] RMX++, and other Sender Authorization
|
||
Schemes.
|
||
|
||
* The creators and maintainers of distributed, collaborative junk mail
|
||
signature repositories, such as [http://rhyolite.com/anti-spam/dcc/] DCC,
|
||
[http://razor.sf.net/] Razor, and [http://pyzor.sf.net/] Pyzor.
|
||
|
||
* The creators and maintainers of various DNS blocklists and whitelists,
|
||
such as [http://www.spamcop.net/] SpamCop, [http://www.spamhaus.org/]
|
||
SpamHaus, [http://www.sorbs.net/] SORBS, [http://cbl.abuseat.org/] CBL,
|
||
and [http://moensted.dk/spam/] many others.
|
||
|
||
* The [http://www.spamassassin.org/full/3.0.x/dist/CREDITS] developers of
|
||
[http://www.spamassassin.org/] SpamAssassin, who have taken giant leaps
|
||
forward in developing and integrating various spam filtering techniques
|
||
into a sophisticated heuristics-based tool.
|
||
|
||
* Tim Jackson <tim (at) timj.co.uk> collated and maintains a list of
|
||
bogus virus warnings for use with SpamAssassin.
|
||
|
||
* A lot of smart people who developed the excellent Exim MTA, including:
|
||
Philip Hazel <ph10 (at) cus.cam.ac.uk>, the maintainer; Tom Kistner <tom
|
||
(at) duncanthrax.net>, who wrote the Exiscan-ACL patch for SMTP-time
|
||
content checks; Andreas Metzler <ametzler (at) debian.org>, who did a
|
||
really good job of building the Exim 4 Debian packages.
|
||
|
||
* Many, many others who contributed ideas, software, and other techniques
|
||
to counter the spam epidemic.
|
||
|
||
* You, for reading this document and your interest in reclaiming e-mail
|
||
as a useful communication tool
|
||
|
||
|
||
-----------------------------------------------------------------------------
|
||
6. Feedback
|
||
|
||
I would love to hear of your experiences with the techniques outlined in
|
||
this document, and of any other comments, questions, suggestions, and/or
|
||
contributions you may have. Please send me an e-mail at: <tor@slett.net>.
|
||
|
||
If you are able to provide implementations for other Mail Transport Agents,
|
||
such as Sendmail or Postfix, please let me know.
|
||
-----------------------------------------------------------------------------
|
||
|
||
7. Translations
|
||
|
||
No translations exist yet. If you would like to create one, please let me
|
||
know.
|
||
-----------------------------------------------------------------------------
|
||
|
||
8. Copyright information
|
||
|
||
Copyright © 2004 Tor Slettnes.
|
||
|
||
This document is free software; you can redistribute it and/or modify it
|
||
under the terms of the GNU General Public License as published by the Free
|
||
Software Foundation; either version 2 of the License, or (at your option) any
|
||
later version.
|
||
|
||
This document is distributed in the hope that it will be useful, but
|
||
WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
|
||
FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more
|
||
details. A copy of the license is included in Appendix B.
|
||
|
||
Read The GNU Manifesto if you want to know why this license was chosen for
|
||
this book.
|
||
|
||
The logos, trademarks and symbols used in this book are the properties of
|
||
their respective owners.
|
||
-----------------------------------------------------------------------------
|
||
|
||
9. What do you need?
|
||
|
||
The techniques described in this document predicate system access to the
|
||
inbound Mail Exchanger(s) for the internet domain where you receive e-mail.
|
||
Essentially, you need to be able to install software and/or modify the
|
||
configuration files for the Mail Transport Agent on that system.
|
||
|
||
Although the discussions in this document are conceptual in nature and can
|
||
be incorporated into a number of different MTAs, a sample Exim 4
|
||
implementation is provided. This implementation, in turn, incorporates other
|
||
software tools, such as [http://www.spamassassin.org/] SpamAssassin. See
|
||
Appendix A for details.
|
||
-----------------------------------------------------------------------------
|
||
|
||
10. Conventions used in this document
|
||
|
||
The following typographic and usage conventions occur in this text:
|
||
|
||
|
||
Table 1. Typographic and usage conventions
|
||
+-------------------------+-------------------------------------------------+
|
||
|Text type |Meaning |
|
||
+-------------------------+-------------------------------------------------+
|
||
|"Quoted text" |Quotes from people, quoted computer output. |
|
||
+-------------------------+-------------------------------------------------+
|
||
|terminal view |Literal computer input and output captured from |
|
||
| |the terminal, usually rendered with a light grey |
|
||
| |background. |
|
||
+-------------------------+-------------------------------------------------+
|
||
|command |Name of a command that can be entered on the |
|
||
| |command line. |
|
||
+-------------------------+-------------------------------------------------+
|
||
|VARIABLE |Name of a variable or pointer to content of a |
|
||
| |variable, as in $VARNAME. |
|
||
+-------------------------+-------------------------------------------------+
|
||
|option |Option to a command, as in "the -a option to the |
|
||
| |ls command". |
|
||
+-------------------------+-------------------------------------------------+
|
||
|argument |Argument to a command, as in "read man ls ". |
|
||
+-------------------------+-------------------------------------------------+
|
||
|command options arguments|Command synopsis or general usage, on a separated|
|
||
| |line. |
|
||
+-------------------------+-------------------------------------------------+
|
||
|filename |Name of a file or directory, for example "Change |
|
||
| |to the /usr/bin directory." |
|
||
+-------------------------+-------------------------------------------------+
|
||
|Key |Keys to hit on the keyboard, such as "type Q to |
|
||
| |quit". |
|
||
+-------------------------+-------------------------------------------------+
|
||
|Button |Graphical button to click, like the OK button. |
|
||
+-------------------------+-------------------------------------------------+
|
||
|Menu->Choice |Choice to select from a graphical menu, for |
|
||
| |instance: "Select Help->About Mozilla in your |
|
||
| |browser." |
|
||
+-------------------------+-------------------------------------------------+
|
||
|Terminology |Important term or concept: "The Linux kernel is |
|
||
| |the heart of the system." |
|
||
+-------------------------+-------------------------------------------------+
|
||
|See Glossary |link to related subject within this guide. |
|
||
+-------------------------+-------------------------------------------------+
|
||
|[http://slett.net/gallery|Clickable link to an external web resource. |
|
||
|/2003-05/IMG_1655] The | |
|
||
|author | |
|
||
+-------------------------+-------------------------------------------------+
|
||
-----------------------------------------------------------------------------
|
||
|
||
11. Organization of this document
|
||
|
||
This document is organized into the following chapters:
|
||
|
||
Background
|
||
General introduction to SMTP time filtering, and to SMTP.
|
||
|
||
Techniques
|
||
Various ways to block junk mail in an SMTP transaction.
|
||
|
||
Considerations
|
||
Issues that pertain to transaction time filtering.
|
||
|
||
Questions & Answers
|
||
My attempt at anticipating your questions, and then answering them.
|
||
|
||
|
||
A sample Exim implementation is provided in Appendix A.
|
||
-----------------------------------------------------------------------------
|
||
|
||
Chapter 1. Background
|
||
|
||
|
||
Here we cover the advantages of filtering mail during an incoming SMTP
|
||
transaction, rather than following the more conventional approach of
|
||
offloading this task to the mail routing and delivery stage. We also
|
||
provide a brief introduction to the SMTP transaction.
|
||
|
||
|
||
-----------------------------------------------------------------------------
|
||
1.1. Why Filter Mail During the SMTP Transaction?
|
||
|
||
1.1.1. Status Quo
|
||
|
||
If you receive spam, raise your hands. Keep them up.
|
||
|
||
If you receive computer virii or other malware, raise your hands too.
|
||
|
||
If you receive bogus Delivery Status Notifications (DSNs), such as "Message
|
||
Undeliverable", "Virus found", "Please confirm delivery", etc, related to
|
||
messages you never sent, raise your hands as well. This is known as
|
||
Collateral Spam.
|
||
|
||
This last form is particularly troublesome, because it is harder to weed
|
||
out than "standard" spam or malware, and because such messages can be quite
|
||
confusing to recipients who do not possess godly skills in parsing message
|
||
headers. In the case of virus warnings, this often causes unnecessary concern
|
||
on the recipient's end; more generally, a common tendency will be to ignore
|
||
all such messages, thereby missing out on legitimate DSNs.
|
||
|
||
Finally, I want those of you who have lost legitimate mail into a big black
|
||
hole - due to misclassification by spam or virus scanners - to lift your
|
||
feet.
|
||
|
||
If you were standing before and are still standing, I suggest that you may
|
||
not be fully aware of what is happening to your mail. If you have been doing
|
||
any type of spam filtering, even by manually moving mails to the trash can in
|
||
your mail reader, let alone by experimenting with primitive filtering
|
||
techniques such as DNS blacklists (SpamHaus, SPEWS, SORBS...), chances are
|
||
that you have lost some valid mail.
|
||
-----------------------------------------------------------------------------
|
||
|
||
1.1.2. The Cause
|
||
|
||
Spam, just like many other artifacts of greed, is a social disease. Call it
|
||
affluenza, or whatever you like; lower life forms seek to destroy a larger
|
||
ecosystem, and if successful, will actually end up ruining their own habitat
|
||
in the end.
|
||
|
||
Larger social issues and philosophy aside: You - the mail system
|
||
administrator - face the very concrete and real life dilemma of finding a way
|
||
to deal with all this junk.
|
||
|
||
As it turns out, there are some limitations with the conventional way that
|
||
mail is being processed and delegated by the various components of mail
|
||
transport and delivery software. In a traditional setup, one or more Mail
|
||
Exchanger(s) accept most or all incoming mail deliveries to addresses within
|
||
a domain. Often, they then forward the mail to one or more internal machines
|
||
for further processing, and/or delivery to the user's mailboxes. If any of
|
||
these servers discovers that it is unable to perform the requested delivery
|
||
or function, it generates and returns a DSN back to the sender address in the
|
||
original mail.
|
||
|
||
As organizations started deploying spam and virus scanners, they often
|
||
found that the path of least resistance was to work these into the message
|
||
delivery path, as mail is transferred from the incoming Mail Exchanger(s) to
|
||
internal delivery hosts and/or software. For instance, a common way filter
|
||
out spam is by routing the mail through SpamAssassin or other software before
|
||
it is delivered to a user's mailbox, and/or rely on spam filtering
|
||
capabilities in the user's Mail User Agent.
|
||
|
||
Options for dealing with mail that is classified as spam or virus at this
|
||
point are limited:
|
||
|
||
* You can return a Delivery Status Notification back to the sender. The
|
||
problem is that nearly all spam and e-mail borne virii are delivered with
|
||
faked sender addresses. If you return this mail, it will invariably go to
|
||
innocent third parties -- perhaps warning a grandmother in Sweden, who
|
||
uses Mac OS X and does not know much about computers, that she is
|
||
infected by the Blaster worm. In other words, you will be generating
|
||
Collateral Spam.
|
||
|
||
* You can drop the message into the bit bucket, without sending any
|
||
notification back to the sender. This is an even bigger problem in the
|
||
case of False Positives, because neither the sender nor the receiver will
|
||
ever know what happened to the message (or in the receiver's case, that
|
||
it ever existed).
|
||
|
||
* Depending on how your users access their mail (for instance, if they
|
||
access it via the IMAP protocol or use a web-based mail reader, but not
|
||
if they retreive it over POP-3), you may be able to file it into a
|
||
separate junk folder for them -- perhaps as an option in their account
|
||
settings.
|
||
|
||
This may be the best of these three options. Even so, the messages may
|
||
remain unseen for some time, or simply overlooked as the receiver
|
||
more-or-less periodically scans through and deletes mail in their "Junk"
|
||
folder.
|
||
|
||
|
||
-----------------------------------------------------------------------------
|
||
1.1.3. The Solution
|
||
|
||
As you would have guessed by now, the One True solution to this problem is
|
||
to do spam and virus filtering during the SMTP dialogue from the remote host,
|
||
as the mail is being received by the inbound mail exchanger for your domain.
|
||
This way, if the mail turns out to be undesirable, you can issue a SMTP
|
||
reject response rather than face the dilemma described above. As a result:
|
||
|
||
* You will be able to stop the delivery of most junk mail early in the
|
||
SMTP transaction, before the actual message data has been received, thus
|
||
saving you both network bandwidth and CPU processing.
|
||
|
||
* You will be able to deploy some spam filtering techniques that are not
|
||
possible later, such as SMTP transaction delays and Greylisting.
|
||
|
||
* You will be able to notify the sender in case of a delivery failure
|
||
(e.g. due to an invalid recipient address) without directly generating
|
||
Collateral Spam
|
||
|
||
We will discuss how you can avoid causing collateral spam indirectly as
|
||
a result of rejecting mail forwarded from trusted sources, such as
|
||
mailing list servers or mail accounts on other sites [1].
|
||
|
||
* You will be able to protect yourself against collateral spam from
|
||
others (such as bogus "You have a virus" messages from anti-virus
|
||
software).
|
||
|
||
|
||
OK, you can lower your hands now. If you were standing, and your feet
|
||
disappeared from under you, you can now also stand up again.
|
||
-----------------------------------------------------------------------------
|
||
|
||
1.2. The Good, The Bad, The Ugly
|
||
|
||
Some filtering techniques are more suitable for use during the SMTP
|
||
transaction than others. Some are simply better than others. Nearly all have
|
||
their proponents and opponents.
|
||
|
||
Needless to say, these controversies extend to the methods described here
|
||
as well. For instance:
|
||
|
||
* Some argue that DNS checks penalize individual mail senders purely
|
||
based on their Internet Service Provider (ISP), not on the merits of
|
||
their particular message.
|
||
|
||
* Some point out that ratware traps like SMTP transaction delays and
|
||
Greylisting are easily overcome and will be less effective over time,
|
||
while continuing to degrade the Quality of Service for legitimate mail.
|
||
|
||
* Some find that Sender Authorization Schemes like the Sender Policy
|
||
Framework give ISPs a way to lock their customers in, and do not
|
||
adequately address users who roam between different networks or who
|
||
forward their e-mail from one host to another.
|
||
|
||
|
||
I will steer away from most of these controversies. Instead, I will try to
|
||
provide a functional description of the various techniques available,
|
||
including their possible side effects, and then talk a little about my own
|
||
experiences using some of them.
|
||
|
||
That said, there are some filtering methods in use today that I
|
||
deliberately omit from this document:
|
||
|
||
* Challenge/response systems (like [http://tmda.net/] TMDA). These are
|
||
not suitable for SMTP time filtering, as they rely on first accepting the
|
||
mail, then returning a confirmation request to the Envelope Sender. This
|
||
technique is therefore outside the scope of this document. [2]
|
||
|
||
* Bayesian Filters. These require training specific to a particular user,
|
||
and/or a particular language. As such, these too are not normally
|
||
suitable for use during the SMTP transaction (But see User Settings and
|
||
Data).
|
||
|
||
* Micropayment Schemes are not really suitable for weeding out junk mail
|
||
until all the world's legitimate mail is sent with a virtual postage
|
||
stamp. (Though in the mean time, they can be used for the opposite
|
||
purpose - that is, to accept mail carrying the stamp that would otherwise
|
||
be rejected).
|
||
|
||
|
||
Generally, I have attempted to offer techniques that are as precise as
|
||
possible, and to go to great lengths to avoid False Positives. People's
|
||
e-mail is important to them, and they spend time and effort writing it. In my
|
||
view, willfully using techniques or tools that reject large amounts of
|
||
legitimate mail is a show of disrespect, both to the people that are directly
|
||
affected and to the Internet as a whole. [3] This is especially true for
|
||
SMTP-time system wide filtering, because end recipients usually have little
|
||
or no control over the criteria being used to filter their mail.
|
||
-----------------------------------------------------------------------------
|
||
|
||
1.3. The SMTP Transaction
|
||
|
||
SMTP is the protocol that is used for mail delivery on the Internet. For a
|
||
detailed description of the protocol, please refer to RFC 2821, as well as
|
||
Dave Crocker's introduction to [http://www.brandenburg.com/specifications/
|
||
draft-crocker-mail-arch-00.htm] Internet Mail Architecture.
|
||
|
||
Mail deliveries involve an SMTP transaction between the connecting host
|
||
(client) and the receiving host (server). For this discussion, the connecting
|
||
host is the peer, and the receiving host is your server.
|
||
|
||
In a typical SMTP transaction, the client issues SMTP commands such as EHLO
|
||
, MAIL FROM:, RCPT TO:, and DATA. Your server responds to each command with a
|
||
3-digit numeric code indicating whether the command was accepted (2xx), was
|
||
subject to a temporary failure or restriction (4xx), or failed definitively/
|
||
permanently (5xx), followed by some human readable explanation. A full
|
||
description of these codes is included in [http://www.ietf.org/rfc/
|
||
rfc2821.txt] RFC 2821.
|
||
|
||
A best case scenario SMTP transaction typically consists of the following
|
||
relevant steps:
|
||
|
||
|
||
Table 1-1. Simple SMTP dialogue
|
||
+-------------------------------------+-------------------------------------+
|
||
|Client |Server |
|
||
+-------------------------------------+-------------------------------------+
|
||
|Initiates a TCP connection to server.|Presents an SMTP banner - that is, a |
|
||
| |greeting that starts with the code |
|
||
| |220 to indicate that it is ready to |
|
||
| |speak SMTP (or usually ESMTP, a |
|
||
| |superset of SMTP): |
|
||
| |220 your.f.q.d.n ESTMP... |
|
||
+-------------------------------------+-------------------------------------+
|
||
|Introduces itself by way of an Hello |Accepts this greeting with a 250 |
|
||
|command, either HELO (now obsolete) |response. If the client used the |
|
||
|or EHLO, followed by its own Fully |extended version of the Hello command|
|
||
|Qualified Domain Name: |(EHLO), your server knows that it is |
|
||
|EHLO peers.f.q.d.n |capable of handling multi-line |
|
||
| |responses, and so will normally send |
|
||
| |back several lines indicating the |
|
||
| |capabilities offered by your server: |
|
||
| |250-your.f.q.d.n Hello ... |
|
||
| |250-SIZE 52428800 |
|
||
| |250-8BITMIME |
|
||
| |250-PIPELINING |
|
||
| |250-STARTTLS |
|
||
| |250-AUTH |
|
||
| |250 HELP |
|
||
| | |
|
||
| |If the PIPELINING capability is |
|
||
| |included in this response, the client|
|
||
| |can from this point forward issue |
|
||
| |several commands at once, without |
|
||
| |waiting for the response to each one.|
|
||
+-------------------------------------+-------------------------------------+
|
||
|Starts a new mail transaction by |Issues a 250 response to indicate |
|
||
|specifying the Envelope Sender: |that the sender is accepted. |
|
||
|MAIL FROM:<sender@address> | |
|
||
| | |
|
||
+-------------------------------------+-------------------------------------+
|
||
|Lists the Envelope Recipients of the |Issues a response to each command ( |
|
||
|message, one at a time, using the |2xx, 4xx, or 5xx, depending on |
|
||
|command: |whether delivery to this recipient |
|
||
|RCPT TO:<receiver@address> |was accepted, subject to a temporary |
|
||
| |failure, or rejected). |
|
||
+-------------------------------------+-------------------------------------+
|
||
|Issues a DATA command to indicate |Responds 354 to indicate that the |
|
||
|that it is ready to send the message.|command has been provisionally |
|
||
| |accepted. |
|
||
+-------------------------------------+-------------------------------------+
|
||
|Transmits the message, starting with |Replies 250 to indicate that the |
|
||
|RFC 2822 compliant header lines (such|message has been accepted. |
|
||
|as: From:, To:, Subject:, Date:, | |
|
||
|Message-ID:). The header and the body| |
|
||
|are separated by an empty line. To | |
|
||
|indicate the end of the message, the | |
|
||
|client sends a single period (".") on| |
|
||
|a separate line. | |
|
||
+-------------------------------------+-------------------------------------+
|
||
|If there are more messages to be |Disconnects. |
|
||
|delivered, issues the next MAIL FROM:| |
|
||
|command. Otherwise, it says QUIT, or | |
|
||
|in rare cases, simply disconnects. | |
|
||
+-------------------------------------+-------------------------------------+
|
||
-----------------------------------------------------------------------------
|
||
|
||
Chapter 2. Techniques
|
||
|
||
|
||
In this chapter, we look at various ways to weed out junk mail during
|
||
the SMTP transaction from remote hosts. We will also try to anticipate
|
||
some of the side effects from deploying these techniques.
|
||
|
||
|
||
-----------------------------------------------------------------------------
|
||
2.1. SMTP Transaction Delays
|
||
|
||
As it turns out, one of the more effective ways of stopping spam is by
|
||
imposing transaction delays during an inbound SMTP dialogue. This is a
|
||
primitive form of teergrubing, see: [http://www.iks-jena.de/mitarb/lutz/
|
||
usenet/teergrube.en.html] http://www.iks-jena.de/mitarb/lutz/usenet/
|
||
teergrube.en.html
|
||
|
||
Most spam and nearly all e-mail borne virii are delivered directly to your
|
||
server by way of specialized SMTP client software, optimized for sending out
|
||
large amounts of mail in a very short time. Such clients are commonly known
|
||
as Ratware.
|
||
|
||
In order to accomplish this task, ratware authors commonly take a few
|
||
shortcuts that, ahem, "diverge" a bit from the RFC 2821 specification. One of
|
||
the intrinsic traits of ratware is that it is notoriously impatient,
|
||
especially with slow-responding mail servers. They may issue the HELO or EHLO
|
||
command before the server has presented the initial SMTP banner, and/or try
|
||
to pipeline several SMTP commands before the server has advertised the
|
||
PIPELINING capability.
|
||
|
||
Certain Mail Transport Agents (such as Exim) automatically treat such SMTP
|
||
protocol violations as synchronization errors, and immediately drop the
|
||
incoming connection. If you happen to be using such an MTA, you may already
|
||
see a lot of entries to this effect in your log files. In fact, chances are
|
||
that if you perform any time-consuming checks (such as DNS checks) prior to
|
||
presenting the initial SMTP banner, such errors will occur frequently, as
|
||
ratware clients simply do not take the time to wait for your server to come
|
||
alive (Things to do, people to spam).
|
||
|
||
We can help along by imposing additional delays. For instance, you may
|
||
decide to wait:
|
||
|
||
* 20 seconds before presenting the initial SMTP banner,
|
||
|
||
* 20 seconds after the Hello (EHLO or HELO) greeting,
|
||
|
||
* 20 seconds, after the MAIL FROM: command, and
|
||
|
||
* 20 seconds after each RCPT TO: command.
|
||
|
||
|
||
Where did 20 seconds come from, you ask. Why not a minute? Or several
|
||
minutes? After all, RFC 2821 mandates that the sending host (client) should
|
||
wait up to several minutes for every SMTP response. The issue is that some
|
||
receiving hosts, particularly those that use Exim, may perform Sender Callout
|
||
Verification in response to incoming mail delivery attempts. If you or one of
|
||
your users send mail to such a host, it will contact the Mail Exchanger (MX
|
||
host) for your domain and start an SMTP dialogue in order to validate the
|
||
sender address. The default timeout of such Sender Callout Verifications is
|
||
30 seconds - if you impose delays this long, the peer's sender callout
|
||
verification would fail, and in turn the original mail delivery from you/your
|
||
user might be rejected (usually with a temporary failure, which means the
|
||
message delivery will be retried for 5 days or so before the mail is finally
|
||
returned to the sender).
|
||
|
||
In other words, 20 seconds is about as long as you can stall before you
|
||
start interfering with legitimate mail deliveries.
|
||
|
||
If you do not like imposing such delays on every SMTP transaction (say, you
|
||
have a very busy site and are low on machine resources), you may choose to
|
||
use "selective" transaction delays. In this case, you could impose the delay:
|
||
|
||
* If there is a problem with the peer's DNS information (see DNS checks).
|
||
|
||
* After detecting some sign of trouble during the SMTP transaction (see
|
||
SMTP checks).
|
||
|
||
* Only in the highest-numbered MX host in your DNS zone, i.e. the mail
|
||
exchanger with the last priority. Often, Ratware specifically target
|
||
these hosts, whereas legitimate MTAs will try the lower-numbered MX hosts
|
||
first.
|
||
|
||
|
||
In fact, selective transaction delays may be a good way to incorporate some
|
||
less conclusive checks that we will discuss in the following sections. You
|
||
probably do not wish to reject the mail outright based the results from e.g.
|
||
the SPEWS blacklist, but on the other hand, it may provide a strong enough
|
||
indication of trouble that you can at least impose transaction delays. After
|
||
all, legitimate mail deliveries are not affected, other than being subjected
|
||
to a slight delay.
|
||
|
||
Conversely, if you find conclusive evidence of spamming (e.g. by way of
|
||
certain SMTP checks), and your server can afford it, you may choose to impose
|
||
an extended delay, e.g. 15 minutes or so, before finally rejecting the
|
||
delivery [4]. This is for little or no benefit other than slowing down the
|
||
spammer a little bit in their quest to reach as many people as possible
|
||
before DNS blacklists and other collaborative network checks catch up. In
|
||
other words, pure altruism on your side. :-)
|
||
|
||
In my own case, selective transaction delays and the resulting SMTP
|
||
synchronization errors account for nearly 50% of rejected incoming delivery
|
||
attempts. This roughly translates into saying that nearly 50% of incoming
|
||
junk mail is stopped by SMTP transaction delays alone.
|
||
|
||
See also What happens when spammers adapt....
|
||
-----------------------------------------------------------------------------
|
||
|
||
2.2. DNS Checks
|
||
|
||
Some indication of the integrity of a particular peer can be gleaned
|
||
directly from the Domain Name System (DNS), even before SMTP commands are
|
||
issued. In particular, various DNS blacklists can be consulted to find out if
|
||
a particular IP address is known to violate or fulfill certain criteria, and
|
||
a simple pair of forward/reverse (DNS/rDNS) lookups can be used as a vague
|
||
indicator of the host's general integrity.
|
||
|
||
Moreover, various data items presented during the SMTP dialogue (such as
|
||
the name presented in the Hello greeting) can be subjected to DNS validation,
|
||
once it becomes available. For a discussion on these items, see the section
|
||
on SMTP checks, below.
|
||
|
||
A word of caution, though. DNS checks are not always conclusive (e.g. a
|
||
required DNS server may not be responding), and not always indicative of
|
||
spam. Moreover, if you have a very busy site, they can be expensive in terms
|
||
of processing time per message. That said, they can provide useful
|
||
information for logging purposes, and/or as part of a more holistic integrity
|
||
check.
|
||
-----------------------------------------------------------------------------
|
||
|
||
2.2.1. DNS Blacklists
|
||
|
||
DNS blacklists (DNSbl's, formerly called "Real-time Black-hole Lists" after
|
||
the original blacklist, "mail-abuse.org") make up perhaps the most common
|
||
tool to perform transaction-time spam blocking. The receiving server performs
|
||
one or more rDNS lookups of the peer's IP address within various DNSbl zones,
|
||
such as "dnsbl.sorbs.net", "opm.blitzed.org", "lists.dsbl.org", and so forth.
|
||
If a matching DNS record is found, a typical action is to reject the mail
|
||
delivery. [5]
|
||
|
||
If in addition to the DNS address ("A" record) you look up the "TXT" record
|
||
of an entry, you will typically receive a one-line description of the
|
||
listing, suitable for inclusion in a SMTP reject response. To try this out,
|
||
you can use the "host" command provided on most Linux and UNIX systems:
|
||
host -t txt 2.0.0.127.dnsbl.sorbs.net
|
||
|
||
There are currently hundreds of these lists available, each with different
|
||
listing criteria, and with different listing/unlisting policies. Some lists
|
||
even combine several listing criteria into the same DNSbl, and issue
|
||
different data in response to the rDNS lookup, depending on which criterion
|
||
affects the address provided. For instance, a rDNS lookup against
|
||
sbl-xbl.spamhaus.org returns 127.0.0.2 for IP addresses that are believed by
|
||
the SpamHaus staff to directly belong to spammers and their providers,
|
||
127.0.0.4 response for Zombie Hosts, or a 127.0.0.6 response for Open Proxy
|
||
servers.
|
||
|
||
Unfortunately, many of these lists contain large blocks of IP addresses
|
||
that are not directly responsible for the alleged violations, don't have
|
||
clear listing / delisting policies, and/or post misleading information about
|
||
which addresses are listed[6]. The blind trust in such lists often cause a
|
||
large amount of what is referred to as Collateral Damage (not to be confused
|
||
with Collateral Spam).
|
||
|
||
For that reason, rather than rejecting mail deliveries outright based on a
|
||
single positive response from DNS blacklists, many administrators prefer to
|
||
use these lists in a more nuanced fashion. They may consult several lists,
|
||
and assign a "score" to each positive response. If the total score for a
|
||
given IP address reaches a given threshold, deliveries from that address are
|
||
rejected. This is how DNS blacklists are used by filtering software such as
|
||
SpamAssassin (Spam Scanners).
|
||
|
||
One could also use such lists as one of several triggers for SMTP
|
||
transaction delays on incoming connections (a.k.a. "teergrubing"). If a host
|
||
is listed in a DNSbl, your server would delay its response to every SMTP
|
||
command issued by the peer for, say, 20 seconds. Several other criteria can
|
||
be used as triggers for such delays; see the section on SMTP transaction
|
||
delays.
|
||
-----------------------------------------------------------------------------
|
||
|
||
2.2.2. DNS Integrity Check
|
||
|
||
Another way to use DNS is to perform a reverse lookup of the peer's IP
|
||
address, then a forward lookup of the resulting name. If the original IP
|
||
address is included in the result, its DNS integrity has been validated.
|
||
Otherwise, the DNS information for the connecting host is not valid.
|
||
|
||
Rejecting mails based on this criterion may be an option if you are a
|
||
militant member of the DNS police, setting up an incoming MX for your own
|
||
personal domain, and don't mind rejecting legitimate mail as a way to impress
|
||
upon the sender that they need to ask their own system administrator to clean
|
||
up their DNS records. For everyone else, the result of a DNS integrity check
|
||
should probably only be used as one data point in a larger set of heuristics.
|
||
Alternatively, as above, using SMTP transaction delays for misconfigured
|
||
hosts may not be a bad idea.
|
||
-----------------------------------------------------------------------------
|
||
|
||
2.3. SMTP checks
|
||
|
||
Once the SMTP dialogue is underway, you can perform various checks on the
|
||
commands and arguments presented by the remote host. For instance, you will
|
||
want to ensure that the name presented in the Hello greeting is valid.
|
||
|
||
However, even if you decide to reject the delivery attempt early in the
|
||
SMTP transaction, you may not want to perform the actual rejection right
|
||
away. Instead, you may stall the sender with SMTP transaction delays until
|
||
after the RCPT TO:, then reject the mail at that point.
|
||
|
||
The reason is that some ratware does not understand rejections early in the
|
||
SMTP transaction; they keep trying. On the other hand, most of them give up
|
||
if the RCPT TO: fails.
|
||
|
||
Besides, this gives a nice opportunity to do a little teergrubing.
|
||
-----------------------------------------------------------------------------
|
||
|
||
2.3.1. Hello (HELO/EHLO) checks
|
||
|
||
Per RFC 2821, the first SMTP command issued by the client should be EHLO
|
||
(or if unsupported, HELO), followed by its primary, Fully Qualified Domain
|
||
Name. This is known as the Hello greeting. If no meaningful FQDN is
|
||
available, the client can supply its IP address enclosed in square brackets:
|
||
"[1.2.3.4]". This last form is known as an IPv4 address "literal" notation.
|
||
|
||
Quite understandably, Ratware rarely present their own FQDN in the Hello
|
||
greeting. Rather, greetings from ratware usually attempt to conceal the
|
||
sending host's identity, and/or to generate confusing and/or misleading
|
||
"Received:" trails in the message header. Some examples of such greetings
|
||
are:
|
||
|
||
* Unqualified names (i.e. names without a period), such as the "local
|
||
part" (username) of the recipient address.
|
||
|
||
* A plain IP address (i.e. not an IP literal); usually yours, but can be
|
||
a random one.
|
||
|
||
* Your domain name, or the FQDN of your server.
|
||
|
||
* Third party domain names, such as yahoo.com and hotmail.com.
|
||
|
||
* Non-existing domain names, or domain names with non-existing name
|
||
servers.
|
||
|
||
* No greeting at all.
|
||
|
||
|
||
-----------------------------------------------------------------------------
|
||
2.3.1.1. Simple HELO/EHLO syntax checks
|
||
|
||
Some of these RFC 2821 violations are both easy to check against, and clear
|
||
indications that the sending host is running some form of Ratware. You can
|
||
reject such greetings -- either right away, or e.g. after the RCPT TO:
|
||
command.
|
||
|
||
First, feel free to reject plain IP addresses in the Hello greeting. Even
|
||
if you wish to generously allow everything RFC 2821 mandates, recommends, and
|
||
suggests, you will note that IP addresses should always be enclosed in square
|
||
brackets when presented in lieu of a name. [7]
|
||
|
||
In particular, you may wish to issue a strongly worded rejection message to
|
||
hosts that introduce themselves using your IP address - or for that matter,
|
||
your host name. They are plainly lying. Perhaps you want to stall the sender
|
||
with an exceedingly long SMTP transaction delay in response to such a
|
||
greeting; say, hours.
|
||
|
||
For that matter, my own experience indicates that no legitimate sites on
|
||
the internet present themselves to other internet sites using an IP address
|
||
literal (the [x.y.z.w] notation) either. Nor should they; all hosts sending
|
||
mail directly on the internet should use their valid Fully Qualified Domain
|
||
Name. The only use of use of IP literals I have come across is from mail user
|
||
agents on my local area network, such as Ximian Evolution, configured to use
|
||
my server as outgoing SMTP server (smarthost). Indeed, I only accept literals
|
||
from my own LAN.
|
||
|
||
You may or may not also wish to reject unqualified host names (host names
|
||
without a period). I find that these are rarely (but not never - how's that
|
||
for double negative negations) legitimate.
|
||
|
||
Similarly, you can reject host names that contain invalid characters. For
|
||
internet domains, only alphanumeric letters and hyphen are valid characters;
|
||
a hyphen is not allowed as the first character. (You may also want to
|
||
consider the underscore a valid character, because it is quite common to see
|
||
this from misconfigured, but ultimately well-meaning, Windows clients).
|
||
|
||
Finally, if you receive a MAIL FROM: command without first having received
|
||
a Hello greeting, well, polite people greet first.
|
||
|
||
On my servers, I reject greetings that fail any of these syntax checks.
|
||
However, the rejection does not actually take place until after the RCPT TO:
|
||
command. In the mean time, I impose a 20 second transaction delay after each
|
||
SMTP command (HELO/EHLO, MAIL FROM:, RCPT TO:).
|
||
-----------------------------------------------------------------------------
|
||
|
||
2.3.1.2. Verifying the Hello greeting via DNS
|
||
|
||
Hosts that make it this far have presented at least a superficially
|
||
credible greeting. Now it is time to verify the provided name via DNS. You
|
||
can:
|
||
|
||
* Perform a forward lookup of the provided name, and match the result
|
||
against the peer's IP address
|
||
|
||
* Perform a reverse lookup of the peer's IP address, and match it against
|
||
name provided in the greeting.
|
||
|
||
|
||
If either of these two checks succeeds, the name has been verified.
|
||
|
||
Your MTA may have a built-in option to perform this check. For instance, in
|
||
Exim (see Appendix A), you want to set "helo_try_verify_hosts = *", and
|
||
create ACLs that take action based on the "verify = helo" condition.
|
||
|
||
This check is a little more expensive in terms of processing time and
|
||
network resources than the simple syntax checks. Moreover, unlike the syntax
|
||
checks, a mismatch does not always indicate ratware; several large internet
|
||
sites, such as hotmail.com, yahoo.com, and amazon.com, frequently present
|
||
unverifiable Hello greetings.
|
||
|
||
On my servers, I do a DNS validation of the Hello greeting if I am not
|
||
already stalling the sender with transaction delays based on prior checks.
|
||
Then, if this check fails, I impose a 20 second delay on every SMTP command
|
||
from this point forward. I also prepare a "X-HELO-Warning:" header that I
|
||
will later add to the message(s), and use to increase the SpamAssassin score
|
||
for possible rejection after the message data has been received.
|
||
-----------------------------------------------------------------------------
|
||
|
||
2.3.2. Sender Address Checks
|
||
|
||
After the client has presented the MAIL FROM: <address> command, you can
|
||
validate the supplied Envelope Sender address as follows. [8]
|
||
-----------------------------------------------------------------------------
|
||
|
||
2.3.2.1. Sender Address Syntax Check
|
||
|
||
Does the supplied address conform to the format <localpart@domain>? Is the
|
||
domain part a syntactically valid Fully Qualified Domain Name?
|
||
|
||
Often, your MTA performs these checks by default.
|
||
-----------------------------------------------------------------------------
|
||
|
||
2.3.2.2. Impostor Check
|
||
|
||
In the case where you and your users send all your outgoing mail only
|
||
through a select few servers, you can reject messages from other hosts in
|
||
which the "domain" of the sender address is your own.
|
||
|
||
A more general alternative to this check is Sender Policy Framework.
|
||
-----------------------------------------------------------------------------
|
||
|
||
2.3.2.3. Simple Sender Address Validation
|
||
|
||
If the address is local, is the "local part" (the part before the @ sign) a
|
||
valid mailbox on your system?
|
||
|
||
If the address is remote, does the "domain" (the part after the @ sign)
|
||
exist?
|
||
-----------------------------------------------------------------------------
|
||
|
||
2.3.2.4. Sender Callout Verification
|
||
|
||
This is a mechanism that is offered by some MTAs, such as Exim and Postfix,
|
||
to validate the "local part" of a remote sender address. In Postfix
|
||
terminology, it is called "Sender Address Verification".
|
||
|
||
Your server contacts the MX for the domain provided in the sender address,
|
||
attempting to initiate a secondary SMTP transaction as if delivering mail to
|
||
this address. It does not actually send any mail; rather, once the RCPT TO:
|
||
command has been either accepted or rejected by the remote host, your server
|
||
sends QUIT.
|
||
|
||
By default, Exim uses an empty envelope sender address for such callout
|
||
verifications. The goal is to determine if a Delivery Status Notification
|
||
would be accepted if returned to the sender.
|
||
|
||
Postfix, on the other hand, defaults to the sender address <
|
||
postmaster@domain> for address verification purposes (domain is taken from
|
||
the $myorigin variable). For this reason, you may wish to treat this sender
|
||
address the same way that you treat the NULL envelope sender (for instance,
|
||
avoid SMTP transaction delays or Greylisting, but require Envelope Sender
|
||
Signatures in recipient addresses). More on this in the implementation
|
||
appendices.
|
||
|
||
You may find that this check alone may not be suitable as a trigger to
|
||
reject incoming mail. Occasionally, legitimate mail, such as a recurring
|
||
billing statement, is sent out from automated services with an invalid return
|
||
address. Also, an unfortunate side effect of spam is that some users tend to
|
||
mangle the return address in their outgoing mails (though this may affect the
|
||
"From:" header in the message itself more often than the Envelope Sender).
|
||
|
||
Moreover, this check only verifies that an address is valid, not that it
|
||
was authentic as the sender of this particular message (but see also Envelope
|
||
Sender Signature).
|
||
|
||
Finally, there are reports of sites, such as "aol.com", that will
|
||
unconditionally blacklist any system from which they discover sender callout
|
||
requests. These sites may be frequent victims of Joe Jobs, and as a result,
|
||
receive storms of sender callout requests. By taking part in these DDoS
|
||
(Distributed Denial-of-Servcie) attacks, you are effectively turning yourself
|
||
into a pawn in the hands of the spammer.
|
||
-----------------------------------------------------------------------------
|
||
|
||
2.3.3. Recipient Address Checks
|
||
|
||
This should be simple, you say. A recipient address is either valid, in
|
||
which case the mail is delivered, or invalid, in which case your MTA takes
|
||
care of the rejection by default.
|
||
|
||
Let us have a look, shall we?
|
||
-----------------------------------------------------------------------------
|
||
|
||
2.3.3.1. Open Relay Prevention
|
||
|
||
Do not relay mail from remote hosts to remote addresses! (Unless the sender
|
||
is authenticated).
|
||
|
||
This may seem obvious to most of us, but apparently this is a frequently
|
||
overlooked consideration. Also, not everyone may have a full grasp of the
|
||
various internet standards related to e-mail addresses and delivery paths
|
||
(consider "percent hack domains", "bang (!) paths", etc).
|
||
|
||
If you are unsure whether your MTA acts as an an Open Relay, you can test
|
||
it via "relay-test.mail-abuse.org". At a shell prompt on your server, type:
|
||
telnet relay-test.mail-abuse.org
|
||
|
||
This is a service that will use various tests to see whether your SMTP
|
||
server appears to forward mail to remote e-mail addresses, and/or any number
|
||
of address "hacks" such as the ones mentioned above.
|
||
|
||
Preventing your servers from acting as open relays is extremely important.
|
||
If your server is an open relay, and spammers find you, you will be listed in
|
||
numerous DNS blacklists instantly. If the maintainers of certain other DNS
|
||
blacklists find you (by probing, and/or by acting on complaints), you will be
|
||
listed in those for an extended period of time.
|
||
-----------------------------------------------------------------------------
|
||
|
||
2.3.3.2. Recipient Address Lookups
|
||
|
||
This, too may seem banal to most of us. It is not always so.
|
||
|
||
If your users' mail accounts and mailboxes are stored directly on your
|
||
incoming mail exchanger, you can simply check that the "local part" of the
|
||
recipient address corresponds to a valid mailbox. No problem here.
|
||
|
||
There are two scenarios where verification of the recipient address is more
|
||
cumbersome:
|
||
|
||
* If your machine is a backup MX for the recipient domain.
|
||
|
||
* If your machine forwards all mail for your domain to another
|
||
(presumably internal) server.
|
||
|
||
|
||
The alternative to recipient address verification is to accept all
|
||
recipient addresses within these respective domains, which in turn means that
|
||
you or the destination server might have to generate a Delivery Status
|
||
Notification for recipient addresses that later turn out to be invalid.
|
||
Ultimately, this means that you would be generating collateral spam.
|
||
|
||
With that in mind, let us see how we can verify the recipient in the
|
||
scenarios listed above.
|
||
-----------------------------------------------------------------------------
|
||
|
||
2.3.3.2.1. Recipient Callout Verification
|
||
|
||
This is a mechanism that is offered by some MTAs, such as Exim and Postfix,
|
||
to verify the "local part" of a remote recipient address (see Sender Callout
|
||
Verification for a description of how this works). In Postfix terminology,
|
||
this is called "Recipient Address Verification".
|
||
|
||
In this case, server attempts to contact the final destination host to
|
||
validate each recipient address before you, in turn, accept the RCPT TO:
|
||
command from your peer.
|
||
|
||
This solution is simple and elegant. It works with any MTA that might be
|
||
running on the final destination host, and without access to any particular
|
||
directory service. Moreover, if that MTA happens to perform a fuzzy match on
|
||
the recipient address (this is the case with Lotus Domino servers), this
|
||
check will accurately reflect whether the recipient address is eventually
|
||
going to be accepted or not - something which may not be true for the
|
||
mechanisms described below.
|
||
|
||
Be sure to keep the original Envelope Sender intact for the recipient
|
||
callout, or the response from the destination host may not be accurate. For
|
||
instance, it may reject bounces (i.e. mail with no envelope sender) for
|
||
system users and aliases, as described in Accept Bounces Only for Real Users.
|
||
|
||
Among major MTAs, Exim and Postfix support this mechanism.
|
||
-----------------------------------------------------------------------------
|
||
|
||
2.3.3.2.2. Directory Services
|
||
|
||
Another good solution would be a directory service (e.g. one or more LDAP
|
||
servers) that can be queried by your MTA. The most common MTAs all support
|
||
LDAP, NIS, and/or various other backends that are commonly used to provide
|
||
user account information.
|
||
|
||
The main sticking point is that unless the final destination host of the
|
||
e-mail already uses such a directory service to map user names to mailboxes,
|
||
there may be some work involved in setting this up.
|
||
-----------------------------------------------------------------------------
|
||
|
||
2.3.3.2.3. Replicated Mailbox Lists
|
||
|
||
If none of the options above are viable, you could fall back to a "poor
|
||
man's directory service", where you would periodically copy a current list of
|
||
mailboxes from the machine where they are located, to your MX host(s). Your
|
||
MTA would then consult this list to validate RCPT TO: commands in incoming
|
||
mail.
|
||
|
||
If the machine(s) that host(s) your mailboxes is/are running on some flavor
|
||
of UNIX or Linux, you could write a script to first generate such a list,
|
||
perhaps from the local "/etc/passwd" file, and then copy it to your MX host
|
||
(s) using the "scp" command from the [http://www.openssh.org/] OpenSSH suite.
|
||
You could then set up a "cron" job (type man cron for details) to
|
||
periodically run this script.
|
||
-----------------------------------------------------------------------------
|
||
|
||
2.3.3.3. Dictionary Attack Prevention
|
||
|
||
Dictionary Attack is a term used to describe SMTP transactions where the
|
||
sending host keeps issuing RCPT TO: commands to probe for possible recipient
|
||
addresses based on common names (often alphabetically starting with "aaron",
|
||
but sometimes starting later in the alphabet, and/or at random). If a
|
||
particular address is accepted by your server, that address is added into the
|
||
spammer's arsenal.
|
||
|
||
Some sites, particularly larger ones, find that they are frequent targets
|
||
of such attacks. From the spammer's perspective, chances of finding a given
|
||
username on a large site is better than on sites with only a few users.
|
||
|
||
One effective way to combat dictionary attacks is to issue increasing
|
||
transaction delays for each failed address. For instance, the first
|
||
non-existing recipient address can be rejected with a 20-second delay, the
|
||
second address with a 30-second delay, and so on.
|
||
-----------------------------------------------------------------------------
|
||
|
||
2.3.3.4. Accept only one recipient for DSNs
|
||
|
||
Legitimate Delivery Status Notifications should be sent to only one
|
||
recipient address - the originator of the original message that triggered the
|
||
notification. You can drop the connection if the Envelope Sender address is
|
||
empty, but there are more than one recipients.
|
||
-----------------------------------------------------------------------------
|
||
|
||
2.4. Greylisting
|
||
|
||
The greylisting concept is presented by Evan Harris in a whitepaper at:
|
||
[http://projects.puremagic.com/greylisting/] http://projects.puremagic.com/
|
||
greylisting/.
|
||
-----------------------------------------------------------------------------
|
||
|
||
2.4.1. How it works
|
||
|
||
Like SMTP transaction delays, greylisting is a simple but highly effective
|
||
mechanism to weed out messages that are being delivered via Ratware. The idea
|
||
is to establish whether a prior relationship exists between the sender and
|
||
the receiver of a message. For most legitimate mail it does, and the delivery
|
||
proceeds normally.
|
||
|
||
On the other hand, if no prior relationship exists, the delivery is
|
||
temporariliy rejected (with a 451 SMTP response). Legitimate MTAs will treat
|
||
this response accordingly, and retry the delivery in a little while[9]. In
|
||
contrast, ratware will either make repeated delivery attempts right away, and
|
||
/or simply give up and move on to the next target in its address list.
|
||
|
||
Three pieces of information from a delivery attempt, referred to a as a
|
||
triplet are used to uniquely identify the relationship between a sender and a
|
||
receiver:
|
||
|
||
* The Envelope Sender.
|
||
|
||
* The sending host's IP address.
|
||
|
||
* The Envelope Recipient.
|
||
|
||
|
||
If a delivery attempt was temporarily rejected, this triplet is cached. It
|
||
remains greylisted for a given amount of time (nominally 1 hour), after which
|
||
it is whitelisted, and new delivery attempts would succeed. If no new
|
||
delivery attempts occur prior to a given timeout (nominally 4 hours), then
|
||
the triplet expires from the cache.
|
||
|
||
If a whitelisted triplet has not been seen for an extended duration (at
|
||
minimum one month, to account for monthly billing statements and the like),
|
||
it is expired. This prevents unlimited growth of the list.
|
||
|
||
These timeouts are taken from Evan Harris' original greylisting whitepaper
|
||
(or should we say, ahem, "greypaper"?) Some people have found that a larger
|
||
timeout may be needed before greylisted triplets expire, because certain ISPs
|
||
(such as earthlink.net) retry deliveries only every 6 hours or similar. [10]
|
||
-----------------------------------------------------------------------------
|
||
|
||
2.4.2. Greylisting in Multiple Mail Exchangers
|
||
|
||
If you operate more than one incoming mail exchangers, and each exchanger
|
||
maintains its own greylisting cache, then:
|
||
|
||
* First-time deliveries from a given sender to one of your users may
|
||
theoretically be delayed up to N times the initial 1-hour delay, where N
|
||
is the number of mail exchangers. This is because the message would
|
||
likely be retried at a different server than the one that issued the 451
|
||
response to the initial delivery. In the worst case, the sender host may
|
||
not get around to retrying the delivery to the first exchanger for 4
|
||
hours, or until after the greylist triplet has expired, thereby causing
|
||
the delivery attempt to be rejected over and over again, until the sender
|
||
gives up (usually after 4 days or so).
|
||
|
||
In practice, this is unlikely. If a delivery attempt temporarily fails,
|
||
the sender host normally retries the delivery immediately, using a
|
||
different MX. Thus, after one hour, any of these MX hosts would accept
|
||
the message.
|
||
|
||
* Even after a triplet has been whitelisted in one of your MXs, the next
|
||
message with the same triplet will be greylisted if it is delivered to a
|
||
different MX.
|
||
|
||
|
||
For these reasons, you may want to implement a solution where the database
|
||
of greylist triplets is shared between your incoming mail exchangers.
|
||
However, since the machine that hosts this database would become a single
|
||
point of failure, you would have to take a sensible action if that machine is
|
||
down (e.g. accept all deliveries). Or you could use database replication
|
||
techniques and have the SMTP server fall back to one of the replicating
|
||
servers for lookups.
|
||
-----------------------------------------------------------------------------
|
||
|
||
2.4.3. Results
|
||
|
||
In my own experience, greylisting gets rid of about 90% of unique junk mail
|
||
deliveries, after most of the SMTP checks previously described are applied!
|
||
If you used greylisting as a first defense, it would likely catch an even
|
||
higher percentage of incoming junk mail.
|
||
|
||
Conversely, there are virtually zero False Positives resulting from this
|
||
technique. All major Mail Transport Agents perform delivery retries after a
|
||
temporary failure, in a manner that will eventually result in a successful
|
||
delivery.
|
||
|
||
The downside to greylisting is a legitimate mail from people who have not
|
||
e-mailed a particular recipient in the past is subject to a one-hour delay
|
||
(or maybe several hours, if you operate several MX hosts).
|
||
|
||
See also What happens when spammers adapt....
|
||
-----------------------------------------------------------------------------
|
||
|
||
2.5. Sender Authorization Schemes
|
||
|
||
Various schemes have been developed for sender verification where not only
|
||
the validity, but also the authenticity, of the sender address is checked.
|
||
The owner of a internet domain specifies certain criteria that must be
|
||
fulfilled in authentic deliveries from senders within that domain.
|
||
|
||
Two early proposed schemes of this kind were:
|
||
|
||
* MAIL-FROM MX records, conceived by Paul Vixie <paul (at) vix.com>
|
||
|
||
* Reverse Mail Exchanger (RMX) records as an addition to DNS itself,
|
||
conceived and published by Hadmut Danisch <hadmut (at) danisch.de>.
|
||
|
||
|
||
Under both of these schemes, all mails from <user@domain.com> had to come
|
||
from the hosts specified in <domain.com>'s DNS zone.
|
||
|
||
These schemes have evolved. Alas, they have also forked.
|
||
-----------------------------------------------------------------------------
|
||
|
||
2.5.1. Sender Policy Framework (SPF)
|
||
|
||
"Server Policy Framework" (previously "Sender Permitted From") is perhaps
|
||
the most well-known scheme for sender authorization. It is loosely based on
|
||
the original schemes described above, but allows for a bit more flexibility
|
||
in the criteria that can be posted by the domain holder.
|
||
|
||
SPF information is published as a TXT record in a domain's top-level DNS
|
||
zone. This record can specify:
|
||
|
||
* which hosts are allowed to send mail from that domain
|
||
|
||
* the mandatory presence of a GPG (GNU Privacy Guard) signature in
|
||
outgoing mail from the domain
|
||
|
||
* other criteria; see [http://spf.pobox.com/] http://spf.pobox.com/ for
|
||
details.
|
||
|
||
|
||
The structure of the TXT record is still undergoing development, however
|
||
basic features to accomplish the above are in place. It starts with the
|
||
string v=spf1, followed by such modifiers as:
|
||
|
||
* a - the IP address of the domain itself is a valid sender host
|
||
|
||
* mx - the incoming mail exchanger for that domain is also a valid sender
|
||
|
||
* ptr - if a rDNS lookup of the sending host's IP address yields a name
|
||
within the domain portion of the sender address, it is a valid sender.
|
||
|
||
|
||
Each of these modifiers may be prefixed with a plus sign (+), minus sign
|
||
(-), question mark (?), or tilde (~) to indicate whether it specifies an
|
||
authorative source, an non-authorative source, a neutral stance, or a likely
|
||
non-authorative source, respectively.
|
||
|
||
Each modifier may also be extended with a colon, followed by an alternate
|
||
domain name. For instance, if you are a Comcast subscriber, your own DNS zone
|
||
may include the string "-ptr:client.comcast.net ptr:comcast.net" to indicate
|
||
that your outgoing e-mail never comes from a host that resolves to
|
||
anything.client.comcast.net, but could come from other hosts that resolve to
|
||
anything.comcast.net.
|
||
|
||
SPF information is currently published for a number of high-profile
|
||
internet domains, such as aol.com, altavista.com, dyndns.org, earthlink.net,
|
||
and google.com.
|
||
|
||
Sender authorization schemes in general and SPF in particular are not
|
||
universally accepted. In particular, one objection is that domain holders may
|
||
effectively establish a monopoly on relaying outgoing mail from their users/
|
||
customers.
|
||
|
||
Another objection is that SPF breaks traditional e-mail forwarding - the
|
||
forwarding host may not have the authority to do so per the SPF information
|
||
in the envelope sender domain. This is partly addressed via [http://
|
||
spf.pobox.com/srs.html] SRS, or Sender Rewriting Scheme, wherein the
|
||
forwarder of the mail will modify the Envelope Sender address to the format:
|
||
user=source.domain@forwarder.domain
|
||
-----------------------------------------------------------------------------
|
||
|
||
2.5.2. Microsoft Caller-ID for E-Mail
|
||
|
||
Similar to SPF, in that acceptance criteria are posted via a TXT record in
|
||
the sending domain's DNS zone. However, rather than relying on simple
|
||
keywords, MS CIDE information consists of fairly large structures encoded in
|
||
XML. The XML schema is published under a license by Microsoft.
|
||
|
||
While SPF would nominally be used to check the Envelope Sender address of
|
||
an e-mail, MS CIDE is mainly a tool to validate the RFC 2822 header of the
|
||
message itself. Thus, the earliest point at which such a check could be
|
||
applied would be after the message data has been delivered, before issuing
|
||
the final 250 response.
|
||
|
||
Quite frankly, dead on arrival. Encumbered by patent issues and sheer
|
||
complexity.
|
||
|
||
That said, Recent SPF tools posted on [http://spf.pobox.com/] http://
|
||
spf.pobox.com/ are capable of checking MS Caller-ID information in addition
|
||
to SPF.
|
||
-----------------------------------------------------------------------------
|
||
|
||
2.5.3. RMX++
|
||
|
||
(part of Simple Caller Authorization Framework - SCAF). This scheme is
|
||
developed by Hadmut Danisch, who also conceived of the original RMX.
|
||
|
||
RMX++ allows for dynamic authorization by way of HTTP servers. The domain
|
||
owner publishes a server location via DNS, and the receiving host contacts
|
||
that server in order to obtain an authorization record to verify the
|
||
authenticity of the caller.
|
||
|
||
This scheme allows the domain owner more fine-grained control of criteria
|
||
used to authenticate the sender address, without having to publicly reveal
|
||
the structure of their network (as with SPF information in static TXT
|
||
records). For instance, an example from Hadmut is an authorization server
|
||
that allows no more than five messages from a given address per day after
|
||
business hours, then issues an alert once the limit has been reached.
|
||
|
||
Moreover, SCAF is not limited to e-mail, but can also be used to provide
|
||
caller authentication for other services such as Voice over IP (VoIP).
|
||
|
||
One possible downside with RMX++, as noted by Rick Stewart <rick.stewart
|
||
(at) theinternetco.net>, is its impact on machine and network resources:
|
||
Replies from HTTP servers are not as widely cached as information obtained
|
||
directly via DNS, and it is signifcantly more expensive to make an HTTP
|
||
request than a DNS request.
|
||
|
||
Further, Rick notes that the dynamic nature of RMX++ makes faults harder to
|
||
track. If there is a five-message-per-day limit, as in the example above, and
|
||
one message gets checked five times, then the limit is hit with a single
|
||
message. It makes re-checking a message impossible.
|
||
|
||
For more information on RMX, RMX++, and SCAF, refer to: [http://
|
||
www.danisch.de/work/security/antispam.html] http://www.danisch.de/work/
|
||
security/antispam.html.
|
||
-----------------------------------------------------------------------------
|
||
|
||
2.6. Message data checks
|
||
|
||
Time has come to look at the content of the message itself. This is what
|
||
conventional spam and virus scanners do, as they normally operate on the
|
||
message after it has been accepted. However, in our case, we perform these
|
||
checks before issuing the final 250 response, so that we have a chance to
|
||
reject the mail on the spot rather than later generating Collateral Spam.
|
||
|
||
If your incoming mail exchangers are very busy (i.e. large site, few
|
||
machines), you may find that performing some or all of these checks directly
|
||
in the mail exchanger is too costly. In particular, running Virus Scanners
|
||
and Spam Scanners do take up a fair amount of CPU bandwidth and time.
|
||
|
||
If so, you will want to set up dedicated machines for these scanning
|
||
operations. Most server-side anti-spam and anti-virus software can be invoked
|
||
over the network, i.e. from your mail exchanger. More on this in the
|
||
following chapters, where we discuss implementation for the various MTAs.
|
||
-----------------------------------------------------------------------------
|
||
|
||
2.6.1. Header checks
|
||
|
||
2.6.1.1. Missing Header Lines
|
||
|
||
RFC 2822 mandates that a message should contain at least the following
|
||
header lines:
|
||
From: ...
|
||
To: ...
|
||
Subject: ...
|
||
Message-ID: ...
|
||
Date: ...
|
||
|
||
The absence of any of these lines means that the message is not generated
|
||
by a mainstream Mail User Agent, and that it is probably junk [11].
|
||
-----------------------------------------------------------------------------
|
||
|
||
2.6.1.2. Header Address Syntax Check
|
||
|
||
Addresses presented in the message header (i.e. the To:, Cc:, From: ...
|
||
fields) should be syntactically valid. Enough said.
|
||
-----------------------------------------------------------------------------
|
||
|
||
2.6.1.3. Simple Header Address Validation
|
||
|
||
For each address in the message header:
|
||
|
||
* If the address is local, is the local part (before the @ sign) a valid
|
||
mailbox?
|
||
|
||
* If the address is remote, does the domain part (after the @ sign)
|
||
exist?
|
||
|
||
|
||
-----------------------------------------------------------------------------
|
||
2.6.1.4. Header Address Callout Verification
|
||
|
||
This works similar to Sender Callout Verification and Recipient Callout
|
||
Verification. Each remote header address is verified by calling the primary
|
||
MX for the corresponding domain to determine if a Delivery Status
|
||
Notification would be accepted.
|
||
-----------------------------------------------------------------------------
|
||
|
||
2.6.2. Junk Mail Signature Repositories
|
||
|
||
One trait of junk mail is that it is sent to a large number of addresses.
|
||
If 50 other recipients have already flagged a particular message as spam, why
|
||
couldn't you use this fact to decide whether or not to accept the message
|
||
when it is delivered to you? Better yet, why not set up Spam Traps that feed
|
||
a public pool of known spam?
|
||
|
||
I am glad you asked. As it turns out, such pools do exist:
|
||
|
||
* [http://razor.sf.net/] Razor
|
||
|
||
* [http://pyzor.sf.net/] Pyzor
|
||
|
||
* Distributed Checksum Clearinghouse (DCC)
|
||
|
||
|
||
These tools have progressed beyond simple signature checks that only
|
||
trigger if you receive an identical copy of a message that is known to be
|
||
junk mail. Rather, they evaluate common patterns, to account for slight
|
||
variations in the message header and body.
|
||
-----------------------------------------------------------------------------
|
||
|
||
2.6.3. Binary garbage checks
|
||
|
||
Messages containing non-printable characters are rare. When they do show
|
||
up, the message is nearly always a virus, or in some cases spam written in a
|
||
non-western language, without the appropriate MIME encoding.
|
||
|
||
One particular case is where the message contains NUL characters (ordinal
|
||
zero). Even if you decide that figuring out what a non-printable character
|
||
means is more complex than beneficial, you might consider checking for this
|
||
character. That is because some Mail Delivery Agents, such as the [http://
|
||
asg.web.cmu.edu/cyrus/] Cyrus Mail Suite, will ultimately reject mails that
|
||
contain it. [12]. If you use such software, you should definitely consider
|
||
getting rid of NUL characters.
|
||
|
||
On the other hand, the (now obsolete) RFC 822 specification did not
|
||
explicitly prohibit NUL characters in the message. For this reason, as an
|
||
alternative to rejecting mails containing it, you may choose to strip these
|
||
characters from the message before delivering it to Cyrus.
|
||
-----------------------------------------------------------------------------
|
||
|
||
2.6.4. MIME checks
|
||
|
||
Similarly, it might be worthwhile to validate the MIME structure of
|
||
incoming message. MIME decoding errors or inconsistencies do not happen very
|
||
often; but when they do, the message is definitely junk. Moreover, such
|
||
errors may indicate potential problems in subsequent checks, such as File
|
||
Attachment Checks, Virus Scanners, or Spam Scanners.
|
||
|
||
In other words, if the MIME encoding is illegal, reject the message.
|
||
-----------------------------------------------------------------------------
|
||
|
||
2.6.5. File Attachment Check
|
||
|
||
When was the last time someone sent you a Windows screensaver (".scr" file)
|
||
or Windows Program Information File (".pif") that you actually wanted?
|
||
|
||
Consider blocking messages with "Windows executable" file attachment(s) -
|
||
i.e. file names that end with a period followed by any of a number of
|
||
three-letter combinations such as the above. This check consumes
|
||
significantly less resources on your server than Virus Scanners, and may also
|
||
catch new virii for which a signature does not yet exist in your anti-virus
|
||
scanner.
|
||
|
||
For a more-or-less comprehensive list of such "file name extensions",
|
||
please visit: [http://support.microsoft.com/default.aspx?scid=kb;EN-US;
|
||
290497] http://support.microsoft.com/default.aspx?scid=kb;EN-US;290497.
|
||
-----------------------------------------------------------------------------
|
||
|
||
2.6.6. Virus Scanners
|
||
|
||
A number of different server-side virus scanners are available. To name a
|
||
few:
|
||
|
||
* [http://www.vanja.com/tools/sophie/] Sophie
|
||
|
||
* [http://www.kapersky.com/] KAVDaemon
|
||
|
||
* [http://clamav.elektrapro.com/] ClamAV
|
||
|
||
* [http://www.sald.com/] DrWeb
|
||
|
||
|
||
In situations where you are not willing to block all potentially dangerous
|
||
files based on their file names alone (consider ".zip" files), such scanners
|
||
are helpful. Also, they will be able to catch virii that are not transmitted
|
||
as file attachments, such as the "Bagle.R" virus that arrived in March, 2004.
|
||
|
||
In most cases, the machine performing the virus scan does not need to be
|
||
your mail exchanger. Most of these anti-virus scanners can be invoked on a
|
||
different host over a network connection.
|
||
|
||
Anti-virus software mainly detect virii based on a set of signatures for
|
||
known virii, or virus definitions. These need to be updated regularly, as new
|
||
virii are developed. Also, the software itself should at any time be up to
|
||
date for maximum accuracy.
|
||
-----------------------------------------------------------------------------
|
||
|
||
2.6.7. Spam Scanners
|
||
|
||
Similarly, anti-spam software can be used to classify messages based on a
|
||
large set of heuristics, including their content, standards compliance, and
|
||
various network checks such as DNS Blacklists and Junk Mail Signature
|
||
Repository. In the end, such software typically assigns a composite "score"
|
||
to each message, indicating the likelihood that the message is spam, and if
|
||
the score is above a certain threshold, would classify it as such.
|
||
|
||
Two of the most popular server-side heuristic anti-spam filters are:
|
||
|
||
* [http://www.spamassassin.org/] SpamAssassin
|
||
|
||
* [http://www.brightmail.com/] BrightMail
|
||
|
||
|
||
These tools undergo a constant evolution as spammers find ways to
|
||
circumvent their various checks. For instance, consider "creative" spelling,
|
||
such as "GR0W lO 1NCH35". So, just like anti-virus software, if you use
|
||
anti-spam software, you should update it frequently for the highest level of
|
||
accuracy.
|
||
|
||
I use SpamAssassin, although to minimize impact on machine resources, it is
|
||
no longer my first line of defense. Out of approximately 500 junk mail
|
||
delivery attempts to my personal address per day, about 50 reach the point
|
||
where they are being checked by SpamAssassin (mainly because they are
|
||
forwarded from one of my other accounts, so the checks described above are
|
||
not effective). Out of these 50 messages, one message ends up in my inbox
|
||
approximately every 2 or 3 days.
|
||
-----------------------------------------------------------------------------
|
||
|
||
2.7. Blocking Collateral Spam
|
||
|
||
Collateral Spam is more difficult to block with the techniques described so
|
||
far, because it normally arrives from legitimate sites using standard mail
|
||
transport software (such as Sendmail, Postfix, or Exim). The challenge is to
|
||
distinguish these messages from valid Delivery Status Notifications returned
|
||
in response to mail sent from your own users. Here are some ways that people
|
||
do this:
|
||
-----------------------------------------------------------------------------
|
||
|
||
2.7.1. Bogus Virus Warning Filter
|
||
|
||
Most of the time, collateral spam is virus warnings generated by anti-virus
|
||
scanners[13]. In turn, the wording in the Subject: line of these virus
|
||
warnings, and/or other characteristics, is usually provided by the anti-virus
|
||
software itself. As such, you could create a list of the more common
|
||
characteristics, and filter out such bogus virus warnings.
|
||
|
||
Well, aren't you in luck - someone already did this for you. :-)
|
||
|
||
Tim Jackson <tim (at) timj.co.uk> maintains a list of bogus virus warnings
|
||
for use with SpamAssassin. This list is available at: [http://www.timj.co.uk/
|
||
linux/bogus-virus-warnings.cf] http://www.timj.co.uk/linux/
|
||
bogus-virus-warnings.cf.
|
||
-----------------------------------------------------------------------------
|
||
|
||
2.7.2. Publish SPF info for your domain
|
||
|
||
The purpose of the Sender Policy Framework is precisely to protect against
|
||
Joe Jobs; i.e. to prevent forgeries of valid e-mail addresses.
|
||
|
||
If you publish SPF records in the DNS zone for your domain, then recipient
|
||
hosts that incorporate SPF checks would not have accepted the forged message
|
||
in the first place. As such, they would not be sending a Delivery Status
|
||
Notification to your site.
|
||
-----------------------------------------------------------------------------
|
||
|
||
2.7.3. Enveloper Sender Signature
|
||
|
||
A different approach that I am currently experimenting with myself is to
|
||
add a signature in the local part of the Envelope Sender address in outgoing
|
||
mail, then check for this signature in the Envelope Recipient address before
|
||
accepting incoming Delivery Status Notifications. For instance, the generated
|
||
sender address might be of the following format:
|
||
localpart=signature@domain
|
||
|
||
Normal message replies are unaffected. These replies go to the address in
|
||
the From: or Reply-To: field of the message, which are left intact.
|
||
|
||
Sounds easy, doesn't it? Unfortunately, generating a signature that is
|
||
suitable for this purpose is a bit more complex than it sounds. There are a
|
||
couple of conflicting considerations to take into account:
|
||
|
||
* To gain any benefit from this method, the signed envelope sender
|
||
address that you generate should be useless in the hands of spammers.
|
||
Typically, this would imply that the signature incorporates a time stamp
|
||
that would eventually expire:
|
||
sender=timestamp=hash@domain
|
||
|
||
* If you send mail to a site that incorporates Greylisting, your envelope
|
||
sender address should remain constant for that particular recipient.
|
||
Otherwise, your mail will continuously be greylisted.
|
||
|
||
With this in mind, you could generate a Envelope Sender based on the
|
||
Envelope Recipient address:
|
||
sender=recipient=recipient.domain=hash@domain
|
||
Although this address does not expire, if you start seeing junk mail to
|
||
it, you will at least know the source of the leak - it is incorported in
|
||
the recipient address. Moreover, you can easily block specific recipient
|
||
address signatures, without affecting normal mail delivery to that same
|
||
recipient.
|
||
|
||
* Two more issues occur with mailing list servers. Usually, replies to
|
||
request mails (such as "subscribe"/"unsubscribe") are sent with no
|
||
envelope sender.
|
||
|
||
+ The first issue pertains to servers that send responses back to the
|
||
Envelope Sender address of the request mail (as in the case of <
|
||
discuss@en.tldp.org>). The problem is that commands for the mailing
|
||
list server (such as subscribe or unsubscribe) are typically sent to
|
||
one or more different addresses (e.g. <discuss-subscribe@en.tldp.org>
|
||
and <discuss-unsubscribe@en.tldp.org>, respectively) than the address
|
||
used for list mail. Hence, the subscriber address will be different
|
||
from the sender address in messages sent to the list itself -- and in
|
||
this example, also different from the address that will be generated
|
||
for unsubscription requests. As a result, you may not be able to post
|
||
to the list, or unsubscribe.
|
||
|
||
The compromise would be to incorporate only the recipient domain in
|
||
the sender signature. The sender address might then look like:
|
||
subscribername=en.tldp.org=hash@subscriber.domain
|
||
|
||
+ The second issue pertains to those that send responses back to the
|
||
reply address in the message header of the request mail (such as <
|
||
spam-l-request@peach.ease.lsoft.com>). Since this address is not
|
||
signed, the response from the list server would be blocked by your
|
||
server.
|
||
|
||
There is not much you can do about this, other than to "whitelist"
|
||
these particular servers in such a way that they are allowed to
|
||
return mail to unsigned recipient addresses.
|
||
|
||
|
||
|
||
At this point, this approach starts losing some of its edge. Moreover, even
|
||
legitimate DSNs are rejected unless the original mail has been sent via your
|
||
server. Thus, you should only consider doing this if for those of your users
|
||
that do not roam, or otherwise send their outgoing mail via servers outside
|
||
your control.
|
||
|
||
That said, in situations where none of the above concerns apply to you,
|
||
this method gives you a good way to not only eliminate collateral spam, but
|
||
also a way to educate the owners of the sites that (presumably unwittingly)
|
||
generate it. Moreover, as a side benefit, sites that perform Sender Callout
|
||
Verification will only get a positive response from you if the original mail
|
||
was, indeed, sent from your site. In essence, you are reducing your exposure
|
||
to sender address forgeries by spammers.
|
||
|
||
You could perhaps allow your users to specify whether to sign outgoing
|
||
mails, and if so, specify which hosts should be allowed to return mails to
|
||
the unsigned version of their address. For instance, if they have system
|
||
accounts on your mail server, you could check for the existence and content,
|
||
respectively, of a given file in their home directory.
|
||
-----------------------------------------------------------------------------
|
||
|
||
2.7.4. Accept Bounces Only for Real Users
|
||
|
||
Even if you check for envelope sender signatures, there may be a loophole
|
||
that allows bogus bounces to be accepted. Specifically, if your users have to
|
||
opt in to the scheme, you are probably not checking for this signature in
|
||
mails sent to system aliases, such as postmaster or mailer-daemon. Moreover,
|
||
since these users do not generate outgoing mail, they should not receive any
|
||
bounces.
|
||
|
||
You can reject mail if it is sent to such system aliases, or alternatively,
|
||
if there is no mailbox for the provided recipient address.
|
||
-----------------------------------------------------------------------------
|
||
|
||
Chapter 3. Considerations
|
||
|
||
|
||
Some specific considerations come into play as a result of system-wide
|
||
SMTP time filtering. Here we cover some of those.
|
||
|
||
|
||
-----------------------------------------------------------------------------
|
||
3.1. Multiple Incoming Mail Exchangers
|
||
|
||
Most domains list more than one incoming Mail Exchangers (a.k.a. "MX hosts"
|
||
). If you do so, then bear in mind that in order to have any effect, any SMTP
|
||
time filtering you incorporate on the primary MX has to be incorporated on
|
||
all the others as well. Otherwise, the sending host would simply sidestep
|
||
filtering by retrying the mail delivery through your backup server(s).
|
||
|
||
If the backup server(s) are not under your control, ask yourself whether
|
||
you need multiple MXs in the first place. In this situation, chances are that
|
||
they serve only as redundant mail servers, and that they in turn forward the
|
||
mail to your primary MX. If so, you probably don't need them. If your host
|
||
happens to be down for a little while, that's OK -- well-behaved sender hosts
|
||
will retry deliveries for several days before giving up [9].
|
||
|
||
A situation where you may need multiple MXs is to perform load balancing
|
||
between several servers - i.e. if you receive so much mail that one machine
|
||
alone could not handle it. In this case, see if you could offload some tasks
|
||
(such as virus and spam scanners) to other machines, in order to reduce or
|
||
eliminate this need.
|
||
|
||
Again, if you do decide to keep using several MXs, your backup servers need
|
||
to be (at least) as restrictive as the primary server, lest filtering in the
|
||
primary MX is useless.
|
||
|
||
See also the section on Greylisting for additional concerns related to
|
||
multiple MX hosts.
|
||
-----------------------------------------------------------------------------
|
||
|
||
3.2. Blocking Access to Other SMTP Servers
|
||
|
||
Any SMTP server that is not listed as a public Mail Exchanger in the DNS
|
||
zone of your domain(s) should not accept incoming connections from the
|
||
internet. All incoming mail traffic should go through your incoming mail
|
||
exchanger(s).
|
||
|
||
This consideration is not unique to SMTP servers. If you have machines that
|
||
only serve an internal purpose within your site, use a firewall to restrict
|
||
access to these.
|
||
|
||
This is a rule, so therefore there must be exceptions. However, if you
|
||
don't know what they are, then the above applies to you.
|
||
-----------------------------------------------------------------------------
|
||
|
||
3.3. Forwarded Mail
|
||
|
||
You should take care not to reject mail as a result of spam filtering if it
|
||
is forwarded from "friendly" sources, such as:
|
||
|
||
* Your backup MX hosts, if any. Supposedly, these have already filtered
|
||
out most of the junk (see Multiple Incoming Mail Exchangers).
|
||
|
||
* Mailing lists, to which you or your users subscribe. You may still
|
||
filter such mail (it may not be as criticial if it ends up in a black
|
||
hole). However, if you reject the mail, you may end up causing the list
|
||
server to automatically unsubscribe the recipient.
|
||
|
||
* Other accounts belonging to the recipient. Again, rejections will
|
||
generate collateral spam, and/or create problems for the host that
|
||
forwards the mail.
|
||
|
||
|
||
You may see a logistical issue with the last two of these sources: They are
|
||
specific to each recipient. How to you allow each user to specify which hosts
|
||
they want to whitelist, and then use such individual whitelists in a
|
||
system-wide SMTP-time filtering setup? If the message is forwarded to several
|
||
recipients at your site (as may often be true in the case of a mailing list),
|
||
how do you decide whose whitelist to use?
|
||
|
||
There is no magic bullet here. This is one of those situations where we
|
||
just have to do a bit of work. You can decide to accept all mails, regardless
|
||
of spam classification, so long as it is sent from a host in the whitelist of
|
||
any one of the recipients. For instance, in response to each RCPT TO:
|
||
command, we can match the sending host against the corresponding user's
|
||
whitelist. If found, set a flag that will prevent a subsequent rejection.
|
||
Effectively, you are using an aggregate of each recipient's whitelist.
|
||
|
||
The implementation appendices cover this in more detail.
|
||
-----------------------------------------------------------------------------
|
||
|
||
3.4. User Settings and Data
|
||
|
||
There are other situations where you may want to support settings and data
|
||
for each user at site. For instance, if you scan incoming mail with
|
||
SpamAssassin (see Spam Scanners), you may want to allow for individual spam
|
||
thresholds, acceptable languages and character sets, and Bayesian training/
|
||
data.
|
||
|
||
A sticking point is that SMTP-time filtering of incoming mail is done at
|
||
the system level, before mail is being delivered to a particular user, and as
|
||
such, does not lend itself too well to individual preferences. A single
|
||
message may have several recipients; and unlike the case with Forwarded Mail,
|
||
using an aggregate of each recipient's preferences is not a good option.
|
||
Consider a scenario where you have users from different linguistic
|
||
backgrounds.
|
||
|
||
As it turns out, though, there is a modification to this truth. The trick
|
||
is to limit the number of recipients in incoming messages to one, so that the
|
||
message can be analyzed in accordance with the settings and data that belongs
|
||
to the corresponding user.
|
||
|
||
To do this, you would accept the first RCPT TO:, then issue a SMTP 451
|
||
(defer) response to subsequent commands. If the caller is a well-behaved MTA,
|
||
it will know how to interpret this response, and try later. (If it is
|
||
confused, then, well, it is probably a sender from which you don't want to
|
||
receive mail in the first place).
|
||
|
||
Obviously, this is a hack. Every mail sent to several users at your site
|
||
will be slowed down by 30 minutes or more per recipient. Especially in
|
||
corporate environments, where it is common to see e-mail discussions
|
||
involving several people on the inside and several others on the outside, and
|
||
where timelines of mail deliveries are essential, this is probably not a good
|
||
solution at all.
|
||
|
||
Another issue that mainly pertains to corporate enterprises and other large
|
||
sites is that incoming mail is often forwarded to internal machines for
|
||
delivery, and that recipients don't normally have accounts on the mail
|
||
exchanger. It may still be possible to support user-specific settings and
|
||
data in these situations (e.g. via database lookups or LDAP queries), but you
|
||
may also want to consider whether it's worth the effort.
|
||
|
||
That said, if you are on a small site, and where you are not afraid of
|
||
delayed deliveries, this may be an acceptable way to allow each user to fine
|
||
tune their filtering criteria.
|
||
-----------------------------------------------------------------------------
|
||
|
||
Chapter 4. Questions & Answers
|
||
|
||
|
||
In this section I try to anticipate some of the questions that may come
|
||
up, and to answer them. If you have questions that are not listed, and/or
|
||
would like to provide extra input in this section, please provide
|
||
feedback.
|
||
|
||
|
||
When Spammers Adapt
|
||
|
||
Q: What happens when spammers adapt and try to get around the techniques
|
||
described in this document?
|
||
|
||
Q: What happens when spammers adapt and try to get around the techniques
|
||
described in this document?
|
||
|
||
A: Well, that depends. :-)
|
||
|
||
Some of the checks described (such as SMTP checks and Greylisting)
|
||
specifically target ratware behavior. It is certainly possible to imagine
|
||
that this behavior will change if enough sites incorporate these checks.
|
||
Hatmut Danisch notes: Ratware contains buggy SMTP protocols because they
|
||
didn't need to do any better. It worked this way, so why should they have
|
||
spent more time? Meanwhile "ratware" has a higher quality, and even the
|
||
quality of spam messages has significantly improved. Once enough people
|
||
reject spam by detecting bad SMTP protocols, spam software authors will
|
||
simply improve their software.
|
||
|
||
That said, there are challenges remaining for such ratware:
|
||
|
||
* To get around SMTP transaction delays, they need to wait for each
|
||
response from the receiving SMTP server. At that point, we have
|
||
collectively accomplished a significant reduction in the rate of mail
|
||
that a given spamming host is able to deliver per unit of time. Since
|
||
spammers are racing against time to deliver as many mails as possible
|
||
before DNS blocklists and collaborative content filters catch up, we are
|
||
improving the effectiveness of these tools.
|
||
|
||
The effect is similar to the goal of Micropayment Schemes, wherein the
|
||
sender spends a few seconds working on a computational challenge for each
|
||
recipient of the mail, and adds a resulting signature to the e-mail
|
||
header for the recipient to validate. The main difference, aside from the
|
||
complexity of these schemes, is that they require the participation of
|
||
virtually everyone in the world before they can effectively be used to
|
||
weed out spam, whereas SMTP transaction delays start being effective with
|
||
the first recipient machine that implements it.
|
||
|
||
* To get around a HELO/EHLO check, they need to provide a proper
|
||
greeting, i.e. identify themselves with a valid Fully Qualified Domain
|
||
Name. This provides for increased traceability, especially with receiving
|
||
Mail Transport Agents that do not automatically insert the results of a
|
||
rDNS lookup into the Received: header of the message.
|
||
|
||
* To get all of the Sender Address Checks, they need to provide their own
|
||
valid sender address (or, at least, a valid sender address within their
|
||
own domain). Nuff said.
|
||
|
||
* To get around Greylisting, they need to retry deliveries to temporarily
|
||
failed recipients addresses after one hour (but before four hours). (As
|
||
far as implementation goes, in order to minimize machine resources,
|
||
rather than keeping a copy of each temporarily failed mail, ratware may
|
||
keep only a list of temporarily failed recipients, and perform a second
|
||
sweep through those addresses after an hour or two).
|
||
|
||
Even so, greylisting will remain fairly effective in conjunction with
|
||
DNS Blacklists that are fed from Spam Traps. That is because the
|
||
mandatory one-hour retry delay will give these lists a chance to list the
|
||
sending host.
|
||
|
||
|
||
Software tools, such as Spam Scanners and Virus Scanners, are in constant
|
||
evolution. As spammers evolve, so do these (and vice versa). As long as you
|
||
use recent versions of these tools, they will remain quite effective.
|
||
|
||
Finally, this document is itself subject to change. As the nature of junk
|
||
mail changes, people will come up with new, creative ways to block it.
|
||
-----------------------------------------------------------------------------
|
||
|
||
Appendix A. Exim Implementation
|
||
|
||
|
||
Here we cover the integration of techniques and tools described in this
|
||
document into the Exim Mail Transport Agent.
|
||
|
||
|
||
-----------------------------------------------------------------------------
|
||
A.1. Prerequisites
|
||
|
||
For these examples, you need the Exim Mail Transport Agent, preferrably
|
||
with Tom Kistner's Exiscan-ACL patch applied. Prebuilt Exim+Exiscan-ACL
|
||
packages exist for the most popular Linux distributions as well as FreeBSD;
|
||
see the [http://duncanthrax.net/exiscan-acl/] Exiscan-ACL home page for
|
||
details[14].
|
||
|
||
The final implementation example at the end incorporates these additional
|
||
tools:
|
||
|
||
* [http://www.spamassassin.org/] SpamAssassin - a popular spam filtering
|
||
tool that analyzes mail content against a large and highly sophisticated
|
||
set of heuristics.
|
||
|
||
* [http://packages.debian.org/unstable/mail/greylistd] greylistd - a
|
||
simple greylisting solution written by yours truly, specifically with
|
||
Exim in mind.
|
||
|
||
|
||
Other optional software is used in examples throughout.
|
||
-----------------------------------------------------------------------------
|
||
|
||
A.2. The Exim Configuration File
|
||
|
||
The Exim configuration file contains global definitions at the top (we will
|
||
call this the main section), followed by several other sections[15]. Each of
|
||
these other sections starts with:
|
||
begin section
|
||
|
||
We will spend most of our time in the acl section (i.e. after begin acl);
|
||
but we will also add and/or modify a few items in the transports and routers
|
||
sections, as well as in the main section at the top of the file.
|
||
-----------------------------------------------------------------------------
|
||
|
||
A.2.1. Access Control Lists
|
||
|
||
As of version 4.xx, Exim incorporates perhaps the most sophisticated and
|
||
flexible mechanism for SMTP-time filtering available anywhere, by way of
|
||
so-called Access Control Lists (ACLs).
|
||
|
||
An ACL can be used to evaluate whether to accept or reject an aspect of an
|
||
incoming message transaction, such as the initial connection from a remote
|
||
host, or the HELO/EHLO, MAIL FROM:, or RCPT TO: SMTP commands. So, for
|
||
instance, you may have an ACL named acl_rcpt_to to validate each RCPT TO:
|
||
command received from the peer.
|
||
|
||
An ACL consists of a series of statements (or rules). Each statement starts
|
||
with an action verb, such as accept, warn, require, defer, or deny, followed
|
||
by a list of conditions, options, and other settings pertaining to that
|
||
statement. Every statement is evaluated in order, until a definitive action
|
||
(besides warn) is taken. There is an implicit deny at the end of the ACL.
|
||
|
||
A sample statement in the acl_rcpt_to ACL above may look like this:
|
||
deny
|
||
message = relay not permitted
|
||
!hosts = +relay_from_hosts
|
||
!domains = +local_domains : +relay_to_domains
|
||
delay = 1m
|
||
|
||
This statement will reject the RCPT TO: command if it was not delivered by
|
||
a host in the "+relay_from_hosts" host list, and the recipient domain is not
|
||
in the "+local_domains" or "+relay_to_domains" domain lists. However, before
|
||
issuing the "550" SMTP response to this command, the server will wait for one
|
||
minute.
|
||
|
||
To evaluate a particular ACL at a given stage of the message transaction,
|
||
you need to point one of Exim's policy controls to that ACL. For instance, to
|
||
use the acl_rcpt_to ACL mentioned above to evaluate the RCPT TO:, the main
|
||
section of your Exim configuration file (before any begin keywords) should
|
||
include:
|
||
acl_smtp_rcpt = acl_rcpt_to
|
||
|
||
For a full list of such policy controls, refer to section 14.11 in the Exim
|
||
specifications.
|
||
-----------------------------------------------------------------------------
|
||
|
||
A.2.2. Expansions
|
||
|
||
A large number of expansion items are available, including run-time
|
||
variables, lookup functions, string/regex manipulations, host/domain lists,
|
||
etc. etc. An exhaustive reference for the last x.x0 release (i.e. 4.20,
|
||
4.30..) can be found in the file "spec.txt"; ACLs are described in section
|
||
38.
|
||
|
||
In particular, Exim provides twenty general purpose expansion variables to
|
||
which we can assign values in an ACL statement:
|
||
|
||
* $acl_c0 - $acl_c9 can hold values that will persist through the
|
||
lifetime of an SMTP connection.
|
||
|
||
* $acl_m0 - $acl_m9 can hold values while a message is being received,
|
||
but are then reset. They are also reset by the HELO, EHLO, MAIL, and RSET
|
||
commands.
|
||
|
||
|
||
-----------------------------------------------------------------------------
|
||
A.3. Options and Settings
|
||
|
||
The main section of the Exim configuration file (before the first begin
|
||
keyword) contains various macros, policy controls, and other general
|
||
settings. Let us start by defining a couple of macros we will use later:
|
||
# Define the message size limit; we will use this in the DATA ACL.
|
||
MESSAGE_SIZE_LIMIT = 10M
|
||
|
||
# Maximum message size for which we will run Spam or Virus scanning.
|
||
# This is to reduce the load imposed on the server by very large messages.
|
||
MESSAGE_SIZE_SPAM_MAX = 1M
|
||
|
||
# Macro defining a secret that we will use to generate various hashes.
|
||
# PLEASE CHANGE THIS!.
|
||
SECRET = some-secret
|
||
|
||
Let us tweak some general Exim settings:
|
||
# Treat DNS failures (SERVFAIL) as lookup failures.
|
||
# This is so that we can later reject sender addresses
|
||
# within non-existing domains, or domains for which no
|
||
# nameserver exists.
|
||
dns_again_means_nonexist = !+local_domains : !+relay_to_domains
|
||
|
||
# Enable HELO verification in ACLs for all hosts
|
||
helo_try_verify_hosts = *
|
||
|
||
# Remove any limitation on the maximum number of incoming
|
||
# connections we can serve at one time. This is so that while
|
||
# we later impose SMTP transaction delays for spammers, we
|
||
# will not refuse to serve new connections.
|
||
smtp_accept_max = 0
|
||
|
||
# ..unless the system load is above 10
|
||
smtp_load_reserve = 10
|
||
|
||
# Do not advertise ESMTP "PIPELINING" to any hosts.
|
||
# This is to trip up ratware, which often tries to pipeline
|
||
# commands anyway.
|
||
pipelining_advertise_hosts = :
|
||
|
||
Finally, we will point some Exim policy controls to five ACLs that we will
|
||
create to evaluate the various stages of an incoming SMTP transaction:
|
||
acl_smtp_connect = acl_connect
|
||
acl_smtp_helo = acl_helo
|
||
acl_smtp_mail = acl_mail_from
|
||
acl_smtp_rcpt = acl_rcpt_to
|
||
acl_smtp_data = acl_data
|
||
-----------------------------------------------------------------------------
|
||
|
||
A.4. Building the ACLs - First Pass
|
||
|
||
In the acl section (following begin acl), we need to define these ACLs. In
|
||
doing so, we will incorporate some of the basic Techniques described earlier
|
||
in this document, namely DNS checks and SMTP checks.
|
||
|
||
In this pass, we will do most of the checks in acl_rcpt_to, and leave the
|
||
other ACLs largely empty. That is because most of the commonly used ratware
|
||
does not understand rejections early in the SMTP transaction - it keeps
|
||
trying. On the other hand, most ratware clients give up if the RCPT TO:
|
||
fails.
|
||
|
||
We create all these ACLs, however, because we will use them later.
|
||
-----------------------------------------------------------------------------
|
||
|
||
A.4.1. acl_connect
|
||
|
||
|
||
# This access control list is used at the start of an incoming
|
||
# connection. The tests are run in order until the connection
|
||
# is either accepted or denied.
|
||
|
||
acl_connect:
|
||
|
||
# In this pass, we do not perform any checks here.
|
||
accept
|
||
-----------------------------------------------------------------------------
|
||
|
||
A.4.2. acl_helo
|
||
|
||
|
||
# This access control list is used for the HELO or EHLO command in
|
||
# an incoming SMTP transaction. The tests are run in order until the
|
||
# greeting is either accepted or denied.
|
||
|
||
acl_helo:
|
||
|
||
# In this pass, we do not perform any checks here.
|
||
accept
|
||
-----------------------------------------------------------------------------
|
||
|
||
A.4.3. acl_mail_from
|
||
|
||
|
||
# This access control list is used for the MAIL FROM: command in an
|
||
# incoming SMTP transaction. The tests are run in order until the
|
||
# sender address is either accepted or denied.
|
||
#
|
||
|
||
acl_mail_from:
|
||
|
||
# Accept the command.
|
||
accept
|
||
-----------------------------------------------------------------------------
|
||
|
||
A.4.4. acl_rcpt_to
|
||
|
||
|
||
# This access control list is used for every RCPT command in an
|
||
# incoming SMTP message. The tests are run in order until the
|
||
# recipient address is either accepted or denied.
|
||
|
||
acl_rcpt_to:
|
||
|
||
# Accept mail received over local SMTP (i.e. not over TCP/IP).
|
||
# We do this by testing for an empty sending host field.
|
||
# Also accept mails received from hosts for which we relay mail.
|
||
#
|
||
# Recipient verification is omitted here, because in many
|
||
# cases the clients are dumb MUAs that don't cope well with
|
||
# SMTP error responses.
|
||
#
|
||
accept
|
||
hosts = : +relay_from_hosts
|
||
|
||
|
||
# Accept if the message arrived over an authenticated connection,
|
||
# from any host. Again, these messages are usually from MUAs, so
|
||
# recipient verification is omitted.
|
||
#
|
||
accept
|
||
authenticated = *
|
||
|
||
|
||
######################################################################
|
||
# DNS checks
|
||
######################################################################
|
||
#
|
||
# The results of these checks are cached, so multiple recipients
|
||
# does not translate into multiple DNS lookups.
|
||
#
|
||
|
||
# If the connecting host is in one of a select few DNSbls, then
|
||
# reject the message. Be careful when selecting these lists; many
|
||
# would cause a large number of false postives, and/or have no
|
||
# clear removal policy.
|
||
#
|
||
deny
|
||
dnslists = dnsbl.sorbs.net : \
|
||
dnsbl.njabl.org : \
|
||
cbl.abuseat.org : \
|
||
bl.spamcop.net
|
||
message = $sender_host_address is listed in $dnslist_domain\
|
||
${if def:dnslist_text { ($dnslist_text)}}
|
||
|
||
|
||
# If reverse DNS lookup of the sender's host fails (i.e. there is
|
||
# no rDNS entry, or a forward lookup of the resulting name does not
|
||
# match the original IP address), then reject the message.
|
||
#
|
||
deny
|
||
message = Reverse DNS lookup failed for host $sender_host_address.
|
||
!verify = reverse_host_lookup
|
||
|
||
|
||
|
||
######################################################################
|
||
# Hello checks
|
||
######################################################################
|
||
|
||
# If the remote host greets with an IP address, then reject the mail.
|
||
#
|
||
deny
|
||
message = Message was delivered by ratware
|
||
log_message = remote host used IP address in HELO/EHLO greeting
|
||
condition = ${if isip {$sender_helo_name}{true}{false}}
|
||
|
||
|
||
# Likewise if the peer greets with one of our own names
|
||
#
|
||
deny
|
||
message = Message was delivered by ratware
|
||
log_message = remote host used our name in HELO/EHLO greeting.
|
||
condition = ${if match_domain{$sender_helo_name}\
|
||
{$primary_hostname:+local_domains:+relay_to_domains}\
|
||
{true}{false}}
|
||
|
||
|
||
deny
|
||
message = Message was delivered by ratware
|
||
log_message = remote host did not present HELO/EHLO greeting.
|
||
condition = ${if def:sender_helo_name {false}{true}}
|
||
|
||
|
||
# If HELO verification fails, we add a X-HELO-Warning: header in
|
||
# the message.
|
||
#
|
||
warn
|
||
message = X-HELO-Warning: Remote host $sender_host_address \
|
||
${if def:sender_host_name {($sender_host_name) }}\
|
||
incorrectly presented itself as $sender_helo_name
|
||
log_message = remote host presented unverifiable HELO/EHLO greeting.
|
||
!verify = helo
|
||
|
||
|
||
|
||
######################################################################
|
||
# Sender Address Checks
|
||
######################################################################
|
||
|
||
# If we cannot verify the sender address, deny the message.
|
||
#
|
||
# You may choose to remove the "callout" option. In particular,
|
||
# if you are sending outgoing mail through a smarthost, it will not
|
||
# give any useful information.
|
||
#
|
||
# Details regarding the failed callout verification attempt are
|
||
# included in the 550 response; to omit these, change
|
||
# "sender/callout" to "sender/callout,no_details".
|
||
#
|
||
deny
|
||
message = <$sender_address> does not appear to be a \
|
||
valid sender address.
|
||
!verify = sender/callout
|
||
|
||
|
||
|
||
######################################################################
|
||
# Recipent Address Checks
|
||
######################################################################
|
||
|
||
# Deny if the local part contains @ or % or / or | or !. These are
|
||
# rarely found in genuine local parts, but are often tried by people
|
||
# looking to circumvent relaying restrictions.
|
||
#
|
||
# Also deny if the local part starts with a dot. Empty components
|
||
# aren't strictly legal in RFC 2822, but Exim allows them because
|
||
# this is common. However, actually starting with a dot may cause
|
||
# trouble if the local part is used as a file name (e.g. for a
|
||
# mailing list).
|
||
#
|
||
deny
|
||
local_parts = ^.*[@%!/|] : ^\\.
|
||
|
||
|
||
# Drop the connection if the envelope sender is empty, but there is
|
||
# more than one recipient address. Legitimate DSNs are never sent
|
||
# to more than one address.
|
||
#
|
||
drop
|
||
message = Legitimate bounces are never sent to more than one \
|
||
recipient.
|
||
senders = : postmaster@*
|
||
condition = $recipients_count
|
||
|
||
|
||
# Reject the recipient address if it is not in a domain for
|
||
# which we are handling mail.
|
||
#
|
||
deny
|
||
message = relay not permitted
|
||
!domains = +local_domains : +relay_to_domains
|
||
|
||
|
||
# Reject the recipient if it is not a valid mailbox.
|
||
# If the mailbox is not on our system (e.g. if we are a
|
||
# backup MX for the recipient domain), then perform a
|
||
# callout verification; but if the destination server is
|
||
# not responding, accept the recipient anyway.
|
||
#
|
||
deny
|
||
message = unknown user
|
||
!verify = recipient/callout=20s,defer_ok
|
||
|
||
|
||
# Otherwise, the recipient address is OK.
|
||
#
|
||
accept
|
||
|
||
-----------------------------------------------------------------------------
|
||
|
||
A.4.5. acl_data
|
||
|
||
|
||
# This access control list is used for message data received via
|
||
# SMTP. The tests are run in order until the recipient address
|
||
# is either accepted or denied.
|
||
|
||
acl_data:
|
||
|
||
# Add Message-ID if missing in messages received from our own hosts.
|
||
warn
|
||
condition = ${if !def:h_Message-ID: {1}}
|
||
hosts = : +relay_from_hosts
|
||
message = Message-ID: <E$message_id@$primary_hostname>
|
||
|
||
|
||
# Accept mail received over local SMTP (i.e. not over TCP/IP).
|
||
# We do this by testing for an empty sending host field.
|
||
# Also accept mails received from hosts for which we relay mail.
|
||
#
|
||
accept
|
||
hosts = : +relay_from_hosts
|
||
|
||
# Accept if the message arrived over an authenticated connection, from
|
||
# any host.
|
||
#
|
||
accept
|
||
authenticated = *
|
||
|
||
|
||
# Enforce a message-size limit
|
||
#
|
||
deny
|
||
message = Message size $message_size is larger than limit of \
|
||
MESSAGE_SIZE_LIMIT
|
||
condition = ${if >{$message_size}{MESSAGE_SIZE_LIMIT}{true}{false}}
|
||
|
||
|
||
# Deny unless the address list header is syntactically correct.
|
||
#
|
||
deny
|
||
message = Your message does not conform to RFC2822 standard
|
||
log_message = message header fail syntax check
|
||
!verify = header_syntax
|
||
|
||
|
||
# Deny non-local messages with no Message-ID, or no Date
|
||
#
|
||
# Note that some specialized MTAs, such as certain mailing list
|
||
# servers, do not automatically generate a Message-ID for bounces.
|
||
# Thus, we add the check for a non-empty sender.
|
||
#
|
||
deny
|
||
message = Your message does not conform to RFC2822 standard
|
||
log_message = missing header lines
|
||
!hosts = +relay_from_hosts
|
||
!senders = : postmaster@*
|
||
condition = ${if or {{!def:h_Message-ID:}\
|
||
{!def:h_Date:}\
|
||
{!def:h_Subject:}} {true}{false}}
|
||
|
||
|
||
# Warn unless there is a verifiable sender address in at least
|
||
# one of the "Sender:", "Reply-To:", or "From:" header lines.
|
||
#
|
||
warn
|
||
message = X-Sender-Verify-Failed: No valid sender in message header
|
||
log_message = No valid sender in message header
|
||
!verify = header_sender
|
||
|
||
|
||
# Accept the message.
|
||
#
|
||
accept
|
||
-----------------------------------------------------------------------------
|
||
|
||
A.5. Adding SMTP transaction delays
|
||
|
||
A.5.1. The simple way
|
||
|
||
The simplest way to add SMTP transaction delays is to append a delay
|
||
control to the final accept statement in each of the ACLs we have declared,
|
||
as follows:
|
||
accept
|
||
delay = 20s
|
||
|
||
In addition, you may want to add progressive delays in the deny statement
|
||
pertaining to invalid recipients ("unknown user") within acl_rcpt_to. This is
|
||
to slow down dictionary attacks. For instance:
|
||
deny
|
||
message = unknown user
|
||
!verify = recipient/callout=20s,defer_ok,use_sender
|
||
delay = ${eval:$rcpt_fail_count*10 + 20}s
|
||
|
||
It should be noted that there is no point in imposing a delay in acl_data,
|
||
after the message data has been received. Ratware commonly disconnect at this
|
||
point, before even receiving a response from your server. In any case,
|
||
whether or not the client disconnects at this point has no bearing on whether
|
||
Exim will proceed with the delivery of the message.
|
||
-----------------------------------------------------------------------------
|
||
|
||
A.5.2. Selective Delays
|
||
|
||
If you are like me, you want to be a little bit more selective about which
|
||
hosts you subject to SMTP transaction delays. For instance, as described
|
||
earlier in this document, you may decide that a match from a DNS blacklist or
|
||
a non-verifiable EHLO/HELO greeting are not conditions that by themselves
|
||
warrant a rejection - but they may well be sufficient triggers for
|
||
transaction delays.
|
||
|
||
In order perform selective delays, we want move some of the checks that we
|
||
previously did in acl_rcpt_to to earlier points in the SMTP transaction. This
|
||
is so that we can start imposing the delays as soon as we see any sign of
|
||
trouble, and thereby increase the chance of causing synchronization errors
|
||
and other trouble for ratware.
|
||
|
||
Specifically, we want to:
|
||
|
||
* Move the DNS checks to acl_connect.
|
||
|
||
* Move the Hello checks to acl_helo. One exception: We cannot yet check
|
||
for a missing Hello greeting at this point, because this ACL is processed
|
||
in response to an EHLO or HELO command. We will do this check in the
|
||
acl_mail_from ACL.
|
||
|
||
* Move the Sender Address Checks checks to acl_mail_from.
|
||
|
||
|
||
However, for reasons described above, we do not want to actually reject the
|
||
mail until after the RCPT TO: command. Instead, in the earlier ACLs, we will
|
||
convert the various deny statements into warn statements, and use Exim's
|
||
general purpose ACL variables to store any error messages or warnings until
|
||
after the RCPT TO: command. We do that as follows:
|
||
|
||
* If we decide to reject the delivery, we store an error message to be
|
||
used in the forthcoming 550 response in $acl_c0 or $acl_m0:
|
||
|
||
+ If we identify the condition before a mail delivery has started
|
||
(i.e. in acl_connect or acl_helo), we use the connection-persistent
|
||
variable $acl_c0
|
||
|
||
+ Once a mail transaction has started (i.e. after the MAIL FROM:
|
||
command), we copy any contents from $acl_c0 into the message-specific
|
||
variable $acl_m0, and use the latter from this point forward. This
|
||
way, any conditions identified in this particular message will not
|
||
affect any subsequent messages received in the same connection.
|
||
|
||
|
||
Also, we store a corresponding log message in $acl_c1 or $acl_m1, in a
|
||
similar manner.
|
||
|
||
* If we come across a condition that does not warrant an outright
|
||
rejection, we only store a warning message in $acl_c1 or $acl_m1. Once a
|
||
mail transaction has started (i.e. in acl_mail_from), we add any content
|
||
in this variable to the message header as well.
|
||
|
||
* If we decide to accept a message without regard to the results of any
|
||
subsequent checks (such as a SpamAssassin scan), we set a flag in $acl_c0
|
||
or $acl_m0, but $acl_c1 and $acl_m1 empty.
|
||
|
||
* At the beginning of every ACL to and including acl_mail_from, we record
|
||
the current timestamp in $acl_m2. At the end of the ACL, we use the
|
||
presence of $acl_c1 or $acl_m1 to trigger a SMTP transaction delay until
|
||
a total of 20 seconds has elapsed.
|
||
|
||
|
||
The following table summarizes our use of these variables:
|
||
|
||
|
||
Table A-1. Use of ACL connection/message variables
|
||
+----------------+----------------------+---------------+
|
||
|Variables: |$acl_[cm]0 unset |$acl_[cm]0 set |
|
||
+----------------+----------------------+---------------+
|
||
|$acl_[cm]1 unset|(No decision yet) |Accept the mail|
|
||
+----------------+----------------------+---------------+
|
||
|$acl_[cm]1 set |Add warning in header |Reject the mail|
|
||
+----------------+----------------------+---------------+
|
||
|
||
As an example of this approach, let us consider two checks that we do in
|
||
response to the Hello greeting; one that will reject mails if the peer greets
|
||
with an IP address, and one that will warn about an unverifiable name in the
|
||
greeting. Previously, we did both of these checks in acl_rcpt_to - now we
|
||
move them to the acl_helo ACL.
|
||
acl_helo:
|
||
# Record the current timestamp, in order to calculate elapsed time
|
||
# for subsequent delays
|
||
warn
|
||
set acl_m2 = $tod_epoch
|
||
|
||
|
||
# Accept mail received over local SMTP (i.e. not over TCP/IP).
|
||
# We do this by testing for an empty sending host field.
|
||
# Also accept mails received from hosts for which we relay mail.
|
||
#
|
||
accept
|
||
hosts = : +relay_from_hosts
|
||
|
||
|
||
# If the remote host greets with an IP address, then prepare a reject
|
||
# message in $acl_c0, and a log message in $acl_c1. We will later use
|
||
# these in a "deny" statement. In the mean time, their presence indicate
|
||
# that we should keep stalling the sender.
|
||
#
|
||
warn
|
||
condition = ${if isip {$sender_helo_name}{true}{false}}
|
||
set acl_c0 = Message was delivered by ratware
|
||
set acl_c1 = remote host used IP address in HELO/EHLO greeting
|
||
|
||
|
||
# If HELO verification fails, we prepare a warning message in acl_c1.
|
||
# We will later add this message to the mail header. In the mean time,
|
||
# its presence indicates that we should keep stalling the sender.
|
||
#
|
||
warn
|
||
condition = ${if !def:acl_c1 {true}{false}}
|
||
!verify = helo
|
||
set acl_c1 = X-HELO-Warning: Remote host $sender_host_address \
|
||
${if def:sender_host_name {($sender_host_name) }}\
|
||
incorrectly presented itself as $sender_helo_name
|
||
log_message = remote host presented unverifiable HELO/EHLO greeting.
|
||
|
||
|
||
#
|
||
# ... additional checks omitted for this example ...
|
||
#
|
||
|
||
|
||
# Accept the connection, but if we previously generated a message in
|
||
# $acl_c1, stall the sender until 20 seconds has elapsed.
|
||
accept
|
||
set acl_m2 = ${if def:acl_c1 {${eval:20 + $acl_m2 - $tod_epoch}}{0}}
|
||
delay = ${if >{$acl_m2}{0}{$acl_m2}{0}}s
|
||
|
||
|
||
Then, in acl_mail_from we transfer the messages from $acl_c{0,1} to $acl_m
|
||
{0,1}. We also add the contents of $acl_c1 to the message header.
|
||
acl_mail_from:
|
||
# Record the current timestamp, in order to calculate elapsed time
|
||
# for subsequent delays
|
||
warn
|
||
set acl_m2 = $tod_epoch
|
||
|
||
|
||
# Accept mail received over local SMTP (i.e. not over TCP/IP).
|
||
# We do this by testing for an empty sending host field.
|
||
# Also accept mails received from hosts for which we relay mail.
|
||
#
|
||
accept
|
||
hosts = : +relay_from_hosts
|
||
|
||
|
||
# If present, the ACL variables $acl_c0 and $acl_c1 contain rejection
|
||
# and/or warning messages to be applied to every delivery attempt in
|
||
# in this SMTP transaction. Assign these to the corresponding
|
||
# $acl_m{0,1} message-specific variables, and add any warning message
|
||
# from $acl_m1 to the message header. (In the case of a rejection,
|
||
# $acl_m1 actually contains a log message instead, but this does not
|
||
# matter, as we will discard the header along with the message).
|
||
#
|
||
warn
|
||
set acl_m0 = $acl_c0
|
||
set acl_m1 = $acl_c1
|
||
message = $acl_c1
|
||
|
||
|
||
#
|
||
# ... additional checks omitted for this example ...
|
||
#
|
||
|
||
# Accept the sender, but if we previously generated a message in
|
||
# $acl_c1, stall the sender until 20 seconds has elapsed.
|
||
accept
|
||
set acl_m2 = ${if def:acl_c1 {${eval:20 + $acl_m2 - $tod_epoch}}{0}}
|
||
delay = ${if >{$acl_m2}{0}{$acl_m2}{0}}s
|
||
|
||
|
||
All the pertinent changes are incorporated in the Final ACLs, to follow.
|
||
-----------------------------------------------------------------------------
|
||
|
||
A.6. Adding Greylisting Support
|
||
|
||
There are several alternate greylisting implementations available for Exim.
|
||
Here we will cover a couple of these.
|
||
-----------------------------------------------------------------------------
|
||
|
||
A.6.1. greylistd
|
||
|
||
This is a Python implementation developed by yours truly. (So naturally,
|
||
this is the implementation I will include in the Final ACLs to follow). It
|
||
operates as a stand-alone daemon, and thus does not depend on any external
|
||
database. Greylist data is stored as simple 32-bit hashes for efficiency.
|
||
|
||
You can find it at [http://packages.debian.org/unstable/mail/greylistd]
|
||
http://packages.debian.org/unstable/mail/greylistd. Debian users can get it
|
||
via APT:
|
||
# apt-get install greylistd
|
||
|
||
To consult greylistd, we insert two statements in acl_rcpt_to ACL that we
|
||
previously declared, right before the final accept statement:
|
||
|
||
|
||
# Consult "greylistd" to obtain greylisting status for this particular
|
||
# peer/sender/recipient triplet.
|
||
#
|
||
# We do not greylist messages with a NULL sender, because sender
|
||
# callout verification would break (and we might not be able to
|
||
# send mail to a host that performs callouts).
|
||
#
|
||
defer
|
||
message = $sender_host_address is not yet authorized to deliver mail \
|
||
from <$sender_address> to <$local_part@$domain>. \
|
||
Please try later.
|
||
log_message = greylisted.
|
||
domains = +local_domains : +relay_to_domains
|
||
!senders = : postmaster@*
|
||
set acl_m9 = $sender_host_address $sender_address $local_part@$domain
|
||
set acl_m9 = ${readsocket{/var/run/greylistd/socket}{$acl_m9}{5s}{}{}}
|
||
condition = ${if eq {$acl_m9}{grey}{true}{false}}
|
||
|
||
Unless you incorporate envelope sender signatures to block bogus Delivery
|
||
Status Notifications, you may want to add a similar statement in your
|
||
acl_data to also greylist messages with a NULL sender.
|
||
|
||
The data we use for greylisting purposes here will be a little different
|
||
than above. In addition to $sender_address being emtpy, neither $local_part
|
||
nor $domain is defined at this point. Instead, the variable $recipients
|
||
contains a comma-separated list of all recipient addresses. For a legitimate
|
||
DSN, there should be only one address.
|
||
# Perform greylisting on messages with no envelope sender here.
|
||
# We did not subject these to greylisting after RCPT TO: because
|
||
# that would interfere with remote hosts doing sender callouts.
|
||
#
|
||
defer
|
||
message = $sender_host_address is not yet authorized to send \
|
||
delivery status reports to <$recipients>. \
|
||
Please try later.
|
||
log_message = greylisted.
|
||
senders = : postmaster@*
|
||
set acl_m9 = $sender_host_address $recipients
|
||
set acl_m9 = ${readsocket{/var/run/greylistd/socket}{$acl_m9}{5s}{}{}}
|
||
condition = ${if eq {$acl_m9}{grey}{true}{false}}
|
||
-----------------------------------------------------------------------------
|
||
|
||
A.6.2. MySQL implementation
|
||
|
||
The following inline implementation was contributed by Johannes Berg <
|
||
johannes (at) sipsolutions.net>, based in part on:
|
||
|
||
* work by Rick Stewart <rick.stewart (at) theinternetco.net>, published
|
||
at [http://theinternetco.net/projects/exim/greylist] http://
|
||
theinternetco.net/projects/exim/greylist, in turn based on
|
||
|
||
* a Postgres implementation created by Tollef Fog Heen <tfheen (at)
|
||
raw.no>, available at [http://raw.no/personal/blog/tech/Debian/
|
||
2004-03-14-15-55_greylisting] http://raw.no/personal/blog/tech/Debian/
|
||
2004-03-14-15-55_greylisting
|
||
|
||
|
||
It requires no external programs - the entire implementation is based on
|
||
these configuration snippets along with a MySQL database.
|
||
|
||
An archive containing up-to-date configuration snippets as well as a README
|
||
file is available at: [http://johannes.sipsolutions.net/wiki/Projects/
|
||
exim-greylist] http://johannes.sipsolutions.net/wiki/Projects/exim-greylist.
|
||
|
||
MySQL needs to be installed on your system. At a MySQL prompt, create an
|
||
exim4 database with two tables named exim_greylist and exim_greylist_log, as
|
||
follows:
|
||
CREATE DATABASE exim4;
|
||
use exim4;
|
||
|
||
CREATE TABLE exim_greylist (
|
||
id bigint(20) NOT NULL auto_increment,
|
||
relay_ip varchar(80) default NULL,
|
||
sender varchar(255) default NULL,
|
||
recipient varchar(255) default NULL,
|
||
block_expires datetime NOT NULL default '0000-00-00 00:00:00',
|
||
record_expires datetime NOT NULL default '9999-12-31 23:59:59',
|
||
create_time datetime NOT NULL default '0000-00-00 00:00:00',
|
||
type enum('AUTO','MANUAL') NOT NULL default 'MANUAL',
|
||
passcount bigint(20) NOT NULL default '0',
|
||
blockcount bigint(20) NOT NULL default '0',
|
||
PRIMARY KEY (id)
|
||
);
|
||
|
||
CREATE TABLE exim_greylist_log (
|
||
id bigint(20) NOT NULL auto_increment,
|
||
listid bigint(20) NOT NULL,
|
||
timestamp datetime NOT NULL default '0000-00-00 00:00:00',
|
||
kind enum('deferred', 'accepted') NOT NULL,
|
||
PRIMARY KEY (id)
|
||
);
|
||
|
||
In the main section of your Exim configuration file, declare the following
|
||
macros:
|
||
# if you don't have another database defined, then define it here
|
||
hide mysql_servers = localhost/exim4/user/password
|
||
|
||
# options
|
||
# these need to be valid as xxx in mysql's DATE_ADD(..,INTERVAL xxx)
|
||
# not valid, for example, are plurals: "2 HOUR" instead of "2 HOURS"
|
||
GREYLIST_INITIAL_DELAY = 1 HOUR
|
||
GREYLIST_INITIAL_LIFETIME = 4 HOUR
|
||
GREYLIST_WHITE_LIFETIME = 36 DAY
|
||
GREYLIST_BOUNCE_LIFETIME = 0 HOUR
|
||
|
||
# you can change the table names
|
||
GREYLIST_TABLE=exim_greylist
|
||
GREYLIST_LOG_TABLE=exim_greylist_log
|
||
|
||
# comment out to the following line to disable greylisting (temporarily)
|
||
GREYLIST_ENABLED=
|
||
|
||
# uncomment the following to enable logging
|
||
#GREYLIST_LOG_ENABLED=
|
||
|
||
# below here, nothing should normally be edited
|
||
|
||
.ifdef GREYLIST_ENABLED
|
||
# database macros
|
||
GREYLIST_TEST = SELECT CASE \
|
||
WHEN now() > block_expires THEN "accepted" \
|
||
ELSE "deferred" \
|
||
END AS result, id \
|
||
FROM GREYLIST_TABLE \
|
||
WHERE (now() < record_expires) \
|
||
AND (sender = '${quote_mysql:$sender_address}' \
|
||
OR (type='MANUAL' \
|
||
AND ( sender IS NULL \
|
||
OR sender = '${quote_mysql:@$sender_address_domain}' \
|
||
) \
|
||
) \
|
||
) \
|
||
AND (recipient = '${quote_mysql:$local_part@$domain}' \
|
||
OR (type = 'MANUAL' \
|
||
AND ( recipient IS NULL \
|
||
OR recipient = '${quote_mysql:$local_part@}' \
|
||
OR recipient = '${quote_mysql:@$domain}' \
|
||
) \
|
||
) \
|
||
) \
|
||
AND (relay_ip = '${quote_mysql:$sender_host_address}' \
|
||
OR (type='MANUAL' \
|
||
AND ( relay_ip IS NULL \
|
||
OR relay_ip = substring('${quote_mysql:$sender_host_address}',1,length(relay_ip)) \
|
||
) \
|
||
) \
|
||
) \
|
||
ORDER BY result DESC LIMIT 1
|
||
|
||
GREYLIST_ADD = INSERT INTO GREYLIST_TABLE \
|
||
(relay_ip, sender, recipient, block_expires, \
|
||
record_expires, create_time, type) \
|
||
VALUES ( '${quote_mysql:$sender_host_address}', \
|
||
'${quote_mysql:$sender_address}', \
|
||
'${quote_mysql:$local_part@$domain}', \
|
||
DATE_ADD(now(), INTERVAL GREYLIST_INITIAL_DELAY), \
|
||
DATE_ADD(now(), INTERVAL GREYLIST_INITIAL_LIFETIME), \
|
||
now(), \
|
||
'AUTO' \
|
||
)
|
||
|
||
GREYLIST_DEFER_HIT = UPDATE GREYLIST_TABLE \
|
||
SET blockcount=blockcount+1 \
|
||
WHERE id = $acl_m9
|
||
|
||
GREYLIST_OK_COUNT = UPDATE GREYLIST_TABLE \
|
||
SET passcount=passcount+1 \
|
||
WHERE id = $acl_m9
|
||
|
||
GREYLIST_OK_NEWTIME = UPDATE GREYLIST_TABLE \
|
||
SET record_expires = DATE_ADD(now(), INTERVAL GREYLIST_WHITE_LIFETIME) \
|
||
WHERE id = $acl_m9 AND type='AUTO'
|
||
|
||
GREYLIST_OK_BOUNCE = UPDATE GREYLIST_TABLE \
|
||
SET record_expires = DATE_ADD(now(), INTERVAL GREYLIST_BOUNCE_LIFETIME) \
|
||
WHERE id = $acl_m9 AND type='AUTO'
|
||
|
||
GREYLIST_LOG = INSERT INTO GREYLIST_LOG_TABLE \
|
||
(listid, timestamp, kind) \
|
||
VALUES ($acl_m9, now(), '$acl_m8')
|
||
.endif
|
||
|
||
Now, in the ACL section (after begin acl), declare a new ACL named
|
||
"greylist_acl":
|
||
.ifdef GREYLIST_ENABLED
|
||
# this acl returns either deny or accept
|
||
# since we use it inside a defer with acl = greylist_acl,
|
||
# accepting here makes the condition TRUE thus deferring,
|
||
# denying here makes the condition FALSE thus not deferring
|
||
greylist_acl:
|
||
# For regular deliveries, check greylist.
|
||
|
||
# check greylist tuple, returning "accepted", "deferred" or "unknown"
|
||
# in acl_m8, and the record id in acl_m9
|
||
|
||
warn set acl_m8 = ${lookup mysql{GREYLIST_TEST}{$value}{result=unknown}}
|
||
# here acl_m8 = "result=x id=y"
|
||
|
||
set acl_m9 = ${extract{id}{$acl_m8}{$value}{-1}}
|
||
# now acl_m9 contains the record id (or -1)
|
||
|
||
set acl_m8 = ${extract{result}{$acl_m8}{$value}{unknown}}
|
||
# now acl_m8 contains unknown/deferred/accepted
|
||
|
||
# check if we know a certain triple, add and defer message if not
|
||
accept
|
||
# if above check returned unknown (no record yet)
|
||
condition = ${if eq{$acl_m8}{unknown}{1}}
|
||
# then also add a record
|
||
condition = ${lookup mysql{GREYLIST_ADD}{yes}{no}}
|
||
|
||
# now log, no matter what the result was
|
||
# if the triple was unknown, we don't need a log entry
|
||
# (and don't get one) because that is implicit through
|
||
# the creation time above.
|
||
.ifdef GREYLIST_LOG_ENABLED
|
||
warn condition = ${lookup mysql{GREYLIST_LOG}}
|
||
.endif
|
||
|
||
# check if the triple is still blocked
|
||
accept
|
||
# if above check returned deferred then defer
|
||
condition = ${if eq{$acl_m8}{deferred}{1}}
|
||
# and note it down
|
||
condition = ${lookup mysql{GREYLIST_DEFER_HIT}{yes}{yes}}
|
||
|
||
# use a warn verb to count records that were hit
|
||
warn condition = ${lookup mysql{GREYLIST_OK_COUNT}}
|
||
|
||
# use a warn verb to set a new expire time on automatic records,
|
||
# but only if the mail was not a bounce, otherwise set to now().
|
||
warn !senders = : postmaster@*
|
||
condition = ${lookup mysql{GREYLIST_OK_NEWTIME}}
|
||
warn senders = : postmaster@*
|
||
condition = ${lookup mysql{GREYLIST_OK_BOUNCE}}
|
||
|
||
deny
|
||
.endif
|
||
|
||
Incorporate this ACL into your acl_rcpt_to to greylist triplets where the
|
||
sender address is non-empty. This is to allow for sender callout
|
||
verifications:
|
||
.ifdef GREYLIST_ENABLED
|
||
defer !senders = : postmaster@*
|
||
acl = greylist_acl
|
||
message = greylisted - try again later
|
||
.endif
|
||
|
||
Also incorporate it into your acl_data block, but this time only if the
|
||
sender address is empty. This is to prevent spammers from getting around
|
||
greylisting by setting the sender address to NULL.
|
||
.ifdef GREYLIST_ENABLED
|
||
defer senders = : postmaster@*
|
||
acl = greylist_acl
|
||
message = greylisted - try again later
|
||
.endif
|
||
-----------------------------------------------------------------------------
|
||
|
||
A.7. Adding SPF Checks
|
||
|
||
Here we cover two different ways to check Sender Policy Framework records
|
||
using Exim. In addition to these explicit mechanisms, the SpamAssassin suite
|
||
will in the near future (around version 2.70) incorporate more sophisticated
|
||
SPF checks, by assigning weighted scores to the various SPF results.
|
||
|
||
Although we could perform this check as early as in the acl_mail_from ACL,
|
||
there is an issue that will affect this decision: SPF is incompatible with
|
||
traditional e-mail forwarding. Unless the forwarding host implements [http://
|
||
spf.pobox.com/srs.html] SRS, you may end up rejecting forwarded mail because
|
||
you receive it from a host that is not authorized to do so per the SPF policy
|
||
of the domain in the Envelope Sender address.
|
||
|
||
To avoid doing this, we need to consult a user-specific list of hosts from
|
||
which forwarded mails should be accepted (as described in Exempting Forwarded
|
||
Mail, to follow). This is only possible after the RCPT TO:, when we know the
|
||
username of the recipient.
|
||
|
||
As such, we will add this check prior to any greylisting checks and/or the
|
||
final accept statement in acl_rcpt_to.
|
||
-----------------------------------------------------------------------------
|
||
|
||
A.7.1. SPF checks via Exiscan-ACL
|
||
|
||
Recent versions of Tom Kistner's Exiscan-ACL patch (see Prerequisites) have
|
||
native support for SPF. [16] Usage is very simple. An spf ACL condition is
|
||
added, and can be compared against any of the keywords pass, fail, softfail,
|
||
none, neutral, err_perm or err_temp.
|
||
|
||
Prior to any greylisting checks and/or the final accept statement in
|
||
acl_rcpt_to, insert the following snippet:
|
||
# Query the SPF information for the sender address domain, if any,
|
||
# to see if the sending host is authorized to deliver its mail.
|
||
# If not, reject the mail.
|
||
#
|
||
deny
|
||
message = [SPF] $sender_host_address is not allowed to send mail \
|
||
from $sender_address_domain
|
||
log_message = SPF check failed.
|
||
spf = fail
|
||
|
||
|
||
# Add a SPF-Received: header to the message
|
||
warn
|
||
message = $spf_received
|
||
|
||
This statement will reject the mail if the owner of the domain in the
|
||
sender address has disallowed deliveries from the calling host. Some people
|
||
find that this gives the domain owner a little bit too much control, even to
|
||
the point of shooting themselves in the foot. A suggested alternative is to
|
||
combine the SPF check with other checks, such as Sender Callout Verification
|
||
(but note that as before, there is no point in doing this if you are sending
|
||
your outgoing mail through a smarthost):
|
||
# Reject the mail if we cannot verify the sender address via callouts,
|
||
# and if SPF information for the sending domain does not grant explicit
|
||
# authority to the sending host.
|
||
#
|
||
deny
|
||
message = The sender address does not seem to be valid, and SPF \
|
||
information does not grant $sender_host_address explicit \
|
||
authority to send mail from $sender_address_domain
|
||
log_message = SPF check failed.
|
||
!verify = sender/callout,random,postmaster
|
||
!spf = pass
|
||
|
||
|
||
# Add a SPF-Received: header to the message
|
||
warn
|
||
message = $spf_received
|
||
-----------------------------------------------------------------------------
|
||
|
||
A.7.2. SPF checks via Mail::SPF::Query
|
||
|
||
Mail::SPF::Query is a the official SPF test suite, available from [http://
|
||
spf.pobox.com/downloads.html] http://spf.pobox.com/downloads.html. Debian
|
||
users, install libmail-spf-query-perl.
|
||
|
||
The Mail::SPF::Query package comes with a daemon (spfd) that listens for
|
||
requests on a UNIX domain socket. Unfortunately, it does not come with an
|
||
"init" script to start this daemon automatically. Therefore, in the following
|
||
example, we will use the standalone spfquery utility to make our SPF
|
||
requests.
|
||
|
||
As above, insert the following prior to any greylisting checks and/or the
|
||
final accept statement in acl_rcpt_to:
|
||
# Use "spfquery" to obtain SPF status for this particular sender/host.
|
||
# If the return code of that command is 1, this is an unauthorized sender.
|
||
#
|
||
deny
|
||
message = [SPF] $sender_host_address is not allowed to send mail \
|
||
from $sender_address_domain.
|
||
log_message = SPF check failed.
|
||
set acl_m9 = -ipv4=$sender_host_address \
|
||
-sender=$sender_address \
|
||
-helo=$sender_helo_name
|
||
set acl_m9 = ${run{/usr/bin/spfquery $acl_m9}}
|
||
condition = ${if eq {$runrc}{1}{true}{false}}
|
||
-----------------------------------------------------------------------------
|
||
|
||
A.8. Adding MIME and Filetype Checks
|
||
|
||
These checks depend on features found in Tom Kistner's Exiscan-ACL patch -
|
||
see Prerequisites for details.
|
||
|
||
Exiscan-ACL includes support for MIME decoding, and file name suffix checks
|
||
(or to use a misnomer from the Windows world, "file extension" checks). This
|
||
check alone will block most Windows virii - but not those that are
|
||
transmitted in .ZIP archives or those that exploit Outlook/MSIE HTML
|
||
rendering vulnerabilities - see the discussion on Virus Scanners.
|
||
|
||
These checks should go into acl_data, before the final accept statement:
|
||
# Reject messages that have serious MIME errors.
|
||
#
|
||
deny
|
||
message = Serious MIME defect detected ($demime_reason)
|
||
demime = *
|
||
condition = ${if >{$demime_errorlevel}{2}{1}{0}}
|
||
|
||
|
||
# Unpack MIME containers and reject file extensions used by worms.
|
||
# This calls the demime condition again, but it will return cached results.
|
||
# Note that the extension list may be incomplete.
|
||
#
|
||
deny
|
||
message = We do not accept ".$found_extension" attachments here.
|
||
demime = bat:btm:cmd:com:cpl:dll:exe:lnk:msi:pif:prf:reg:scr:vbs:url
|
||
|
||
You will note that the demime condition is invoked twice in the example
|
||
above. However, the results are cached, so the message is not actually
|
||
processed twice.
|
||
-----------------------------------------------------------------------------
|
||
|
||
A.9. Adding Anti-Virus Software
|
||
|
||
Exiscan-ACL plugs into a number of different virus scanners directly, or
|
||
any other scanner that can be run from the command line via its cmdline
|
||
backend.
|
||
|
||
To use this feature, the main section of your Exim configuration file must
|
||
specify which virus scanner to use, along with any options you wish to pass
|
||
to that scanner. The basic syntax is:
|
||
av_scanner = scanner-type:option1:option:...
|
||
|
||
For instance:
|
||
av_scanner = sophie:/var/run/sophie
|
||
av_scanner = kavdaemon:/opt/AVP/AvpCtl
|
||
av_scanner = clamd:127.0.0.1 1234
|
||
av_scanner = clamd:/opt/clamd/socket
|
||
av_scanner = cmdline:/path/to/sweep -all -rec -archive %s:found:'(.+)'
|
||
...
|
||
|
||
|
||
In the DATA ACL, you then want to use the malware condition to perform the
|
||
actual scanning:
|
||
deny
|
||
message = This message contains a virus ($malware_name)
|
||
demime = *
|
||
malware = */defer_ok
|
||
|
||
The included file exiscan-acl-spec.txt contains full usage information.
|
||
-----------------------------------------------------------------------------
|
||
|
||
A.10. Adding SpamAssassin
|
||
|
||
Invoking SpamAssassin at SMTP-time is commonly done in either of two ways
|
||
in Exim:
|
||
|
||
* Via the spam condition offered by Exiscan-ACL. This is the mechanism we
|
||
will cover here.
|
||
|
||
* Via SA-Exim, another utility written by Marc Merlins (<marc (at)
|
||
merlins.org>), specifically for running SpamAssassin at SMTP time in
|
||
Exim. This program operates through Exim's local_scan() interface, either
|
||
patched directly into the Exim source code, or via Marc's own dlopen()
|
||
plugin (which, by the way, is included in Debian's exim4-daemon-light and
|
||
exim4-daemon-heavy packages).
|
||
|
||
SA-Exim offers some other features as well, namely greylisting and
|
||
teergrubing. However, because the scan happens after the message data has
|
||
been received, neither of these two features may be as useful as they
|
||
would be earlier in the SMTP transaction.
|
||
|
||
SA-Exim can be found at: [http://marc.merlins.org/linux/exim/sa.html]
|
||
http://marc.merlins.org/linux/exim/sa.html.
|
||
|
||
|
||
-----------------------------------------------------------------------------
|
||
A.10.1. Invoke SpamAssassin via Exiscan
|
||
|
||
Exiscan-ACL's "spam" condition passes the message through either
|
||
SpamAssassin or Brightmail, and triggers if these indicate that the message
|
||
is junk. By default, it connects to a SpamAssassin daemon (spamd) running on
|
||
localhost. The host address and port can be changed by adding a spamd_address
|
||
setting in the main section of the Exim configuration file. For more
|
||
information, see the exiscan-acl-spect.txt file included with the patch.
|
||
|
||
In our implementation, we are going to reject messages classified as spam.
|
||
However, we would like to keep a copy of such messages in a separate mail
|
||
folder, at least for the time being. This is so that the user can
|
||
periodically scan for False Positives.
|
||
|
||
Exim offers controls that can be applied to a message that is accepted,
|
||
such as freeze. The Exiscan-ACL patch adds one more of these controls, namely
|
||
fakereject. This causes the following SMTP response:
|
||
550-FAKEREJECT id=message-id
|
||
550-Your message has been rejected but is being kept for evaluation.
|
||
550 If it was a legit message, it may still be delivered to the target recipient(s).
|
||
|
||
We can incorporate this feature into our implementation, by inserting the
|
||
following snippet in acl_data, prior to the final accept statement:
|
||
# Invoke SpamAssassin to obtain $spam_score and $spam_report.
|
||
# Depending on the classification, $acl_m9 is set to "ham" or "spam".
|
||
#
|
||
# If the message is classified as spam, pretend to reject it.
|
||
#
|
||
warn
|
||
set acl_m9 = ham
|
||
spam = mail
|
||
set acl_m9 = spam
|
||
control = fakereject
|
||
logwrite = :reject: Rejected spam (score $spam_score): $spam_report
|
||
|
||
# Add an appropriate X-Spam-Status: header to the message.
|
||
#
|
||
warn
|
||
message = X-Spam-Status: \
|
||
${if eq {$acl_m9}{spam}{Yes}{No}} (score $spam_score)\
|
||
${if def:spam_report {: $spam_report}}
|
||
logwrite = :main: Classified as $acl_m9 (score $spam_score)
|
||
|
||
|
||
In this example, $acl_m9 is initially set to "ham". Then SpamAssassin is
|
||
invoked as the user mail. If the message is classified as spam, then $acl_m9
|
||
is set to "spam", and the FAKEREJECT response above is issued. Finally, an
|
||
X-Spam-Status: header is added to the message. The idea is that the Mail
|
||
Delivery Agent or the recipient's Mail User Agent can use this header to
|
||
filter junk mail into a separate folder.
|
||
-----------------------------------------------------------------------------
|
||
|
||
A.10.2. Configure SpamAssassin
|
||
|
||
By default, SpamAssassin presents its report in a verbose, table-like
|
||
format, mainly suitable for inclusion in or attachment to the message body.
|
||
In our case, we want a terse report, suitable for the X-Spam-Status: header
|
||
in the example above. To do this, we add the following snippet in its site
|
||
specific configuration file (/etc/spamassassin/local.cf, /etc/mail/
|
||
spamassassin/local.cf, or similar):
|
||
### Report template
|
||
clear_report_template
|
||
report "_TESTSSCORES(, )_"
|
||
|
||
Also, a Bayesian scoring feature is built in, and is turned on by default.
|
||
We normally want to turn this off, because it requires training that will be
|
||
specific to each user, and thus is not suitable for system-wide SMTP time
|
||
filtering:
|
||
### Disable Bayesian scoring
|
||
use_bayes 0
|
||
|
||
For these changes to take effect, you have to restart the SpamAssassin
|
||
daemon (spamd).
|
||
-----------------------------------------------------------------------------
|
||
|
||
A.10.3. User Settings and Data
|
||
|
||
Say you have a number of users that want to specify their individual
|
||
SpamAssassin preferences, such as the spam threshold, acceptable languages
|
||
and character sets, white/blacklisted senders, and so on. Or perhaps they
|
||
really want to be able to make use of SpamAssassin's native Bayesian scoring
|
||
(though I don't see why[17]).
|
||
|
||
As discussed in the User Settings and Data section earlier in the document,
|
||
there is a way for this to happen. We need to limit the number of recipients
|
||
we accept per incoming mail delivery to one. We accept the first RCPT TO:
|
||
command issued by the caller, then defer subsequent ones using a 451 SMTP
|
||
response. As with greylisting, if the caller is a well-behaved MTA it will
|
||
know how to interpret this response, and retry later.
|
||
-----------------------------------------------------------------------------
|
||
|
||
A.10.3.1. Tell Exim to accept only one recipient per delivery
|
||
|
||
In the acl_rcpt_to, we insert the following statement after validating the
|
||
recipient address, but before any accept statements pertaining to
|
||
unauthenticated deliveries from remote hosts to local users (i.e. before any
|
||
greylist checks, envelope signature checks, etc):
|
||
# Limit the number of recipients in each incoming message to one
|
||
# to support per-user settings and data (e.g. for SpamAssassin).
|
||
#
|
||
# NOTE: Every mail sent to several users at your site will be
|
||
# delayed for 30 minutes or more per recipient. This
|
||
# significantly slow down the pace of discussion threads
|
||
# involving several internal and external parties.
|
||
#
|
||
defer
|
||
message = We only accept one recipient at a time - please try later.
|
||
condition = $recipients_count
|
||
|
||
-----------------------------------------------------------------------------
|
||
|
||
A.10.3.2. Pass the recipient username to SpamAssassin
|
||
|
||
In acl_data, we modify the spam condition given in the previous section, so
|
||
that it passes on to SpamAssassin the username specified in the local part of
|
||
the recipient address.
|
||
# Invoke SpamAssassin to obtain $spam_score and $spam_report.
|
||
# Depending on the classification, $acl_m9 is set to "ham" or "spam".
|
||
#
|
||
# We pass on the username specified in the recipient address,
|
||
# i.e. the portion before any '=' or '@' character, converted
|
||
# to lowercase. Multiple recipients should not occur, since
|
||
# we previously limited delivery to one recipient at a time.
|
||
#
|
||
# If the message is classified as spam, pretend to reject it.
|
||
#
|
||
warn
|
||
set acl_m9 = ham
|
||
spam = ${lc:${extract{1}{=@}{$recipients}{$value}{mail}}}
|
||
set acl_m9 = spam
|
||
control = fakereject
|
||
logwrite = :reject: Rejected spam (score $spam_score): $spam_report
|
||
|
||
|
||
Note that instead of using Exim's ${local_part:...} function to get the
|
||
username, we manually extracted the portion before any "@" or "=" character.
|
||
This is because we will use the latter character in our envelope signature
|
||
scheme, to follow.
|
||
-----------------------------------------------------------------------------
|
||
|
||
A.10.3.3. Enable per-user settings in SpamAssassin
|
||
|
||
Let us now again look at SpamAssassin. First of all, you may choose to
|
||
remove the use_bayes 0 setting that we previously added in its site-wide
|
||
configuration file. In any case, each user will now have the ability to
|
||
decide whether to override this setting for themselves.
|
||
|
||
If mailboxes on your system map directly to local UNIX accounts with home
|
||
directories, you are done. By default, the SpamAssassin daemon (spamd)
|
||
performs a setuid() to the username we pass to it, and stores user data and
|
||
settings in that user's home directory.
|
||
|
||
If this is not the case (for instance, if your mail accounts are managed by
|
||
Cyrus SASL or by another server), you need to tell SpamAssassin where to find
|
||
each user's preferences and data files. Also, spamd needs to keep running as
|
||
a specific local user instead of attempting to setuid() to a non-existing
|
||
user.
|
||
|
||
We do these things by specifying the options passed to spamd at startup:
|
||
|
||
* On a Debian system, edit the OPTIONS= setting in /etc/default/
|
||
spamassassin.
|
||
|
||
* On a RedHat system, edit the SPAMDOPTIONS= setting in /etc/sysconfig/
|
||
spamassassin.
|
||
|
||
* Others, figure it out.
|
||
|
||
|
||
The options you need are:
|
||
|
||
* -u username - specify the user under which spamd will run (e.g. mail)
|
||
|
||
* -x - disable configuration files in user's home directory.
|
||
|
||
* --virtual-config-dir=/var/lib/spamassassin/%u - specify where per-user
|
||
settings and data are stored. "%u" is replaced with the calling username.
|
||
spamd must be able to create or modify this directory:
|
||
# mkdir /var/lib/spamassassin
|
||
# chown -R mail:mail /var/lib/spamassassin
|
||
|
||
|
||
Needless to say, after making these changes, you need to restart spamd.
|
||
-----------------------------------------------------------------------------
|
||
|
||
A.11. Adding Envelope Sender Signatures
|
||
|
||
Here we implement Envelope Sender Signature in our outgoing mail, and check
|
||
for these signatures before accepting incoming "bounces" (i.e. mail with no
|
||
envelope sender).
|
||
|
||
The envelope sender address of outgoing mails from your host will be
|
||
modified as follows:
|
||
sender=recipient=recipient.domain=hash@sender.domain
|
||
|
||
However, because this scheme may produce unintended consequences (e.g. in
|
||
the case of mailing list servers), we make it optional for your users. We
|
||
sign the envelope sender address of outgoing mail only if we find a file
|
||
named ".return-path-sign" in the sender's home directory, and only if the
|
||
domain we are sending to is matched in that file. If the file exists, but is
|
||
empty, all domains match.
|
||
|
||
Similarly, we only require the recipient address to be signed in incoming
|
||
"bounce" messages (i.e. messages with no envelope sender) if the same file
|
||
exists in recipient's home directory. Users can exempt specific hosts from
|
||
this check via their user specific whitelist, as described in Exempting
|
||
Forwarded Mail.
|
||
|
||
Also, because this scheme involves tweaking with routers and transports in
|
||
addition to ACLs, we do not include it in the Final ACLs to follow. If you
|
||
are able to follow the instructions pertaining to those sections, you should
|
||
also be able to add the ACL section as described here.
|
||
-----------------------------------------------------------------------------
|
||
|
||
A.11.1. Create a Transport to Sign the Sender Address
|
||
|
||
First we create an Exim transport that will be used to sign the envelope
|
||
sender for remote deliveries:
|
||
remote_smtp_signed:
|
||
debug_print = "T: remote_smtp_signed for $local_part@$domain"
|
||
driver = smtp
|
||
max_rcpt = 1
|
||
return_path = $sender_address_local_part=$local_part=$domain=\
|
||
${hash_8:${hmac{md5}{SECRET}{${lc:\
|
||
$sender_address_local_part=$local_part=$domain}}}}\
|
||
@$sender_address_domain
|
||
|
||
The "local part" of the sender address now consists of the following
|
||
components, separated by equal signs ("="):
|
||
|
||
* the sender's username, i.e. the original local part,
|
||
|
||
* the local part of the recipient address,
|
||
|
||
* the domain part of the recipient address,
|
||
|
||
* a string unique to this sender/recipient combination, generated by:
|
||
|
||
+ encrypting the three prior components of the rewritten sender
|
||
address, using Exim's ${hmac{md5}...} function along with the SECRET
|
||
we declared in the main section, [18]
|
||
|
||
+ hashing the result into 8 lowercase letters, using Exim's $
|
||
{hash...} function.
|
||
|
||
|
||
|
||
If you need authentication for deliveries to "smarthosts", add an
|
||
appropriate hosts_try_auth line here as well. (Take it from your existing
|
||
smarthost transport).
|
||
-----------------------------------------------------------------------------
|
||
|
||
A.11.2. Create a New Router for Remote Deliveries
|
||
|
||
Add a new router prior to the existing router(s) that currently handles
|
||
your outgoing mail. This router will use the transport above for remote
|
||
deliveries, but only if the file ".return-path-sign" exists in the sender's
|
||
home directory, and if the recipient's domain is matched in that file. For
|
||
instance, if you send mail directly over the internet to the final
|
||
destination:
|
||
# Sign the envelope sender address (return path) for deliveries to
|
||
# remote domains if the sender's home directory contains the file
|
||
# ".return-path-sign", and if the remote domain is matched in that
|
||
# file. If the file exists, but is empty, the envelope sender
|
||
# address is always signed.
|
||
#
|
||
dnslookup_signed:
|
||
debug_print = "R: dnslookup_signed for $local_part@$domain"
|
||
driver = dnslookup
|
||
transport = remote_smtp_signed
|
||
senders = ! : *
|
||
domains = ! +local_domains : !+relay_to_domains : \
|
||
${if exists {/home/$sender_address_local_part/.return-path-sign}\
|
||
{/home/$sender_address_local_part/.return-path-sign}\
|
||
{!*}}
|
||
no_more
|
||
|
||
Or if you use a smarthost:
|
||
# Sign the envelope sender address (return path) for deliveries to
|
||
# remote domains if the sender's home directory contains the file
|
||
# ".return-path-sign", and if the remote domain is matched in that
|
||
# file. If the file exists, but is empty, the envelope sender
|
||
# address is always signed.
|
||
#
|
||
smarthost_signed:
|
||
debug_print = "R: smarthost_signed for $local_part@$domain"
|
||
driver = manualroute
|
||
transport = remote_smtp_signed
|
||
senders = ! : *
|
||
route_list = * smarthost.address
|
||
host_find_failed = defer
|
||
domains = ! +local_domains : !+relay_to_domains : \
|
||
${if exists {/home/$sender_address_local_part/.return-path-sign}\
|
||
{/home/$sender_address_local_part/.return-path-sign}\
|
||
{!*}}
|
||
no_more
|
||
|
||
Add other options as you see fit (e.g. same_domain_copy_routing = yes),
|
||
perhaps modelled after your existing routers.
|
||
|
||
Note that we do not use this router for mails with no envelope sender
|
||
address - we wouldn't want to tamper with those! [19]
|
||
-----------------------------------------------------------------------------
|
||
|
||
A.11.3. Create New Redirect Router for Local Deliveries
|
||
|
||
Next, you need to tell Exim that incoming recipient addresses that match
|
||
the format above should be delivered to the mailbox identified by the portion
|
||
before the first equal ("=") sign. For this purpose, you want to insert a
|
||
redirect router early in the routers section of your configuration file -
|
||
before any other routers pertaining to local deliveries (such as a system
|
||
alias router):
|
||
hashed_local:
|
||
debug_print = "R: hashed_local for $local_part@$domain"
|
||
driver = redirect
|
||
domains = +local_domains
|
||
local_part_suffix = =*
|
||
data = $local_part@$domain
|
||
|
||
Recipient addresses that contain a equal sign are rewritten such that the
|
||
portion of the local part that follows the equal sign are stripped off. Then
|
||
all routers are processed again.
|
||
-----------------------------------------------------------------------------
|
||
|
||
A.11.4. ACL Signature Check
|
||
|
||
The final part of this scheme is to tell Exim that mails delivered to valid
|
||
recipient addresses with this signature should always be accepted, and that
|
||
other messages with a NULL envelope sender should be rejected if the
|
||
recipient has opted in to this scheme. No greylisting should be done in
|
||
either case.
|
||
|
||
The following snippet should be placed in acl_rcpt_to, prior to any SPF
|
||
checks, greylisting, and/or the final accept statement:
|
||
# Accept the recipient addresss if it contains our own signature.
|
||
# This means this is a response (DSN, sender callout verification...)
|
||
# to a message that was previously sent from here.
|
||
#
|
||
accept
|
||
domains = +local_domains
|
||
condition = ${if and {{match{${lc:$local_part}}{^(.*)=(.*)}}\
|
||
{eq{${hash_8:${hmac{md5}{SECRET}{$1}}}}{$2}}}\
|
||
{true}{false}}
|
||
|
||
# Otherwise, if this message claims to be a bounce (i.e. if there
|
||
# is no envelope sender), but if the receiver has elected to use
|
||
# and check against envelope sender signatures, reject it.
|
||
#
|
||
deny
|
||
message = This address does not match a valid, signed \
|
||
return path from here.\n\
|
||
You are responding to a forged sender address.
|
||
log_message = bogus bounce.
|
||
senders = : postmaster@*
|
||
domains = +local_domains
|
||
set acl_m9 = /home/${extract{1}{=}{${lc:$local_part}}}/.return-path-sign
|
||
condition = ${if exists {$acl_m9}{true}}
|
||
|
||
|
||
You will have an issue when sending mail to hosts that perform callout
|
||
verification on addresses in the message header, such as the one provided in
|
||
the From: field of your outgoing mail. The deny statement here will
|
||
effectively give a negative response to such a verification attempt.
|
||
|
||
For that reason, you may want to convert the last deny statement into a
|
||
warn statement, store the rejection message in $acl_m0, and perform the
|
||
actual rejection after the DATA command, in a fashion similar to previously
|
||
described:
|
||
# Otherwise, if this message claims to be a bounce (i.e. if there
|
||
# is no envelope sender), but if the receiver has elected to use
|
||
# and check against envelope sender signatures, store a reject
|
||
# message in $acl_m0, and a log message in $acl_m1. We will later
|
||
# use these to reject the mail. In the mean time, their presence
|
||
# indicate that we should keep stalling the sender.
|
||
#
|
||
warn
|
||
senders = : postmaster@*
|
||
domains = +local_domains
|
||
set acl_m9 = /home/${extract{1}{=}{${lc:$local_part}}}/.return-path-sign
|
||
condition = ${if exists {$acl_m9}{true}}
|
||
set acl_m0 = The recipient address <$local_part@$domain> does not \
|
||
match a valid, signed return path from here.\n\
|
||
You are responding to a forged sender address.
|
||
set acl_m1 = bogus bounce for <$local_part@$domain>.
|
||
|
||
Also, even if the recipient has chosen to use envelope sender signatures in
|
||
their outgoing mail, they may want to exempt specific hosts from having to
|
||
provide this signature in incoming mail, even if the mail has no envelope
|
||
sender address. This may be required for specific mailing list servers, see
|
||
the discussion on Envelope Sender Signature for details.
|
||
-----------------------------------------------------------------------------
|
||
|
||
A.12. Accept Bounces Only for Real Users
|
||
|
||
As discussed in Accept Bounces Only for Real Users, there is now a loophole
|
||
that prevents us from catching bogus Delivery Status Notification sent to
|
||
system users and aliases, such as postmaster. Here we cover two alternate
|
||
ways to ensure that bounces are only accepted for users that actually send
|
||
outgoing mail.
|
||
-----------------------------------------------------------------------------
|
||
|
||
A.12.1. Check for Recipient Mailbox
|
||
|
||
The first method is performed in the acl_rcpt_to ACL. Here, we check that
|
||
the recipient address corresponds to a local mailbox:
|
||
# Deny mail for users that do not have a mailbox (i.e. postmaster,
|
||
# webmaster...) if no sender address is provided. These users do
|
||
# not send outgoing mail, so they should not receive returned mail.
|
||
#
|
||
deny
|
||
message = This address never sends outgoing mail. \
|
||
You are responding to a forged sender address.
|
||
log_message = bogus bounce for system user <$local_part@$domain>
|
||
senders = : postmaster@*
|
||
domains = +local_domains
|
||
!mailbox check
|
||
|
||
Unfortunately, how we perform the mailbox check will depend on how you
|
||
deliver your mail (as before, we extract the portion before the first "="
|
||
sign of the recipient address, to accomodate for Envelope Sender Signatures):
|
||
|
||
* If mailboxes map to local user accounts on your server, we can check
|
||
that the recipient name maps to a user ID that corresponds to "regular"
|
||
users on your system, e.g. in the range 500 - 60000:
|
||
set acl_m9 = ${extract{1}{=}{${lc:$local_part}}}
|
||
set acl_m9 = ${extract{2}{:}{${lookup passwd {$acl_m9}{$value}}}{0}}
|
||
condition = ${if and {{>={$acl_m9}{500}} {<${acl_m9}{60000}}} {true}}
|
||
|
||
|
||
* If you deliver mail to the [http://asg.web.cmu.edu/cyrus/] Cyrus IMAP
|
||
suite, you can use the provided mbpath command-line utility to check that
|
||
the mailbox exists. You will want to make sure that the Exim user has
|
||
permission to check for mailboxes (for instance, you may add it to the
|
||
cyrus group: # adduser exim4 cyrus).
|
||
set acl_m9 = ${extract{1}{=}{${lc:$local_part}}}
|
||
condition = ${run {/usr/sbin/mbpath -q -s user.$acl_m9} {true}}
|
||
|
||
|
||
* If you forward all mail to a remote machine for delivery, you may need
|
||
to perform a Recipient Callout Verification and let that machine decide
|
||
whether to accept the mail. You need to keep the original envelope sender
|
||
intact in the callout:
|
||
verify = recipient/callout=use_sender
|
||
|
||
|
||
|
||
Since in the case of locally delivered mail, this mailbox check duplicates
|
||
some of the logic that is performed in the routers, and since it is specific
|
||
to the mail delivery mechanism on our site, it is perhaps a bit kludgy for
|
||
the perfectionists among us. So we will now provide an alternate way.
|
||
-----------------------------------------------------------------------------
|
||
|
||
A.12.2. Check for Empty Sender in Aliases Router
|
||
|
||
You probably have a router named system_aliases or similar, to redirect
|
||
mail for users such as postmaster and mailer-demon. Typically, these aliases
|
||
are not used in the sender address of outgoing mail. As such, you can ensure
|
||
that incoming Delivery Status Notifications are not routed through it by
|
||
adding the following condition to the router:
|
||
!senders = : postmaster@*
|
||
|
||
A sample aliases router may now look like this:
|
||
system_aliases:
|
||
driver = redirect
|
||
domains = +local_domains
|
||
!senders = : postmaster@*
|
||
allow_fail
|
||
allow_defer
|
||
data = ${lookup{$local_part}lsearch{/etc/aliases}}
|
||
user = mail
|
||
group = mail
|
||
file_transport = address_file
|
||
pipe_transport = address_pipe
|
||
|
||
Although we now block bounces to some system aliases, other aliases were
|
||
merely shadowing existing system users (such as "root", "daemon", etc). If
|
||
you deliver local mail through the the accept driver, and use
|
||
check_local_user to validate the recipient address, you may now find yourself
|
||
routing mail directly to these system accounts.
|
||
|
||
To fix this problem, we now want to add an additional condition in the
|
||
router that handles your local mail (e.g. local_user) to ensure that the
|
||
recipient not only exists, but is a "regular" user. For instance, as above,
|
||
we can check that the user ID is in the range 500 - 60000:
|
||
condition = ${if and {{>={$local_user_uid}{500}}\
|
||
{<{$local_user_uid}{60000}}}\
|
||
{true}}
|
||
|
||
A sample router for local delivery may now look like this:
|
||
local_user:
|
||
driver = accept
|
||
domains = +local_domains
|
||
check_local_user
|
||
condition = ${if and {{>={$local_user_uid}{500}}\
|
||
{<{$local_user_uid}{60000}}}\
|
||
{true}}
|
||
transport = transport
|
||
|
||
Beware that if you implement this method, the reject response from your
|
||
server in response to bogus bounce mail for system users will be the same as
|
||
for unknown recipients (550 Unknown User in our case).
|
||
-----------------------------------------------------------------------------
|
||
|
||
A.13. Exempting Forwarded Mail
|
||
|
||
After adding all these checks in the SMTP transaction, we may find
|
||
ourselves indirectly creating collateral spam as a result of rejecting mails
|
||
forwarded from trusted sources, such as mailing lists and mail accounts on
|
||
other sites (see the discussion on Forwarded Mail for details). We now need
|
||
to whitelist these hosts in order to exempt them from SMTP rejections -- at
|
||
least those rejections that are caused by our spam and/or virus filtering.
|
||
|
||
In this example, we will consult two files in response to each RCPT TO:
|
||
command:
|
||
|
||
* A global whitelist in /etc/mail/whitelist-hosts, containing backup MX
|
||
hosts and other whitelisted senders [9], and
|
||
|
||
* A user-specific list in /home/user/.forwarders, specifying hosts from
|
||
which that particuar user will receive forwarded mail (e.g. mailing list
|
||
servers, outgoing mail servers for accounts elsewhere...)
|
||
|
||
|
||
If your mail users do not have local user accounts and home directories,
|
||
you may want to modify the file paths and/or lookup mechanisms to something
|
||
more suitable for your system (e.g. database lookups or LDAP queries).
|
||
|
||
If the sender host is found in one of these whitelists, we save the word
|
||
"accept" in $acl_m0, and clear the contents of $acl_m1, as described in the
|
||
previous section on Selective Delays. This will indicate that we should not
|
||
reject the mail in subsequent statements.
|
||
|
||
In the acl_rcpt_to, we insert the following statement after validating the
|
||
recipient address, but before any accept statements pertaining to
|
||
unauthenticated deliveries from remote hosts to local users (i.e. before any
|
||
greylist checks, envelope signature checks, etc):
|
||
# Accept the mail if the sending host is matched in the global
|
||
# whitelist file. Temporarily set $acl_m9 to point to this file.
|
||
# If the host is found, set a flag in $acl_m0 and clear $acl_m1 to
|
||
# indicate that we should not reject this mail later.
|
||
#
|
||
accept
|
||
set acl_m9 = /etc/mail/whitelist-hosts
|
||
hosts = ${if exists {$acl_m9}{$acl_m9}}
|
||
set acl_m0 = accept
|
||
set acl_m1 =
|
||
|
||
|
||
# Accept the mail if the sending host is matched in the ".forwarders"
|
||
# file in the recipient's home directory. Temporarily set $acl_m9 to
|
||
# point to this file. If the host is found, set a flag in $acl_m0 and
|
||
# clear $acl_m1 to indicate that we should not reject this mail later.
|
||
#
|
||
accept
|
||
domains = +local_domains
|
||
set acl_m9 = /home/${extract{1}{=}{${lc:$local_part}}}/.forwarders
|
||
hosts = ${if exists {$acl_m9}{$acl_m9}}
|
||
set acl_m0 = accept
|
||
set acl_m1 =
|
||
|
||
In various statements in the acl_data ACL, we check the contents of $acl_m0
|
||
to avoid rejecting the mail if this is set as per above. For instance, to
|
||
avoid rejecting mail from whitelisted hosts due to a missing RFC2822 header:
|
||
deny
|
||
message = Your message does not conform to RFC2822 standard
|
||
log_message = missing header lines
|
||
!hosts = +relay_from_hosts
|
||
!senders = : postmaster@*
|
||
condition = ${if !eq {$acl_m0}{accept}{true}}
|
||
condition = ${if or {{!def:h_Message-ID:}\
|
||
{!def:h_Date:}\
|
||
{!def:h_Subject:}} {true}{false}}
|
||
|
||
The appropriate checks are embedded in the Final ACLs, next.
|
||
-----------------------------------------------------------------------------
|
||
|
||
A.14. Final ACLs
|
||
|
||
OK, time to wake up! This has been very long reading - but congratulations
|
||
on making it this far!
|
||
|
||
The following ACLs incorporate all of the checks we have described so in
|
||
this implementation. However, some have been commented out, for the following
|
||
reasons:
|
||
|
||
* Greylisting. This either requires additional software to be installed,
|
||
or fairly complex inline configuration by way of additional ACLs and
|
||
definitions in the Exim configuration file. I highly recommend it,
|
||
though.
|
||
|
||
* Virus scanning. There is no ubiquitous scanner that nearly everyone
|
||
uses, similar to SpamAssassin for spam identification. On the other hand,
|
||
the documentation that comes with Exiscan-ACL should be easy to follow.
|
||
|
||
* Per-user settings for SpamAssassin. This is a trade-off that for many
|
||
is unacceptable, as it involves deferring mail to all but the first
|
||
recipient of a message.
|
||
|
||
* Envelope Sender Signatures. There are consequences, e.g. for roaming
|
||
users. Also, it involves configuring routers and transports as well as
|
||
ACLs. See that section for details.
|
||
|
||
* Accepting Bounces Only for Real Users. There are several ways of doing
|
||
this, and determining which users are real is specific to how mail is
|
||
delivered.
|
||
|
||
|
||
Without further ado, here comes the final result we have all been waiting
|
||
for.
|
||
-----------------------------------------------------------------------------
|
||
|
||
A.14.1. acl_connect
|
||
|
||
|
||
# This access control list is used at the start of an incoming
|
||
# connection. The tests are run in order until the connection is
|
||
# either accepted or denied.
|
||
|
||
acl_connect:
|
||
# Record the current timestamp, in order to calculate elapsed time
|
||
# for subsequent delays
|
||
warn
|
||
set acl_m2 = $tod_epoch
|
||
|
||
|
||
# Accept mail received over local SMTP (i.e. not over TCP/IP). We do
|
||
# this by testing for an empty sending host field.
|
||
# Also accept mails received over a local interface, and from hosts
|
||
# for which we relay mail.
|
||
accept
|
||
hosts = : +relay_from_hosts
|
||
|
||
|
||
# If the connecting host is in one of several DNSbl's, then prepare
|
||
# a warning message in $acl_c1. We will later add this message to
|
||
# the mail header. In the mean time, its presence indicates that
|
||
# we should keep stalling the sender.
|
||
#
|
||
|
||
warn
|
||
!hosts = ${if exists {/etc/mail/whitelist-hosts} \
|
||
{/etc/mail/whitelist-hosts}}
|
||
dnslists = list.dsbl.org : \
|
||
dnsbl.sorbs.net : \
|
||
dnsbl.njabl.org : \
|
||
bl.spamcop.net : \
|
||
dsn.rfc-ignorant.org : \
|
||
sbl-xbl.spamhaus.org : \
|
||
l1.spews.dnsbl.sorbs.net
|
||
set acl_c1 = X-DNSbl-Warning: \
|
||
$sender_host_address is listed in $dnslist_domain\
|
||
${if def:dnslist_text { ($dnslist_text)}}
|
||
|
||
|
||
# Likewise, if reverse DNS lookup of the sender's host fails (i.e.
|
||
# there is no rDNS entry, or a forward lookup of the resulting name
|
||
# does not match the original IP address), then generate a warning
|
||
# message in $acl_c1. We will later add this message to the mail
|
||
# header.
|
||
warn
|
||
condition = ${if !def:acl_c1 {true}{false}}
|
||
!verify = reverse_host_lookup
|
||
set acl_m9 = Reverse DNS lookup failed for host $sender_host_address
|
||
set acl_c1 = X-DNS-Warning: $acl_m9
|
||
|
||
|
||
# Accept the connection, but if we previously generated a message in
|
||
# $acl_c1, stall the sender until 20 seconds has elapsed.
|
||
accept
|
||
set acl_m2 = ${if def:acl_c1 {${eval:20 + $acl_m2 - $tod_epoch}}{0}}
|
||
delay = ${if >{$acl_m2}{0}{$acl_m2}{0}}s
|
||
-----------------------------------------------------------------------------
|
||
|
||
A.14.2. acl_helo
|
||
|
||
|
||
# This access control list is used for the HELO or EHLO command in
|
||
# an incoming SMTP transaction. The tests are run in order until the
|
||
# greeting is either accepted or denied.
|
||
|
||
acl_helo:
|
||
# Record the current timestamp, in order to calculate elapsed time
|
||
# for subsequent delays
|
||
warn
|
||
set acl_m2 = $tod_epoch
|
||
|
||
|
||
# Accept mail received over local SMTP (i.e. not over TCP/IP).
|
||
# We do this by testing for an empty sending host field.
|
||
# Also accept mails received from hosts for which we relay mail.
|
||
#
|
||
accept
|
||
hosts = : +relay_from_hosts
|
||
|
||
|
||
# If the remote host greets with an IP address, then prepare a reject
|
||
# message in $acl_c0, and a log message in $acl_c1. We will later use
|
||
# these in a "deny" statement. In the mean time, their presence indicate
|
||
# that we should keep stalling the sender.
|
||
#
|
||
warn
|
||
condition = ${if isip {$sender_helo_name}{true}{false}}
|
||
set acl_c0 = Message was delivered by ratware
|
||
set acl_c1 = remote host used IP address in HELO/EHLO greeting
|
||
|
||
|
||
# Likewise if the peer greets with one of our own names
|
||
#
|
||
warn
|
||
condition = ${if match_domain{$sender_helo_name}\
|
||
{$primary_hostname:+local_domains:+relay_to_domains}\
|
||
{true}{false}}
|
||
set acl_c0 = Message was delivered by ratware
|
||
set acl_c1 = remote host used our name in HELO/EHLO greeting.
|
||
|
||
|
||
# If HELO verification fails, we prepare a warning message in acl_c1.
|
||
# We will later add this message to the mail header. In the mean time,
|
||
# its presence indicates that we should keep stalling the sender.
|
||
#
|
||
warn
|
||
condition = ${if !def:acl_c1 {true}{false}}
|
||
!verify = helo
|
||
set acl_c1 = X-HELO-Warning: Remote host $sender_host_address \
|
||
${if def:sender_host_name {($sender_host_name) }}\
|
||
incorrectly presented itself as $sender_helo_name
|
||
log_message = remote host presented unverifiable HELO/EHLO greeting.
|
||
|
||
|
||
# Accept the greeting, but if we previously generated a message in
|
||
# $acl_c1, stall the sender until 20 seconds has elapsed.
|
||
accept
|
||
set acl_m2 = ${if def:acl_c1 {${eval:20 + $acl_m2 - $tod_epoch}}{0}}
|
||
delay = ${if >{$acl_m2}{0}{$acl_m2}{0}}s
|
||
|
||
-----------------------------------------------------------------------------
|
||
|
||
A.14.3. acl_mail_from
|
||
|
||
|
||
# This access control list is used for the MAIL FROM: command in an
|
||
# incoming SMTP transaction. The tests are run in order until the
|
||
# sender address is either accepted or denied.
|
||
#
|
||
|
||
acl_mail_from:
|
||
# Record the current timestamp, in order to calculate elapsed time
|
||
# for subsequent delays
|
||
warn
|
||
set acl_m2 = $tod_epoch
|
||
|
||
|
||
# Accept mail received over local SMTP (i.e. not over TCP/IP).
|
||
# We do this by testing for an empty sending host field.
|
||
# Also accept mails received from hosts for which we relay mail.
|
||
#
|
||
# Sender verification is omitted here, because in many cases
|
||
# the clients are dumb MUAs that don't cope well with SMTP
|
||
# error responses.
|
||
#
|
||
accept
|
||
hosts = : +relay_from_hosts
|
||
|
||
|
||
# Accept if the message arrived over an authenticated connection,
|
||
# from any host. Again, these messages are usually from MUAs.
|
||
#
|
||
accept
|
||
authenticated = *
|
||
|
||
|
||
# If present, the ACL variables $acl_c0 and $acl_c1 contain rejection
|
||
# and/or warning messages to be applied to every delivery attempt in
|
||
# in this SMTP transaction. Assign these to the corresponding
|
||
# $acl_m{0,1} message-specific variables, and add any warning message
|
||
# from $acl_m1 to the message header. (In the case of a rejection,
|
||
# $acl_m1 actually contains a log message instead, but this does not
|
||
# matter, as we will discard the header along with the message).
|
||
#
|
||
warn
|
||
set acl_m0 = $acl_c0
|
||
set acl_m1 = $acl_c1
|
||
message = $acl_c1
|
||
|
||
|
||
# If sender did not provide a HELO/EHLO greeting, then prepare a reject
|
||
# message in $acl_m0, and a log message in $acl_m1. We will later use
|
||
# these in a "deny" statement. In the mean time, their presence indicate
|
||
# that we should keep stalling the sender.
|
||
#
|
||
warn
|
||
condition = ${if def:sender_helo_name {0}{1}}
|
||
set acl_m0 = Message was delivered by ratware
|
||
set acl_m1 = remote host did not present HELO/EHLO greeting.
|
||
|
||
|
||
# If we could not verify the sender address, create a warning message
|
||
# in $acl_m1 and add it to the mail header. The presence of this
|
||
# message indicates that we should keep stalling the sender.
|
||
#
|
||
# You may choose to omit the "callout" option. In particular, if
|
||
# you are sending outgoing mail through a smarthost, it will not
|
||
# give any useful information.
|
||
#
|
||
warn
|
||
condition = ${if !def:acl_m1 {true}{false}}
|
||
!verify = sender/callout
|
||
set acl_m1 = Invalid sender <$sender_address>
|
||
message = X-Sender-Verify-Failed: $acl_m1
|
||
log_message = $acl_m1
|
||
|
||
|
||
# Accept the sender, but if we previously generated a message in
|
||
# $acl_c1, stall the sender until 20 seconds has elapsed.
|
||
accept
|
||
set acl_m2 = ${if def:acl_c1 {${eval:20 + $acl_m2 - $tod_epoch}}{0}}
|
||
delay = ${if >{$acl_m2}{0}{$acl_m2}{0}}s
|
||
|
||
-----------------------------------------------------------------------------
|
||
|
||
A.14.4. acl_rcpt_to
|
||
|
||
|
||
# This access control list is used for every RCPT command in an
|
||
# incoming SMTP message. The tests are run in order until the
|
||
# recipient address is either accepted or denied.
|
||
|
||
acl_rcpt_to:
|
||
|
||
# Accept mail received over local SMTP (i.e. not over TCP/IP).
|
||
# We do this by testing for an empty sending host field.
|
||
# Also accept mails received from hosts for which we relay mail.
|
||
#
|
||
# Recipient verification is omitted here, because in many
|
||
# cases the clients are dumb MUAs that don't cope well with
|
||
# SMTP error responses.
|
||
#
|
||
accept
|
||
hosts = : +relay_from_hosts
|
||
|
||
|
||
# Accept if the message arrived over an authenticated connection,
|
||
# from any host. Again, these messages are usually from MUAs, so
|
||
# recipient verification is omitted.
|
||
#
|
||
accept
|
||
authenticated = *
|
||
|
||
|
||
# Deny if the local part contains @ or % or / or | or !. These are
|
||
# rarely found in genuine local parts, but are often tried by people
|
||
# looking to circumvent relaying restrictions.
|
||
#
|
||
# Also deny if the local part starts with a dot. Empty components
|
||
# aren't strictly legal in RFC 2822, but Exim allows them because
|
||
# this is common. However, actually starting with a dot may cause
|
||
# trouble if the local part is used as a file name (e.g. for a
|
||
# mailing list).
|
||
#
|
||
deny
|
||
local_parts = ^.*[@%!/|] : ^\\.
|
||
|
||
|
||
# Deny if we have previously given a reason for doing so in $acl_m0.
|
||
# Also stall the sender for another 20s first.
|
||
#
|
||
deny
|
||
message = $acl_m0
|
||
log_message = $acl_m1
|
||
condition = ${if and {{def:acl_m0}{def:acl_m1}} {true}}
|
||
delay = 20s
|
||
|
||
|
||
# If the recipient address is not in a domain for which we are handling
|
||
# mail, stall the sender and reject.
|
||
#
|
||
deny
|
||
message = relay not permitted
|
||
!domains = +local_domains : +relay_to_domains
|
||
delay = 20s
|
||
|
||
|
||
# If the address is in a local domain or in a domain for which are
|
||
# relaying, but is invalid, stall and reject.
|
||
#
|
||
deny
|
||
message = unknown user
|
||
!verify = recipient/callout=20s,defer_ok,use_sender
|
||
delay = ${if def:sender_address {1m}{0s}}
|
||
|
||
|
||
|
||
# Drop the connection if the envelope sender is empty, but there is
|
||
# more than one recipient address. Legitimate DSNs are never sent
|
||
# to more than one address.
|
||
#
|
||
drop
|
||
message = Legitimate bounces are never sent to more than one \
|
||
recipient.
|
||
senders = : postmaster@*
|
||
condition = $recipients_count
|
||
delay = 5m
|
||
|
||
|
||
# --------------------------------------------------------------------
|
||
# Limit the number of recipients in each incoming message to one
|
||
# to support per-user settings and data (e.g. for SpamAssassin).
|
||
#
|
||
# NOTE: Every mail sent to several users at your site will be
|
||
# delayed for 30 minutes or more per recipient. This
|
||
# significantly slow down the pace of discussion threads
|
||
# involving several internal and external parties.
|
||
# Thus, it is commented out by default.
|
||
#
|
||
#defer
|
||
# message = We only accept one recipient at a time - please try later.
|
||
# condition = $recipients_count
|
||
# --------------------------------------------------------------------
|
||
|
||
|
||
# Accept the mail if the sending host is matched in the ".forwarders"
|
||
# file in the recipient's home directory. Temporarily set $acl_m9 to
|
||
# point to this file. If the host is found, set a flag in $acl_m0 and
|
||
# clear $acl_m1 to indicate that we should not reject this mail later.
|
||
#
|
||
accept
|
||
domains = +local_domains
|
||
set acl_m9 = /home/${extract{1}{=}{${lc:$local_part}}}/.forwarders
|
||
hosts = ${if exists {$acl_m9}{$acl_m9}}
|
||
set acl_m0 = accept
|
||
set acl_m1 =
|
||
|
||
|
||
# Accept the mail if the sending host is matched in the global
|
||
# whitelist file. Temporarily set $acl_m9 to point to this file.
|
||
# If the host is found, set a flag in $acl_m0 and clear $acl_m1 to
|
||
# indicate that we should not reject this mail later.
|
||
#
|
||
accept
|
||
set acl_m9 = /etc/mail/whitelist-hosts
|
||
hosts = ${if exists {$acl_m9}{$acl_m9}}
|
||
set acl_m0 = accept
|
||
set acl_m1 =
|
||
|
||
|
||
# --------------------------------------------------------------------
|
||
# Envelope Sender Signature Check.
|
||
# This is commented out by default, because it requires additional
|
||
# configuration in the 'transports' and 'routers' sections.
|
||
#
|
||
# Accept the recipient addresss if it contains our own signature.
|
||
# This means this is a response (DSN, sender callout verification...)
|
||
# to a message that was previously sent from here.
|
||
#
|
||
#accept
|
||
# domains = +local_domains
|
||
# condition = ${if and {{match{${lc:$local_part}}{^(.*)=(.*)}}\
|
||
# {eq{${hash_8:${hmac{md5}{SECRET}{$1}}}}{$2}}}\
|
||
# {true}{false}}
|
||
#
|
||
# Otherwise, if this message claims to be a bounce (i.e. if there
|
||
# is no envelope sender), but if the receiver has elected to use
|
||
# and check against envelope sender signatures, reject it.
|
||
#
|
||
#deny
|
||
# message = This address does not match a valid, signed \
|
||
# return path from here.\n\
|
||
# You are responding to a forged sender address.
|
||
# log_message = bogus bounce.
|
||
# senders = : postmaster@*
|
||
# domains = +local_domains
|
||
# set acl_m9 = /home/${extract{1}{=}{${lc:$local_part}}}/.return-path-sign
|
||
# condition = ${if exists {$acl_m9}{true}}
|
||
# --------------------------------------------------------------------
|
||
|
||
|
||
# --------------------------------------------------------------------
|
||
# Deny mail for local users that do not have a mailbox (i.e. postmaster,
|
||
# webmaster...) if no sender address is provided. These users do
|
||
# not send outgoing mail, so they should not receive returned mail.
|
||
#
|
||
# NOTE: This is commented out by default, because the condition is
|
||
# specific to how local mail is delivered. If you want to
|
||
# enable this check, uncomment one and only one of the
|
||
# conditions below.
|
||
#
|
||
#deny
|
||
# message = This address never sends outgoing mail. \
|
||
# You are responding to a forged sender address.
|
||
# log_message = bogus bounce for system user <$local_part@$domain>
|
||
# senders = : postmaster@*
|
||
# domains = +local_domains
|
||
# set acl_m9 = ${extract{1}{=}{${lc:$local_part}}}
|
||
#
|
||
# --- Uncomment the following 2 lines if recipients have local accounts:
|
||
# set acl_m9 = ${extract{2}{:}{${lookup passwd {$acl_m9}{$value}}}{0}}
|
||
# !condition = ${if and {{>={$acl_m9}{500}} {<${acl_m9}{60000}}} {true}}
|
||
#
|
||
# --- Uncomment the following line if you deliver mail to Cyrus:
|
||
# condition = ${run {/usr/sbin/mbpath -q -s user.$acl_m9} {true}}
|
||
# --------------------------------------------------------------------
|
||
|
||
|
||
|
||
# Query the SPF information for the sender address domain, if any,
|
||
# to see if the sending host is authorized to deliver its mail.
|
||
# If not, reject the mail.
|
||
#
|
||
deny
|
||
message = [SPF] $sender_host_address is not allowed to send mail \
|
||
from $sender_address_domain
|
||
log_message = SPF check failed.
|
||
spf = fail
|
||
|
||
|
||
# Add a SPF-Received: line to the message header
|
||
warn
|
||
message = $spf_received
|
||
|
||
|
||
# --------------------------------------------------------------------
|
||
# Check greylisting status for this particular peer/sender/recipient.
|
||
# Before uncommenting this statement, you need to install "greylistd".
|
||
# See: http://packages.debian.org/unstable/main/greylistd
|
||
#
|
||
# Note that we do not greylist messages with NULL sender, because
|
||
# sender callout verification would break (and we might not be able
|
||
# to send mail to a host that performs callouts).
|
||
#
|
||
#defer
|
||
# message = $sender_host_address is not yet authorized to deliver mail \
|
||
# from <$sender_address> to <$local_part@$domain>. \
|
||
# Please try later.
|
||
# log_message = greylisted.
|
||
# domains = +local_domains : +relay_to_domains
|
||
# !senders = : postmaster@*
|
||
# set acl_m9 = $sender_host_address $sender_address $local_part@$domain
|
||
# set acl_m9 = ${readsocket{/var/run/greylistd/socket}{$acl_m9}{5s}{}{}}
|
||
# condition = ${if eq {$acl_m9}{grey}{true}{false}}
|
||
# delay = 20s
|
||
# --------------------------------------------------------------------
|
||
|
||
# Accept the recipient.
|
||
accept
|
||
-----------------------------------------------------------------------------
|
||
|
||
A.14.5. acl_data
|
||
|
||
|
||
# This access control list is used for message data received via
|
||
# SMTP. The tests are run in order until the recipient address
|
||
# is either accepted or denied.
|
||
|
||
acl_data:
|
||
# Log some header lines
|
||
warn
|
||
logwrite = Subject: $h_Subject:
|
||
|
||
|
||
# Add Message-ID if missing in messages received from our own hosts.
|
||
warn
|
||
condition = ${if !def:h_Message-ID: {1}}
|
||
hosts = +relay_from_hosts
|
||
message = Message-ID: <E$message_id@$primary_hostname>
|
||
|
||
|
||
# Accept mail received over local SMTP (i.e. not over TCP/IP).
|
||
# We do this by testing for an empty sending host field.
|
||
# Also accept mails received from hosts for which we relay mail.
|
||
#
|
||
accept
|
||
hosts = : +relay_from_hosts
|
||
|
||
# Accept if the message arrived over an authenticated connection, from
|
||
# any host.
|
||
#
|
||
accept
|
||
authenticated = *
|
||
|
||
|
||
# Deny if we have previously given a reason for doing so in $acl_m0.
|
||
# Also stall the sender for another 20s first.
|
||
#
|
||
deny
|
||
message = $acl_m0
|
||
log_message = $acl_m1
|
||
condition = ${if and {{def:acl_m0}{def:acl_m1}} {true}{false}}
|
||
delay = 20s
|
||
|
||
|
||
# enforce a message-size limit
|
||
#
|
||
deny
|
||
message = Message size $message_size is larger than limit of \
|
||
MESSAGE_SIZE_LIMIT
|
||
condition = ${if >{$message_size}{MESSAGE_SIZE_LIMIT}{yes}{no}}
|
||
|
||
|
||
# Deny unless the addresses in the header is syntactically correct.
|
||
#
|
||
deny
|
||
message = Your message does not conform to RFC2822 standard
|
||
log_message = message header fail syntax check
|
||
!verify = header_syntax
|
||
|
||
|
||
# Uncomment the following to deny non-local messages without
|
||
# a Message-ID:, Date:, or Subject: header.
|
||
#
|
||
# Note that some specialized MTAs, such as certain mailing list
|
||
# servers, do not automatically generate a Message-ID for bounces.
|
||
# Thus, we add the check for a non-empty sender.
|
||
#
|
||
#deny
|
||
# message = Your message does not conform to RFC2822 standard
|
||
# log_message = missing header lines
|
||
# !hosts = +relay_from_hosts
|
||
# !senders = : postmaster@*
|
||
# condition = ${if !eq {$acl_m0}{accept}{true}}
|
||
# condition = ${if or {{!def:h_Message-ID:}\
|
||
# {!def:h_Date:}\
|
||
# {!def:h_Subject:}} {true}{false}}
|
||
|
||
|
||
# Warn unless there is a verifiable sender address in at least
|
||
# one of the "Sender:", "Reply-To:", or "From:" header lines.
|
||
#
|
||
warn
|
||
message = X-Sender-Verify-Failed: No valid sender in message header
|
||
log_message = No valid sender in message header
|
||
!verify = header_sender
|
||
|
||
|
||
|
||
# --------------------------------------------------------------------
|
||
# Perform greylisting on messages with no envelope sender here.
|
||
# We did not subject these to greylisting after RCPT TO: because
|
||
# that would interfere with remote hosts doing sender callouts.
|
||
# Note that the sender address is empty, so we don't bother using it.
|
||
#
|
||
# Before uncommenting this statement, you need to install "greylistd".
|
||
# See: http://packages.debian.org/unstable/main/greylistd
|
||
#
|
||
#defer
|
||
# message = $sender_host_address is not yet authorized to send \
|
||
# delivery status reports to <$recipients>. \
|
||
# Please try later.
|
||
# log_message = greylisted.
|
||
# senders = : postmaster@*
|
||
# condition = ${if !eq {$acl_m0}{accept}{true}}
|
||
# set acl_m9 = $sender_host_address $recipients
|
||
# set acl_m9 = ${readsocket{/var/run/greylistd/socket}{$acl_m9}{5s}{}{}}
|
||
# condition = ${if eq {$acl_m9}{grey}{true}{false}}
|
||
# delay = 20s
|
||
# --------------------------------------------------------------------
|
||
|
||
|
||
|
||
# --- BEGIN EXISCAN configuration ---
|
||
|
||
# Reject messages that have serious MIME errors.
|
||
#
|
||
deny
|
||
message = Serious MIME defect detected ($demime_reason)
|
||
demime = *
|
||
condition = ${if >{$demime_errorlevel}{2}{1}{0}}
|
||
|
||
|
||
# Unpack MIME containers and reject file extensions used by worms.
|
||
# This calls the demime condition again, but it will return cached results.
|
||
# Note that the extension list may be incomplete.
|
||
#
|
||
deny
|
||
message = We do not accept ".$found_extension" attachments here.
|
||
demime = bat:btm:cmd:com:cpl:dll:exe:lnk:msi:pif:prf:reg:scr:vbs:url
|
||
|
||
|
||
# Messages larger than MESSAGE_SIZE_SPAM_MAX are accepted without
|
||
# spam or virus scanning
|
||
accept
|
||
condition = ${if >{$message_size}{MESSAGE_SIZE_SPAM_MAX} {true}}
|
||
logwrite = :main: Not classified \
|
||
(message size larger than MESSAGE_SIZE_SPAM_MAX)
|
||
|
||
|
||
# --------------------------------------------------------------------
|
||
# Anti-Virus scanning
|
||
# This requires an 'av_scanner' setting in the main section.
|
||
#
|
||
#deny
|
||
# message = This message contains a virus ($malware_name)
|
||
# demime = *
|
||
# malware = */defer_ok
|
||
# --------------------------------------------------------------------
|
||
|
||
|
||
|
||
# Invoke SpamAssassin to obtain $spam_score and $spam_report.
|
||
# Depending on the classification, $acl_m9 is set to "ham" or "spam".
|
||
#
|
||
# If the message is classified as spam, and we have not previously
|
||
# set $acl_m0 to indicate that we want to accept it anyway, pretend
|
||
# reject it.
|
||
#
|
||
warn
|
||
set acl_m9 = ham
|
||
# ------------------------------------------------------------------
|
||
# If you want to allow per-user settings for SpamAssassin,
|
||
# uncomment the following line, and comment out "spam = mail".
|
||
# We pass on the username specified in the recipient address,
|
||
# i.e. the portion before any '=' or '@' character, converted
|
||
# to lowercase. Multiple recipients should not occur, since
|
||
# we previously limited delivery to one recipient at a time.
|
||
#
|
||
# spam = ${lc:${extract{1}{=@}{$recipients}{$value}{mail}}}
|
||
# ------------------------------------------------------------------
|
||
spam = mail
|
||
set acl_m9 = spam
|
||
condition = ${if !eq {$acl_m0}{accept}{true}}
|
||
control = fakereject
|
||
logwrite = :reject: Rejected spam (score $spam_score): $spam_report
|
||
|
||
|
||
|
||
# Add an appropriate X-Spam-Status: header to the message.
|
||
#
|
||
warn
|
||
message = X-Spam-Status: \
|
||
${if eq {$acl_m9}{spam}{Yes}{No}} (score $spam_score)\
|
||
${if def:spam_report {: $spam_report}}
|
||
logwrite = :main: Classified as $acl_m9 (score $spam_score)
|
||
|
||
|
||
# --- END EXISCAN configuration ---
|
||
|
||
|
||
# Accept the message.
|
||
#
|
||
accept
|
||
|
||
Glossary
|
||
|
||
|
||
These are definitions for some of the words and terms that are used
|
||
throughout this document.
|
||
|
||
|
||
B
|
||
|
||
Bayesian Filters
|
||
A filter that assigns a probability of spam based on the recurrence of
|
||
words (or, more recently, word constellations/phrases) between messages.
|
||
|
||
You initially train the filter by feeding it known junk mail (spam) and
|
||
known legitimate mail (ham). A bayesian score is then be assigned to each
|
||
word (or phrase) in each message, indicating whether this particular word
|
||
or phrase occurs most commonly in ham or in spam. The word, along with
|
||
its score, is stored in a bayesian index.
|
||
|
||
Such filters may catch indicators that may be missed by human
|
||
programmers trying to manually create keyword-based filters. At the very
|
||
least, they automate this task.
|
||
|
||
Bayesian word indexes are most certainly specific to the language in
|
||
which they received training. Moreover, they are specific to individual
|
||
users. Thus, they are perhaps more suitable for individual content
|
||
filters (e.g. in Mail User Agents) than they are for system-wide,
|
||
SMTP-time filtering.
|
||
|
||
Moreover, spammers have developed techniques to defeat simple bayesian
|
||
filters, by including random dictionary words and/or short stories in
|
||
their messages. This decreases the spam probability assigned by a
|
||
baynesian filter, and in the long run, degrades the quality of the
|
||
bayesian index.
|
||
|
||
See also: [http://www.everything2.com/index.pl?node=Bayesian] http://
|
||
www.everything2.com/index.pl?node=Bayesian.
|
||
|
||
|
||
C
|
||
|
||
Collateral Damage
|
||
Blocking of a legitimate sender host due to an entry in a DNS
|
||
blocklist.
|
||
|
||
Some blocklists (like SPEWS) routinely list the entire IP address space
|
||
of an ISP if they feel the ISP is not responsive to abuse complaints,
|
||
thereby affecting all its customers.
|
||
|
||
See also: False Positive
|
||
|
||
Collateral Spam
|
||
Automated messages sent in response to an original message (mostly spam
|
||
or malware) where the sender address is forged. Typical examples of
|
||
collateral spam include virus scan reports ("You have a virus") or other
|
||
Delivery Status Notifications).
|
||
|
||
|
||
D
|
||
|
||
Domain Name System
|
||
(abbrev: DNS) The de-facto standard for obtaining information about
|
||
internet domain names. Examples of such information include IP addresses
|
||
of its servers (so-called A records), the dedication of incoming mail
|
||
exchangers (MX records), generic server information (SRV records), and
|
||
miscellaneous text information (TXT records).
|
||
|
||
DNS is a hierarctical, distributed system; each domain name is
|
||
associated with a set of one or more DNS servers that provide information
|
||
about that domain - including delegation of name service for its
|
||
subdomains.
|
||
|
||
For instance, the top-level domain "org" is operated by The Public
|
||
Interest Registry; its DNS servers delegate queries for the domain name
|
||
"tldp.org" to specific name servers for The Linux Documentation Project.
|
||
In turn, TLDPs name server (actually operated by UNC) may or may not
|
||
delegate queries for third-level names, such as "www.tldp.org".
|
||
|
||
DNS lookups are usually performed by forwarding name servers, such as
|
||
those provided by an Internet Service Provider (e.g. via DHCP).
|
||
|
||
Delivery Status Notification
|
||
(abbrev: DSN) A message automatically created by an MTA or MDA, to
|
||
inform the sender of an original messsage (usually included in the DSN)
|
||
about its status. For instance, DSNs may inform the sender of the
|
||
original message that it could not be delivered due to a temporary or
|
||
permanent problem, and/or whether or not and for how long delivery
|
||
attempts will continue.
|
||
|
||
Delivery Status Notifications are sent with an empty Envelope Sender
|
||
address.
|
||
|
||
|
||
E
|
||
|
||
Envelope Sender
|
||
The e-mail address given as sender of a message during the SMTP
|
||
transaction, using the MAIL FROM: command. This may be different from the
|
||
address provided in the "From:" header of the message itself.
|
||
|
||
One special case is Delivery Status Notification (bounced message,
|
||
return receipt, vacation message..). For such mails, the Envelope Sender
|
||
is empty. This is to prevent Mail Loops, and generally to be able to
|
||
distinguish these from "regular" mails.
|
||
|
||
See also: The SMTP Transaction
|
||
|
||
Envelope Recipient
|
||
The e-mail address(es) to which the message is sent. These are provided
|
||
during the SMTP transaction, using the RCPT TO command. These may be
|
||
different from the addresses provided in the "To:" and "Cc:" headers of
|
||
the message itself.
|
||
|
||
See also: The SMTP Transaction
|
||
|
||
|
||
F
|
||
|
||
False Negative
|
||
Junk mail (spam, virus, malware) that is misclassified as legitimate
|
||
mail (and consequently, not filtered out).
|
||
|
||
False Positive
|
||
Legitimate mail that is misclassified as junk (and consequently,
|
||
blocked).
|
||
|
||
See also: Collateral Damage.
|
||
|
||
Fully Qualified Domain Name
|
||
(a.k.a. "FQDN"). A full, globally unique, internet name, including DNS
|
||
domain. For instance: "www.yahoo.com".
|
||
|
||
A FQDN does not always point to a single host. For instance, common
|
||
service names such as "www" often point to many IP addresses, in order to
|
||
provide some load balancing on the servers. However, the primary host
|
||
name of a given machine should always be unique to that machine; for
|
||
instance: "p16.www.scd.yahoo.com".
|
||
|
||
A FQDN always contains a period ("."). The part before the first period
|
||
is the unqualified name, and is not globally unique.
|
||
|
||
|
||
J
|
||
|
||
Joe Job
|
||
A spam designed to look like it came from someone else's valid address,
|
||
often in a malicous attempt at generating complaints from third parties
|
||
and/or cause other damage to the owner of that address.
|
||
|
||
See also: [http://www.everything2.com/index.pl?node=Joe%20Job] http://
|
||
www.everything2.com/index.pl?node=Joe%20Job
|
||
|
||
|
||
M
|
||
|
||
Mail Delivery Agent
|
||
(abbrev: MDA) Software that runs on the machine where a users' mailbox
|
||
is located, to deliver mail into that mailbox. Often, that delivery is
|
||
performed directly by the MTA Mail Transport Agent, which then serves a
|
||
secondary role as an MDA. Examples of separate Mail Delivery Agents
|
||
include: Deliver, Procmail, Cyrmaster and/or Cyrdeliver (from the Cyrus
|
||
IMAP suite).
|
||
|
||
Mail Loop
|
||
A situation where one automated message triggers another, which
|
||
directly or indirectly triggers the first message over again, and so on.
|
||
|
||
Imagine a mailing list where one of the subscribers is the address of
|
||
the list itself. This situation is often dealt with by the list server
|
||
adding an "X-Loop:" line in the message header, and not processing mails
|
||
that already have one.
|
||
|
||
Another equivalent term is Ringing.
|
||
|
||
Mail Transport Agent
|
||
(abbrev: MTA) Software that runs on a mail server, such as the mail
|
||
exchanger(s) of a internet domain, to send mail to and receive mail from
|
||
other hosts. Popular MTAs include: Sendmail, Postfix, Exim, Smail.
|
||
|
||
Mail User Agent
|
||
(abbrev: MUA; a.k.a. Mail Reader) User software to access, download,
|
||
read, and send mail. Examples include Microsoft Outlook/Outlook Express,
|
||
Apple Mail.app, Mozilla Thunderbird, Ximian Evolution.
|
||
|
||
Mail Exchanger
|
||
(abbrev: MX) A machine dedicated to (sending and/or) receiving mail for
|
||
an internet domain.
|
||
|
||
The DNS zone information for a internet domain normally contains a list
|
||
of Fully Qualified Domain Names that act as incoming mail exchangers for
|
||
that domain. Each such listing is called an "MX record", and it also
|
||
contains a number indicating its "priority" among several "MX records".
|
||
The listing with the lowest number has the first priority, and is
|
||
considered the "primary mail exchanger" for that domain.
|
||
|
||
Micropayment Schemes
|
||
(a.k.a. sender pay schemes). The sender of a message expends some
|
||
machine resources to create a virtual postage stamp for each recipient of
|
||
a message - usually by solving a mathematical challenge that requires a
|
||
large number of memory read/write operations, but is relatively CPU speed
|
||
insensitive. This stamp is then added to the headers of the message, and
|
||
the recipient would validate the stamp through a much simpler decoding
|
||
operation.
|
||
|
||
The idea is that because the message requires a postage stamp for every
|
||
recipient address, spamming hundreds or thousands of users at once would
|
||
be prohibitively "expensive".
|
||
|
||
Two such systems are:
|
||
|
||
+ [http://www.camram.org/] Camram
|
||
|
||
+ Microsoft's Penny Black Project
|
||
|
||
|
||
|
||
O
|
||
|
||
Open Proxy
|
||
A proxy which openly accepts TCP/IP connections from anywhere, and
|
||
forwards them anywhere.
|
||
|
||
These are typically exploited by spammers and virii, who use them to
|
||
conceal their own IP address, and/or to more effectively distribute
|
||
transmission loads across several hosts and networks.
|
||
|
||
See also: Zombie Host
|
||
|
||
Open Relay
|
||
A Relay which openly accepts mail from anywhere, and forwards them to
|
||
anywhere.
|
||
|
||
In the 1980s, virtually every public SMTP server was an Open Relay.
|
||
Messages would often travel between multiple third-party machines before
|
||
it reached the intended recipient. Now, legitimate mail are almost
|
||
exclusively sent directly from an outgoing Mail Transport Agent on the
|
||
sender's end to the incoming Mail Exchanger(s) for the recipient's
|
||
domain.
|
||
|
||
Conversely, Open Relay servers that still exist on the internet are
|
||
almost exclusively exploited by spammers to hide their own identity, and
|
||
to perform some load balancing on the task of sending out millions of
|
||
messages, presumably before DNS blocklists have a chance to get all of
|
||
these machines listed.
|
||
|
||
See also the discussion on Open Relay Prevention.
|
||
|
||
|
||
P
|
||
|
||
proxy
|
||
A machine that acts on behalf of someone else. It may forward e.g. HTTP
|
||
requests or TCP/IP connections, usually to or from the internet. For
|
||
instance, companies - or sometimes entire countries - often use "Web
|
||
Proxy Servers" to filter outgoing HTTP requests from their internal
|
||
network. This may or may not be transparent to the end user.
|
||
|
||
See also: Open Proxy, Relay.
|
||
|
||
|
||
R
|
||
|
||
Ratware
|
||
Mass-mailing virii and e-mail software used by spammers, specifically
|
||
designed to deliver large amounts of mail in a very short time.
|
||
|
||
Most ratware implementations incorporate only as much SMTP client code
|
||
as strictly neccessary to deliver mail in the best-case scenario. They
|
||
provide false or inaccurate information in the SMTP dialogue with the
|
||
receiving host. They do not wait for responses from the receiver before
|
||
issuing commands, and disconnect if no response has been received in a
|
||
few seconds. They do not follow normal retry mechanisms in case of
|
||
temporary failures.
|
||
|
||
Relay
|
||
A machine that forwards e-mail, usually to or from the internet. One
|
||
example of a relay is the "smarthost" that an ISP provides to its
|
||
customers for sending outgoing mail.
|
||
|
||
See also: Open Relay, proxy
|
||
|
||
Request for Comments
|
||
(abbrev: RFC) From [http://www.rfc-editor.org/] http://
|
||
www.rfc-editor.org/: " The Request for Comments (RFC) document series is
|
||
a set of technical and organizational notes about the internet [...].
|
||
Memos in the RFC series discuss many aspects of computer networking,
|
||
incluing protocols, procedures, programs, and concepts, as well as
|
||
meeting notes, opinions, and sometimes humor. "
|
||
|
||
These documents make up the "rules" internet conduct, including
|
||
descriptions of protocols and data formats. Of particular interest for
|
||
mail deliveries are:
|
||
|
||
+ RFC 2821, "Simple Mail transfer Protocol", and
|
||
|
||
+ RFC 2822, "Internet Message Format".
|
||
|
||
|
||
|
||
S
|
||
|
||
Spam Trap
|
||
An e-mail address that is seeded to address-harvesting robots via
|
||
public locations, then used to feed collaborative tools such as DNS
|
||
Blacklists and Junk Mail Signature Repository.
|
||
|
||
Mails sent to these addresses are normally spam or malware. However,
|
||
some of it will be collateral, spam - i.e. Delivery Status Notification
|
||
to faked sender addresses. Thus, unless the spam trap has safeguards in
|
||
place to disregard such messages, the resulting tool may not be
|
||
completely reliable.
|
||
|
||
|
||
Z
|
||
|
||
Zombie Host
|
||
A machine with an internet connection that is infected by a
|
||
mass-mailing virus or worm. Such machines invariably run a flavor of the
|
||
Microsoft® Windows® operating system, and are almost always in
|
||
"residential" IP address blocks. Their owners either do not know or do
|
||
not care that the machines are infected, and often, their ISP will not
|
||
take any actions to shut them down.
|
||
|
||
Fortunately, there are various DNS blocklists, such as
|
||
"dul.dnsbl.sorbs.net", that incorporate such "residential" address
|
||
blocks. You should be able to use these blocklists to reject incoming
|
||
mail. Legitimate mail from residential users should normally go through
|
||
their ISP's "smarthost".
|
||
|
||
|
||
-----------------------------------------------------------------------------
|
||
Appendix B. GNU General Public License
|
||
|
||
B.1. Preamble
|
||
|
||
The licenses for most software are designed to take away your freedom to
|
||
share and change it. By contrast, the GNU General Public License is intended
|
||
to guarantee your freedom to share and change free software - to make sure
|
||
the software is free for all its users. This General Public License applies
|
||
to most of the Free Software Foundation's software and to any other program
|
||
whose authors commit to using it. (Some other Free Software Foundation
|
||
software is covered by the GNU Library General Public License instead.) You
|
||
can apply it to your programs, too.
|
||
|
||
When we speak of free software, we are referring to freedom, not price. Our
|
||
General Public Licenses are designed to make sure that you have the freedom
|
||
to distribute copies of free software (and charge for this service if you
|
||
wish), that you receive source code or can get it if you want it, that you
|
||
can change the software or use pieces of it in new free programs; and that
|
||
you know you can do these things.
|
||
|
||
To protect your rights, we need to make restrictions that forbid anyone to
|
||
deny you these rights or to ask you to surrender the rights. These
|
||
restrictions translate to certain responsibilities for you if you distribute
|
||
copies of the software, or if you modify it.
|
||
|
||
For example, if you distribute copies of such a program, whether gratis or
|
||
for a fee, you must give the recipients all the rights that you have. You
|
||
must make sure that they, too, receive or can get the source code. And you
|
||
must show them these terms so they know their rights.
|
||
|
||
We protect your rights with two steps:
|
||
|
||
1. copyright the software, and
|
||
|
||
2. offer you this license which gives you legal permission to copy,
|
||
distribute and/or modify the software.
|
||
|
||
|
||
Also, for each author's protection and ours, we want to make certain that
|
||
everyone understands that there is no warranty for this free software. If the
|
||
software is modified by someone else and passed on, we want its recipients to
|
||
know that what they have is not the original, so that any problems introduced
|
||
by others will not reflect on the original authors' reputations.
|
||
|
||
Finally, any free program is threatened constantly by software patents. We
|
||
wish to avoid the danger that redistributors of a free program will
|
||
individually obtain patent licenses, in effect making the program
|
||
proprietary. To prevent this, we have made it clear that any patent must be
|
||
licensed for everyone's free use or not licensed at all.
|
||
|
||
The precise terms and conditions for copying, distribution and modification
|
||
follow.
|
||
-----------------------------------------------------------------------------
|
||
|
||
B.2. TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
|
||
|
||
B.2.1. Section 0
|
||
|
||
This License applies to any program or other work which contains a notice
|
||
placed by the copyright holder saying it may be distributed under the terms
|
||
of this General Public License. The "Program", below, refers to any such
|
||
program or work, and a "work based on the Program " means either the Program
|
||
or any derivative work under copyright law: that is to say, a work containing
|
||
the Program or a portion of it, either verbatim or with modifications and/or
|
||
translated into another language. (Hereinafter, translation is included
|
||
without limitation in the term "modification ".) Each licensee is addressed
|
||
as "you".
|
||
|
||
Activities other than copying, distribution and modification are not covered
|
||
by this License; they are outside its scope. The act of running the Program
|
||
is not restricted, and the output from the Program is covered only if its
|
||
contents constitute a work based on the Program (independent of having been
|
||
made by running the Program). Whether that is true depends on what the
|
||
Program does.
|
||
-----------------------------------------------------------------------------
|
||
|
||
B.2.2. Section 1
|
||
|
||
You may copy and distribute verbatim copies of the Program's source code as
|
||
you receive it, in any medium, provided that you conspicuously and
|
||
appropriately publish on each copy an appropriate copyright notice and
|
||
disclaimer of warranty; keep intact all the notices that refer to this
|
||
License and to the absence of any warranty; and give any other recipients of
|
||
the Program a copy of this License along with the Program.
|
||
|
||
You may charge a fee for the physical act of transferring a copy, and you may
|
||
at your option offer warranty protection in exchange for a fee.
|
||
-----------------------------------------------------------------------------
|
||
|
||
B.2.3. Section 2
|
||
|
||
You may modify your copy or copies of the Program or any portion of it, thus
|
||
forming a work based on the Program, and copy and distribute such
|
||
modifications or work under the terms of Section 1 above, provided that you
|
||
also meet all of these conditions:
|
||
|
||
1. You must cause the modified files to carry prominent notices stating that
|
||
you changed the files and the date of any change.
|
||
|
||
2. You must cause any work that you distribute or publish, that in whole or
|
||
in part contains or is derived from the Program or any part thereof, to
|
||
be licensed as a whole at no charge to all third parties under the terms
|
||
of this License.
|
||
|
||
3. If the modified program normally reads commands interactively when run,
|
||
you must cause it, when started running for such interactive use in the
|
||
most ordinary way, to print or display an announcement including an
|
||
appropriate copyright notice and a notice that there is no warranty (or
|
||
else, saying that you provide a warranty) and that users may redistribute
|
||
the program under these conditions, and telling the user how to view a
|
||
copy of this License.
|
||
|
||
Note Exception:
|
||
If the Program itself is interactive but does not normally print
|
||
such an announcement, your work based on the Program is not required
|
||
to print an announcement.)
|
||
|
||
|
||
These requirements apply to the modified work as a whole. If identifiable
|
||
sections of that work are not derived from the Program, and can be reasonably
|
||
considered independent and separate works in themselves, then this License,
|
||
and its terms, do not apply to those sections when you distribute them as
|
||
separate works. But when you distribute the same sections as part of a whole
|
||
which is a work based on the Program, the distribution of the whole must be
|
||
on the terms of this License, whose permissions for other licensees extend to
|
||
the entire whole, and thus to each and every part regardless of who wrote it.
|
||
|
||
Thus, it is not the intent of this section to claim rights or contest your
|
||
rights to work written entirely by you; rather, the intent is to exercise the
|
||
right to control the distribution of derivative or collective works based on
|
||
the Program.
|
||
|
||
In addition, mere aggregation of another work not based on the Program with
|
||
the Program (or with a work based on the Program) on a volume of a storage or
|
||
distribution medium does not bring the other work under the scope of this
|
||
License.
|
||
-----------------------------------------------------------------------------
|
||
|
||
B.2.4. Section 3
|
||
|
||
You may copy and distribute the Program (or a work based on it, under Section
|
||
2 in object code or executable form under the terms of Sections 1 and 2 above
|
||
provided that you also do one of the following:
|
||
|
||
1. Accompany it with the complete corresponding machine-readable source
|
||
code, which must be distributed under the terms of Sections 1 and 2 above
|
||
on a medium customarily used for software interchange; or,
|
||
|
||
2. Accompany it with a written offer, valid for at least three years, to
|
||
give any third party, for a charge no more than your cost of physically
|
||
performing source distribution, a complete machine-readable copy of the
|
||
corresponding source code, to be distributed under the terms of Sections
|
||
1 and 2 above on a medium customarily used for software interchange; or,
|
||
|
||
3. Accompany it with the information you received as to the offer to
|
||
distribute corresponding source code. (This alternative is allowed only
|
||
for noncommercial distribution and only if you received the program in
|
||
object code or executable form with such an offer, in accord with
|
||
Subsection b above.)
|
||
|
||
|
||
The source code for a work means the preferred form of the work for making
|
||
modifications to it. For an executable work, complete source code means all
|
||
the source code for all modules it contains, plus any associated interface
|
||
definition files, plus the scripts used to control compilation and
|
||
installation of the executable. However, as a special exception, the source
|
||
code distributed need not include anything that is normally distributed (in
|
||
either source or binary form) with the major components (compiler, kernel,
|
||
and so on) of the operating system on which the executable runs, unless that
|
||
component itself accompanies the executable.
|
||
|
||
If distribution of executable or object code is made by offering access to
|
||
copy from a designated place, then offering equivalent access to copy the
|
||
source code from the same place counts as distribution of the source code,
|
||
even though third parties are not compelled to copy the source along with the
|
||
object code.
|
||
-----------------------------------------------------------------------------
|
||
|
||
B.2.5. Section 4
|
||
|
||
You may not copy, modify, sublicense, or distribute the Program except as
|
||
expressly provided under this License. Any attempt otherwise to copy, modify,
|
||
sublicense or distribute the Program is void, and will automatically
|
||
terminate your rights under this License. However, parties who have received
|
||
copies, or rights, from you under this License will not have their licenses
|
||
terminated so long as such parties remain in full compliance.
|
||
-----------------------------------------------------------------------------
|
||
|
||
B.2.6. Section 5
|
||
|
||
You are not required to accept this License, since you have not signed it.
|
||
However, nothing else grants you permission to modify or distribute the
|
||
Program or its derivative works. These actions are prohibited by law if you
|
||
do not accept this License. Therefore, by modifying or distributing the
|
||
Program (or any work based on the Program), you indicate your acceptance of
|
||
this License to do so, and all its terms and conditions for copying,
|
||
distributing or modifying the Program or works based on it.
|
||
-----------------------------------------------------------------------------
|
||
|
||
B.2.7. Section 6
|
||
|
||
Each time you redistribute the Program (or any work based on the Program),
|
||
the recipient automatically receives a license from the original licensor to
|
||
copy, distribute or modify the Program subject to these terms and conditions.
|
||
You may not impose any further restrictions on the recipients' exercise of
|
||
the rights granted herein. You are not responsible for enforcing compliance
|
||
by third parties to this License.
|
||
-----------------------------------------------------------------------------
|
||
|
||
B.2.8. Section 7
|
||
|
||
If, as a consequence of a court judgment or allegation of patent infringement
|
||
or for any other reason (not limited to patent issues), conditions are
|
||
imposed on you (whether by court order, agreement or otherwise) that
|
||
contradict the conditions of this License, they do not excuse you from the
|
||
conditions of this License. If you cannot distribute so as to satisfy
|
||
simultaneously your obligations under this License and any other pertinent
|
||
obligations, then as a consequence you may not distribute the Program at all.
|
||
For example, if a patent license would not permit royalty-free redistribution
|
||
of the Program by all those who receive copies directly or indirectly through
|
||
you, then the only way you could satisfy both it and this License would be to
|
||
refrain entirely from distribution of the Program.
|
||
|
||
If any portion of this section is held invalid or unenforceable under any
|
||
particular circumstance, the balance of the section is intended to apply and
|
||
the section as a whole is intended to apply in other circumstances.
|
||
|
||
It is not the purpose of this section to induce you to infringe any patents
|
||
or other property right claims or to contest validity of any such claims;
|
||
this section has the sole purpose of protecting the integrity of the free
|
||
software distribution system, which is implemented by public license
|
||
practices. Many people have made generous contributions to the wide range of
|
||
software distributed through that system in reliance on consistent
|
||
application of that system; it is up to the author/donor to decide if he or
|
||
she is willing to distribute software through any other system and a licensee
|
||
cannot impose that choice.
|
||
|
||
This section is intended to make thoroughly clear what is believed to be a
|
||
consequence of the rest of this License.
|
||
-----------------------------------------------------------------------------
|
||
|
||
B.2.9. Section 8
|
||
|
||
If the distribution and/or use of the Program is restricted in certain
|
||
countries either by patents or by copyrighted interfaces, the original
|
||
copyright holder who places the Program under this License may add an
|
||
explicit geographical distribution limitation excluding those countries, so
|
||
that distribution is permitted only in or among countries not thus excluded.
|
||
In such case, this License incorporates the limitation as if written in the
|
||
body of this License.
|
||
-----------------------------------------------------------------------------
|
||
|
||
B.2.10. Section 9
|
||
|
||
The Free Software Foundation may publish revised and/or new versions of the
|
||
General Public License from time to time. Such new versions will be similar
|
||
in spirit to the present version, but may differ in detail to address new
|
||
problems or concerns.
|
||
|
||
Each version is given a distinguishing version number. If the Program
|
||
specifies a version number of this License which applies to it and "any later
|
||
version", you have the option of following the terms and conditions either of
|
||
that version or of any later version published by the Free Software
|
||
Foundation. If the Program does not specify a version number of this License,
|
||
you may choose any version ever published by the Free Software Foundation.
|
||
-----------------------------------------------------------------------------
|
||
|
||
B.2.11. Section 10
|
||
|
||
If you wish to incorporate parts of the Program into other free programs
|
||
whose distribution conditions are different, write to the author to ask for
|
||
permission. For software which is copyrighted by the Free Software
|
||
Foundation, write to the Free Software Foundation; we sometimes make
|
||
exceptions for this. Our decision will be guided by the two goals of
|
||
preserving the free status of all derivatives of our free software and of
|
||
promoting the sharing and reuse of software generally.
|
||
-----------------------------------------------------------------------------
|
||
|
||
B.2.12. NO WARRANTY Section 11
|
||
|
||
BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY FOR THE
|
||
PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE
|
||
STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE
|
||
PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED,
|
||
INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
|
||
FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND
|
||
PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE,
|
||
YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
|
||
-----------------------------------------------------------------------------
|
||
|
||
B.2.13. Section 12
|
||
|
||
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL
|
||
ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR REDISTRIBUTE
|
||
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
|
||
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE
|
||
OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR
|
||
DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR
|
||
A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH
|
||
HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
|
||
|
||
END OF TERMS AND CONDITIONS
|
||
-----------------------------------------------------------------------------
|
||
|
||
B.3. How to Apply These Terms to Your New Programs
|
||
|
||
If you develop a new program, and you want it to be of the greatest
|
||
possible use to the public, the best way to achieve this is to make it free
|
||
software which everyone can redistribute and change under these terms.
|
||
|
||
To do so, attach the following notices to the program. It is safest to
|
||
attach them to the start of each source file to most effectively convey the
|
||
exclusion of warranty; and each file should have at least the "copyright"
|
||
line and a pointer to where the full notice is found.
|
||
|
||
<one line to give the program's name and a brief idea of what it does.>
|
||
Copyright (C) <year> <name of author>
|
||
|
||
This program is free software; you can redistribute it and/or modify it
|
||
under the terms of the GNU General Public License as published by the Free
|
||
Software Foundation; either version 2 of the License, or (at your option) any
|
||
later version.
|
||
|
||
This program is distributed in the hope that it will be useful, but WITHOUT
|
||
ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
|
||
FOR A PARTICULAR PURPOSE. See the GNU General Public License for more
|
||
details.
|
||
|
||
You should have received a copy of the GNU General Public License along
|
||
with this program; if not, write to the Free Software Foundation, Inc., 59
|
||
Temple Place, Suite 330, Boston, MA 02111-1307 USA
|
||
|
||
Also add information on how to contact you by electronic and paper mail.
|
||
|
||
If the program is interactive, make it output a short notice like this when
|
||
it starts in an interactive mode:
|
||
|
||
Gnomovision version 69, Copyright (C) year name of author Gnomovision comes
|
||
with ABSOLUTELY NO WARRANTY; for details type `show w'. This is free
|
||
software, and you are welcome to redistribute it under certain conditions;
|
||
type `show c' for details.
|
||
|
||
The hypothetical commands `show w' and `show c' should show the appropriate
|
||
parts of the General Public License. Of course, the commands you use may be
|
||
called something other than `show w' and `show c'; they could even be
|
||
mouse-clicks or menu items--whatever suits your program.
|
||
|
||
You should also get your employer (if you work as a programmer) or your
|
||
school, if any, to sign a "copyright disclaimer" for the program, if
|
||
necessary. Here is a sample; alter the names:
|
||
|
||
Yoyodyne, Inc., hereby disclaims all copyright interest in the program
|
||
`Gnomovision' (which makes passes at compilers) written by James Hacker.
|
||
|
||
<signature of Ty Coon>, 1 April 1989 Ty Coon, President of Vice
|
||
|
||
This General Public License does not permit incorporating your program into
|
||
proprietary programs. If your program is a subroutine library, you may
|
||
consider it more useful to permit linking proprietary applications with the
|
||
library. If this is what you want to do, use the GNU Library General Public
|
||
License instead of this License.
|
||
|
||
Notes
|
||
|
||
[1] Untrusted third party hosts may still generate collateral spam if you
|
||
reject the mail. However, unless that host is an Open Proxy or Open
|
||
Relay, it presumably delivers mail only from legitimate senders, whose
|
||
addresses are valid. If it is an Open Proxy or SMTP Relay - well, it is
|
||
better that you reject the mail and let it freeze in their outgoing mail
|
||
queue than letting it freeze in yours. Eventually, this ought to give
|
||
the owners of that server a clue.
|
||
[2] Personally I do not think challenge/response systems are a good idea in
|
||
any case. They generate Collateral Spam, they require special attention
|
||
for mail sent from automated sources such as monthly bank statements,
|
||
and they degrade the usability of e-mail as people need to jump through
|
||
hoops to get in touch with each other. Many times, senders of legitimate
|
||
mail will not bother to or know that they need to follow up to the
|
||
confirmation request, and the mail is lost.
|
||
[3] My view stands in sharp contrast to that of a large number of "spam
|
||
hacktivists", such as the maintainers of the [http://www.spews.org/]
|
||
SPEWS blacklist. One of the stated aims of this list is precisely to
|
||
inflict Collateral Damage as a means of putting pressure on ISPs to
|
||
react on abuse complaints. Listing complaints are typically met with
|
||
knee-jerk responses such as "bother your ISP, not us", or "get another
|
||
ISP".
|
||
|
||
Often, these are not viable options. Consider developing countries. For
|
||
that matter, consider the fact that nearly everywhere, broadband
|
||
providers are regulated monopolies. I believe that these attitudes
|
||
illustrate the exact crux of the problem with trusting these groups.
|
||
|
||
Put plainly, there are much better and more accurate ways available to
|
||
filter junk mail.
|
||
[4] Beware that while you are holding up an incoming SMTP delivery, you are
|
||
also holding up a TCP socket on your server, as well as memory and other
|
||
server resources. If your server is generally busy, imposing SMTP
|
||
transaction delays will make you more vulnerable to Denial-of-Service
|
||
attacks. A more "scalable" option may be to drop the connection once you
|
||
have conclusive evidence that the sender is a ratware client.
|
||
[5] Similar lists exist for different purposes. For instance,
|
||
"bondedsender.org" is a DNS whitelist (DNSwl), containing "trusted" IP
|
||
addresses, whose owners have posted a financial bond that will be
|
||
debited in the event that spam originates from that address. Other lists
|
||
contain IP addresses in use by specific countries, specific ISPs, etc.
|
||
[6] For instance, the outgoing mail exchangers ("smart hosts") of the
|
||
world's largest Internet Service Provider (ISP), comcast.net, is as of
|
||
the time of this writing included in the SPEWS Level 1 list. Not wholly
|
||
undeserved from the viewpoint that Comcast needs to more effectively
|
||
enforce their own AUP, but this listing does affect 30% of all US
|
||
internet users, mostly "innocent" subscribers such as myself.
|
||
|
||
To make matters worse, information published in the [http://spews.org/
|
||
faq.html] SPEWS FAQ states: The majority of the Level 1 list is made up
|
||
of netblocks owned by the spammers or spam support operations
|
||
themselves, with few or no other legitimate customers detected.
|
||
Technically, this information is accurate if (a) you consider Comcast a
|
||
"spam support operation", and (b) pay attention to the word "other".
|
||
Word parsing aside, this information is clearly misleading.
|
||
[7] Although this check is normally quite effective at weeding out junk,
|
||
there are reports of buggy L-Soft [http://www.lsoft.com/products/
|
||
default.asp?item=listserv] listserv installations that greet with the
|
||
plain IP address of the list server.
|
||
[8] A special case is the NULL envelope sender address (i.e. MAIL FROM: <>)
|
||
used in Delivery Status Notifications and other automatically generated
|
||
responses. This address should always be accepted.
|
||
[9] Although rare, some "legitimate" bulk mail senders, such as
|
||
groups.yahoo.com, will not retry temporarily failed deliveries. Evan
|
||
Harris has compiled a list of such senders, suitable for whitelisting
|
||
purposes: [http://cvs.puremagic.com/viewcvs/greylisting/schema/
|
||
whitelist_ip.txt?view=markup] http://cvs.puremagic.com/viewcvs/
|
||
greylisting/schema/whitelist_ip.txt?view=markup.
|
||
[10] Large sites often use multiple servers to handle outgoing mail. For
|
||
instance, one server or pool of servers may be used for immediate
|
||
delivery. If the first delivery attempt fails, the mail is handed off to
|
||
a fallback server which has been tuned for large queues. Hence, from
|
||
such sites, the first two delivery attempts will fail.
|
||
[11] Some specialized MTAs, such as certain mailing list servers, do not
|
||
automatically generate a Message-ID: header for "bounced" messages (
|
||
Delivery Status Notifications). These messages are identified by an
|
||
empty Envelope Sender.
|
||
[12] The IMAP protocol does not allow for NUL characters to be transmitted to
|
||
the mail user agent, so the Cyrus developers decided that the easiest
|
||
way to deal with mails containing it was to reject them.
|
||
[13] Why on earth the authors of anti-virus software are stupid enough to
|
||
trust the sender address in an e-mail containing a virus is perhaps a
|
||
topic for a closer psychoanalytic study.
|
||
[14] In particular, Exim is perhaps most popular among users of [http://
|
||
www.debian.org/] Debian GNU/Linux, as it is the default MTA in that
|
||
distribution. If you use Debian ("Sarge" or later), you can obtain
|
||
Exim+Exiscan-ACL by installing the exim4-daemon-heavy package:
|
||
# apt-get install exim4-daemon-heavy
|
||
[15] Debian users: The exim4-config package gives you a choice between
|
||
splitting the Exim configuration into several small chunks distributed
|
||
within subdirectories below /etc/exim4/conf.d, or to keep the entire
|
||
configuration in a single file.
|
||
|
||
If you chose the former option (I recommend this!), you can keep your
|
||
customization well separated from the stock configuration provided with
|
||
the exim4-config package by creating new files within these
|
||
subdirectories, rather than modifying the existing ones. For instance,
|
||
you may create a file named /etc/exim4/conf.d/acl/
|
||
80_local-config_rcpt_to to declare your own ACL for the RCPT TO: command
|
||
(see below).
|
||
|
||
The Exim "init" script (/etc/init.d/exim4) will automatically
|
||
consolidate all these files into a single large run-time configuration
|
||
file next time you (re)start.
|
||
[16] Debian users: As of July 14th, 2004, the version of Exiscan-ACL that is
|
||
included in the exim4-daemon-heavy package does not yet have support for
|
||
SPF. In the mean time, you may choose the other SPF implementation;
|
||
install libmail-spf-query-perl.
|
||
[17] Although it is true that Bayesian training is specific to each user, it
|
||
should be noted that SpamAssassin's Bayesian classifier is, IMHO, not
|
||
that stellar in any case. Especially I find this to be the case since
|
||
spammers have learned to defeat such systems by seeding random
|
||
dictionary words or stories in their mail (e.g. in the metadata of HTML
|
||
messages).
|
||
[18] If you think this is an overkill, would I tend to agree on the surface.
|
||
In previous versions of this document, I simply used ${hash_8:SECRET
|
||
=....} to generate the last component of the signature. However, with
|
||
this it would be technically possible, with a bit of insight into Exim's
|
||
${hash...} function and some samples of your outgoing mail sent to
|
||
different recipients, to forge the signature. Matthew Byng-Maddic <mbm
|
||
(at) colondot.net> notes: What you're writing is a document that you
|
||
expect many people to just copy. Given that, kerchoff's principle starts
|
||
applying, and all of your secrecy should be in the key. If the key can
|
||
be reversed out, as seems likely with a few return paths, then the
|
||
spammer kan once again start emitting valid return-paths from that
|
||
domain, and you're back to where you started. [...] Better, IMO, to have
|
||
it being strong from the start.
|
||
[19] In the examples above, the senders condition is actually redundant,
|
||
since the file /home//.return-path-sign is not likely to exist. However,
|
||
we make the condition explicit for clarity.
|