403 lines
7.8 KiB
HTML
403 lines
7.8 KiB
HTML
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
|
|
<HTML
|
|
><HEAD
|
|
><TITLE
|
|
>The Good, The Bad, The Ugly</TITLE
|
|
><META
|
|
NAME="GENERATOR"
|
|
CONTENT="Modular DocBook HTML Stylesheet Version 1.7"><LINK
|
|
REL="HOME"
|
|
TITLE="Spam Filtering for Mail Exchangers"
|
|
HREF="index.html"><LINK
|
|
REL="UP"
|
|
TITLE="Background"
|
|
HREF="background.html"><LINK
|
|
REL="PREVIOUS"
|
|
TITLE="Why Filter Mail During the SMTP Transaction?"
|
|
HREF="whysmtptime.html"><LINK
|
|
REL="NEXT"
|
|
TITLE="The SMTP Transaction"
|
|
HREF="smtpintro.html"></HEAD
|
|
><BODY
|
|
CLASS="section"
|
|
BGCOLOR="#FFFFFF"
|
|
TEXT="#000000"
|
|
LINK="#0000FF"
|
|
VLINK="#840084"
|
|
ALINK="#0000FF"
|
|
><DIV
|
|
CLASS="NAVHEADER"
|
|
><TABLE
|
|
SUMMARY="Header navigation table"
|
|
WIDTH="100%"
|
|
BORDER="0"
|
|
CELLPADDING="0"
|
|
CELLSPACING="0"
|
|
><TR
|
|
><TH
|
|
COLSPAN="3"
|
|
ALIGN="center"
|
|
>Spam Filtering for Mail Exchangers: </TH
|
|
></TR
|
|
><TR
|
|
><TD
|
|
WIDTH="10%"
|
|
ALIGN="left"
|
|
VALIGN="bottom"
|
|
><A
|
|
HREF="whysmtptime.html"
|
|
ACCESSKEY="P"
|
|
>Prev</A
|
|
></TD
|
|
><TD
|
|
WIDTH="80%"
|
|
ALIGN="center"
|
|
VALIGN="bottom"
|
|
>Chapter 1. Background</TD
|
|
><TD
|
|
WIDTH="10%"
|
|
ALIGN="right"
|
|
VALIGN="bottom"
|
|
><A
|
|
HREF="smtpintro.html"
|
|
ACCESSKEY="N"
|
|
>Next</A
|
|
></TD
|
|
></TR
|
|
></TABLE
|
|
><HR
|
|
ALIGN="LEFT"
|
|
WIDTH="100%"></DIV
|
|
><DIV
|
|
CLASS="section"
|
|
><H1
|
|
CLASS="section"
|
|
><A
|
|
NAME="goodbadugly"
|
|
></A
|
|
>1.2. The Good, The Bad, The Ugly</H1
|
|
><P
|
|
> Some filtering techniques are more suitable for use during the
|
|
SMTP transaction than others. Some are simply better than
|
|
others. Nearly all have their proponents and opponents.
|
|
</P
|
|
><P
|
|
> Needless to say, these controversies extend to the methods
|
|
described here as well. For instance:
|
|
</P
|
|
><P
|
|
></P
|
|
><UL
|
|
><LI
|
|
><P
|
|
> Some argue that <A
|
|
HREF="dnschecks.html"
|
|
>DNS checks</A
|
|
> penalize
|
|
individual mail senders purely based on their Internet
|
|
Service Provider (ISP), not on the merits of their
|
|
particular message.
|
|
</P
|
|
></LI
|
|
><LI
|
|
><P
|
|
> Some point out that ratware traps like <A
|
|
HREF="smtpdelays.html"
|
|
>SMTP transaction delays</A
|
|
> and <A
|
|
HREF="greylisting.html"
|
|
>Greylisting</A
|
|
> are
|
|
easily overcome and will be less effective over time, while
|
|
continuing to degrade the Quality of Service for legitimate
|
|
mail.
|
|
</P
|
|
></LI
|
|
><LI
|
|
><P
|
|
> Some find that <A
|
|
HREF="senderauth.html"
|
|
>Sender Authorization Schemes</A
|
|
> like the <A
|
|
HREF="senderauth.html#spf"
|
|
>Sender Policy Framework</A
|
|
> give ISPs a way to lock their customers in,
|
|
and do not adequately address users who roam between
|
|
different networks or who forward their e-mail from one host
|
|
to another.
|
|
</P
|
|
></LI
|
|
></UL
|
|
><P
|
|
> I will steer away from most of these controversies. Instead, I
|
|
will try to provide a functional description of the various
|
|
techniques available, including their possible side effects, and
|
|
then talk a little about my own experiences using some of them.
|
|
</P
|
|
><P
|
|
> That said, there are some filtering methods in use today that I
|
|
deliberately omit from this document:
|
|
</P
|
|
><P
|
|
></P
|
|
><UL
|
|
><LI
|
|
><P
|
|
> Challenge/response systems (like <A
|
|
HREF="http://tmda.net/"
|
|
TARGET="_top"
|
|
>TMDA</A
|
|
>). These are not
|
|
suitable for SMTP time filtering, as they rely on first
|
|
accepting the mail, then returning a confirmation request to
|
|
the <A
|
|
HREF="gloss.html#envfrom"
|
|
><I
|
|
CLASS="glossterm"
|
|
>Envelope Sender</I
|
|
></A
|
|
>. This technique is therefore
|
|
outside the scope of this document.
|
|
<A
|
|
NAME="AEN452"
|
|
HREF="#FTN.AEN452"
|
|
><SPAN
|
|
CLASS="footnote"
|
|
>[1]</SPAN
|
|
></A
|
|
>
|
|
</P
|
|
></LI
|
|
><LI
|
|
><P
|
|
> <A
|
|
HREF="gloss.html#bayesian"
|
|
><I
|
|
CLASS="glossterm"
|
|
>Bayesian Filters</I
|
|
></A
|
|
>. These require training specific
|
|
to a particular user, and/or a particular language. As
|
|
such, these too are not normally suitable for use during the
|
|
SMTP transaction (But see <A
|
|
HREF="usersettings.html"
|
|
>User Settings and Data</A
|
|
>).
|
|
</P
|
|
></LI
|
|
><LI
|
|
><P
|
|
> <A
|
|
HREF="gloss.html#micropay"
|
|
><I
|
|
CLASS="glossterm"
|
|
>Micropayment Schemes</I
|
|
></A
|
|
> are not really suitable for
|
|
weeding out junk mail until all the world's legitimate mail
|
|
is sent with a virtual <EM
|
|
>postage stamp</EM
|
|
>.
|
|
(Though in the mean time, they can be used for the opposite
|
|
purpose - that is, to accept mail carrying the stamp that
|
|
would otherwise be rejected).
|
|
</P
|
|
></LI
|
|
></UL
|
|
><P
|
|
> Generally, I have attempted to offer techniques that are as
|
|
precise as possible, and to go to great lengths to avoid <A
|
|
HREF="gloss.html#falsepos"
|
|
><I
|
|
CLASS="glossterm"
|
|
>False Positive</I
|
|
></A
|
|
>s. People's e-mail is important to them,
|
|
and they spend time and effort writing it. In my view,
|
|
willfully using techniques or tools that reject large amounts of
|
|
legitimate mail is a show of disrespect, both to the people that
|
|
are directly affected and to the Internet as a whole.
|
|
<A
|
|
NAME="AEN465"
|
|
HREF="#FTN.AEN465"
|
|
><SPAN
|
|
CLASS="footnote"
|
|
>[2]</SPAN
|
|
></A
|
|
>
|
|
|
|
This is especially true for SMTP-time system wide filtering,
|
|
because end recipients usually have little or no control over
|
|
the criteria being used to filter their mail.
|
|
</P
|
|
></DIV
|
|
><H3
|
|
CLASS="FOOTNOTES"
|
|
>Notes</H3
|
|
><TABLE
|
|
BORDER="0"
|
|
CLASS="FOOTNOTES"
|
|
WIDTH="100%"
|
|
><TR
|
|
><TD
|
|
ALIGN="LEFT"
|
|
VALIGN="TOP"
|
|
WIDTH="5%"
|
|
><A
|
|
NAME="FTN.AEN452"
|
|
HREF="goodbadugly.html#AEN452"
|
|
><SPAN
|
|
CLASS="footnote"
|
|
>[1]</SPAN
|
|
></A
|
|
></TD
|
|
><TD
|
|
ALIGN="LEFT"
|
|
VALIGN="TOP"
|
|
WIDTH="95%"
|
|
><P
|
|
> Personally I do not think challenge/response systems are
|
|
a good idea in any case. They generate <A
|
|
HREF="gloss.html#colspam"
|
|
><I
|
|
CLASS="glossterm"
|
|
>Collateral Spam</I
|
|
></A
|
|
>, they require special attention for
|
|
mail sent from automated sources such as monthly bank
|
|
statements, and they degrade the usability of e-mail as
|
|
people need to jump through hoops to get in touch with
|
|
each other. Many times, senders of legitimate mail will
|
|
not bother to or know that they need to follow up to the
|
|
confirmation request, and the mail is lost.
|
|
</P
|
|
></TD
|
|
></TR
|
|
><TR
|
|
><TD
|
|
ALIGN="LEFT"
|
|
VALIGN="TOP"
|
|
WIDTH="5%"
|
|
><A
|
|
NAME="FTN.AEN465"
|
|
HREF="goodbadugly.html#AEN465"
|
|
><SPAN
|
|
CLASS="footnote"
|
|
>[2]</SPAN
|
|
></A
|
|
></TD
|
|
><TD
|
|
ALIGN="LEFT"
|
|
VALIGN="TOP"
|
|
WIDTH="95%"
|
|
><P
|
|
> My view stands in sharp contrast to that of a large number
|
|
of <SPAN
|
|
CLASS="QUOTE"
|
|
>"spam hacktivists"</SPAN
|
|
>, such as the maintainers
|
|
of the <A
|
|
HREF="http://www.spews.org/"
|
|
TARGET="_top"
|
|
>SPEWS</A
|
|
>
|
|
<A
|
|
HREF="dnschecks.html#dnsbl"
|
|
>blacklist</A
|
|
>. One of the stated
|
|
aims of this list is precisely to inflict <A
|
|
HREF="gloss.html#coldamage"
|
|
><I
|
|
CLASS="glossterm"
|
|
>Collateral Damage</I
|
|
></A
|
|
> as a means of putting pressure on ISPs
|
|
to react on abuse complaints. Listing complaints are
|
|
typically met with knee-jerk responses such as <SPAN
|
|
CLASS="QUOTE"
|
|
>"bother
|
|
your ISP, not us"</SPAN
|
|
>, or <SPAN
|
|
CLASS="QUOTE"
|
|
>"get another ISP"</SPAN
|
|
>.
|
|
</P
|
|
><P
|
|
> Often, these are not viable options. Consider developing
|
|
countries. For that matter, consider the fact that nearly
|
|
everywhere, broadband providers are regulated monopolies. I
|
|
believe that these attitudes illustrate the exact crux of
|
|
the problem with trusting these groups.
|
|
</P
|
|
><P
|
|
> Put plainly, there are much better and more accurate ways
|
|
available to filter junk mail.
|
|
</P
|
|
></TD
|
|
></TR
|
|
></TABLE
|
|
><DIV
|
|
CLASS="NAVFOOTER"
|
|
><HR
|
|
ALIGN="LEFT"
|
|
WIDTH="100%"><TABLE
|
|
SUMMARY="Footer navigation table"
|
|
WIDTH="100%"
|
|
BORDER="0"
|
|
CELLPADDING="0"
|
|
CELLSPACING="0"
|
|
><TR
|
|
><TD
|
|
WIDTH="33%"
|
|
ALIGN="left"
|
|
VALIGN="top"
|
|
><A
|
|
HREF="whysmtptime.html"
|
|
ACCESSKEY="P"
|
|
>Prev</A
|
|
></TD
|
|
><TD
|
|
WIDTH="34%"
|
|
ALIGN="center"
|
|
VALIGN="top"
|
|
><A
|
|
HREF="index.html"
|
|
ACCESSKEY="H"
|
|
>Home</A
|
|
></TD
|
|
><TD
|
|
WIDTH="33%"
|
|
ALIGN="right"
|
|
VALIGN="top"
|
|
><A
|
|
HREF="smtpintro.html"
|
|
ACCESSKEY="N"
|
|
>Next</A
|
|
></TD
|
|
></TR
|
|
><TR
|
|
><TD
|
|
WIDTH="33%"
|
|
ALIGN="left"
|
|
VALIGN="top"
|
|
>Why Filter Mail During the SMTP Transaction?</TD
|
|
><TD
|
|
WIDTH="34%"
|
|
ALIGN="center"
|
|
VALIGN="top"
|
|
><A
|
|
HREF="background.html"
|
|
ACCESSKEY="U"
|
|
>Up</A
|
|
></TD
|
|
><TD
|
|
WIDTH="33%"
|
|
ALIGN="right"
|
|
VALIGN="top"
|
|
>The SMTP Transaction</TD
|
|
></TR
|
|
></TABLE
|
|
></DIV
|
|
></BODY
|
|
></HTML
|
|
> |