[SATLUG] spam-fighting techniques

[Bob Tracy]

Many thanks for the pointer to "dspam".  Yeah, it looks painful, but
few things in life that are worthwhile don't have at least *some* pain
associated with them.

My anti-spam measures are largely influenced by the thinking of a man
I am proud to acknowledge as a mentor -- Bill Kennedy (deceased).  His
philosophy was "once the DATA command has been issued in a SMTP
conversation with a spammer, the battle has been lost," which translates
to filtering based on the message envelope during the delivery attempt
rather than on message content.  Yes, there are weaknesses/limitations
inherent in this approach:

(1) DNS weaknesses / limitations become filtering weaknesses.
(2) spammers who play by the rules (rare, but it happens) will succeed
    at least temporarily.
(3) cannot block spam arriving via a mail list to which someone in your
    user base is subscribed.  Unfortunately, this requires content
    filtering ala "dspam".

How do the above limitations manifest themselves in practice?  Locally,
>99% of the spam that makes it past my filters does so due to (2) and (3)
above (the remaining <1% is due to (1)), and (3) is far and away the largest
hole.  Item (2) isn't much of a problem due to the empirically demonstrable
fact that spammers are both cheap and lazy, i.e., it takes both money and
effort to register a domain that's going to end up on a blacklist within
minutes of it being used to deliver spam -- the potential upside doesn't
begin to outweigh the downside, and while I will happily categorize spammers
as cheap and lazy, "stupid" doesn't apply to the ones I'd love to introduce
to the business end of a baseball bat at high speed.  Want more evidence of
cheap and lazy?  More than half of the attempted spam deliveries I see each
day come from IP addresses that aren't resolvable: Postfix's "reject_unknown_
client" restriction wins big, and I'm sure other MTAs have something similar.
I don't intend to give the impression I'm glossing over (1), but its effects
are negligible, and under no circumstances do my filters turn a DNS problem
into a mail problem: the worst case scenario means the sender is effectively
greylisted for the duration of the DNS problem.  Unfortunately, I've seen
first-hand the evidence that the more clueful spammers are looking for and
actively exploiting misconfigured DNS records, particularly improperly
constructed SPF records -- that's the <1% leakage mentioned earlier.  Carefully
constructed blacklists cure this problem, and I have exactly two list entries
after nearly a year of having the envelope filters on for *all* incoming mail.

As far as false positives, they *do* happen, and usually for the following reasons:

(1) DNS misconfiguration: sometimes willful, sometimes not.  This issue
    looms large for domain owners who can't maintain their corresponding
    DNS records except via a third party.
(2) Sender domains that, in the absence of valid SPF and/or MX records,
    cannot reasonably be correlated with the domain of the SMTP client
    attempting to deliver the message, e.g., mail from user at surname.com
    being handled by a RoadRunner mail host.  I suppose this could be
    considered a sub-case of (1), but it's an error of omission rather
    than commission.

Both of the above cases have happened, but are infrequent: simple white-
listing handles them, and the lists are shorter than you might imagine
for a site that handles over 5,000 messages per day for two legitimate
recipients :-).

Someone will say that envelope filtering doesn't scale to the enterprise
level, and my response is simply "you're wrong."  Bill authored and sold
a commercial anti-spam product called "Ironhand" that was based on
envelope filtering.  Satisfied customers included large computer
manufacturers and oil companies with tens of thousands of users.

-- 
-----------------------------------------------------------------------
Bob Tracy                   WTO + WIPO = DMCA? http://www.anti-dmca.org
rct at frus.com
-----------------------------------------------------------------------