[SATLUG] Fwd: Undelivered Mail Returned to Sender

[Brad Knowles]

At 11:42 AM -0600 2/10/07, Bob Tracy wrote:

>  I checked the link you provided:
>  <http://bradknowles.typepad.com/considered_harmful/2004/05/spf.html>.
>  I might have missed something, but saw nothing I hadn't seen before on
>  the subject.

For 90% of the servers out there, I can poison their DNS in less than 
a second, and stuff in whatever data I want.  All it takes is 
connecting to their port 25.  I don't even have to send a message.  I 
just want them to do a reverse DNS lookup on my IP address, and I own 
them.

In most cases, I can subvert the DNS of the individuals sending the 
message, so that their own SPF records that come directly from their 
own nameservers would be intentionally mis-directed to some other 
machines, thus causing all receiving servers which properly implement 
SPF to throw away all mail from these machines.


Just in case your servers are secure and I can't poison them, I can 
still poison 90% of the other servers out there, and make them 
*think* that your SPF records point somewhere else, and throw away 
all your mail.


SPF records are difficult to craft correctly.  Many people craft them 
incorrectly, so even if your server is configured to implement them 
in the appropriate fashion, you would still throw away their 
legitimate mail.


SPF is hard to implement correctly.  Even if all the nameservers in 
the world were hardened and impossible to poison, and even if it was 
trivially easy for everyone in the world to correctly craft SPF 
records, there would still be a lot of broken machines out there 
which throw away legitimate mail, simply because they do not behave 
correctly.

This is the case for your server.


That's four pretty damn bloody big huge strikes.  Isn't that enough?

>  I've never bought the line of reasoning that says thus and such shouldn't
>  be used because the underlying technology is imperfect.  In this
>  particular case, the old argument against using SPF because of the
>  insecure (unreliable?) DNS underpinnings doesn't wash.

You're building inherently insecure systems on top of inherently 
insecure technologies, and then you're taking mission-critical binary 
decisions based on incomplete information that is highly likely to be 
flawed.

Don't you want to rethink that idea?

>                                                            People haven't
>  quit using DNS, and the security issues are increasingly being addressed.

I know about DNSSEC.  It's always been just around the corner, and 
people are discovering more and more operational problems with it, as 
they try to use the mechanisms that have been developed.

I've been writing a book on basic DNS operational security, filling 
in all the stuff that Cricket Liu didn't cover in his book.  I've 
known Cricket for years, and I was a technical reviewer of his book. 
The situation is *not* getting better.  The surveys from kc claffy 
and her co-workers at CAIDA have proven that -- and yes, she does 
spell her name in all-lowercase.

The folks at Nominum and Men & Mice also have some nice surveys that 
show how bad the problem is, but then if you're going to ignore this 
problem, I guess you should feel free to ignore everyone.

>                      The spammers have ruined a lot of things :-(.  On
>  the plus side, I reiterate that spammers are inherently lazy.

But not all bad people are spammers.  There are plenty of people out 
there who are not spammers, and who just want to make your life 
miserable.  In MMOGs, they are called "griefers".

And one of the things that we learned early at AOL is that a small 
number of determined attackers can take out any site anywhere on the 
entire Internet.  No site is immune, no matter how large.

>                                                                Poisoning
>  DNS caches is certainly possible, but requires effort, and is a bit like
>  peeing in the swimming pool: the perp doesn't get away clean -- everyone
>  suffers.

It doesn't matter.  He gets to ruin all your e-mail, for as long as he wants.

>  There is no MAY, MUST, or SHOULD clause in the above that indicates any
>  particular action an RFC4408-compliant SMTP server may, must, or should
>  take when a particular sender domain has no associated SPF records.  It
>  simply states the obvious, i.e., that the checking software cannot
>  ascertain whether or not the client host is authorized ON THE BASIS OF
>  SPF (capitalized words added for emphasis, because SPF isn't the only
>  envelope test method available).  That makes it a local policy decision
>  for the owner / operator of the SMTP server.  My local policy decision
>  is to employ other tests on the envelope.

If you want to break all e-mail coming in from domains that don't 
have SPF records, I guess that's your perogative.

Of course, you might want to consider that more than 99% of all 
e-mail sent from SPF-enabled domains is actually spam, but I guess 
you can ignore that, too.

>  I don't insist on SPF-compliance, however you define it, because not
>  having an SPF record is both legitimate and in compliance with the
>  experimental RFC.  I simply elect to apply other criteria in the
>  absence of SPF records to attempt to determine whether the delivery host
>  is somehow affiliated with the sender's domain.

So, you break all e-mail from all sites where virtual domains are 
hosted on a webfarm?  That's like 90% of all legitimate virtual 
domains in existence.  At AOL (one of my former employers), Belgacom 
Skynet (the largest ISP in Belgium, and another former employer), and 
all the other ISPs I've seen, they host literally thousands of 
domains on a single machine, and everything for that domain is 
handled by that one box, which uses just one name for itself and 
that's unrelated to any of the virtual domains hosted there.


I know the guy that wrote the postfix patches you're using.  Ron F. 
Guilmette is a guy I've been running into for a long time -- some of 
his ideas are good, and some of them are really bad.  He created your 
patches in August of 2001, and got flamed pretty heavily for them -- 
see the thread at 
<http://archives.neohapsis.com/archives/postfix/2001-08/thread.html#434>.

There's a reason why his postfix patches page at 
<http://www.monkeys.com/anti-spam/filtering/additions.html> has had 
all this stuff removed -- even Ron finally figured out that this was 
a bad idea, although he doesn't quite say as much on that page.

>  Please reread my explanation of what happened to you: SPF didn't spear
>  you -- my additional envelope checks employed in the absence of SPF did.
>  The best I can do for you is put your posting sender address (omitted
>  here to avoid giving even minimal help to would-be spammers) in a
>  whitelist, which I have done.

My addresses have been available to spammers for decades.  I think 
they've got them by now.

Moreover, thanks to my involvement with the Mailman project, I know 
that the spammers have figured out how to parse the "obscured" 
address format used in most archives, so there's not much of anything 
you're going to be able to do to help keep my addresses out of the 
hands of spammers.


My ISP is already doing a first level of outsourced spam filtering 
via Postini, which catches hundreds to thousands of spams per day for 
my addresses alone.  They add an additional layer of filtering that 
they perform on their own servers (which tags most everything else), 
and then my MUA does a third level of spam filtering internally.

The biggest problem I have is that sometimes overly aggressive spam 
filters will tag a legitimate message as spam, and I won't see it. 
Traditionally, the false positive problem is the single biggest one 
facing all so-called solutions to the spam problem.

>                                 I welcome the opportunity to review your
>  spamfighting series and provide feedback.  Methinks our common ground is
>  far more important than our current disagreement.

Paul Vixie has said for many years now that the biggest problem in 
this field is not the spammers, but the anti-spammers -- by which he 
means the ones that fundamentally misunderstand the problem and apply 
inappropriate solutions, and then defend their decisions to the death.

-- 
Brad Knowles <brad at shub-internet.org>, Consultant & Author
Co-author of SAGE Booklet #15 "Internet Postmaster: Duties and 
Responsibilities"
Founding Member and Platinum Individual Sponsor of LOPSA: 
<http://www.lopsa.org>
Papers: <http://tinyurl.com/tj6q4> LinkedIn Profile: 
<http://tinyurl.com/y8kpxu>