OT: obscene message sizes and idiocy - a solution!

[Bill Campbell]

On Wed, Mar 08, 2006, D. Thomas Podnar wrote:
>On Wed, Mar 08, 2006 at 12:40:24PM -0500, Mark said a bunch of stuff
>which he summarized as:
>
>> I'm sure their solution is everything you say it is.  It's just not
>> for us, same as mine isn't for everyone.  I don't think there's one uniform
>> solution--it's contextual.
>
>We agree completely. We've both just shared our positive experiences
>with the list in the hope that some will find them useful, and yet
>others will perhaps tell us about their solutions.

I have developed, and we are installing systems that cut the amount of
incoming spam down to fairly reasonable levels (I see maybe 20 messages a
day in my spam folder, down from hundreds every 8 hours or so).  This is
being marketed under the name Atrica, by one of our largest regional ISP
customers, Blue Mountain Internet, and is based on the systems we've been
developing over almost 20 years of dealing with e-mail.

	http://www.atricaserver.com

These systems are based on SuSE Linux Enterprise 9, with most of the server
components replaced with the OpenPKG.org versions, plus a fair amount of
code I've written to handle incoming mail and other network chores.  The
heart of the e-mail system is postfix, amavisd-new, clamav, spamassassin,
courier-imap, and a system I've built to allow individual users to
automatically sort incoming mail into Maildir folders, a simple rule set
based on incoming mail headers.

Spam filtering is selectable on a each mail folder (e.g. I can turn it off
totally for my postmaster and security mail folders),  It allows one to put
spam into different folders based on the spamassassin scoring, and send
messages above a user-specified score to /dev/null.

For people using a POP interface instead of IMAP, it sends periodic
messages to each user with a summary of messages in their current spam mail
folder with a link to the webmail interface so they can review them for
false positives.  The user can select the schedule for these messages,
daily or on specified day(s) of the week.

False positives found in the spam folder(s) can be simply dropped into the
spam.falsepositive or spam.whitelist folders, and within fifteen minutes a
cron job will update their bayesian filters or spamassassin user_prefs
files respectively.

Likewise, spam that gets by the filters may be dropped into the folder,
spam.missed to update the bayesian filters.

We have these systems installed now in sites with fewer than 10 users, and
at BMI in a distributed cluster of servers serving well over 7,000 mail
accounts where it's replaced their postini pre-filtering.  Their users seem
to be much happier with this solution than the previous postini filtering,
saying they're now receiving legitimate mail that postini was dropping, and
they have far more control of their individual mail delivery.

In the BMI system we have one publically facing server that handles all
incoming mail using a combination of postfix and DNSRBL filtering then
running everything through the clamav anti-``virus'' software before
forwarding to four internal servers that do spamassassin checking and
delivery into the user's mail stores.  The external server rejects about
750,000 connections and accepts about 250,000 daily based on the initial
DNSRBLs and postfix checking.  FWIW, the postfix.sum daily mail log report
on this machine was 75,687,062 bytes for yesterday -- a bit too large to
peruse in its entirety :-).

The border machine has a load average of about 0.7 running on a fairly
standard Intel box with 2GB of RAM.  The internal servers doing the
spamassassin processing have 1GB RAM, and run a load average between 2.00
and 4.00 most of the time.

There's more information on the clustering servers here:

	http://atricacluster.com

Bill
--
INTERNET:   bill at Celestial.COM  Bill Campbell; Celestial Systems, Inc.
URL: http://www.celestial.com/  PO Box 820; 6641 E. Mercer Way
FAX:            (206) 232-9186  Mercer Island, WA 98040-0820; (206) 236-1676

``Microsoft IIS has more holes than a wheel of Swiss Cheese after a shotgun
blast'' -- John Dvorak