[IMC-Tech] Re: [Imc-web] Email and imsahp Tue Aug 22 10:42 CDT reboot

[Zach M]

Friends of mine that run large email servers have spoken very highly
of the effectiveness of implementing rules that stop spam before the
DATA command is even sent by looking at envelope information and
rejecting malformed headers, blacklisting dynamic IP blocks and known
spam sources, etc. There's LOTS of stuff we can do to reduce spam but
anything we do will require admins to pay close attention to what is
being caught so they can adjust rules if there is a lot of ham being
caught. This is time I don't have so I've opted for a configuration
that lets more spam through at the cost of ham very rarely getting
caught.

On 8/23/06, dan blah <dan.blah at gmail.com> wrote:
> ya i am not going to lie, i see more spam on this server than of any i
> have ever worked on.  in its defense, this server is one of the most
> exposed/used server of any i have worked on.  i agree on all points.
> i will work on fixing apache to allow more resources to be used for
> mail.  until then i will wait to add any large sa rule sets that will
> only bog it down any more.
>
> On 8/23/06, Zach M <leftistmuppet at gmail.com> wrote:
> > It should be further noted that some of our spam problems are
> > unavoidable. Even gmail's fairly advanced spam filter has been letting
> > through more and more spam every day (even as it captures more and
> > more as well....just like our spam filter...it stop thousands of spams
> > daily...that we see hundreds of spams getting through may be bad but
> > it could be MUCH worse). The spammers are getting smart faster than
> > the anti-spam software is getting effective.
> >
> > One thing we need to look into is some software that will allow users
> > to have access to their stuff that gets identified as spam so that ham
> > can be found without sysadmin intervention.
> >
> > On 8/23/06, Zach M <leftistmuppet at gmail.com> wrote:
> > > I moved away from spamd/spamc....amavis is a much more advanced
> > > framework for using spamassassin. It uses the perl libraries directly
> > > so it's like spamd on crack because it also does the anti-virus stuff
> > > which is very important. It also integrates much more nicely with
> > > postfix.
> > >
> > > The configuration is a very good one. The problem we had today was
> > > because of a known and patched security problem. There really isn't a
> > > problem with our mail config except that we need to put better SA
> > > rules in place and also add in some smarter envelope-level filtering
> > > options in postfix. I highly recommend getting the O'Reilley postfix
> > > book!
> > >
> > > One limitting factor is that because of all the CPU and memory that
> > > apache eats up we are limitted in how many concurrent amavis processes
> > > we can run. Mail would run much more quickly if could run twice as
> > > many amavis processes concurrently. Once again everything on this
> > > system is pretty well tuned EXCEPT that apache sucks all the system
> > > resources. Fix apache and lots of other problems fix themselves.
> > >
> > > On 8/23/06, dan blah <dan.blah at gmail.com> wrote:
> > > > awesome... thanks for the info.  what do you think about using the sa
> > > > blacklists and moving to spamd/spamc?  also, i was thinking about
> > > > using temfs for the spool directory used to create the tmp files for
> > > > scanning.  dunno if all of this will help or hurt performance.  as it
> > > > seems to me fixing/moving apache will solve lots.
> > > >
> > > > On 8/23/06, Zach M <leftistmuppet at gmail.com> wrote:
> > > > > I just upgraded clamav which had a recently discovered buffer overrun
> > > > > that allowed a denial of service attack. The mail queue is now empty.
> > > > > I suspect this may have been the problem.
> > > > >
> > > > > There is no customer list. Someone has to generate one. Look at all
> > > > > the mailman lists and all the virtual hosts in apache. Figure out who
> > > > > runs each organization. Ask me about ones you can't figure out.
> > > > >
> > > > > Mail works like this:
> > > > >
> > > > > Postfix receives a message via smtpd.
> > > > >
> > > > > The message is first passed to postgrey which does greylisting (if no
> > > > > messages have been previously received from this
> > > > > <sender,recipient,sender-ip> triple before  then delivery acceptance
> > > > > is deferred for 300 seconds. Correctly configured mail daemons resend
> > > > > every 5-15 minutes. Some spammers tend not to resend. Postgray is a
> > > > > service that gets a message a message on a localhost port and simply
> > > > > returns an answer of whether the message should be accepted or
> > > > > deferred.
> > > > >
> > > > > When the message gets past postgrey it is sent to amavisd. Which is a
> > > > > spam and virus filtering daemon that coordinates spamassassin and
> > > > > clamav. Amavis receives an SMTP style message on a localhost port and
> > > > > then re-delivers an SMTP style message back to a special localhost
> > > > > port on postfox.
> > > > >
> > > > > Amavis sends the message to clamav for a virus check.
> > > > >
> > > > > If the message passes clamav, Amavis sends the message through
> > > > > spamassassin for a spam check.
> > > > >
> > > > > If the message passed spamassassin, amavis sends the message back to
> > > > > postfix for final delivery.
> > > > >
> > > > > Things you can tweak: There are a number of envelope checks that
> > > > > postfix can do that you could enable. You could prevent postfix from
> > > > > ever even accepting certain messages from certain kinds of spam
> > > > > sources (using black hole lists, using checks for non-RFC compliant
> > > > > SMTP behavior, etc). Every one of these checks introduces the
> > > > > possibility that ham will be lost forever without sender or recipient
> > > > > knowing that it was mis-identified. These policies must be very
> > > > > carefully researched and considered.
> > > > >
> > > > > Spam assassin rules can be adjusted/updated.
> > > > >
> > > > > Spamassassin rules live in /etc/spamassassin. In particulate check out
> > > > > /etc/spamassassin/local.cf
> > > > >
> > > > > Postfix configs live in /etc/postfix. In particularly check out
> > > > > main.cf and master.cf.
> > > > >
> > > > > Amavis configs live in /etc/amavis/amavisd.conf
> > > > >
> > > > > Caught spam and virus mails live in /var/lib/amavis/virusmails (which
> > > > > I occasionally rotate out since it has tens of thousands of messages
> > > > > in it).
> > > > >
> > > > > I have several utilities for redelivering caught spam at:
> > > > > /home/wolfgang/bin/process_spam.pl and /etc/wolfgang/bin/redeliver_non_spam.pl
> > > > >
> > > > > If you are going to use the spam redelivery programs PLEASE read them
> > > > > carefully and understand what they do first. They are quick hacks that
> > > > > I put together myself and I need you to understand them before running
> > > > > them.
> > > > >
> > > >
> > > >
> > > > --
> > > > Daniel
> > > > _______________________________________________
> > > > IMC-Tech mailing list
> > > > IMC-Tech at lists.ucimc.org
> > > > http://lists.chambana.net/cgi-bin/listinfo/imc-tech
> > > >
> > > >
> > >
> >
>
>
> --
> Daniel
>