pfSense Gold Subscription

Author Topic: Anti-Spam Package (dspam or SpamAssasin)  (Read 36870 times)

0 Members and 1 Guest are viewing this topic.

Offline illizit

  • Newbie
  • *
  • Posts: 22
  • Karma: +0/-0
    • View Profile
Anti-Spam Package (dspam or SpamAssasin)
« on: August 22, 2011, 11:46:57 pm »
Hello,

After running into issues with spamd, I would like to have a package that has a true SpamFilter (not greylisting). I think running SpamAssasin or dspam would be a great contribution towards pfsense and would make it a complete UTM (with the other packages that are already included).

I'm willing to contribute $300 towards this project.

Offline wallabybob

  • Hero Member
  • *****
  • Posts: 5240
  • Karma: +11/-1
    • View Profile
Re: Anti-Spam Package (dspam or SpamAssasin)
« Reply #1 on: August 23, 2011, 04:40:01 pm »
I'm not familiar with the details of any of these packages mentioned.

The FreeBSD man page for spamd says it is a daemonized version of spamassassin so if spamd doesn't do what you want how would spamassassin help?

dspam is already available as a FreeBSD package so should be able to be installed by pkg_add. You are also looking for a GUI interface?

Offline illizit

  • Newbie
  • *
  • Posts: 22
  • Karma: +0/-0
    • View Profile
Re: Anti-Spam Package (dspam or SpamAssasin)
« Reply #2 on: August 23, 2011, 08:47:19 pm »
The spamd package included in pfsense is not the daemonized version of spamassassin. It is a tar-pit (graylisting) fake SMTP server that throttles SMTP connections in an attempt to catch spammers. Nothing like SpamAssasin (although I did see the package you are referring to which I do not know why it wasn't included in Pfsense). The spamd package included in Pfsense is similar to the package spamd for OpenBSD. I guess they ported it from there.

I have not tried installing dspam manually, I rather have the GUI able to manage it, view logs, etc.

Offline jimp

  • Administrator
  • Hero Member
  • *****
  • Posts: 21397
  • Karma: +1432/-26
    • View Profile
Re: Anti-Spam Package (dspam or SpamAssasin)
« Reply #3 on: August 31, 2011, 11:34:17 am »
There is a big difference in configuring a greylisting daemon and configuring a full spam filter package. An actual spam filter package would involve running a full mail server on the firewall, plus all of the spam scanning software, plus coming up with some kind of GUI for it, and a way to manage not only the global spam filter settings but probably per-address options, and updating the spamassassin rules, etc, etc.

You'd realistically be looking at a several thousand dollars to have someone do that properly.

(And I still wouldn't want to run that on my firewall... :-)
Need help fast? Commercial Support!

Co-Author of pfSense: The Definitive Guide. - Check the Doc Wiki for FAQs.

Do not PM for help!

Offline Bai Shen

  • Full Member
  • ***
  • Posts: 220
  • Karma: +1/-0
    • View Profile
Re: Anti-Spam Package (dspam or SpamAssasin)
« Reply #4 on: August 31, 2011, 11:59:51 am »
There is a big difference in configuring a greylisting daemon and configuring a full spam filter package. An actual spam filter package would involve running a full mail server on the firewall, plus all of the spam scanning software, plus coming up with some kind of GUI for it, and a way to manage not only the global spam filter settings but probably per-address options, and updating the spamassassin rules, etc, etc.

You'd realistically be looking at a several thousand dollars to have someone do that properly.

(And I still wouldn't want to run that on my firewall... :-)

That doesn't sound right.  I know there was a SpamAssassin package for IPCop and I don't think it runs a mail server.

It seems like there should be a way to scan and mark the emails.  Like what Untangle or IPCop with the above mentioned package do.

Offline jimp

  • Administrator
  • Hero Member
  • *****
  • Posts: 21397
  • Karma: +1432/-26
    • View Profile
Re: Anti-Spam Package (dspam or SpamAssasin)
« Reply #5 on: August 31, 2011, 12:21:28 pm »
Sorry, but you are not correct.

I have setup spamassassin on mail servers before, and run it in a couple locations. SpamAssassin on its own is not a mail server, cannot accept messages, it cannot send them out, it requires another program to feed it the messages and then deal with the result. A common way is via something like amavisd, which is essentially a mail server (rather, an MTA, to be more precise). Other places run it through scripts hooked in from postfix/exim/sendmail/etc.

Something has to accept the entire message, then let spamassin scan it, and then based on that result, send it to your mail server.

It can't just grab the traffic stream, scan it live, and then forward/reject it after the whole message has been received, not without a program receiving the message and forwarding it to your internal mail server. If it scanned the stream as it flowed through the message would already be delivered to your mail server by the time SpamAssassin scanned it. That just isn't how it works.
Need help fast? Commercial Support!

Co-Author of pfSense: The Definitive Guide. - Check the Doc Wiki for FAQs.

Do not PM for help!

Offline Bai Shen

  • Full Member
  • ***
  • Posts: 220
  • Karma: +1/-0
    • View Profile
Re: Anti-Spam Package (dspam or SpamAssasin)
« Reply #6 on: August 31, 2011, 12:46:33 pm »
Sorry, but you are not correct.

Won't be the first nor last time. :)

Quote
I have setup spamassassin on mail servers before, and run it in a couple locations. SpamAssassin on its own is not a mail server, cannot accept messages, it cannot send them out, it requires another program to feed it the messages and then deal with the result. A common way is via something like amavisd, which is essentially a mail server (rather, an MTA, to be more precise). Other places run it through scripts hooked in from postfix/exim/sendmail/etc.

Something has to accept the entire message, then let spamassin scan it, and then based on that result, send it to your mail server.

It can't just grab the traffic stream, scan it live, and then forward/reject it after the whole message has been received, not without a program receiving the message and forwarding it to your internal mail server. If it scanned the stream as it flowed through the message would already be delivered to your mail server by the time SpamAssassin scanned it. That just isn't how it works.

I've only done Spamassassin in between my POP server and my local client.  In those cases, it simply marked messages with a SPAM tag.  So it wasn't rejecting anything, but making it easier for the client to do filtering.

Offline jimp

  • Administrator
  • Hero Member
  • *****
  • Posts: 21397
  • Karma: +1432/-26
    • View Profile
Re: Anti-Spam Package (dspam or SpamAssasin)
« Reply #7 on: August 31, 2011, 01:02:59 pm »
I've only done Spamassassin in between my POP server and my local client.  In those cases, it simply marked messages with a SPAM tag.  So it wasn't rejecting anything, but making it easier for the client to do filtering.

In that kind of case, it's the client downloading the message, feeding it through SA, and then injecting the result into your mailbox (more or less). Or it's a proxy where something pulls the messages, lets SA scan them, and then your client gets the results from the proxy. There are some other variations but that's the basic method.

Much different than doing it in between two mail servers that have to obey proper standards/protocols for delivering mail to each other.
Need help fast? Commercial Support!

Co-Author of pfSense: The Definitive Guide. - Check the Doc Wiki for FAQs.

Do not PM for help!

Offline Bai Shen

  • Full Member
  • ***
  • Posts: 220
  • Karma: +1/-0
    • View Profile
Re: Anti-Spam Package (dspam or SpamAssasin)
« Reply #8 on: August 31, 2011, 01:59:41 pm »
I've only done Spamassassin in between my POP server and my local client.  In those cases, it simply marked messages with a SPAM tag.  So it wasn't rejecting anything, but making it easier for the client to do filtering.

In that kind of case, it's the client downloading the message, feeding it through SA, and then injecting the result into your mailbox (more or less). Or it's a proxy where something pulls the messages, lets SA scan them, and then your client gets the results from the proxy. There are some other variations but that's the basic method.

Much different than doing it in between two mail servers that have to obey proper standards/protocols for delivering mail to each other.

I believe IPCop and Untangle use the proxy method.  And honestly, that's all I really need from pfSense.  Not sure about the OP.

Offline jimp

  • Administrator
  • Hero Member
  • *****
  • Posts: 21397
  • Karma: +1432/-26
    • View Profile
Re: Anti-Spam Package (dspam or SpamAssasin)
« Reply #9 on: August 31, 2011, 02:03:07 pm »
If they proxy POP3/IMAP then that's one thing, and wouldn't work with SSL/TLS. Proxying SMTP is entirely different. Not sure if that's even feasible in this kind of scenario with a proxy of sorts.
Need help fast? Commercial Support!

Co-Author of pfSense: The Definitive Guide. - Check the Doc Wiki for FAQs.

Do not PM for help!

Offline dhatz

  • Hero Member
  • *****
  • Posts: 1000
  • Karma: +5/-0
    • View Profile
Re: Anti-Spam Package (dspam or SpamAssasin)
« Reply #10 on: August 31, 2011, 02:37:18 pm »
I have been running SpamAssassin since several years with various MTAs (in recent years mostly with Postfix) together with ClamAV anti-virus and various other tools that enhance SA's effectiveness.

In my experience such a system really needs to be a full-blown mail-server to do the job. It is not something I would want to run on a router/firewall.
« Last Edit: August 31, 2011, 02:39:55 pm by dhatz »

submicron

  • Guest
Re: Anti-Spam Package (dspam or SpamAssasin)
« Reply #11 on: August 31, 2011, 07:19:53 pm »
For clarification, the IPCop SPAM filter uses ProxSMTP to proxy SMTP traffic and pass it off to spamassassin.  It also uses p3Scan to prox pop3 connections.  In fact, in looking at the Copfilter plugin, it looks like they use half a dozen little proxy programs to proxy different protocols, pass them off to ClamAV and/or Spamassassin.  I would expect this kind of system to be exceptionally messy, resource intensive and not at all appropriate for a firewall.  Its one thing to use a small amount of CPU to stem the tide of spam waves using a greylisting daemon, and quite another to be proxying, queing, scanning and rejecting|forwarding|dropping all of these kinds of connections. 

Not surprisingly, I agree with jimp on this, such a package would require several thousand dollars at least and probably closer to ten thousand to do anything close to correctly.  Even then, it would require herculean effort to keep up to date and would never be appropriate for anything more than a small network.

Offline Metu69salemi

  • Hero Member
  • *****
  • Posts: 1556
  • Karma: +2/-0
    • View Profile
Re: Anti-Spam Package (dspam or SpamAssasin)
« Reply #12 on: August 31, 2011, 10:56:54 pm »
And when this is done by firewall you have usually no possibility to have those attachments or what ever is going to get proxied off.
Like in these little redboxes(what are greatly used also in pfsense environment) those can proxy a smtp trafic, but when it kicks in, no one can get those mails nor attachments.

Offline Bai Shen

  • Full Member
  • ***
  • Posts: 220
  • Karma: +1/-0
    • View Profile
Re: Anti-Spam Package (dspam or SpamAssasin)
« Reply #13 on: September 01, 2011, 07:27:17 am »
What about those of us who aren't running a mail server and want SpamAssassin for marking our pop3 mail as it comes in?  What would you recommend?

Offline marcelloc

  • Hero Member
  • *****
  • Posts: 13398
  • Karma: +589/-7
    • View Profile
Re: Anti-Spam Package (dspam or SpamAssasin)
« Reply #14 on: September 18, 2011, 03:21:45 pm »
There is a big difference in configuring a greylisting daemon and configuring a full spam filter package. An actual spam filter package would involve running a full mail server on the firewall, plus all of the spam scanning software, plus coming up with some kind of GUI for it, and a way to manage not only the global spam filter settings but probably per-address options, and updating the spamassassin rules, etc, etc.

You'd realistically be looking at a several thousand dollars to have someone do that properly.

(And I still wouldn't want to run that on my firewall... :-)

I like the idea of 'several thousand dollars'  ;)

I'm working on it and I agree that firewall is not the best place for a antispam system, but you can put it on another pfSense box/vm and use the same great pfSense GUI/rules to configure it.

What is done:
First stage antispam checks:
  • Detect zombies
  • check RBL
  • check SPF

Second stage antispam check:
  • Subject
  • Valid Recipients
  • Valid Domains
  • Body Content
  • Attachment types

What is almost done:
Third stage antispam checks(high cpu usage):
  • Mailscanner
  • SpamAssassin
  • Clamav
  • pyzor

If any one are willing to contribute with the project, the time has come.

Not surprisingly, I agree with jimp on this, such a package would require several thousand dollars at least and probably closer to ten thousand to do anything close to correctly.  Even then, it would require herculean effort to keep up to date and would never be appropriate for anything more than a small network.
Yes it is difficult but not impossible. Many checks nowadays are based on sourceip or live lists(white, gray,black,etc).
Spamd just like the two biggest commercial antispam tools use a mundial database for ip reputation and other mail stuff.
pyzor can check a global server too.
I`m planning that sa-update will be updated via cron.
« Last Edit: September 18, 2011, 06:21:09 pm by marcelloc »