pfSense Gold Subscription

Author Topic: Working OpenBSD spamd package ** now $800USD - $1000USD **  (Read 20940 times)

0 Members and 1 Guest are viewing this topic.

Offline lambert

  • Jr. Member
  • **
  • Posts: 26
  • Karma: +0/-0
    • View Profile
Re: Working OpenBSD spamd package ** now $800USD - $1000USD **
« Reply #15 on: April 16, 2008, 03:39:47 pm »
In the hopes that relating my experience with spamd will motivate some people to contribute to this bounty, and ultimately motivate a developer to take the bounty, here is the story.

I have put up a FreeBSD 7.x box where I want the pfSense box(es) to see how much of a difference spamd is going to make.   So far, it looks like it is going to knock about half the load off my Barracuda boxes.

I moved one unit behind the spamd box yesterday and saw a drop in message attempts per hour on that box drop from 18,000 per hour to about 3,000 per hour.

I left it running that way for about 20 hours and it worked it's way up to about 4,000 per hour for the hour before I began moving the second barracuda.

Servers were not being very successful in un-greylisting themselves because when box 1 told them to shove off, they immediately sent the message via box 2.  At 11am I disabled the second box.  In a couple of hours, we were seeing about 8,300 message attempts per hour on the protected box.  The number of whitelisted entries climbed very quickly.  (Before the spamd box was inline, one box could not have survived handling the e-mail load by itself.  It ran about 2  hours 50 minutes taking the full load without breaking a sweat.)

We now have both boxes behind the spamd box and are seeing just under 6,000 message attempts per hour per box.  The number of allowed messages per hour per box has gone from about 2,000 to about 1,000.  If things stay this way, my customers are going to start wondering where their spam is.   So far, we have had no complaints of missing e-mail.  However, we are only 25 hours in.  I suspect more and more spam hosts will successfully white list themselves, but the Barracudas should take care of them when they get through.  I have hope that we will stabilize at around 10,000 message send attempts per hour per box.  That will be an almost 50% reduction in load on the barracudas.  This is a very impressive layer to add to the e-mail security onion.

The spamd box is a Via C7 1GHz with 256MB RAM and 3 Intel gigabit NICs.   I still have about 20MB free memory and we are at about 20% cpu utilization system wide.  The spamd processes themselves are using about 17% of the CPU.

All of that to say, having spamd working on a pfSense box could very well save your spam scanning hardware from needing to be upgraded for a year or two.  I spent 2 weeks trying to get the pfSense development environment figured out and finally punted and had the raw FreeBSD solution up in about a day with configuring everything by hand.

Getting someone to make this all clickable in pfSense will be very much worth the money I have pledged to the bounty.  I sure hope someone takes my money.  They will have earned it.

Offline sullrich

  • Hero Member
  • *****
  • Posts: 5110
  • Karma: +3/-0
    • View Profile
    • pfSense
Re: Working OpenBSD spamd package ** now $800USD - $1000USD **
« Reply #16 on: April 16, 2008, 05:14:57 pm »
I am going to pick up this Bounty since I have some downtime waiting on another project.  I just uploaded 4.1.2 to files.pfsense.org and re-enabled the package.   We'll take it from there once it shows up in the packages section.

Offline sullrich

  • Hero Member
  • *****
  • Posts: 5110
  • Karma: +3/-0
    • View Profile
    • pfSense
Re: Working OpenBSD spamd package ** now $800USD - $1000USD **
« Reply #17 on: April 16, 2008, 08:22:34 pm »
SpamD is working!

Free_the_mallocs, I was able to think up a "trick" to allow multiple mail servers behind the firewall.  It's easy.

Simply:

1. Create an alias of hosts, add an ip for each mail server living behind the firewall.  Call it mailservers.
2. Set Next MTA to the alias name, but surround it in {}, so it would be {mailservers} in this case.

Please test and let me know what all else needs fixing up.

NOTE: I might change this in the future (which will auto-upgrade itself) but this was done so that we could get this working in 1.2-REL without a new filter.inc.
« Last Edit: April 16, 2008, 10:39:02 pm by sullrich »

Offline lambert

  • Jr. Member
  • **
  • Posts: 26
  • Karma: +0/-0
    • View Profile
Re: Working OpenBSD spamd package ** now $800USD - $1000USD **
« Reply #18 on: April 16, 2008, 10:28:48 pm »
Sweet!  I'll have to try it out tomorrow on a second machine, while I wait for Paypal to link to the new bank account.  It's been a while since I used it...

Offline sullrich

  • Hero Member
  • *****
  • Posts: 5110
  • Karma: +3/-0
    • View Profile
    • pfSense
Re: Working OpenBSD spamd package ** now $800USD - $1000USD **
« Reply #19 on: April 16, 2008, 10:37:20 pm »
 1-3 should be working (hopefully) and I'll implement #4 by EOD friday.

Offline submicron

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 741
  • Karma: +0/-0
  • I like pie!
    • View Profile
    • BSDPerimeter
Re: Working OpenBSD spamd package ** now $800USD - $1000USD **
« Reply #20 on: April 17, 2008, 08:52:55 am »
Great news all around!  We're setting up a test MTA right now and we'll probably start testing things here shortly.
I do not respond to PMs demanding help.

Offline submicron

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 741
  • Karma: +0/-0
  • I like pie!
    • View Profile
    • BSDPerimeter
Re: Working OpenBSD spamd package ** now $800USD - $1000USD **
« Reply #21 on: April 17, 2008, 03:08:26 pm »
I opened a topic in the forum for people to test this package and report back:

http://forum.pfsense.org/index.php/topic,8952.0.html
I do not respond to PMs demanding help.

Offline sullrich

  • Hero Member
  • *****
  • Posts: 5110
  • Karma: +3/-0
    • View Profile
    • pfSense
Re: Working OpenBSD spamd package ** now $800USD - $1000USD **
« Reply #22 on: April 18, 2008, 01:09:10 pm »
The package is ready for testing.     Paypal me please at sullrich@gmail.com

Thanks!!

Offline submicron

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 741
  • Karma: +0/-0
  • I like pie!
    • View Profile
    • BSDPerimeter
Re: Working OpenBSD spamd package ** now $800USD - $1000USD **
« Reply #23 on: April 18, 2008, 05:18:31 pm »
I'm going to count this as a feature request rather than a bug.  The requirement to encapsulate an alias inside of {}'s is inconsistent with the pfSense UI.  Ideally, there would be a drop down field to specify a single IP or an alias, and if alias is selected, then the alias field would autopopulate just like in other places throughout the UI. 
I do not respond to PMs demanding help.

Offline sullrich

  • Hero Member
  • *****
  • Posts: 5110
  • Karma: +3/-0
    • View Profile
    • pfSense
Re: Working OpenBSD spamd package ** now $800USD - $1000USD **
« Reply #24 on: April 19, 2008, 10:58:14 am »
I'm going to count this as a feature request rather than a bug.  The requirement to encapsulate an alias inside of {}'s is inconsistent with the pfSense UI.  Ideally, there would be a drop down field to specify a single IP or an alias, and if alias is selected, then the alias field would autopopulate just like in other places throughout the UI. 

Absolutely.  But that is the only way to work with 1.2 without shipping a new filter.inc.   And it has changed to $aliasname instead of {}.   As I stated previously this will change going forward where the item will automatically be changed into a name without the $.  But for 1.2 that is all we can do...

From an earlier post in this thread... "NOTE: I might change this in the future (which will auto-upgrade itself) but this was done so that we could get this working in 1.2-REL without a new filter.inc."

Offline sullrich

  • Hero Member
  • *****
  • Posts: 5110
  • Karma: +3/-0
    • View Profile
    • pfSense
Re: Working OpenBSD spamd package ** now $800USD - $1000USD **
« Reply #25 on: April 30, 2008, 03:12:40 pm »
I have received payment from submicron and lambert.  Everyone else that committed to this bounty, please pony up or face the wrath of the bounty pig.

Offline Liam

  • Newbie
  • *
  • Posts: 10
  • Karma: +0/-0
    • View Profile
Re: Working OpenBSD spamd package ** now $800USD - $1000USD **
« Reply #26 on: May 01, 2008, 09:16:16 am »
Pardon me for my ignorance, I am new to pfsense.  Is a bounty package only available to those who contribute to the bounty?  This port of spamd is a main reason why I chose pfsense, so I would like to know what the process is to get access to it.

Thank you
Cheers, Liam

Offline submicron

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 741
  • Karma: +0/-0
  • I like pie!
    • View Profile
    • BSDPerimeter
Re: Working OpenBSD spamd package ** now $800USD - $1000USD **
« Reply #27 on: May 01, 2008, 09:38:53 am »
This particular package is available to anyone now and will be freely available going forward.  In fact, we'd encourage you to grab it and test it.  Generally bounty projects are available to anyone, the only real exception has been the Traffic Shaper, which will be freely available to anyone in pfSense 1.3, but for those who contributed to the bounty project, its available in a special 1.2 release so that it can be fully tested and support methodologies developed. 
I do not respond to PMs demanding help.

Offline sullrich

  • Hero Member
  • *****
  • Posts: 5110
  • Karma: +3/-0
    • View Profile
    • pfSense
Re: $100 -- Working OpenBSD spamd package
« Reply #28 on: May 01, 2008, 12:43:14 pm »
i would be very interested in spamd being brought back into pfSense. We have two pfSense CARP routers sitting infront of a mail server that takes in 30-50k emails per day most of which is spam. I would love to see and have spamd implemented on pfSense. This would also be significantly helpful so we don't have to expose Exchange machines to the internet. right now we use postfix at our colo and deliver it ot the exchange machine via ipsec.
I am willing to throw down $250 for this in the next few weeks. i would like to give the money directly to the project and have them payout someone who completes this bounty.
thanks,

Please paypal me sullrich@gmail.com

Offline iced98lx

  • Jr. Member
  • **
  • Posts: 55
  • Karma: +0/-0
  • Package Abuser
    • View Profile
Re: $100 -- Working OpenBSD spamd package
« Reply #29 on: July 01, 2008, 11:13:40 am »
Please paypal me sullrich@gmail.com

I just sent you $50, I know I wasn't part of the bounty originally but I appreciate your work and would like to see development continue and some issues worked out in it such as the whitelist/blacklist count fixed and other nitpicky things. Thanks for your work on it.