Netgate SG-1000 microFirewall

Author Topic: DNSBL Enable TLD RAM/freezing issues  (Read 164 times)

0 Members and 1 Guest are viewing this topic.

Offline sjtorrie

  • Newbie
  • *
  • Posts: 2
  • Karma: +0/-0
    • View Profile
DNSBL Enable TLD RAM/freezing issues
« on: November 16, 2017, 06:45:24 am »
Is anyone else using DNSBL with with TLD enabled using lists from http://www.squidblacklist.org/ and having RAM/freezing on CRON update issues?

Squidblacklist pulls in a hearty 1947496 entries and everything is fine without TLD enabled - however, we really need TLD enabled for it to be an effective webfilter. Once enabled (with 8GB RAM) it struggles to do an initial CRON and then freezes constantly.

If I ramp it up to 10GB then it's stable, but with very high RAM use, but when manual/scheduled CRON runs it freezes up again, all users are locked out from the internet and I have to reboot the device.

I REALLY want to use this for filtering and I don't really want to use a smaller list - We've considered using squidguard as a filtered webproxy instead but it doesn't seem to be as advanced, and we lose all the IP list functons (we're using FireHOL to lockdown on this level)


(pfSense 2.4.1 & pfbockerng 2.1.2_1)


Offline BBcan177

  • Moderator
  • Hero Member
  • *****
  • Posts: 2553
  • Karma: +798/-5
    • View Profile
    • Click for Support
Re: DNSBL Enable TLD RAM/freezing issues
« Reply #1 on: November 16, 2017, 10:06:08 am »
Unbound creates a pointer in memory for each "redirect" zone and this is why it uses more memory.

Keep in mind that the pkg will do a validation of the database after each cron event and it will require memory also for that purpose.... So initially it loads ok, but when cron runs, Unbound is already using quite a bit of memory and you need that much more for the validation process...

So you will need to bump the memory in the box to be able to use 2M domains... Not much I can do about that... I have worked with the Unbound devs but so far there is no change on how Unbound loads these domains into memory.
"Experience is something you don't get until just after you need it."

 | http://pfblockerng.com | Twitter @BBcan177  | #pfBlockerNG |

Offline f34rinc

  • Jr. Member
  • **
  • Posts: 50
  • Karma: +21/-0
    • View Profile
    • legoclan
Re: DNSBL Enable TLD RAM/freezing issues
« Reply #2 on: November 16, 2017, 10:42:40 am »
Another idea would be to use 3rd party DNS filtering for the TLD blocking.

Offline sjtorrie

  • Newbie
  • *
  • Posts: 2
  • Karma: +0/-0
    • View Profile
Re: DNSBL Enable TLD RAM/freezing issues
« Reply #3 on: November 17, 2017, 04:20:03 am »
Another idea would be to use 3rd party DNS filtering for the TLD blocking.

I'm all ears for suggestions on what 3rd parties might be out there(?!)... ideally I want to keep everything on a single pfSense VM for each internet breakout > in the mean time I'll push the RAM a little higher - unfortunately the hosts I have out and about in our branch sites are somewhat limited...