The pfSense Store

Author Topic: pfBlockerNG v2.1 w/TLD  (Read 33805 times)

0 Members and 1 Guest are viewing this topic.

Offline someuser123

  • Newbie
  • *
  • Posts: 17
  • Karma: +9/-0
    • View Profile
Re: pfBlockerNG v2.1 w/TLD
« Reply #15 on: July 29, 2016, 07:21:26 pm »
pfBlockerNG-2.1.1_1 is working like charm, On 2.3.3-DEVELOPMENT (amd64) no issues.

TLD Blacklist is really handy, Thanks BBcan177

Offline BBcan177

  • Moderator
  • Hero Member
  • *****
  • Posts: 2554
  • Karma: +797/-5
    • View Profile
    • Click for Support
Re: pfBlockerNG v2.1 w/TLD
« Reply #16 on: July 29, 2016, 11:52:05 pm »
Here are the links for Malware Corpus Tracker which can be used w/ pfBlockerNG DNSBL:

Site:
http://track.h3x.eu/about/400

Available Feeds:
https://tracker.h3x.eu/api/sites_1month.php
https://tracker.h3x.eu/api/sites_1week.php
https://tracker.h3x.eu/api/sites_1day.php
https://tracker.h3x.eu/api/sites_1hour.php

DO NOT Select all of these Feeds. You should pick only one Feed. For example: the "1Month" will include the "1Week/1Day/1Hour".

 [ Edit - change to https ]

Twitter:
https://twitter.com/h3x2b
« Last Edit: August 21, 2016, 09:30:35 pm by BBcan177 »
"Experience is something you don't get until just after you need it."

 | http://pfblockerng.com | Twitter @BBcan177  | #pfBlockerNG |

Offline ntct

  • Jr. Member
  • **
  • Posts: 64
  • Karma: +8/-0
    • View Profile
Re: pfBlockerNG v2.1 w/TLD
« Reply #17 on: July 30, 2016, 01:06:56 am »
Hi BBcan177,

I can't update h3x feed from available feeds list in pfBlockerNG v2.1.

It show below.

Code: [Select]
[ h3x ] Downloading update .. 200 OK
 Remote timestamp missing
 No Domains Found

And I can't let TLD Exclusion List working. Can you give a example or check it works?
« Last Edit: July 30, 2016, 01:10:10 am by ntct »

Offline RonpfS

  • Hero Member
  • *****
  • Posts: 684
  • Karma: +96/-2
    • View Profile
Re: pfBlockerNG v2.1 w/TLD
« Reply #18 on: July 30, 2016, 04:26:47 am »
Hi BBcan177,

I can't update h3x feed from available feeds list in pfBlockerNG v2.1.

It show below.

Code: [Select]
[ h3x ] Downloading update .. 200 OK
 Remote timestamp missing
 No Domains Found
Same here

And I can't let TLD Exclusion List working. Can you give a example or check it works?
Did you do a Force Reload after changing the list ?
2.3.5-RELEASE (amd64)
Intel Core2 Quad CPU Q8400 @ 2.66GHz 8GB
pfBlockerNG 2.1.2_1/Dev, suricata 4.0.1

Offline hulleyrob

  • Jr. Member
  • **
  • Posts: 27
  • Karma: +1/-0
    • View Profile
Re: pfBlockerNG v2.1 w/TLD
« Reply #19 on: July 30, 2016, 06:34:35 am »
Code: [Select]
[ 1month ] Downloading update .. 200 OK
  Remote timestamp missing
 No Domains Found

[ 1week ] Downloading update [ 07/30/16 12:31:20 ] .. 200 OK
  Remote timestamp missing
 No Domains Found

[ 1day ] Downloading update .. 200 OK
  Remote timestamp missing
 No Domains Found

[ 1hour ] Downloading update .. 200 OK
  Remote timestamp missing
 No Domains Found

Me three, anyone post how exactly you get these list working?

Offline BBcan177

  • Moderator
  • Hero Member
  • *****
  • Posts: 2554
  • Karma: +797/-5
    • View Profile
    • Click for Support
Re: pfBlockerNG v2.1 w/TLD
« Reply #20 on: July 30, 2016, 07:40:22 am »
Here is a patch to fix the H3X Feed...  Sorry about that  ...

Here are the links for Malware Corpus Tracker which can be used w/ pfBlockerNG DNSBL:

UPDATE:

Guess the internal QA testing didn't work too well when I tested this Feed.
Please follow these instructions below to patch the code to get the following feed to parse:

Edit     /usr/local/pkg/pfblockerng/pfblockerng.inc

Goto Line 3368 which contains the following:
Code: [Select]
$h3x_feed = TRUE;
Reference:
https://github.com/pfsense/FreeBSD-ports/blob/devel/net/pfSense-pkg-pfBlockerNG/files/usr/local/pkg/pfblockerng/pfblockerng.inc#L3368

and add the following line after line 3368:
Code: [Select]
$liteparser = TRUE;
Then follow that with a    "Force Update"
"Experience is something you don't get until just after you need it."

 | http://pfblockerng.com | Twitter @BBcan177  | #pfBlockerNG |

Offline BBcan177

  • Moderator
  • Hero Member
  • *****
  • Posts: 2554
  • Karma: +797/-5
    • View Profile
    • Click for Support
Re: pfBlockerNG v2.1 w/TLD
« Reply #21 on: July 30, 2016, 07:42:06 am »
And I can't let TLD Exclusion List working. Can you give a example or check it works?

Can you provide more detail about what you're trying to accomplish?
"Experience is something you don't get until just after you need it."

 | http://pfblockerng.com | Twitter @BBcan177  | #pfBlockerNG |

Offline hulleyrob

  • Jr. Member
  • **
  • Posts: 27
  • Karma: +1/-0
    • View Profile
Re: pfBlockerNG v2.1 w/TLD
« Reply #22 on: July 30, 2016, 07:42:32 am »
Works for me.

For the lazy:

Code: [Select]
vi +3368 /usr/local/pkg/pfblockerng/pfblockerng.inc
to go straight to the line.

Thanks BBcan

Offline BBcan177

  • Moderator
  • Hero Member
  • *****
  • Posts: 2554
  • Karma: +797/-5
    • View Profile
    • Click for Support
Re: pfBlockerNG v2.1 w/TLD
« Reply #23 on: July 30, 2016, 03:37:21 pm »
I have posted a PR #164 to fix the H3x parser issue noted above.
https://github.com/pfsense/FreeBSD-ports/pull/164‎

Once this is merged the pkg will be at version 2.1.1_2. 

If you manually edited the file noted above, or not, you do not need to make any further changes with this version.
"Experience is something you don't get until just after you need it."

 | http://pfblockerng.com | Twitter @BBcan177  | #pfBlockerNG |

Offline oddworld19

  • Jr. Member
  • **
  • Posts: 27
  • Karma: +1/-0
    • View Profile
Re: pfBlockerNG v2.1 w/TLD
« Reply #24 on: July 30, 2016, 06:31:42 pm »
......and I'm buying another 8 gigs RAM tonight (from 8G to 16G) now that unbound is VIRT 12.3G and I've swapped 6G.

Worth it though.
« Last Edit: July 30, 2016, 07:14:08 pm by oddworld19 »
Supermicro SYS-5018A-FTN4 (Atom c2758)
pfSense 2.3.2

Offline Andrew453

  • Full Member
  • ***
  • Posts: 122
  • Karma: +5/-0
    • View Profile
Re: pfBlockerNG v2.1 w/TLD
« Reply #25 on: July 31, 2016, 04:32:23 am »
Hi BBcan177

Thanks for implementing this.  Would you be able to explain a bit more what the role of the /usr/local/pkg/pfblockerng/dnsbl_tld file is please?

I was expecting it to contain a pure list of TLDs which pfblockerng can then use to work out whether any given domain is a second level domain or higher.  But it seems itself to contain some second level domains?

That said, when I've looked that the /var/unbound/pfb_dnsbl.conf on my set up that pfblockerng has created, it does contain exactly what I would expect to see (i.e. full blocking of the entire domain for second level domains, but only specific blocking for higher level domains).  So it does seem to be doing exactly what I'd like it to, but I'm not sure how the dnsbl_tld file is working to do that.

Thanks.

Offline Qinn

  • Full Member
  • ***
  • Posts: 135
  • Karma: +5/-1
    • View Profile
Re: pfBlockerNG v2.1 w/TLD
« Reply #26 on: July 31, 2016, 04:44:47 am »
Hi BBcan177,

Is there any good install/setup/configure instruction (video or guide) for the last version op pfblockerNG, that you could/would recommend?

Thanks for your advice, cheers Qinn

Offline BBcan177

  • Moderator
  • Hero Member
  • *****
  • Posts: 2554
  • Karma: +797/-5
    • View Profile
    • Click for Support
Re: pfBlockerNG v2.1 w/TLD
« Reply #27 on: July 31, 2016, 04:53:38 am »
I was expecting it to contain a pure list of TLDs which pfblockerng can then use to work out whether any given domain is a second level domain or higher.  But it seems itself to contain some second level domains?

Hi Andrew453,

If I only used the TLD, it would be a simple process of looking at any listed Domain and seeing if it had only a second-level Domain (SLD) then block the entire Domain. However, there are suffixes like "uk.com" which is what I would call the TLD that is used to determine if there is one more level. So all of the TLDs (suffixes) in that file are known TLDs which is used in the determination process. Most of the file was taken from the "Public Suffix Registry".
"Experience is something you don't get until just after you need it."

 | http://pfblockerng.com | Twitter @BBcan177  | #pfBlockerNG |

Offline BBcan177

  • Moderator
  • Hero Member
  • *****
  • Posts: 2554
  • Karma: +797/-5
    • View Profile
    • Click for Support
Re: pfBlockerNG v2.1 w/TLD
« Reply #28 on: July 31, 2016, 04:55:32 am »
Hi BBcan177,

Is there any good install/setup/configure instruction (video or guide) for the last version op pfblockerNG, that you could/would recommend?

Thanks for your advice, cheers Qinn

There is a pfSense Hangout that I did which can be used for an overview of the pkg functionality. However, apart from the three main pfBlockerNG threads in this forum, there isn't any other documentation.
"Experience is something you don't get until just after you need it."

 | http://pfblockerng.com | Twitter @BBcan177  | #pfBlockerNG |

Offline Qinn

  • Full Member
  • ***
  • Posts: 135
  • Karma: +5/-1
    • View Profile
Re: pfBlockerNG v2.1 w/TLD
« Reply #29 on: July 31, 2016, 05:01:54 am »
Thanks for the quick reply. Darn  :( I found this one can you can agree to this one?

https://www.youtube.com/watch?v=YLhDOaH0q5U