Netgate SG-1000 microFirewall

Author Topic: firehol level 1 list blocking LAN resources  (Read 1212 times)

0 Members and 2 Guests are viewing this topic.

Offline davidology

  • Newbie
  • *
  • Posts: 4
  • Karma: +0/-0
    • View Profile
firehol level 1 list blocking LAN resources
« on: August 15, 2017, 11:50:20 pm »
The FireHol level 1 list (https://raw.githubusercontent.com/ktsaou/blocklist-ipsets/master/firehol_level1.netset)

has all local IP addresses (like 10.0.0.0/8 and 192.168.0.0/24 etc) listed. When I subscribe to it in pfBlockerNG under IPv4, it obviously blocks my access to my LAN resources. Is there a way to have it disregard those entries without compromising overall security?

Sorry if this is a basic question. I've been searching. I have a couple ideas, but don't want to lock myself out again so some advice from someone who has implemented this successfully is welcome!

Thanks in advance..

Offline BBcan177

  • Moderator
  • Hero Member
  • *****
  • Posts: 2554
  • Karma: +797/-5
    • View Profile
    • Click for Support
Re: firehol level 1 list blocking LAN resources
« Reply #1 on: August 16, 2017, 09:23:48 am »
Lvl1 contains bogons and shouldn't be used to block Outbound.... You can also manually add the same feeds that comprise the Lvl1 feed directly..... also can enable "Suppression" in the General Tab which will filter out RFC1918 and loopback addresses...
"Experience is something you don't get until just after you need it."

 | http://pfblockerng.com | Twitter @BBcan177  | #pfBlockerNG |

Offline davidology

  • Newbie
  • *
  • Posts: 4
  • Karma: +0/-0
    • View Profile
Re: firehol level 1 list blocking LAN resources
« Reply #2 on: August 17, 2017, 02:32:59 pm »
Great idea on recreating the feed without the local addresses. Worked flawlessly.

Thank you!

Offline frankvh

  • Newbie
  • *
  • Posts: 4
  • Karma: +2/-0
    • View Profile
Re: firehol level 1 list blocking LAN resources
« Reply #3 on: November 08, 2017, 12:38:53 pm »
This is interesting because I recently had the same experience. I lost access to my LAN because I'd used the level 1 list. I switched to Firehol level 3 but it's a shorter list.

As a "lazy man's" alternative, am I safe if I use the Firehol level 1 list, but also enable "supression" in the pfBlockNG general tab?

Thanks.

Offline charvey

  • Newbie
  • *
  • Posts: 16
  • Karma: +0/-0
    • View Profile
Re: firehol level 1 list blocking LAN resources
« Reply #4 on: November 09, 2017, 06:33:36 pm »
I had similar issues with firehol lvl 1, but with broadcast packets.

See https://forum.pfsense.org/index.php?topic=138877.0

Offline frankvh

  • Newbie
  • *
  • Posts: 4
  • Karma: +2/-0
    • View Profile
Re: firehol level 1 list blocking LAN resources
« Reply #5 on: November 09, 2017, 11:32:47 pm »
I ended up taking BBcan's advice (good advice it always is) and simply put the lists that comprise firehol1 into pfblocker (minus the bogons list of course).

Offline BBcan177

  • Moderator
  • Hero Member
  • *****
  • Posts: 2554
  • Karma: +797/-5
    • View Profile
    • Click for Support
Re: firehol level 1 list blocking LAN resources
« Reply #6 on: November 12, 2017, 05:17:12 pm »
I ended up taking BBcan's advice (good advice it always is) and simply put the lists that comprise firehol1 into pfblocker (minus the bogons list of course).

Using the Lvl 1 Feed is going to cause grief... Just don't do it  ;) ;)
"Experience is something you don't get until just after you need it."

 | http://pfblockerng.com | Twitter @BBcan177  | #pfBlockerNG |

Offline iorx

  • Full Member
  • ***
  • Posts: 144
  • Karma: +4/-0
    • View Profile
Re: firehol level 1 list blocking LAN resources
« Reply #7 on: November 15, 2017, 07:41:46 pm »
The level1 list looks like this, excluding the bogons.
(I really hope that I understood the firehol list content correctly. If that is the case, now someone else don't have to recreate this wheel again.)

Here goes:
Code: [Select]
http://www.spamhaus.org/drop/drop.txt
http://www.spamhaus.org/drop/edrop.txt
http://feeds.dshield.org/block.txt
https://feodotracker.abuse.ch/blocklist/?download=ipblocklist
https://sslbl.abuse.ch/blacklist/sslipblacklist.csv
https://zeustracker.abuse.ch/blocklist.php?download=badips
http://osint.bambenekconsulting.com/feeds/c2-ipmasterlist.txt

Offline BBcan177

  • Moderator
  • Hero Member
  • *****
  • Posts: 2554
  • Karma: +797/-5
    • View Profile
    • Click for Support
Re: firehol level 1 list blocking LAN resources
« Reply #8 on: November 16, 2017, 10:06:34 am »
Try to use HTTPS for all sites that support it...
"Experience is something you don't get until just after you need it."

 | http://pfblockerng.com | Twitter @BBcan177  | #pfBlockerNG |

Offline adoucette

  • Jr. Member
  • **
  • Posts: 32
  • Karma: +0/-0
    • View Profile
Re: firehol level 1 list blocking LAN resources
« Reply #9 on: November 18, 2017, 09:16:47 am »
The level1 list looks like this, excluding the bogons.
(I really hope that I understood the firehol list content correctly. If that is the case, now someone else don't have to recreate this wheel again.)

Here goes:
Code: [Select]
http://www.spamhaus.org/drop/drop.txt
http://www.spamhaus.org/drop/edrop.txt
http://feeds.dshield.org/block.txt
https://feodotracker.abuse.ch/blocklist/?download=ipblocklist
https://sslbl.abuse.ch/blacklist/sslipblacklist.csv
https://zeustracker.abuse.ch/blocklist.php?download=badips
http://osint.bambenekconsulting.com/feeds/c2-ipmasterlist.txt

iorx, thank you for compiling and posting that.
Is the above list sufficient for the residential user? Or are there additional sources I should add?

On the other hand, given the great work BBcan has done with the "suppression" option etc, is it safe to just use the firehol level 1 list just in case the sources it pulls from change over time?

(just one small piece of 1) protecting the kids online and 2) protecting our networked computers from what the kids may inadvertently do online!)

Thank you all.
BBCan -- donation coming your way.

Ari

Offline seanr22a

  • Newbie
  • *
  • Posts: 22
  • Karma: +2/-0
    • View Profile
Re: firehol level 1 list blocking LAN resources
« Reply #10 on: November 19, 2017, 02:16:24 pm »
This is the IP lists Iím using. I started with a few and over time added more and more lists Iíve found. I use a script that download all lists with wget, unpack those that need and put all of them in one file. The resulting file is filtered leaving only ip addresses and checked and cleaned for duplicates and sorted. Finally, the file is cleaned from RFC 1918 and RFC 4193 addresses and some common important ip addresses such as 8.8.8.8  8.8.4.4 and so on. The finished file is put on a small internal webserver. Pfblocker get the file from there. I do the same with DNSBL lists, I can post all DNSBL lists I have if someone is interested. The ones here is only IP lists.
I put the resulting IP list and DNSBL list on an external webserver as well currently updated once a week. You have them here: http://dnsbl.dyndns.org:9080/MyBlocklist.txt and http://dnsbl.dyndns.org:9080/mydnsblfeed.txt  The DNSBL file is big so it can take a while to download.

https://gist.githubusercontent.com/BBcan177/d7105c242f17f4498f81/raw/90eb2ac8bdc01af3008d728b7c0f10dc7b2506b4/MS-3
https://gist.githubusercontent.com/BBcan177/bf29d47ea04391cb3eb0/raw/b344ebc9475acdea1fae38a12c4ea9332838a184/MS-1
https://rules.emergingthreats.net/fwrules/emerging-Block-IPs.txt
https://www.spamhaus.org/drop/drop.txt
https://pfblockerlists.smallbusinesstech.net/hackerlist.txt
https://ransomwaretracker.abuse.ch/downloads/RW_IPBL.txt
https://rules.emergingthreats.net/blockrules/compromised-ips.txt
http://www.abuseat.org/iotcc.txt
https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/firehol_level1.netset
https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/firehol_level2.netset
http://cinsscore.com/list/ci-badguys.txt
https://ransomwaretracker.abuse.ch/downloads/LY_PS_IPBL.txt
https://ransomwaretracker.abuse.ch/downloads/TL_C2_IPBL.txt
https://zeustracker.abuse.ch/blocklist.php?download=badips
https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/firehol_level3.netset
http://danger.rulez.sk/projects/bruteforceblocker/blist.php
https://lists.blocklist.de/lists/all.txt
http://malc0de.com/bl/IP_Blacklist.txt
https://feodotracker.abuse.ch/blocklist/?download=ipblocklist
https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/firehol_level4.netset
http://feeds.dshield.org/top10-2.txt
https://feeds.dshield.org/block.txt
http://list.iblocklist.com/?list=dgxtneitpuvgqqcpfulq&fileformat=p2p&archiveformat=gz
http://list.iblocklist.com/?list=dgxtneitpuvgqqcpfulq&fileformat=p2p&archiveformat=gz
http://list.iblocklist.com/?list=dgxtneitpuvgqqcpfulq&fileformat=p2p&archiveformat=gz
http://list.iblocklist.com/?list=llvtlsjyoyiczbkjsxpf&fileformat=p2p&archiveformat=gz
http://list.iblocklist.com/?list=llvtlsjyoyiczbkjsxpf&fileformat=p2p&archiveformat=gz
http://list.iblocklist.com/?list=llvtlsjyoyiczbkjsxpf&fileformat=p2p&archiveformat=gz
http://list.iblocklist.com/?list=usrcshglbiilevmyfhse&fileformat=p2p&archiveformat=gz
http://list.iblocklist.com/?list=usrcshglbiilevmyfhse&fileformat=p2p&archiveformat=gz
http://list.iblocklist.com/?list=usrcshglbiilevmyfhse&fileformat=p2p&archiveformat=gz
http://list.iblocklist.com/?list=xpbqleszmajjesnzddhv&fileformat=p2p&archiveformat=gz
http://list.iblocklist.com/?list=xpbqleszmajjesnzddhv&fileformat=p2p&archiveformat=gz
http://list.iblocklist.com/?list=xpbqleszmajjesnzddhv&fileformat=p2p&archiveformat=gz
http://www.spamhaus.org/drop/edrop.txt
https://sslbl.abuse.ch/blacklist/sslipblacklist.csv
http://osint.bambenekconsulting.com/feeds/c2-ipmasterlist.txt


If you want to use the Firehol level 1 only you could use the same approach with a script to filter out those addresses you don't want.

[EDIT]
added three more lists to my script from this thread
Forgot that the webserver is on port 9080 Fixed now
« Last Edit: November 25, 2017, 11:31:08 pm by seanr22a »

Offline Presbuteros

  • Jr. Member
  • **
  • Posts: 63
  • Karma: +4/-0
    • View Profile
Re: firehol level 1 list blocking LAN resources
« Reply #11 on: November 20, 2017, 02:15:41 am »
I put the resulting IP list and DNSBL list on an external webserver as well currently updated once a week. You have them here: https://dnsbl.dyndns.org/modules/mymod/MyBlocklist.txt and https://dnsbl.dyndns.org/modules/mymod/mydnsblfeed.txt  The DNSBL file is big so it can take a while to download.

The links are not working.

Code: [Select]
Connection timed out after 15039 milliseconds Retry in 5 seconds...
. cURL Error: 28
Connection timed out after 15015 milliseconds Retry in 5 seconds...
. cURL Error: 28
Connection timed out after 15021 milliseconds Retry in 5 seconds...
.. unknown http status code
Download FAIL [ 11/20/17 08:12:54 ]
  Firewall and/or IDS are not blocking download.

The Following list has been REMOVED
 

Thanks for sharing. Let us know when the webserver is working again.

Offline seanr22a

  • Newbie
  • *
  • Posts: 22
  • Karma: +2/-0
    • View Profile
Re: firehol level 1 list blocking LAN resources
« Reply #12 on: November 20, 2017, 03:14:23 am »

The links are not working.


Sorry, forgot that the server is on port 9080. Changed in the original post.

[edit]
Changed to http port 9080
« Last Edit: November 20, 2017, 04:48:37 am by seanr22a »

Offline Presbuteros

  • Jr. Member
  • **
  • Posts: 63
  • Karma: +4/-0
    • View Profile
Re: firehol level 1 list blocking LAN resources
« Reply #13 on: November 20, 2017, 03:24:27 am »
Tried again with updated URL

https://dnsbl.dyndns.org:9443/modules/mymod/MyBlocklist.txt

Code: [Select]
[ Comp ] Downloading update .
**Saving configuration [ 11/20/17 09:21:40 ] ...
 cURL Error: 51
SSL: no alternative certificate subject name matches target host name 'dnsbl.dyndns.org' Retry in 5 seconds...
. cURL Error: 51
SSL: no alternative certificate subject name matches target host name 'dnsbl.dyndns.org' Retry in 5 seconds...
. cURL Error: 51
SSL: no alternative certificate subject name matches target host name 'dnsbl.dyndns.org' Retry in 5 seconds...
.. unknown http status code

Offline seanr22a

  • Newbie
  • *
  • Posts: 22
  • Karma: +2/-0
    • View Profile
Re: firehol level 1 list blocking LAN resources
« Reply #14 on: November 20, 2017, 04:54:41 am »
This got far more complicated than I thought ..... Now I've put up a new small webserver for external access so I don't have to mess around with our production systems for this.

http://dnsbl.dyndns.org:9080/MyBlocklist.txt
http://dnsbl.dyndns.org:9080/mydnsblfeed.txt

I hope everything ok now. I modified my previous posts  :P


Info about all the public available IP and DNSBL lists I'm using
http://dnsbl.dyndns.org:9080/info.txt
« Last Edit: November 20, 2017, 06:06:49 am by seanr22a »

Offline adoucette

  • Jr. Member
  • **
  • Posts: 32
  • Karma: +0/-0
    • View Profile
Re: firehol level 1 list blocking LAN resources
« Reply #15 on: November 24, 2017, 12:53:38 pm »
Thank you seanr22a -- that's very generous of you to compile, filter, host and post. Very generous of iorx as well, above.

I put together a comparison in Excel of the three lists (FireHol lvl 1, iorx's compilation above, and seanr22a's compilation above) and the initial conclusions I draw are:
37% of FireHol Lvl 1's IPs are in iorx's list and 100% are in seanr22a's list
99% of iorx's values are reflected in FireHol's and 65% of the ips are reflected in seanr22a's.
Seanr22a's list may be more comprehensive. It is a 3MB download and has almost 200,000 IPs (the dbsbl list is a 45MB download  :o - may reflect lots of effort on his part).

Excel file attached.

My questions, as a layman, would be:
With the larger list, is there a substantially increased potential for false-positives?
Will this larger list slow down the pfSense box?

Ari
« Last Edit: November 25, 2017, 08:06:52 am by adoucette »

Offline Presbuteros

  • Jr. Member
  • **
  • Posts: 63
  • Karma: +4/-0
    • View Profile
Re: firehol level 1 list blocking LAN resources
« Reply #16 on: November 29, 2017, 02:06:25 am »
Yes, thanks to all who are contributing here.

With the larger list, is there a substantially increased potential for false-positives?
Will this larger list slow down the pfSense box?
Ari

I loaded the list from seanr22a. I took a while to download and compile. It did not "appear" to slow down the pfSense box. My mobo is a Gigabyte GA-J1900N-D3V so a Celeron quad-core 2Ghz and 8 GB of RAM. What I did notice was a lot of issues loading news sites like cnn, foznews, drudgereport, etc. Videos were stalling out and certain essential elements of the page would not load. However, I would like to hear from others on their use of the list and if they had any obvious issues. I simply disabled the list until I have more time to test later.

Offline seanr22a

  • Newbie
  • *
  • Posts: 22
  • Karma: +2/-0
    • View Profile
Re: firehol level 1 list blocking LAN resources
« Reply #17 on: November 29, 2017, 05:43:11 am »
Yes, thanks to all who are contributing here.

I loaded the list from seanr22a. I took a while to download and compile. It did not "appear" to slow down the pfSense box. My mobo is a Gigabyte GA-J1900N-D3V so a Celeron quad-core 2Ghz and 8 GB of RAM. What I did notice was a lot of issues loading news sites like cnn, foznews, drudgereport, etc. Videos were stalling out and certain essential elements of the page would not load. However, I would like to hear from others on their use of the list and if they had any obvious issues. I simply disabled the list until I have more time to test later.

Check the Pfblocker logs for what is blocked related to the sites you visit. You have logs for both DNSBL and IP lists.
As you can see in the info.txt file the lists are made from public available lists maintained by many different people and organizations. Unfortunately there is no list that fits everyone. Simply whitelist those sites that causes you problem, I've done that for many ip's and domains to make it work for me so start dig in to the logs :)

Offline iorx

  • Full Member
  • ***
  • Posts: 144
  • Karma: +4/-0
    • View Profile
Re: firehol level 1 list blocking LAN resources
« Reply #18 on: December 03, 2017, 06:53:54 pm »
I ended up taking BBcan's advice (good advice it always is) and simply put the lists that comprise firehol1 into pfblocker (minus the bogons list of course).

Using the Lvl 1 Feed is going to cause grief... Just don't do it  ;) ;)

Even with Suppression enabled?