Netgate SG-1000 microFirewall

Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Messages - Locked

Pages: [1] 2
I have successfully done this entirely with aliases and pfsense rules only, on specific LAN interfaces. It is a very time consuming task to do this effectively by just using "Firewall Rules".

To be successful, you must be running Wireshark on a workstation on the LAN. Set the filter in Wireshark to DNS only, and resolve names while you browse the target website.  Observe ALL the coincident domains and Content Distribution Service providers ( CDN networks) needed to deliver the target website (Akamai, Fastly, ....)

Smaller independent sites are relatively easy to isolate. Anything hosted on AWS is virtually impossible isolate.

In some cases the best solution is to derive IP lists in pfblockerng using the ASN lookup feature to create "Aliase permit" list rules which  you can refer to from the firewall configuration screens. For example Facebook has its own ASN so its very easy to filter it either by blocks or permits. (ASN = Autonomous System Number)

Anyway that is the concept methodology in broad terms, to achieve your objective by just using "Firewall Rules". It's as much as I can help you with

Good luck !!

Hmmm  to be specific regarding Sync and my import export dilemma

.... I need to take portions of a huge Old config, port it to the new machine and then modify it. A lot of it is IPv4 ASN lookups for outbound rules

This is more of a one time replication of portions of an old config, which does have unknown misconfigurations in it that I dont want to propagate to the new machine.

I dont think this going to be possible

You are truly a hero Ron !
Thank you

I have been scolded by Netgate support for entering FQDNs into the TLD Blacklist - though it does work

Creating a DNSBL feed with custom a FQDNs list that never needs updating will greatly reduce the overhead on Unbounds workload during the pfBlockerNG Update process.

The unbound configuration doesn't need to know anything other than the FQDNs to ignore.

TLD Blacklist probably has to do many lookups to create the lists for Unbound

On another issue - how do I port pfBlockerNG config to another machine - there is zero capacity for pfBlockerNG in Backup/Restore under Diagnostics

DHCP and DNS / Re: Unbound DNS intermittent failure
« on: Today at 11:32:07 am »
I think I may have stumbled upon something in the ISP modem config that could be causing this, though the times are different than the pfSense 5 minute issues.
In the IP-passthrough page, there is a Passthrough DHCP Lease. Default value is 10 minutes.  I changed to 1 day, hopefully this is the root cause and will fix things.

FYI, the modem is this one:

Manufacturer   ARRIS
Model Number   BGW210-700

I have many of the problems discussed here on this thread and also an ARRIS modem on a poor signal quality cable ISP connection.
Maybe we can share remedies and results

Some of the steps to remedy the situation I have taken are extreme for the time being:

Removed as many FQDNs from my firewall rules aliases tables as possible and used specific IP #'s instead
Disabled CRON automatic updates in pfblockerng (with 2 TLD Blacklist entries)
Disabled Gateway Pinger
Disabled Gateway monitoring "Action"
Disabled default blocks on RFC 1918 on WAN - my ISP uses 192.168.0 to establish DHCP
Defined about 7 or 8 public resolvers, including the ISP assigned ones for Unbound to forward Queries to

I am not happy about having to do any of this but perhaps all I need to do is disable gateway monitoring action on WAN to prevent all the subsequent issues cause by unbound restarting

How did you get into the ARRIS to increase the length of DHCP leases ?
My solution was to spoof a fixed IP config in the WAN interface - which seems to work for a while but I have backed that out as a solution

Perhaps if we studied the WAN DHCP client Advanced options in pfSense there might be something there of value to us ? I don't know much about what is listed there as of now.

Solved the problem - it seems when I started looking at others peoples problems and offering suggestions I saw mine in a new light

I had Alexa TLD exclusions selected - several of them
I removed all exclusions and TLD is back working just fine

Though I will be putting custom FQDN's I want to block into the proper category - DNSBL Feeds -  from now on

To summarize - packet loss was the first issue + configuration error on my part the second

pfBlockerNG / Re: DNSBL not working, easylist works
« on: Yesterday at 08:14:14 pm »
Turn on Global logging, for debugging purposes, and try inbound and outbound on just the LAN for starters

Choose the 2nd or 3rd "Rule Order" option so all your firewall pass rules are evaluated first

pfBlockerNG rules on the WAN can complicate updates

I prefer floating rule sets

my 2 cents

Specify "Floating Rules" under general setup

Thanks for the suggestion, but this was the first place I looked. 

It is an error message about the filter reload - I think I may have lost the message by re-sizing the system.log, before I realized the importance of the message about the filter error bad characters. Which I am writing off as an error due to packet loss creating download file corruption. Something I saw a lot of that day, bad cable connection.

I am still having other problems with pfBlockerNG including being totally unable to use TLD blacklist with normal TLD's such as CN, RU - etc bizarre errors such as

/var/unbound/pfb_dnsbl.conf:5: error: unknown keyword '.cn'
/var/unbound/pfb_dnsbl.conf:5: error: unknown keyword  '60'

when I force an update

I know TLD Blacklist is meant for TLD's only, but it used to work for any FQDN as well - I understand now that Custom DNSBL Lists for FQDNs are where I do that.

I have also tried completely removing and then installing pfBlockerNG to no avail. This module has been very problematic for me recently, I don't know why. I ditched 2 years worth of config and have started from scratch, but still getting the above stated unknown keyword error is frustrating

So now TLD Blacklist  wont work for anything. --  Going to reflash the entire device, since I do have the USB for that already made, and start all over again ..... with a restore.

Need to re-size the swap file anyway

Installation and Upgrades / Re: All pass rules appear disabled
« on: March 19, 2018, 05:10:32 pm »
To summarize and conclude PACKET LOSS issues caused 2 of 3 problems

I now realize that all instances of filter failure (except filesystem full (#3)) can be attributed to packet loss at my router which is a residential cable connection.  It's been an issue here for years. Gateway pinger has been documenting it very well.

In one instance of filter failure (#1)We discovered intermittent DNS failure to resolve names which populate aliases used to evaluate pass rules. The evaluation fails because the alias has a null value and the rule "appears to be ignored" - I am now 100% certain this intermitent DNS failure has been caused by packet loss.  This is my original condition and the basic premise for this thread.

In the second instance of filter failure after downloading pfb_NAmerica GeoIP datasets by maxmind, the error message before filter failure was something to  the effect of "bad characters in ..." and the application of the new block rules failed because of that, which took out the entire set of firewall rules. Once again I will attribute this to packet loss creating a faulty download and subsequent IP data set applied to the filter crashed it.

I feel Negate Support should be compensated for the time which has been spent for me to arrive at this conclusion (and inspire me to program my new firewall rules to avoid future problems)

James or Steve please get in touch and lets agree on an invoice amount to be paid, I believe in fairness and healthy client / provider relationships - Netgate got the short end of the straw here when they stepped up to the plate

Installation and Upgrades / Re: All pass rules appear disabled
« on: March 19, 2018, 01:38:55 pm »

Today another Filter corruption / failure / error / hack has occurred
Earlier today, all filter rules were dropped. And this time there are clues.

I've posted more specific pfBlockerNG questions here

So bearing in mind that this most recent Filter Failure occurred with a pfBlockerNG configuration which solely consisted of GEOIP blocks ... NO DNSBL entries what so ever.

There has been a revelation - an error on my part:
In all previous versions of pfBlockerNG - TLD Blacklist - I discovered I was able to enter FQDN's as well as TLD's, so I kept doing so unaware that FQDN's can be entered as a custom block list under DNSBL Feeds.  OK - everything worked fine until recent versions of pfBlockerNG corrected the ability to make non TLD entries in the TLD Blacklist.

So the conclusion is that historical mis-configurations of the TLD Blacklist (FQDNS) under newer versions crash the FILTER or wipe it completely. Netgate has closed the support on "all pass rules dropped" mystery.  Fair enough.

EXCEPT refer back to item 1) which is not explained by 2)

So the next instance of "all filter rules being dropped" - Which as also happened to me during a "File system Full" incident - I will gladly open a PAID INCIDENT support ticket.  Because we still have not got to the bottom of this. But I now believe it is a BUG and not a HACK, which has not yet been resolved.

Thanks very much to James and Steve for the free support, I've got no problem with pulling out the credit card the next time my filter blows up. And the incident will be titled "Filter blows up"

Installation and Upgrades / Re: All pass rules appear disabled
« on: March 18, 2018, 02:17:13 pm »
Good advice, but I have Debian so hardened that the enemy has no choice but to go after the firewall. Using Same methodology, I believe they attack from the inside with altered scripts disguised as updates.

The last two Debian Kernels have been Excellent in terms of security, and I have customized my own Firefox apparmor profile, plus utterly destroyed any ability to add extensions or pluging's to firefox from the system level. I love GUFW, very simple to use.  I'd like to spend time hardening sysctl.config for my own purposes but can't find the time to do so

Yes I hear there are a lot of complaints about systemd but my beef is with the TLD root servers, rogue NOCTION IP BGP attacks, and AKAMAI CDN IP mappings to Japan and Honk Kong from Vancouver.

Debian is back in the game and we await your return webtyro !

But I'll consider OpenBSD for my dedicated pfSense administrators workstation

Firstly pfBlockerNG is very powerful and I wish to commend the developer for his work. I don't see how one person is able to manage development and maintenance of such a package for free ...

I just had an issue where I added North America IPv4 selections from GeoIP and the update process completed successfully.  Moments later a screen notification flashed on the WEBGui indicating bad characters in the NAmerica filter set. Then I looked at the rules tables for all interfaces and they had vanished.

Unfortunately I did not pay enough attention at the time to message, and can't find the log file it would reside in under /var/log on pfSense

So my questions are how are we validating downloaded data sets for integrity ?  Would a form of sand boxing be beneficial ? 
Does MaxMind provide a checksum file for downloads so a developer could easily run hashes on the downloads as an initial integrity check. 
Is there a undiscovered bug in the update filter command ?  Is it possible to turn on verbose logging for the filter update command and log it somewhere ?

Thanks in Advance

Installation and Upgrades / Re: All pass rules appear disabled
« on: March 18, 2018, 01:10:35 am »
Thanks very much to Netgate Global support for the assistance but we were unable to to fully reproduce the error or find evidence of a vector - So I am back to community support for assistance

Today another Filter corruption / failure / error / hack has occurred

Earlier today, all filter rules were dropped. And this time there are clues.

How do I find the log file that has the record of notifications which are displayed in the upper right hand corner of the WEBGUI ?

I've posted more specific pfBlockerNG questions here

Installation and Upgrades / Re: All pass rules appear disabled
« on: March 04, 2018, 02:55:04 pm »

 I just sent in now, a bunch of log files I pulled off of the firewall last week under the correct ticket ID this morning.

Additional analysis and a suggestion from me regarding a new piece of companion hardware to deal with this problem, which will compliment the Netgate suite of solutions,  will be emailed to support in the next day or two .

It's impossible for me to sufficiently express my appreciation regarding Netgates response to this issue

Thanks very much

Pages: [1] 2