Netgate SG-1000 microFirewall

Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Topics - bmeeks

Pages: [1] 2 3 4 5 ... 8
IDS/IPS / Important Info About Passlists with Suricata Inline IPS Mode
« on: March 15, 2018, 04:50:36 pm »
If you use Suricata and have a network card that fully supports Inline IPS Mode, then this notice is important to you so read on.  If you use Legacy Mode blocking,  then none of what is said below applies to you and you can skip it if you desire.

History of Passlists
The idea of passlists sprang from the old Legacy Mode Blocking used in Snort and later in Suricata when that package was created.  Legacy Mode uses a custom plugin that runs within the Suricata binary.  This plugin intercepts every ALERT triggered by the rules.  The plugin extracts the IP addresses from the ALERT (source and destination) and then inserts one or both of them into the snort2c table that lives in the pf (packet filter) firewall engine in pfSense.  Any IP address inserted into that pf table will be subsequently blocked by the firewall.  So that is how Legacy Mode blocking really works.  IP addresses extracted from ALERTS are put into that snort2c table and the pf firewall takes it from there.  Some options were added to the Legacy Mode plugin that allow the user to specify which IP address from the alerting packet to block (SRC, DST or BOTH).  Another requested feature was the abilty to exempt certain IP addresses from ever being blocked even if they triggered an ALERT.  Thus was born the idea of the Passlist.  The Passlist is simply a list of IP addresses or IP network subnets that should never be blocked.  The Legacy Mode plugin consults this list of Passlist IP addresses before sending the IP from an alert packet over to the pf firewall's snort2c table.  If the IP is on the Passlist, then it is not sent over to pf and thus is not blocked.

A passlist was needed in Legacy Mode because once an IP address gets into the snort2c table, all further traffic coming from or going to that IP address is blocked.  This includes even non-malicious traffic.  So "blocked" really means blocked until the address is physically removed from the snort2c table.  This can be done by the user manually, by rebooting the firewall or by means of a cron task that periodically removes addresses from that table.

Why Inline IPS Mode Generally Negates the Need for a Passlist
Inline IPS Mode is a different animal when it comes to blocking traffic.  It does not use the firewall at all.  Instead, it creates a netmap pipe between the NIC driver and the pfSense kernel and pf firewall.  You can visualize this as literally a pipeline that connects these two endpoints.  All network traffic to and from the network interface must go through that pipe.  Suricata sits in the middle of that pipe and acts like a gatekeeper.  If a Suricata rule with the DROP action is triggered, the packet that triggered the rule is literally just dropped by Suricata and not passed on to the other end of the pipe.  So assume a packet was coming from the NIC on the way to the kernel and pf firewall for processing and that packet triggered a Suricata DROP rule; then that packet is just ignored by Suricata and never forwarded on to the kernel and pf firewall for futher processing.  However, if the packet triggers just an ALERT rule or no rule at all; then it is passed on to the kernel and pf firewall for futher processing.

One key point to note here is that this inspection and passing/dropping is done on a packet-by-packet basis.  So since no IP address gets put into the firewall table, you don't have to worry about subsequent non-malicious traffic to or from the same IP host being blocked.  This is an important point!  Bad packets are dropped for an IP address, but good packets pass for that same IP address.  Compare this to the old Legacy Mode where once an IP address was put into the snort2c table all traffic with that IP address in it was blocked (both malicious and non-malicious).  So this difference in the way blocks are handled minimizes the need for a Passlist with Inline IPS Mode.  In fact, in most instances, you should never need or want a Passlist with Inline IPS Mode.

Suricata 4.0.3 Inline IPS Mode and Default Automatic Passlists
Okay, this is the section where I admit making a mistake by allowing myself to be talked into including a default Passlist option when using Suricata Inline IPS Mode.  I added that capability a while back (I think it was when 4.0.1 came out).  Anyway, you have the option at the moment on the INTERFACE SETTINGS tab of choosing "none", "default" or any other user-created Passlist for the Passlist Setting on the interface when using Inline IPS Mode.  The help text associated with the Passlist control recommends you choose "none", and so do I!  Here's why.  If you choose "default", then a potentially bad thing happens.  Some PASS rules get automatically created that effectively whitelist your entire LAN.  By "whitelist" I mean no alerts or drops from anything whose IP address is in your LAN or any other locally-attached firewall networks.  And what's worse, this whitelist extends to traffic flow in both directions:  (1) FROM a device on your LAN to elsewhere; and (2) from elsewhere TO a device on your LAN.  This is not good as bad stuff can flow to/from your LAN hosts and other locally-attached firewall networks and Suricata won't care!  This bad situation came courtesy of the existing default passlist code used for Legacy Mode operation.

Here are some examples of how the automatic default Passlist rules can be bad --

The current "default" passlist includes locally-attached networks if you select that option when creating a pass list (and it is checked by default).  So assume your LAN is the subnet  So Suricata will create this pass rule for the default Passlist when the option to include locally-attached networks is checked:

Code: [Select]
pass ip any <-> any any (msg: "Pass List Entry - allow all traffic to/from"; sid:1000006;
This rule says when the SRC or DST is an IP address in the subnet, then let it pass with no inspection.  Pass rules in Suricata mean just that -- pass the traffic unconditionally with no further inspection.  And pass rules are the very first rules the Suricata engine evaluates.  So this default pass rule is bad (very bad actually).  Some of the other automatic default passlist rules are more restrictive and specify only a single IP address instead of the whole subnet, but even then it still is generally not a good idea to totally whitelist most hosts.

So unless you are totally convinced you need a Passlist with Inline IPS Mode, go right now and set your PassList entry on the INTERFACE SETTINGS tab to "none", save the change and then restart Suricata.  Choosing "none" turns off the automatic passlist.  In the next Suricata GUI package update, the ability to assign and use a passlist will be removed from Inline IPS Mode operation to prevent users from inadvertently neutering their IPS when running Inline IPS Mode.

What If I Really Need a Passlist with Inline IPS Mode?
If you truly do need to whitelist a specific host for some reason (or perhaps more rarely an entire subnet), then you should probably create your own Custom Rules for this and forget about using a passlist.  Custom rules allow you to be more specific.  For example, you could specify an IP, a flow direction and even a port or port range to limit the traffic that bypasses normal Suricata inspection (remember that a passlist when using Inline IPS Mode means passlist addresses get evaluated by no rules; any host or network on the passlist bypasses all Suricata inspection).  The very first test in the Suricata engine is to check if the SRC or DST addresses, ports and protocol match any PASS rules.  If there is a match, the packet is passed on to the netmap pipe endpoint and bypasses all of the other Suricata rules.

So a rule like this is much more limited than the earlier rule:

Code: [Select]
pass ip 80 <- any any (msg: "Pass List Entry - allow all traffic to/from"; sid:1000006;
This rule only passes traffic where the destination is IP address and the port is 80.  Only traffic matching that specific criteria bypasses further Suricata inspection.

To create custom passlist rules like this, go to the RULES tab for the interface, choose CUSTOM RULES in the Category drop-down and then type in the rules you need.  There are plenty of examples on the web.  You can add restrictions by protocol, port and source or destination IP address.  Just really think about what your rule is allowing when creating it.  It's easy to bypass your Suricata protection with a Passlist!  That's why I intend to remove that capability for Inline IPS Mode operation in the next update.  Folks needing to whitelist a host or network can do so with custom PASS rules instead.

Realistically, about the only application I can imagine for a Passlist is if you are running a honeypot host and you actually want bad stuff to find its way to that host.  In that situation, a passlist makes sense.  For about any other case, it does not.  So get ready to see Pass Lists disappear from Suricata in the next version when using Inline IPS Mode, and get prepared to use custom PASS rules instead if you really need passlist functionality.


IDS/IPS / Snort Package Notes
« on: February 06, 2018, 12:34:41 pm »
Users of Snort will notice an update for the package to version  This update only includes a binary fix for users of the Netgate SG-3100 appliance.  There are no GUI changes and no binary changes for Intel x86-based users.

So unless you want to run Snort on the SG-3100 appliance from Netgate, there is no compelling reason to upgrade to the Snort package.


IDS/IPS / Snort Package v3.2.9.6 - Release Notes
« on: January 26, 2018, 08:28:27 am »
Snort Package Update to v3.2.9.6 (binary version

An update for the Snort package has been posted.  The binary is updated to version and the GUI package to version

It is strongly recommended that you install this update by removing the Snort package and then installing it again instead of using the "upgrade" icon.  This is because a couple of the files in the new update will be cached by the PHP process if you simply "upgrade" using the reinstall icon.  The older version of the cached file will be used during the post-install steps and your rules may fail to update properly.  If you remove the package completely and then install it again, there will be no cached files issue.  So long as you have the "Save Settings" checkbox ticked on the GLOBAL SETTINGS tab, your Snort configuration will be retained when removing the package.  That box is checked by default, but if you have ever unchecked it for some reason, be sure to check it before removing the package.

If you read this warning afer you've already tried the reinstall icon, then simply manually update your rules on the UPDATES tab, start Snort if it failed to start after the upgrade, and you should be fine.

This update to the Snort GUI package incorporates six bug fixes and two new features. The GUI package now supports the latest version of Snort. References to the text "Snort VRT rules" within hints, help messages, log entries and titles within the GUI have been changed to read "Snort Subscriber Rules" to align with the naming convention preferred by Talos and Cisco.  This version of Snort runs without crashing on Netgate SG-3100 and similar ARM-based hardware.

New Features
  • Added dynamic updating of service status to INTERFACES tab for Snort and Barnyard2. When starting Snort on an interface, the task is launched as a background job and the GUI monitors the task status to update the icons on the INTERFACES tab.
  • Added support for the new "Max-Detect" IPS Policy available with Snort Subscriber Rules. This new policy is designed mainly for testing purposes as it is maximizes detection (as the name implies) but also raises the number of potential false positives. The new mode is not recommended for production systems!
Bug Fixes
  • FQDN aliases are accepted without flagging an error, but do not process and result in no parts of the alias being used at runtime when an FQDN alias is nested within a normal static IP alias. With the fix, a warning message is printed to the system log and a safe default value is used (if applicable).
  • Bogus gettext() header info displayed on PASS LISTS tab in Alias column when alias is empty.
  • HOME_NET and EXTERNAL_NET custom lists ignore the setting to exclude locally-attached networks.
  • Fix syntax error on RULES tab causing rule status icons to display twice for 'User Force Disabled" rules.
  • Barnyard2 configuration is not properly configured to allow full packet dumps.
  • Modify SNORT_BIN_VERSION constant's calculated value to account for longer version number string in latest Snort binary (such as

IDS/IPS / Snort updates coming soon
« on: January 23, 2018, 03:47:20 pm »
Updates for the Snort binary and GUI package are coming soon.  The binary will update to version and the GUI package will update to  The pull request for the GUI update is posted here:

Official release notes will be posted along with a release announcement when the updates are ready.  Will probably be next week before the requests are approved, the new packages built and then posted to Package Manager.


IDS/IPS / Suricata package version 4.0.3 -- Release Notes
« on: January 12, 2018, 08:01:51 am »
Suricata v4.0.3 Release Notes

This update to the Suricata GUI package contains three bug fixes and five new or updated features. The version is changed to 4.0.3 to match that of the underlying Suricata binary.

New Features:
  • Add the SIP_SERVERS variable to the list of other automatically included rules variables on the VARIABLES tab.
  • Add some help/hint text to the INTERFACE SETTINGS tab in the blocking section to emphasize Inline mode needs rule actions changed to DROP.
  • Add notice to "Clear Blocks Interval" setting on GLOBAL SETTINGS tab that clearing only works for Legacy Mode.
  • Make INTERFACES tab dynamic by periodically polling Suricata binary status and updating status icons for each configured interface.
  • Configure ET-Pro rules download URL path based on full Suricata engine version. See this post on Forum:

Bug Fixes:
  • Barnyard2 start/stop toggle icon not working on INTERFACES tab.
  • Customized HOME_NET passlist ignores option to exclude locally-attached networks.  Redmine bug report #8214.
  • Fix display of bogus header info from gettext() when displaying an "empty" alias on PASS LISTS tab.


IDS/IPS / Suricata GUI package update coming soon
« on: January 11, 2018, 05:28:38 pm »
An update for the Suricata GUI package has been posted for review and approval by the pfSense team.  The new version will be 4.0.3 to match that of the underlying Suricata binary.  The update will include a few minor new features and some bug fixes.  Details are listed in the comments for the Pull Request posted here:

Look for this update to show up in Package Manager in the near future.  Once it is approved by the developers and merged into the RELEASE and DEVELOPMENT branches, I will post up the official release notes.


IDS/IPS / Future deprecation of some Suricata features
« on: December 21, 2017, 09:55:24 am »
The Suricata developers have published a list of deprecated features.  Most of the listed features will disappear from Suricata within the next two years, although some are only one year from removal.  The list and details are here:

The one most likely impact to the pfSense Suricata package is the removal of unified2 binary log outputs.  This is the log format that Barnyard2 depends upon, so once unified2 binary logging is removed from Suricata the Barnyard tab will cease to have any benefit.  So expect Barnyard2 support to disappear in the future and plan accordingly (if you use it).  I suspect most Suricata users instead make use of EVE logging options and feed logs to an external ELK stack for analysis.

For those of you that have asked about CUDA support on pfSense in the past, notice that CUDA support is also being deprecated and will soon be removed from Suricata.


IDS/IPS / Snort - Release Notes
« on: December 05, 2017, 08:28:27 am »

This update to the Snort package changes the URL for downloading the OpenAppID open rules package.  This rules package is maintained by a volunteer contributor and was formerly hosted at a University web site in Brazil.  That web site employs geo-protection, and thus Snort users in some countries were unable to download the OpenAppID rules.  The pfSense team has worked out an agreement with the rules package creator to host the package on pfSense infrastructure.  This will eliminate the geo-blocking issue.  No action is required on your part.  The URL change is automatic.

Note to SG-3100 and other ARM hardware users:
This package update does not address the problems within the Snort binary on ARM CPUs.  We are still trying to fix that issue.


IDS/IPS / Suricata Package 4.0.1_1 - Release Notes
« on: December 05, 2017, 08:21:54 am »
Suricata 4.0.1_1

This update to Suricata makes a change to the URL used for downloading ET-Pro and ET-Open rules packages.  The change was necessitated by a new folder structure on the Proofpoint web site.  Proofpoint now has tailored rules for different Suricata versions.

There are no other changes in this Suricata update.


IDS/IPS / Suricata Package 4.0.1 Update - Release Notes
« on: November 29, 2017, 10:25:49 am »
Suricata v4.0.1 Update

The Suricata package has been updated to the latest upstream binary version 4.0.1.  There are no GUI package changes in this release.  All the bug fixes are within the underlying Suricata binary code.  Details on Suricata version 4.0.1 can be found here:


IDS/IPS / Snort GUI Package v3.2.9.5_1 Update - Release Notes
« on: September 14, 2017, 03:36:32 pm »
Snort GUI Package

This update fixes the XMLRPC package sync code so that it works in both 2.3 and 2.4 versions of pfSense.

New Features

Bug Fixes

1.  XMLRPC package sync does not work in pfSense 2.4-RC due to changes in the XMLRPC client.

IDS/IPS / Snort Package v3.2.9.5 Update -- Release Notes
« on: August 23, 2017, 04:30:07 pm »
Snort v3.2.9.5 -- Release Notes

An update for the Snort package has been posted.  There is both a GUI code update and an underlying binary bug fix.

I strongly suggest that the best way to upgrade the Snort package is to first uninstall it from the Package Manager, and then reinstall it.  You won't lose any settings so long as you have the "Save Settings on Deinstall" checkbox on the GLOBAL SETTINGS tab checked.  It is checked by default.

New Features

The ARP Spoofing preprocessor is now exposed in the GUI as a configurable option on the PREPROCESSORS tab. Settings are available for toggling the enabled state of the preprocessor and for enabling detection of unicast ARP requests. A multi-entry table is provided that allows for entry of MAC address-to-IP address pairs to monitor for ARP spoofing incidents. New MAC/IP address pairs can be added to the table and existing MAC/IP pairs can be edited or deleted from the table.

Bug Fixes

  • Fix display of IPv6 addresses so they wrap correctly when displayed in the SRC IP and DST IP columns on the ALERTS tab.
  • Fix the DOWNLOAD button on the ALERTS and BLOCKS tabs so it works. Also fix the ALERTS download so that it only includes alert logs and not all log files in the directory.
  • Restore the installation of the attribute_table.dtd validation file to the Snort conf directory.
  • Do not show Reverse DNS Lookup and Track-by-IP icons on the ALERTS tab for alert entries that do not contain an IP Header and thus have no IP addresses (usually from alerts generated by the ARP Spoof preprocessor).
  • Improve package uninstall procedure by manually cleaning up files created or altered by the GUI code. The default pkg uninstall code does not remove files modified by others.
  • Remove the shared object rules files when uninstalling the package. This should fix errors during package upgrades when older versions of these files still exist.
  • Add a warning to the OpenAppID rules file download section on the GLOBAL SETTINGS tab about Geo-IP blocking at the volunteer hosting web site. This hosting block may impact users in some countries when they attempt to enable the OpenAppID rules download. (NOTE - this is for the rules only. The OpenAppID detectors are maintained by the Snort VRT and should always download and install. However, the detectors need the rules in order to be fully functional.)
  • Fix deletion of multi-configuration engines for preprocessors that support them such as Frag3, Stream5, HttpInspect, FTP and ARP Spoof.
  • Correct a bug in the custom blocking plugin for the Snort binary. Failure to validate that the IP header information in an alerting packet is valid before attempting to insert the IP addressess into the "snort2c" table could cause Signal 11 faults in the Snort binary. Certain alerts, particularly from the ARP Spoofing detection preprocessor, may not contain valid IP header information. The IP header information is now validated before attempting to insert the addresses into the "snort2c" table. Alert packets with empty or invalid IP header information are now ignored by the blocking plugin since it could not do anything useful with them anyway.


IDS/IPS / Snort GUI Package Update On the Way
« on: August 21, 2017, 10:10:43 pm »
An update for the Snort GUI package has been posted for review and approval by the pfSense developer team.  This update adds support for the ARP Spoofing Detection preprocessor on the PREPROCESSORS tab and fixes several user-reported bugs in the GUI code.  Details can be found in the Pull Request posted here:

When this request is approved and merged, you should see an update available for Snort in the Package Manager to version


IDS/IPS / About Pass Lists in Suricata
« on: August 17, 2017, 08:09:30 am »
There seem to always be some questions and/or misconceptions about how Pass Lists work in Suricata.  That's especially true now that Suricata offers two quite different IPS modes:  (1) Legacy Mode and (2) Inline IPS.

Quick Review of Legacy vs Inline IPS Modes
Legacy Mode operation uses a custom output module I wrote that gets patched into the Suricata binary when the package is built for pfSense.  The custom module is called alert-pf.  Suricata Legacy Mode on pfSense uses the libpcap library to capture network packets as they traverse the firewall.  Those copied packets are analyzed by Suricata to determine if alerts should be generated.  The alert-pf module gets a copy of every alert event generated by the Suricata engine.  It pulls the IP addresses from the alert event and makes a FreeBSD system call to insert them into a packet filter (pf) firewall table called "snort2c".  That table is created at boot-up by the pfSense code.  It always exists on a pfSense firewall whether Suricata and Snort are installed or not.  IP addresses placed in that pf table are blocked by the firewall.  Removing an IP address from that table clears the "block".  Because the Legacy Mode process is using copies of the packets to make decisions, there is some leakage of data through the firewall before a block/no-block decision is made.

The new Inline IPS mode solves the leakage problem by making use of a new ability within FreeBSD called Netmap.  Netmap is a mechanism allowing very high-speed processing of network packets.  It basically creates an intercept point between the network hardware driver and the FreeBSD kernel.  Packets going to and from the network driver must pass through the Netmap conduit.  In order for this conduit to work properly, there must be support within the network driver.  Thus Netmap does not work for just any network card.  It depends on some driver implementation support.  If that support is missing or incomplete, then Netmap will either not work at all, or will work in an extremely buggy manner all the way up to crashing the kernel.  Netmap support within Suricata itself is provided from upstream (meaning it comes baked into the binary source code already).  No patching of any kind is done on the pfSense side in order to implement Netmap in Suricata.  It is just enabled at compile time.  This is different from the Legacy Mode custom blocking module mentioned earier.

Pass List Operation
A pass list is just another term for "whitelist".  I chose Pass List so as not to get things confused with the whitelisting function within the Snort IP Reputation preprocessor.  A pass list is simply a collection of IP addresses that are never to be blocked.  The IP addresses can be for individual hosts, or entire CIDR blocks can be defined using the standard syntax supported by the underlying IDS/IPS engine (either Suricata or Snort).  This next sentence is important!  Only Legacy Mode blocking operation supports a true Pass List.  This is because Legacy Mode uses the custom plugin I created, and that plugin understands what a pass list is for and how to use it.  It consults the pass list right before inserting an IP address in that "snort2c" table mentioned above.  If the IP is on the pass list, then it is not put in the table and thus no block happens.

A pass list only works with static IP addresses.  Go back and read that sentence again -- only static IP addresses work in a pass list.  FQDN Aliases or any other kind of dynamic IP address is not suitable for a pass list.  That's because the list is static in memory once created, and it's only created at startup of Suricata.  So it can't know when a host in the pass list gets a new address.  It won't follow FQDN aliases that auto-update.  I've been looking at some options to maybe work with FQDN aliases in a pass list, but there is a fine line to tread before performance of the blocking plugin degrades and packet processing speed would drop.

Pass lists are not supported when Inline IPS mode is enabled because that mode uses the built-in Netmap functionality of Suricata, and the native Netmap code is not patched to recognize and work with pass lists.  So that's why they don't work using Inline IPS mode.  Making them work with Inline IPS mode would require yet more custom patches be created and applied to Suricata when built for pfSense.  We are trying to get away from that kind of customization because it naturally makes keeping in sync with upstream releases more difficult.  There have been several occasions where upstream releases have broken the patch for the Legacy Mode plugin, and time has to be allocated and spent to recode the patch to work with newer Suricata versions.

To mimic Pass List functionality when using Inline IPS mode, you should create custom PASS rules.  These are rules that have PASS as the action keyword instead of ALERT or DROP.  The configuration of Suricata on pfSense ensures that PASS rules are always evaluated first, and the first matching rule wins (just as with the firewall rules).  Some examples can be found with a Google search or at the Suricata documentation Wiki site here:


IDS/IPS / About the New Block-on-Drops Only Option in Suricata 4.0.0
« on: August 17, 2017, 07:33:28 am »
Version 4.0.0 of the Suricata package contains a new option on the INTERFACE SETTINGS tab in the section where blocking is configured.  The new option is called Block Drops Only.  It applies only to Legacy Blocking Mode operation.  If you use the Inline IPS mode for blocking, then this new option is not used and is in fact hidden by the GUI code when Inline IPS mode is enabled.

The new option allows Legacy Blocking Mode users the same flexibility with rule actions as Inline IPS Mode users enjoy.  That would be the ability to use the SID MGMT tab options to turn individual rules (or entire rule categories) from ALERT to DROP action.  So now with the new option enabled, rules that have the ALERT action keyword (the first word of the rule signature text) will only generate alerts in the log on the ALERTS tab but no blocks.  Only rules with the DROP action keyword will generate blocks that show up on the BLOCKS tab.  You have to specifically enable this behavior by checking the box to enable this new option on the INTERFACE SETTINGS tab.  Then you will need to restart Suricata for the change to take effect.  If you leave this new option unchecked, which is the default, then Legacy Mode operation continues to function as it always has whereby every rule firing will generate a block (assuming the IP address of the offender is not in a Pass List configured on the interface).


Pages: [1] 2 3 4 5 ... 8