pfSense Gold Subscription

Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Topics - bmeeks

Pages: [1] 2 3 4 5 ... 8
IDS/IPS / Snort - Release Notes
« on: December 05, 2017, 08:28:27 am »

This update to the Snort package changes the URL for downloading the OpenAppID open rules package.  This rules package is maintained by a volunteer contributor and was formerly hosted at a University web site in Brazil.  That web site employs geo-protection, and thus Snort users in some countries were unable to download the OpenAppID rules.  The pfSense team has worked out an agreement with the rules package creator to host the package on pfSense infrastructure.  This will eliminate the geo-blocking issue.  No action is required on your part.  The URL change is automatic.

Note to SG-3100 and other ARM hardware users:
This package update does not address the problems within the Snort binary on ARM CPUs.  We are still trying to fix that issue.


IDS/IPS / Suricata Package 4.0.1_1 - Release Notes
« on: December 05, 2017, 08:21:54 am »
Suricata 4.0.1_1

This update to Suricata makes a change to the URL used for downloading ET-Pro and ET-Open rules packages.  The change was necessitated by a new folder structure on the Proofpoint web site.  Proofpoint now has tailored rules for different Suricata versions.

There are no other changes in this Suricata update.


IDS/IPS / Suricata Package 4.0.1 Update - Release Notes
« on: November 29, 2017, 10:25:49 am »
Suricata v4.0.1 Update

The Suricata package has been updated to the latest upstream binary version 4.0.1.  There are no GUI package changes in this release.  All the bug fixes are within the underlying Suricata binary code.  Details on Suricata version 4.0.1 can be found here:


IDS/IPS / Snort GUI Package v3.2.9.5_1 Update - Release Notes
« on: September 14, 2017, 03:36:32 pm »
Snort GUI Package

This update fixes the XMLRPC package sync code so that it works in both 2.3 and 2.4 versions of pfSense.

New Features

Bug Fixes

1.  XMLRPC package sync does not work in pfSense 2.4-RC due to changes in the XMLRPC client.

IDS/IPS / Snort Package v3.2.9.5 Update -- Release Notes
« on: August 23, 2017, 04:30:07 pm »
Snort v3.2.9.5 -- Release Notes

An update for the Snort package has been posted.  There is both a GUI code update and an underlying binary bug fix.

I strongly suggest that the best way to upgrade the Snort package is to first uninstall it from the Package Manager, and then reinstall it.  You won't lose any settings so long as you have the "Save Settings on Deinstall" checkbox on the GLOBAL SETTINGS tab checked.  It is checked by default.

New Features

The ARP Spoofing preprocessor is now exposed in the GUI as a configurable option on the PREPROCESSORS tab. Settings are available for toggling the enabled state of the preprocessor and for enabling detection of unicast ARP requests. A multi-entry table is provided that allows for entry of MAC address-to-IP address pairs to monitor for ARP spoofing incidents. New MAC/IP address pairs can be added to the table and existing MAC/IP pairs can be edited or deleted from the table.

Bug Fixes

  • Fix display of IPv6 addresses so they wrap correctly when displayed in the SRC IP and DST IP columns on the ALERTS tab.
  • Fix the DOWNLOAD button on the ALERTS and BLOCKS tabs so it works. Also fix the ALERTS download so that it only includes alert logs and not all log files in the directory.
  • Restore the installation of the attribute_table.dtd validation file to the Snort conf directory.
  • Do not show Reverse DNS Lookup and Track-by-IP icons on the ALERTS tab for alert entries that do not contain an IP Header and thus have no IP addresses (usually from alerts generated by the ARP Spoof preprocessor).
  • Improve package uninstall procedure by manually cleaning up files created or altered by the GUI code. The default pkg uninstall code does not remove files modified by others.
  • Remove the shared object rules files when uninstalling the package. This should fix errors during package upgrades when older versions of these files still exist.
  • Add a warning to the OpenAppID rules file download section on the GLOBAL SETTINGS tab about Geo-IP blocking at the volunteer hosting web site. This hosting block may impact users in some countries when they attempt to enable the OpenAppID rules download. (NOTE - this is for the rules only. The OpenAppID detectors are maintained by the Snort VRT and should always download and install. However, the detectors need the rules in order to be fully functional.)
  • Fix deletion of multi-configuration engines for preprocessors that support them such as Frag3, Stream5, HttpInspect, FTP and ARP Spoof.
  • Correct a bug in the custom blocking plugin for the Snort binary. Failure to validate that the IP header information in an alerting packet is valid before attempting to insert the IP addressess into the "snort2c" table could cause Signal 11 faults in the Snort binary. Certain alerts, particularly from the ARP Spoofing detection preprocessor, may not contain valid IP header information. The IP header information is now validated before attempting to insert the addresses into the "snort2c" table. Alert packets with empty or invalid IP header information are now ignored by the blocking plugin since it could not do anything useful with them anyway.


IDS/IPS / Snort GUI Package Update On the Way
« on: August 21, 2017, 10:10:43 pm »
An update for the Snort GUI package has been posted for review and approval by the pfSense developer team.  This update adds support for the ARP Spoofing Detection preprocessor on the PREPROCESSORS tab and fixes several user-reported bugs in the GUI code.  Details can be found in the Pull Request posted here:

When this request is approved and merged, you should see an update available for Snort in the Package Manager to version


IDS/IPS / About Pass Lists in Suricata
« on: August 17, 2017, 08:09:30 am »
There seem to always be some questions and/or misconceptions about how Pass Lists work in Suricata.  That's especially true now that Suricata offers two quite different IPS modes:  (1) Legacy Mode and (2) Inline IPS.

Quick Review of Legacy vs Inline IPS Modes
Legacy Mode operation uses a custom output module I wrote that gets patched into the Suricata binary when the package is built for pfSense.  The custom module is called alert-pf.  Suricata Legacy Mode on pfSense uses the libpcap library to capture network packets as they traverse the firewall.  Those copied packets are analyzed by Suricata to determine if alerts should be generated.  The alert-pf module gets a copy of every alert event generated by the Suricata engine.  It pulls the IP addresses from the alert event and makes a FreeBSD system call to insert them into a packet filter (pf) firewall table called "snort2c".  That table is created at boot-up by the pfSense code.  It always exists on a pfSense firewall whether Suricata and Snort are installed or not.  IP addresses placed in that pf table are blocked by the firewall.  Removing an IP address from that table clears the "block".  Because the Legacy Mode process is using copies of the packets to make decisions, there is some leakage of data through the firewall before a block/no-block decision is made.

The new Inline IPS mode solves the leakage problem by making use of a new ability within FreeBSD called Netmap.  Netmap is a mechanism allowing very high-speed processing of network packets.  It basically creates an intercept point between the network hardware driver and the FreeBSD kernel.  Packets going to and from the network driver must pass through the Netmap conduit.  In order for this conduit to work properly, there must be support within the network driver.  Thus Netmap does not work for just any network card.  It depends on some driver implementation support.  If that support is missing or incomplete, then Netmap will either not work at all, or will work in an extremely buggy manner all the way up to crashing the kernel.  Netmap support within Suricata itself is provided from upstream (meaning it comes baked into the binary source code already).  No patching of any kind is done on the pfSense side in order to implement Netmap in Suricata.  It is just enabled at compile time.  This is different from the Legacy Mode custom blocking module mentioned earier.

Pass List Operation
A pass list is just another term for "whitelist".  I chose Pass List so as not to get things confused with the whitelisting function within the Snort IP Reputation preprocessor.  A pass list is simply a collection of IP addresses that are never to be blocked.  The IP addresses can be for individual hosts, or entire CIDR blocks can be defined using the standard syntax supported by the underlying IDS/IPS engine (either Suricata or Snort).  This next sentence is important!  Only Legacy Mode blocking operation supports a true Pass List.  This is because Legacy Mode uses the custom plugin I created, and that plugin understands what a pass list is for and how to use it.  It consults the pass list right before inserting an IP address in that "snort2c" table mentioned above.  If the IP is on the pass list, then it is not put in the table and thus no block happens.

A pass list only works with static IP addresses.  Go back and read that sentence again -- only static IP addresses work in a pass list.  FQDN Aliases or any other kind of dynamic IP address is not suitable for a pass list.  That's because the list is static in memory once created, and it's only created at startup of Suricata.  So it can't know when a host in the pass list gets a new address.  It won't follow FQDN aliases that auto-update.  I've been looking at some options to maybe work with FQDN aliases in a pass list, but there is a fine line to tread before performance of the blocking plugin degrades and packet processing speed would drop.

Pass lists are not supported when Inline IPS mode is enabled because that mode uses the built-in Netmap functionality of Suricata, and the native Netmap code is not patched to recognize and work with pass lists.  So that's why they don't work using Inline IPS mode.  Making them work with Inline IPS mode would require yet more custom patches be created and applied to Suricata when built for pfSense.  We are trying to get away from that kind of customization because it naturally makes keeping in sync with upstream releases more difficult.  There have been several occasions where upstream releases have broken the patch for the Legacy Mode plugin, and time has to be allocated and spent to recode the patch to work with newer Suricata versions.

To mimic Pass List functionality when using Inline IPS mode, you should create custom PASS rules.  These are rules that have PASS as the action keyword instead of ALERT or DROP.  The configuration of Suricata on pfSense ensures that PASS rules are always evaluated first, and the first matching rule wins (just as with the firewall rules).  Some examples can be found with a Google search or at the Suricata documentation Wiki site here:


IDS/IPS / About the New Block-on-Drops Only Option in Suricata 4.0.0
« on: August 17, 2017, 07:33:28 am »
Version 4.0.0 of the Suricata package contains a new option on the INTERFACE SETTINGS tab in the section where blocking is configured.  The new option is called Block Drops Only.  It applies only to Legacy Blocking Mode operation.  If you use the Inline IPS mode for blocking, then this new option is not used and is in fact hidden by the GUI code when Inline IPS mode is enabled.

The new option allows Legacy Blocking Mode users the same flexibility with rule actions as Inline IPS Mode users enjoy.  That would be the ability to use the SID MGMT tab options to turn individual rules (or entire rule categories) from ALERT to DROP action.  So now with the new option enabled, rules that have the ALERT action keyword (the first word of the rule signature text) will only generate alerts in the log on the ALERTS tab but no blocks.  Only rules with the DROP action keyword will generate blocks that show up on the BLOCKS tab.  You have to specifically enable this behavior by checking the box to enable this new option on the INTERFACE SETTINGS tab.  Then you will need to restart Suricata for the change to take effect.  If you leave this new option unchecked, which is the default, then Legacy Mode operation continues to function as it always has whereby every rule firing will generate a block (assuming the IP address of the offender is not in a Pass List configured on the interface).


IDS/IPS / Suricata GUI Package Version 4.0.0 Release Notes
« on: August 17, 2017, 07:02:25 am »
Suricata Package v4.0.0 Release Notes

This updates the GUI package to version 4.0.0 to sync with the new upstream binary version. There are several bug fixes and one new feature in this update.

New Features
  • The custom alert-pf plugin utilized for Legacy Mode blocking operation has a new option to cause blocking of host IP addresses only when the offending IP address is detected in a rule with the DROP action keyword. This option is now exposed in the GUI on the INTERACE SETTINGS tab. It is "off" by default, which results in the legacy behavior of blocking for all rules generating alerts. When the new option is enabled in the GUI, only rules having the DROP action will result in blocks being inserted in the packet filter firewall for the offending IP address. IP addresses contained within a Pass List will never be blocked in Legacy Blocking mode.

    Check out this Sticky Topic for more information:

Bug Fixes
  • Icon on ALERTS tab for removing blocked hosts not working, Bug #7578.
  • Log rotation and cleanup code in suricata_check_dir_size_limit() needs to be improved. Bug #7756. Thank you to user Orion Poplawski for the patch to fix this!
  • Icon for download of alert logs on ALERTS tab actually downloads all log files and not just alerts.


An update for the Suricata binary to the latest 4.0.0 version has been posted as a pull request for the pfSense developer team to review and approve.  This update includes a new feature for Legacy Mode operation.  Details can be found in the pull request here:

The new feature will allow Legacy Mode blocking users to have the option of blocking traffic only for rules with the DROP action keyword in the rule's signature.  Rules with ALERT as the action will just alert and not block.  This will be a configurable option within the GUI.  The default will be the existing Legacy Mode behavior where any ALERT rule firing will cause a block of the offending IP address (if the offender is not a pass-listed host).

For users capable of using the new inline IPS mode, and that have that mode enabled, this new option is not used.  Inline IPS mode already uses only the rule signature's action keyword to determine when to drop packets.

An update to the GUI package will be coming soon as well to enable access to this new option.  The updated GUI package will also feature some much needed enhancements to the log rotation code submitted by user @opoplawski over on Github.


IDS/IPS / Snort Update - Release Notes
« on: May 26, 2017, 06:43:53 pm »
Snort GUI Package Update - Release Notes

This update for the Snort GUI package implements support for the latest version of the Snort binary. Release Notes for Snort can be found here:

Note: some users are reporting a failure to start due to a probable syntax error in an Emerging Threats Exploit rule.  This is not caused by a problem with the Snort update.  It is an issue in the ET Open Exploit rules category.  I suspect the ET guys will get it sorted out soon.

Thanks to @pfcode for identifying the errant rule.  Here is his post:  Here is my post explaining how to interpret the error message in order to find the rule on your own:  Each person seeing the error will probably have a different line number reported.  This is because where the errant rule is located within the snort.rules file depends on how many rules you have enabled in your configuration.  That snort.rules file contains all of your "enabled" rules.  The path to the rules file contains your physical interface name along with a UUID number, so the path in your error message is going to be different.  If you are impacted by the error, simply disable the rule using either the icons on the RULES tab or via the configuration on the SID MGMT tab.  The SID of the rule is given in @pfcode's post (following the link given earlier).

GUI Package New Features:

GUI Package Bug Fixes:

When using the download buttons on the ALERTS, BLOCKED and SID MGT tabs, the downloaded files either have HTML appended to them (if downloading individual files) or when downloading a gzip archive it shows as corrupt on Windows.

Redmine Bug #7555, translation data shown in breadcrumb link when no interfaces are defined for Snort and one of the interface settings tabs is selected.

IDS/IPS / FYI -- Snort update to the binary is coming soon
« on: May 23, 2017, 07:15:10 am »
Just FYI guys.  I will be updating Snort to the latest binary in the coming days.  It will be available for the 2.4-BETA tree first, and then a bit later in the 2.3.4 release and 2.3.5 snapshot trees.

The update to was posted to FreeBSD ports yesterday, so that is my trigger to then update the pfSense package.


IDS/IPS / Suricata 3.2.1 Package Update -- Release Notes
« on: April 12, 2017, 01:33:23 pm »
Suricata 3.2.1 Package Update

This updates the Suricata package on pfSense to version 3.2.1. The underlying Suricata binary is also versioned up to 3.2.1.  This update is initially available for pfSense 2.4-BETA snapshots, but will become available for the 2.3.x Release versions of pfSense shortly. Some Suricata GUI configuration parameters were changed as a result of the update.  See the Release Notes below for details.

Important Upgrade Information
The recommended way to upgrade the Suricata package is to first remove it and then reinstall it.  This bypasses any caching that may occur with the PHP code files.  This is particularly important for this update as some suricata.yaml configuration parameters have changed and must be migrated to new values during the installation of the package.  Removing and then reinstalling the package ensures the latest PHP code files are used to perform the migration of impacted configuration parameters.

Release Notes:

1. Suricata 3.2.1 now supports hyperscan for the pattern matcher algorithm. Hyperscan is a high-performance regex pattern matching library. Several older pattern matching algorithms were deprecated. If your existing Suricata configuration is using any pattern matcher algorithm not shown in the list of acceptable values below, the setting will be migrated to "Auto". If your existing configuration is "AC", then it will be left at that value and you will need to manually change the Pattern Matcher setting on the INTERFACE SETTINGS tab. The new valid options for Pattern Matching are:
 Auto   - will use hyperscan when available, else defaults to AC
 AC     - Aho-Corasick (default implementation)
 AC-BS  - Aho-Corasick (reduced memory implementation)
 AC-KS  - Aho-Corasick (Ken Steele variant)
 HS     - Hyperscan (available when built with hyperscan support)

Please note that hyperscan is only available with 64-bit builds of pfSense.  There is no hyperscan support available on 32-bit versions of pfSense.  This is a limitation of the hyperscan library.  If you have a 32-bit system and attempt to force hyperscan mode, it will not work.  Leaving the setting in AC or Auto is suggested for 32-bit installations.

You should generally leave the Pattern Matcher setting on the INTERFACE SETTINGS tab set to "Auto".  With this setting, hyperscan will be used if available; otherwise "AC" will used.  For existing installations where your Pattern Matcher setting was "AC", you should change the setting to "Auto" after upgrading.  I made the choice not to automatically make this change during the upgrade installation in case a user had chosen "AC" for a specific reason.  "AC" is a safe default.  If you have a 64-bit build of pfSense and wish to use hyperscan pattern matching, make the change on the INTERFACE SETTINGS tab, save it, and then restart Suricata on the interface.

2. Two additional hashing algorithms (SHA1 and SHA256) were added to the Tracked Files option. The old binary config parameter for switching MD5 hashing of tracked files ON or OFF is changed to a select drop-down with choices of "None", "MD5", "SHA1" and "SHA256".  This option is part of the logging options on the INTERFACE SETTINGS tab.  Formerly it was an On/Off checkbox to toggle MD5 hashing on or off.  The option is now a select drop-down.  Choose "None" if you wish to disable hashing for logged Tracked Files, otherwise choose one of the three available hashing algorithms.  The default for this option is "None".

3. Two new EVE JSON logging options were added for logging SMTP traffic and DROPPED traffic. These are enabled by default when EVE JSON logging is enabled. Note that the DROPPED traffic option can consume quite a large amount of disk space on a busy network. This option logs all packets that are dropped when using inline IPS mode in JSON format. The DROPPED traffic option is hidden and not used if Legacy Mode is chosen for the IPS Mode.


IDS/IPS / Suricata 3.2.1 update coming soon with hyperscan support
« on: April 10, 2017, 01:44:20 pm »
Just FYI to let everyone know I'm working now on updating the Suricata binary on pfSense to version 3.2.1.  This new version will enable hyperscan support on AMD64 architectures.  Please note that hyperscan will not work with 32-bit systems.  This is a limitation of the hyperscan library.  The new default mpm_algo setting (pattern matching algorithm) in Suricata 3.2.1 will be "auto".  This will use hyperscan when it is available, and if no hyperscan support is available, then it will default to the AC setting for pattern matching.

The new 3.2.1 version of the Suricata package will also include a few updates to the structure of the suricata.yaml file to catch up with changes from upstream.  There are a number of leftover legacy settings in the suricata.yaml file for the pfSense package that are now deprecated.  These will be removed.  The 3.2.1 Suricata binary also includes a number of bug fixes from upstream.


IDS/IPS / Suricata Package Updated to 3.1.2_1 -- Release Notes
« on: January 20, 2017, 09:59:40 am »
Suricata v3.1.2_1

The Suricata package has been updated to v3.1.2_1 to fix issues with Pass List implementation when using inline IPS Mode.

Bug Fixes
  • Remove automatic inclusion of all locally-attached networks in the default Pass List when using IPS mode. This had the unintended side-effect of essentially whitelisting all traffic to and from local hosts.
  • Add the capability to completely disable use of the default Pass List when running with inline IPS Mode. Formerly, the default Pass List would be used when no custom list was specified.
  • Remove automatic inclusion of the WAN interface IP address in the default Pass List when using IPS mode. This had the unintended side-effect of essentially whitelisting all inbound NAT traffic because the destination IP would be the WAN IP.
  • Increase default value of Host Memcap on IP REPUTATION tab to 32 MB as most IP lists today are quite large. This is effective only for newly created interfaces.
  • The checkbox for including/excluding the WAN IP from a custom Pass List was inadvertently removed during the Bootstrap conversion of the GUI code. This checkbox is now restored.

Implications of Using a Pass List with Inline IPS Mode
There is a new choice in the Pass List drop-down selector on the INTERFACE SETTINGS tab.  That new choice is "none", and when that option is selected no Pass List of any type will be used on the Suricata interface.  This is the recommended setting for those of you using Inline IPS Mode.  To understand why that is the recommended choice, read on below.

When you use a Pass List with Inline IPS Mode, you need to be aware of some potential pitfalls.  The most significant pitfall is that a host IP in a Pass List is totally unprotected by Suricata.  No rule but the auto-generated PASS rule will ever match, and thus the host is open to attack and with no logging of the attack.  Read that previous sentence one more time and let it sink in!  It is the number one reason why you generally don't want to use a Pass List with Inline IPS Mode.

If you still have a need to use a Pass List, consider carefully which of the auto-generated IP addresses you allow in your custom list.  In a NAT environment you would never want to include the WAN IP in the list.  Doing so will exempt all inbound traffic from the WAN from inspection because Suricata sees the traffic before the inbound NAT has happened.  Thus the inbound "destination IP address" will be the firewall's WAN IP.  If that IP is in the Pass List, then the traffic is passed without further inspection!  That's probably not what you want.  So with inline IPS mode, you rarely want to include the WAN IP.  Carefully consider also whether or not you choose to include the other defaults of locally-attached networks, the DNS servers specified for the firewall and the Virtual IPs and any VPNs.  Consider the impact in the same terms as described for the WAN IP.  You can easily exempt vast swaths of your network from even being scanned by Suricata.  You should also generally uncheck the option to include "All Local Networks" when creating a custom Pass List.  This option, when enabled, will include the entire subnet for each directly-attached local network (meaning all networks defined on all firewall interfaces with the exception of the WAN) in the Pass List.  This would generally be a bad thing because, again, the auto-generated PASS rules would exempt all hosts on those subnets from being examined by Suricata.

The v3.1.2_1 version of the Suricata package does make some changes in the composition of the "default" Pass List.  It no longer automatically includes the WAN IP nor Local Networks in the default list.  It does include the single firewall interface IP of every interface on the firewall with the exception of the WAN, though.  All of the old defaults discussed in this thread are usually OK for Legacy Mode.  This is due to the way Legacy Mode implements blocking.  It does not use PASS rules like IPS Mode does.  The custom blocking plugin used in Legacy Mode can be more selective when blocking because it is able to block one of the IP addresses in a packet and not the other.  For example, it might let the Pass List host IP go but yet still block the other end of the conversation.  So if the SRC IP was in the Pass List but the DST IP was not, the SRC IP would not be blocked but the DST IP would.  This would still protect the host from the bad guy.  IPS Mode is different because the entire packet is simply dropped.  You can't drop one IP in a conversation but not the other when using IPS Mode.

So what is the better way of protecting hosts from being blocked (or having their traffic dropped)?  You might want to consider using the Suppress List feature to selectively bypass only the rules giving you trouble for specific hosts.  You can use the icons on the ALERTS tab to accomplish this.  Hover over the plus signs (+) in the alert rows to see a tooltip popup describing what clicking the icon does.  A Suppress List entry bypasses a single rule GID:SID.  It can be bypassed for all hosts, or only select hosts by either source IP or destination IP.  A Pass List entry, on the other hand, will bypass all rules for the host IP addresses in the list.


Pages: [1] 2 3 4 5 ... 8