Netgate SG-1000 microFirewall

Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.


Messages - bmeeks

Pages: [1] 2 3 4 5 ... 212
1
IDS/IPS / Re: netmap errors in console
« on: Yesterday at 04:01:17 pm »
I did a comparison of the hundreds of netmap errors I have seen over the last week and noticed some common occurrences.
Most happen at the same minute of every hour. Almost like on a schedule, but Cron does not have any events at the same time.
This can't be a coincidence.
Are there any scheduled events outside of the ones that Cron performs in PFsense?
Maybe the ISP runs something on a scheduled basis. I am going to do a packet capture during the next period I expect to see more netmap errors.

Strange thing is my other PFsense box with identical hardware doesn't see these netmap errors, but it doesn't see nearly the same volume of traffic either.

There are some other tasks that happen on a periodic basis, but I do not know what all they are.  That would be a question better answered by a member of the pfSense developer team.

Bill

2
IDS/IPS / Re: Pass List adds unwanted IPv6 addresses
« on: Yesterday at 03:58:30 pm »
The passlist alias would be used to set the HOME_NET even with Inline, correct?

Yes, but the list name would need to be selected in the HOME_NET drop-down selector on the INTERFACE SETTINGS tab.  I was talking specifically about the Pass List drop-down, but that control should be hidden when Inline IPS Mode is selected.

Bill

3
Packages / Re: Snort FATAL ERROR
« on: Yesterday at 03:24:49 pm »
The switch to Suricata was pretty simple. In the pfsense webGUI it's mostly identical. It took me a little googling to figure out how to point it to the correct snort tar.gz file, but it seems to be working fine.

Hopefully your Google searching sent you to this Sticky Post in the IDS/IPS forum here:  https://forum.pfsense.org/index.php?topic=124054.0.

Bill

4
Packages / Re: Snort FATAL ERROR
« on: Yesterday at 03:23:21 pm »
The rules contained in this download file are not formatted properly. There are many rules missing proper syntax. If anybody knows who this volunteer is, please
From what I've seen, Suricata doesn't have the option to enable AppID Open Rules. This option seems only available in Snort.

True ... only Snort has OpenAppID functionality.

Here is a quick and dirty comparison of Snort and Suricata functionality --

OpenAppID (application detection and alerting)
    Snort - Yes
    Suricata - No

Inline IPS Mode (dropping traffic)
    Snort - No
    Suricata - Yes

Multi-threaded Detection Engine
   Snort - No
   Suricata - Yes

Uses All Snort VRT Rules Without Error
    Snort - Yes
    Suricata - No (some Snort VRT rules have keywords not recognized by Suricata and will be ignored)

Continues Startup After Encountering Rule Syntax Errors
    Snort - No
    Suricata - Yes

In terms of relative protection, both packages are really about equal.  As you can see in the list above, each package has its pros and cons.  If something like application detection and alerting is important to you, then Snort is what you need.  If Inline IPS Mode dropping of packets (as opposed to implementing firewall block rules) is critical for you, then Suricata is what you need (if your NIC supports Netmap operation).  Both packages can use the entire rule set from Emerging Threats (now Proofpoint), but Suricata will ignore about 70-100 Snort VRT rule because of unsupported keywords.  Suricata will print an error about these rules and then skip loading them.

Bill

5
Packages / Re: Snort FATAL ERROR
« on: Yesterday at 03:14:24 pm »
Thanks bmeeks, that definitely helped. I now know how to find these errors and correct them. The problem is when I fixed that one, I'm getting another one and another one. What are the chances of multiple rules in multiples files being messed up with syntax errors like this? I find this a bit odd. I'm not gonna sit here trying to fix each one either, this can't be normal.

I guess I may have to give Suricata a try. Can I use my snort disablesid.conf in Suricata? I'm not familiar with Suricata at all.

Yes, the SID MGMT files can be easily used in Suricata.  Both Snort and Suricata share almost identical code for the SID MGMT tab.

P.S. -- as for the errors, my understanding from a third-hand source is the rule maintainer for those OpenAppID rules made some updates recently.  Apparently that's when all the syntax errors crept in.

Bill

6
General Questions / Re: Annoying Snort Issue
« on: Yesterday at 03:04:42 pm »
I can understand that.   I'm curious when they might clear up the issue.   It's been 3 days.  I would hate to see sourcefire have the same issue at work.

I believe this was identified as an error in one of the volunteer-maintained OpenAppID rules.  That rules package was created and is maintained by an individual in Brazil.  The pfSense team just recently moved the hosting site from a Brazilian University over to pfSense infrastructure.  The text OpenAppID rules are not maintained by the Snort VRT.

I was under the impression this rule typo had been corrected a couple of days ago.  You could try reaching out to the pfSense team for more information, or temporarily turn off the OpenAppID rules and see if the error goes away.  I think it will.

Snort has one failing compared to Suricata.  With Suricata, when a rule syntax error is encountered, the binary will print an error message but then skip the offending rule and load the others.  Snort, on the other hand, will print an error and exit when encountering a rule syntax error.  This behavior is baked into the underlying binary and is not something the pfSense GUI package can influence.

Bill

7
IDS/IPS / Re: Pass List adds unwanted IPv6 addresses
« on: Yesterday at 10:54:20 am »
Sorry, my intent was not adversarial, which is why I hate emails.

Yes, I do see IPv6 Link Local addresses displayed. And you gave a good answer, which is I have no control over it. Maybe extra overhead for netmap, which is what I was trying to avoid.

Thanks

No problem.  Emails (or forum posts) can often be misconstrued because we don't have visual or auditory clues from getting and hearing the information first hand.

There is no overhead for Netmap in regards to those addresses.  However, also realize that a Pass List has no function when using Inline IPS Mode.  And if you are not using Inline IPS Mode, then Netmap is not even turned on.  So those two are not connected.  If you have not reviewed this Sticky Post, you will want to check it out as it explains some key differences in the two modes:  https://forum.pfsense.org/index.php?topic=135331.0.

Bill

8
IDS/IPS / Re: NIC's with Suricata Inline mode
« on: Yesterday at 10:49:47 am »

Now, a point I must make here is that these NIC's were tested on a high traffic interface. When I use on a low traffic interface, I do not see any netmap issues.


This would be a good point to highlight in a Redmine bug report posting for pfSense.  The sensitivity to traffic loading might be a valuable clue for a FreeBSD or pfSense kernel developer.  Please consider posting a bug report on the pfSense Redmine site here:  https://redmine.pfsense.org/projects/pfsense.

I don't currently use Suricata and thus not Inline IPS Mode.  My home connection is also probably much too slow and has much too little traffic to make the issues with Netmap surface.

Bill

9
IDS/IPS / Re: netmap errors in console
« on: Yesterday at 10:45:40 am »
Thanks for your answer. Yes all the offloading is disabled. This is why I posted a thread for users to come forward with which NIC's have been error free with Suricata Inline. Unfortunately there have been no responses to that.

I am using a very common Intel NIC. i350T2V2. I have also tried an Intel i219 with same results.

But what I really wanted to know is if these netmap errors can just be ignored, and what they represent. Am I losing data? Or are these actual bad packets that have hit the interface. Does anyone know this? I am sure that actual bad packets arrive on occasion. I just want to know if it is an issue of netmap not being able to handle a good packet or if it is just reporting a legitimate bad packet. That's all I want to know. Since the error does not display the IP, I have no way to tell. And besides, wouldn't the packet just be resent since there would be no ACK? Sort of like an SA, RA, or PA protocol log entry from the firewall.

I did put these issues in with FreeBSD and do realize that this is not a PFsense issue.

Thanks

I am not knowledgeable enough about the FreeBSD kernel to say with certainty, but my guess would be Netmap may be falsely reporting bad packets.  It's true you will get some now and then, but I would not think you would see very many at all in a switched LAN environment.

Bill

10
IDS/IPS / Re: Suricata keeps crashing since 2.4.2 upgrade
« on: Yesterday at 10:43:04 am »
               Crash report begins.  Anonymous machine information:

amd64
11.1-RELEASE-p6
FreeBSD 11.1-RELEASE-p6 #421 r313908+a5b33c9d1c4(RELENG_2_4): Tue Dec 12 09:20:59 CST 2017     root@buildbot2.netgate.com:/builder/ce/tmp/obj/builder/ce/tmp/FreeBSD-src/sys/pfSense

Crash report details:

PHP Errors:
[13-Dec-2017 16:03:42 America/Los_Angeles] PHP Warning:  filesize(): stat failed for /usr/local/etc/suricata/suricata_18353_em0/rules/suricata.rules in /usr/local/pkg/suricata/suricata_generate_yaml.php on line 855
[13-Dec-2017 16:03:42 America/Los_Angeles] PHP Warning:  filesize(): stat failed for /usr/local/etc/suricata/suricata_18353_em0/rules/flowbit-required.rules in /usr/local/pkg/suricata/suricata_generate_yaml.php on line 857
[13-Dec-2017 16:03:42 America/Los_Angeles] PHP Warning:  filesize(): stat failed for /usr/local/etc/suricata/suricata_18353_em0/rules/custom.rules in /usr/local/pkg/suricata/suricata_generate_yaml.php on line 859
[13-Dec-2017 16:03:42 America/Los_Angeles] PHP Warning:  filesize(): stat failed for /usr/local/etc/suricata/suricata_57646_em1/rules/suricata.rules in /usr/local/pkg/suricata/suricata_generate_yaml.php on line 855
[13-Dec-2017 16:03:42 America/Los_Angeles] PHP Warning:  filesize(): stat failed for /usr/local/etc/suricata/suricata_57646_em1/rules/flowbit-required.rules in /usr/local/pkg/suricata/suricata_generate_yaml.php on line 857
[13-Dec-2017 16:03:42 America/Los_Angeles] PHP Warning:  filesize(): stat failed for /usr/local/etc/suricata/suricata_57646_em1/rules/custom.rules in /usr/local/pkg/suricata/suricata_generate_yaml.php on line 859


Filename: /var/crash/minfree
2048
            

this happens after I reinstall the whole package

What type of hardware is this?  Those errors indicate problems within the file system.  Another possibility, if you have recently upgraded your hardware and imported an old config, is the interface names have changed (the em1 part of the error path).  So for example if your NIC driver is now say igb1 instead of em1, then you will get this error.  To fix it you will need to either delete the interface and recreate it from scratch, or manually go into your config.xml file and change all the instances of the strings "em0" and "em1" to match whatever the new name is for your physical interfaces.

11
IDS/IPS / Re: Pass List adds unwanted IPv6 addresses
« on: Yesterday at 10:37:53 am »
I setup an alias for the IP's I want to use with the passlist in Suricata (Legacy).
Then I set the HOME_NET to that passlist value in the WAN interface.
When I 'View List', many IPv6 addresses were added to the HOME_NET that I did not ask for and do not want in there.
How can I remove these? And please don't say, just ignore them. I am trying to tune the interface and these are unnecessary entries.

I do not use IPv6 on any interface. All Interface have IPv6 set to none.
Please advise.

You can't remove them, so since you said

And please don't say, just ignore them[/i]

there is nothing for us to help you with.

OK, the reply above was bit "tongue-in-cheek" since your post seems to have a bit of an attitude (maybe unintended).  Here is the scoop.  Suricata and Snort automatically ask pfSense for all the locally assigned interface addresses.  The IPv6 ones you see are really assigned to the interface, but they are automatic.  If they begin with FE80:, then they are link-local addresses.  Click STATUS and INTERFACES from the pfSense menu and I bet you will see them listed for the interface.  They truly cause no harm being in the Pass List.  Suricata does not generate them, it simply asks pfSense (and by extension, the FreeBSD kernel) for all the IP addresses assigned to the local interfaces.

Bill

12
IDS/IPS / Re: Suricata signature rule - email alert
« on: December 13, 2017, 09:03:14 am »
Hello everybody

Is it possible to configure pfsense+suricate to make a e-mail alert when some signature rule is met? Means no watchdog, but e-mail alert when selected signature is detected.

Best regards
Michal

No, this capability does not exist.  Sounds like you need a third-party alert correlator on separate server if you want that level of functionality. 

Bill

13
IDS/IPS / Re: netmap errors in console
« on: December 12, 2017, 03:49:10 pm »
When you say you followed the tuning guide, that includes turning off all the hardware checksum offloading, LRO and segmentation stuff.  Have you done that?  If so, you just may have one of those NICs that is partially supported.  If it seems to work and is just spamming the console, maybe you live with it until Netmap is more mature.  This may be something you post to the FreeBSD folks as that's where the driver will get updated.

Bill

14
IDS/IPS / Re: Snort alert log entry timestamp delta between GUI and syslog
« on: December 12, 2017, 03:41:52 pm »
Hi Bill,

Yeah - really strange.  I considered the clog aspect as well but if that were part of this, then you would expect there to be skew consistent across the whole file which I do not see.

I do think the 5m delay for the block resulting from the 12:00 related syslog message is due to the rules updating - I figure maybe the BLOCK_THIS IPC message somehow got head-of-line blocked due to Snort grinding through rule updates.  I believe Snort is single-threaded and if so, then this might make even more sense.  Would be curious to hear your comments on that possibility...

In any event, still doesn't explain the different timestamps on the syslog messages... *scratches head*

Snort is indeed single-threaded ... at least the 2.x and older versions.  The new 3.0-ALPHA is multi-threaded, but it's not released as stable yet and is not in the FreeBSD ports collection.

Bill

15
IDS/IPS / Re: Snort OpenappID Rules - Syntax errors
« on: December 12, 2017, 03:40:00 pm »
Anyone know who the "volunteer maintainer" is for the file hosted at http://files.pfsense.org/openappid/appid_rules.tar.gz??

There are syntax errors in the rules (missing the closing ")" on several rules) which causes snort to fail to start until you manually chase down each one. I did the work identify and disable the troublesome rules so I could use the rest and so will share the details below on what rules to disable and what categories they belong to to save you guys some time until this is fixed.

The error produced is FATAL ERROR: /usr/local/etc/snort/snort_{0}_igb{0}/rules/snort.rules({0}) Rule options must be enclosed in '(' and ')'.

file_storage.rules >>>> alert tcp $HOME_NET any -> $EXTERNAL_NET any (msg:"scribd_upload";flow:from_client;appid:scribd_upload; sid:71443 ; classtype:misc-activity; rev:1
ads.rules >>>> alert tcp $HOME_NET any -> $EXTERNAL_NET any (msg:"inskin_media";flow:from_client;appid:inskin_media; sid:71780 ; classtype:misc-activity; rev:1;
network_protocol.rules >>>> alert tcp $HOME_NET any -> $EXTERNAL_NET any (msg:"cisco_sysmaint";flow:from_client;appid:cisco_sysmaint; sid:70052 ; classtype:misc-activity; rev:1;
social_networking.rules >>>> alert tcp $HOME_NET any -> $EXTERNAL_NET any (msg:"networker";flow:from_client;appid:networker; sid:71392 ; classtype:misc-activity; rev:1;
social_networking.rules >>>> alert tcp $HOME_NET any -> $EXTERNAL_NET any (msg:"sway";flow:from_client;appid:sway; sid:72795 ; classtype:misc-activity; rev:1;
streaming_media.rules >>>> alert tcp $HOME_NET any -> $EXTERNAL_NET any (msg:"crackle";flow:from_client;appid:crackle; sid:70785 ; classtype:misc-activity; rev:1;
webbrowser.rules >>>> alert tcp $HOME_NET any -> $EXTERNAL_NET any (msg:"epiphany";flow:from_client;appid:epiphany; sid:71186 ; classtype:misc-activity; rev:1;
web_services.rules >>>> alert tcp $HOME_NET any -> $EXTERNAL_NET any (msg:"ensighten";flow:from_client;appid:ensighten; sid:71488 ; classtype:misc-activity; rev:1;

If someone knows where to file bug reports specifically for this hosted ruleset please let me know so they can be made aware and fix the errors.

As a side note, volunteer maintainer or not (often the case with open source projects), if you are going to take ownership of something you should probably test it on your own system at least one time before you post updates for the whole community. This was one simple test away from preventing a bug in the "wild"; just check all the categories and reload snort and see if it actually loads if you don't have time to do anything more robust...

Thanks

Agree with you.  I do know the OpenAppID rules were updated yesterday.  The pfSense team has contact with the developer.  Maybe they can get them fixed.

My other sore point with Snort is the binary will complain and die when encountering bad rule syntax.  Suricata, on the other hand, will flag the error and then ignore the offending rule and continue loading.  Snort needs to do that.

Just in case folks are confused, the issues discussed above are about the underlying Snort and Suricata binaries and not the GUI package you interact with on pfSense.  All the pfSense GUI package does is provide a pretty wrapper to help you create the Snort or Suricata config files the underlying binary uses to actually do the work.

Bill

Pages: [1] 2 3 4 5 ... 212