Suricata hash matching Please Help

[bmeeks]

Hi Meeks.

Im guessing this is normal behavior since suricata is merely intercepting a tcp stream and examining it but ive noticed that with the file hash comparison and blocking turned on the ips will always allow the first instance of a blacklisted file theough.  After that the ips engine will block the DST address and subsequent attempts to download the same file will be blocked along with the entire host.

You know of any way to force it not to allow the first instance through?

Also. Do you know if theres a way to have the executable name included in the alerts section without having to go to the eve.json log?


Thanks

What you are seeing is an artifact of the Legacy Mode blocking.  Suricata, in that mode, is using libpcap to get copies of each packet traversing the interface.  It examines the copy (and in many cases may need to see a number of copies of additional packets) to make a block/no-block decision.  Once it decides to block, it makes a system call to insert the offender's IP address into a built-in pfSense firewall table called snort2c.  That will result in subsequent traffic from that IP address getting blocked until the IP is removed from that pf table.  Suricata will also optionally kill all open states associated with the IP (the default is to kill open states).  However, remember I said "copies of packets", so that initial download can get started as the original packets continued on to the firewall and then the host behind it (assuming the traffic was solicited by the host in the first place).  This is what we call "packet leakage" and is inherent in Legacy Mode.

To prevent this you would need to run with Inline IPS Mode, but that mode uses native Netmap functionality in FreeBSD and also requires the NIC hardware driver to fully support Netmap operation.  Some do, but many to do not.  Also, you must remember that Suricata needs to download the entire file in order to compute the checksum to check against the list.  So a lot of memory might be required for buffering.

Bill

[steve40]

Hi Meeks.

Im guessing this is normal behavior since suricata is merely intercepting a tcp stream and examining it but ive noticed that with the file hash comparison and blocking turned on the ips will always allow the first instance of a blacklisted file theough.  After that the ips engine will block the DST address and subsequent attempts to download the same file will be blocked along with the entire host.

You know of any way to force it not to allow the first instance through?

Also. Do you know if theres a way to have the executable name included in the alerts section without having to go to the eve.json log?


Thanks

What you are seeing is an artifact of the Legacy Mode blocking.  Suricata, in that mode, is using libpcap to get copies of each packet traversing the interface.  It examines the copy (and in many cases may need to see a number of copies of additional packets) to make a block/no-block decision.  Once it decides to block, it makes a system call to insert the offender's IP address into a built-in pfSense firewall table called snort2c.  That will result in subsequent traffic from that IP address getting blocked until the IP is removed from that pf table.  Suricata will also optionally kill all open states associated with the IP (the default is to kill open states).  However, remember I said "copies of packets", so that initial download can get started as the original packets continued on to the firewall and then the host behind it (assuming the traffic was solicited by the host in the first place).  This is what we call "packet leakage" and is inherent in Legacy Mode.

To prevent this you would need to run with Inline IPS Mode, but that mode uses native Netmap functionality in FreeBSD and also requires the NIC hardware driver to fully support Netmap operation.  Some do, but many to do not.  Also, you must remember that Suricata needs to download the entire file in order to compute the checksum to check against the list.  So a lot of memory might be required for buffering.

Bill

Hi Bill,

thanks for the reply. I had a feeling that that would be the case so I used Inline mode and saw the same behavior. I belive the interface is a realtek card based on the "re0" ...

[bmeeks]

When using Inline IPS mode, you would have to manually change the associated rule or rules to DROP action, but I'm still not sure what the impact is with these file matching rules.  As I stated earlier, it's obvious Suricata will have to let the entire file download before it can check the checksum to see if the file is on a blacklist or whitelist.  So the part I'm not sure about is what Suricata is doing with the parts of the file as it is downloaded and before the checksum can be calculated.  If Suricata buffered the entire file, then exactly what will it tell the LAN host that started the transfer?  If that host does not get any packets during the buffering, it will assume the download is failing.  If Suricata feeds it the file parts, then it can't block the download.  I've never investigated this feature, though, so I really am not sure how it works.

Bill

[steve40]

I see where you're coming from.

Oh well, so much for trying to use suricata as a content filter   lol

I wonder how clamav gets around the buffering/leaving client in limbo issue. I wrote a md5 signature for the exact same file and injected it into the .ndb files clam uses and it picked it up and blocked it on first attempt. No issues.

Only thing is, for performance reasons id rather have done the checking lower down the stack