Netgate SG-1000 microFirewall

Author Topic: Snort VRT Not Dowloading -Snort VRT rules md5 download failed Error  (Read 3695 times)

0 Members and 1 Guest are viewing this topic.

Offline phantonuser

  • Newbie
  • *
  • Posts: 12
  • Karma: +1/-0
    • View Profile
Hi All,

For some time I am not able to download Snort VRT rules and today I realized this error in System Logs:

Snort VRT rules md5 download failed...

Does anyone know if there is a fix for this without ignoring the md5 checksum of the rules for downloading them?

Is there something with the snort package? I saw some people saying they can download it fine.

Thanks for your Help!


Offline virgiliomi

  • Sr. Member
  • ****
  • Posts: 566
  • Karma: +74/-4
    • View Profile
Re: Snort VRT Not Dowloading -Snort VRT rules md5 download failed Error
« Reply #1 on: July 22, 2016, 09:06:58 am »
Make sure your Snort package is up to date. Snort EOL'ed an old version at the end of July (2.9.8.0), which had been a version used in pfSense. bmeeks has since updated the package to a newer version of Snort (2.9.8.3).

Offline phantonuser

  • Newbie
  • *
  • Posts: 12
  • Karma: +1/-0
    • View Profile
Re: Snort VRT Not Dowloading -Snort VRT rules md5 download failed Error
« Reply #2 on: July 22, 2016, 11:37:24 am »
Hi virgiliomi,

I am using version 3.2.9.1_14 for what I can see in my package manager and it was recently installed due an error with libraries and that is now fully corrected and functional. Thank you for all for this help by the way.

Is there anything wrong with my snort package, should I try reinstalling it?

Thank you for your reply.

Thank You all for the Help.

You have a lot a of patience with a newbie. I am very grateful for this.




Offline bmeeks

  • Hero Member
  • *****
  • Posts: 3298
  • Karma: +862/-0
    • View Profile
Re: Snort VRT Not Dowloading -Snort VRT rules md5 download failed Error
« Reply #3 on: July 22, 2016, 12:01:25 pm »
Hi virgiliomi,

I am using version 3.2.9.1_14 for what I can see in my package manager and it was recently installed due an error with libraries and that is now fully corrected and functional. Thank you for all for this help by the way.

Is there anything wrong with my snort package, should I try reinstalling it?

Thank you for your reply.

Thank You all for the Help.

You have a lot a of patience with a newbie. I am very grateful for this.

Are you running any other packages?  Several folks have reported one of the IP block lists in pfBlockerNG erroneously blocks the Snort VRT rules download site (which is hosted on an Amazon Web Services server).

Bill

Offline phantonuser

  • Newbie
  • *
  • Posts: 12
  • Karma: +1/-0
    • View Profile
Re: Snort VRT Not Dowloading -Snort VRT rules md5 download failed Error
« Reply #4 on: July 22, 2016, 02:49:02 pm »
Hi bmeeks,

No I am not using pfBlockerNG, I have only Snort, Squid and Lightsquid installed.

I maybe doing some wrong configuration, is there somewhere else that this "IP block lists in pfBlockerNG" can be erroneous configured by me in some of the packages I have installed?

Thank You for your Help. I really appreciate it.

Regards.

Offline bmeeks

  • Hero Member
  • *****
  • Posts: 3298
  • Karma: +862/-0
    • View Profile
Re: Snort VRT Not Dowloading -Snort VRT rules md5 download failed Error
« Reply #5 on: July 22, 2016, 04:39:04 pm »
Hi bmeeks,

No I am not using pfBlockerNG, I have only Snort, Squid and Lightsquid installed.

I maybe doing some wrong configuration, is there somewhere else that this "IP block lists in pfBlockerNG" can be erroneous configured by me in some of the packages I have installed?

Thank You for your Help. I really appreciate it.

Regards.

No, assuming you have a valid Snort VRT rules subscription Oinkcode, then there is really nothing to configure wrong in pfSense.  Is there any other message in your logs?  What about the View Log button on the UPDATES tab?  Anything showing when viewing that log?

You could always try to manually download the file from the command line on the firewall to test connectivity.  Something is preventing your firewall from connecting to the Snort VRT rules site.  The rules updates are hosted on Amazon Web Services servers.  Don't know where you are located, but make sure that netblock is accessible from where you are.

Bill

Offline phantonuser

  • Newbie
  • *
  • Posts: 12
  • Karma: +1/-0
    • View Profile
Re: Snort VRT Not Dowloading -Snort VRT rules md5 download failed Error
« Reply #6 on: July 22, 2016, 11:52:55 pm »
Hi Bill,

I have recently changed my Oinkcode, it was a few days ago, I don't believe it is the problem but I can try changing it again to see if it works.

About the View Logs in the Update Tab it shows:

      Downloading Snort VRT rules md5 file snortrules-snapshot-2983.tar.gz.md5...
   Snort VRT rules md5 download failed.
   Server returned error code 422.
   Server error message was:
   Snort VRT rules will not be updated.

It is similar to the message that I see in System Logs, for some reason the md5 checksum of the rules is not being downloaded, in my understanding and I don't feel like it is a good idea to download the rules without the md5 checksum.

" You could always try to manually download the file from the command line on the firewall to test connectivity."

It seems a good troubleshooting approach but I am not very good at command line, I could try it and see if works. Could you tell me how to do it?

I prefer it doing automatically like it was before that's why I am trying to fix this issue in my PfSense box without having to reinstall it.

"The rules updates are hosted on Amazon Web Services servers. Don't know where you are located, but make sure that netblock is accessible from where you are."

I did not see any problems with other servers of Amazon Web Services from my location and the issue started without me noticing any changes at my connection or any changes on nothing, not even when the package was updated what was my first guess, maybe can be this particular server where the md5 checksum and rules are hosted.

About the netblock I did not understood very well, if it can be related to my ISP. I kind of doubt it because I saw a similar post in the forum about the Snort VRT rules issue and I did not noticed if they have it solved without removing the download of the md5 checksum.

I use PfSense for almost 8 years now, the forum helps a lot, I learned almost everything by myself just using it and I see you do a very nice work on this free platform, the Squid Package is always being updated with the latest ClamAV when possible, of course, and I find it a very nice feature.

I miss Bind by the way. My lack of programming skills keeps in the way of engaging myself in getting involved with projects.

But I use and recommend the ones I like and PfSense is a very good Firewall and it has great features.

PfSense was installed in several places like other Free Software and OS. Not every person or companies can afford the payed ones.

I would appreciate if you could continue trying to help me on this and also all your job on developing the software.

Thanks a lot.

Phanton
« Last Edit: July 23, 2016, 12:19:44 am by phantonuser »

Offline bmeeks

  • Hero Member
  • *****
  • Posts: 3298
  • Karma: +862/-0
    • View Profile
Re: Snort VRT Not Dowloading -Snort VRT rules md5 download failed Error
« Reply #7 on: July 24, 2016, 08:57:50 pm »
Hi Bill,

I have recently changed my Oinkcode, it was a few days ago, I don't believe it is the problem but I can try changing it again to see if it works.

About the View Logs in the Update Tab it shows:

      Downloading Snort VRT rules md5 file snortrules-snapshot-2983.tar.gz.md5...
   Snort VRT rules md5 download failed.
   Server returned error code 422.
   Server error message was:
   Snort VRT rules will not be updated.

It is similar to the message that I see in System Logs, for some reason the md5 checksum of the rules is not being downloaded, in my understanding and I don't feel like it is a good idea to download the rules without the md5 checksum.

" You could always try to manually download the file from the command line on the firewall to test connectivity."

It seems a good troubleshooting approach but I am not very good at command line, I could try it and see if works. Could you tell me how to do it?

I prefer it doing automatically like it was before that's why I am trying to fix this issue in my PfSense box without having to reinstall it.

"The rules updates are hosted on Amazon Web Services servers. Don't know where you are located, but make sure that netblock is accessible from where you are."

I did not see any problems with other servers of Amazon Web Services from my location and the issue started without me noticing any changes at my connection or any changes on nothing, not even when the package was updated what was my first guess, maybe can be this particular server where the md5 checksum and rules are hosted.

About the netblock I did not understood very well, if it can be related to my ISP. I kind of doubt it because I saw a similar post in the forum about the Snort VRT rules issue and I did not noticed if they have it solved without removing the download of the md5 checksum.

I use PfSense for almost 8 years now, the forum helps a lot, I learned almost everything by myself just using it and I see you do a very nice work on this free platform, the Squid Package is always being updated with the latest ClamAV when possible, of course, and I find it a very nice feature.

I miss Bind by the way. My lack of programming skills keeps in the way of engaging myself in getting involved with projects.

But I use and recommend the ones I like and PfSense is a very good Firewall and it has great features.

PfSense was installed in several places like other Free Software and OS. Not every person or companies can afford the payed ones.

I would appreciate if you could continue trying to help me on this and also all your job on developing the software.

Thanks a lot.

Phanton

Are you running any other packages on your firewall besides Snort?  You mentioned Squid in your recent reply.  Are you running the Squid package or the pfBlockerNG package?  If so, try temporarily disabling one or both of those and try the Snort rules download again.

You have a configuration issue on your end.  If the Snort package itself was messed up, there would be hundreds of posts here about the problem.  There have been a few posts about rules update issues, but they were all caused by a false positive block by one of the IP lists downloaded and used by pfBlockerNG.  The only other issue occurred maybe 10 days ago and was due to a corrupeted file on the Snort VRT web site itself.  That issue was corrected by the Snort VRT within 12 hours.

Bill

Offline Bill Taroli

  • Newbie
  • *
  • Posts: 5
  • Karma: +1/-0
    • View Profile
Re: Snort VRT Not Dowloading -Snort VRT rules md5 download failed Error
« Reply #8 on: August 03, 2016, 03:02:17 am »
I've been having some trouble with this as well. The error seems to be "505 HTTP Version Not Supported". I'm not sure what information this error is intended to impart. I'm running latest pfSense release (2.3.2).

Code: [Select]
Downloading Snort VRT rules md5 file snortrules-snapshot-2983.tar.gz.md5...
Snort VRT rules md5 download failed.
Server returned error code 505.
Server error message was: 505 HTTP Version Not Supported
Snort VRT rules will not be updated.

I have successfully downloaded the rules file using the examples from https://www.snort.org/oinkcodes and the same 2983 version of the file and the MD5.

Code: [Select]
[2.3.2-RELEASE][root@]/tmp: curl -L -o test.tgz "https://www.snort.org/rules/snortrules-snapshot-2983.tar.gz?oinkcode=..."

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   293    0   293    0     0    650      0 --:--:-- --:--:-- --:--:--   649
100 40.6M  100 40.6M    0     0   650k      0  0:01:04  0:01:04 --:--:--  674k


[2.3.2-RELEASE][root@]/tmp: curl -L -o test.md5 "https://www.snort.org/rules/snortrules-snapshot-2983.tar.gz.md5?oinkcode=..."

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    32    0    32    0     0     67      0 --:--:-- --:--:-- --:--:--    67

What I find odd is that it refers to "HTTP version". Does this mean it's trying to use http instead of https? I don't see anything in the configuration that suggests it would be doing that. If I run the same curl with http:// it completes fine as well. So not really sure what to think.

« Last Edit: August 04, 2016, 02:20:38 pm by Bill Taroli »

Offline phantonuser

  • Newbie
  • *
  • Posts: 12
  • Karma: +1/-0
    • View Profile
Re: Snort VRT Not Dowloading -Snort VRT rules md5 download failed Error
« Reply #9 on: August 04, 2016, 02:01:50 pm »
Hi Bill,

I've stopped package squid, snort and changed my DNS. None of that worked.

Then I tried generating a new Oinkcode and it worked with the new one. The VRT rules updated.

I had to remove the old Oinkcode, disable the VRT rules, save and then enable again a copy the new Oinkcode and save.

Then Force Update. And Voila.

I feel a little stupid since I have done this before. It was so simple, probably just a configuration issue in my package.

Thanks for the Help.

Hope this post helps anyone else with the same issue.

:-)




Offline Bill Taroli

  • Newbie
  • *
  • Posts: 5
  • Karma: +1/-0
    • View Profile
Re: Snort VRT Not Dowloading -Snort VRT rules md5 download failed Error
« Reply #10 on: August 04, 2016, 02:21:37 pm »
Good to hear! I may try that myself and see if it helps. But given mine is failing for an HTTP version issue -- whatever THAT means -- I'm not sure if the oinkcode is really my problem.  :(

Offline ngnrpu@gmx.com

  • Newbie
  • *
  • Posts: 2
  • Karma: +0/-0
    • View Profile
Re: Snort VRT Not Dowloading -Snort VRT rules md5 download failed Error
« Reply #11 on: August 05, 2016, 01:24:42 pm »
 Once i faced same issue. I resolved it by deleting the snort package completely and then reinstalling package from scratch.

Offline Bill Taroli

  • Newbie
  • *
  • Posts: 5
  • Karma: +1/-0
    • View Profile
Re: Snort VRT Not Dowloading -Snort VRT rules md5 download failed Error
« Reply #12 on: August 06, 2016, 07:00:53 pm »
Tried that, making sure that the values in general settings would remove config and blocked lists, and reinstalled. Regenerated my oinkcode, entered it in the settings (which I noted *was* pre-populated, making me wonder what stuff the package didn't remove) and got the following when I triggered update. Everything except the VRT rules downloaded.

Code: [Select]
Starting rules update...  Time: 2016-08-06 16:57:41
Downloading Snort VRT rules md5 file snortrules-snapshot-2983.tar.gz.md5...
Snort VRT rules md5 download failed.
Server returned error code 505.
Server error message was: 505 HTTP Version Not Supported
Snort VRT rules will not be updated.

Offline Bill Taroli

  • Newbie
  • *
  • Posts: 5
  • Karma: +1/-0
    • View Profile
Re: Snort VRT Not Dowloading -Snort VRT rules md5 download failed Error
« Reply #13 on: August 06, 2016, 07:19:51 pm »
OK. Required a bit of extra shell action. After removing package, hunted down leftover bits in the filesystem.

Code: [Select]
rm -rf /usr/local/etc/snort
rm -rf /usr/local/lib/snort_dynamicrules
rm /var/cache/pkg/*snort*

Also grep'ed globally to find references to snort. In config.xml I found that it still had stuff about snort *and* there were two sqlite databases that contained references. I didn't bother with those, but I did open up config.xml and found all the basic setting properties in there. So removing doesn't really remove. That's not cool. But I left it there not wanting to break anything.

I did notice that there was a space in front of my oinkcode though! :) When reinstalling the package, I made sure to remove that and when I did the update it went fine.

Offline 6pac

  • Newbie
  • *
  • Posts: 1
  • Karma: +0/-0
    • View Profile
Re: Snort VRT Not Dowloading -Snort VRT rules md5 download failed Error
« Reply #14 on: January 22, 2018, 01:36:03 pm »

I did notice that there was a space in front of my oinkcode though! :) When reinstalling the package, I made sure to remove that and when I did the update it went fine.


You're the man Bill!  8)

Mine had two spaces in front; deleting those sorted the problem.

Cheers!

Offline bmeeks

  • Hero Member
  • *****
  • Posts: 3298
  • Karma: +862/-0
    • View Profile
Re: Snort VRT Not Dowloading -Snort VRT rules md5 download failed Error
« Reply #15 on: January 22, 2018, 05:42:40 pm »
OK. Required a bit of extra shell action. After removing package, hunted down leftover bits in the filesystem.

Code: [Select]
rm -rf /usr/local/etc/snort
rm -rf /usr/local/lib/snort_dynamicrules
rm /var/cache/pkg/*snort*

Also grep'ed globally to find references to snort. In config.xml I found that it still had stuff about snort *and* there were two sqlite databases that contained references. I didn't bother with those, but I did open up config.xml and found all the basic setting properties in there. So removing doesn't really remove. That's not cool. But I left it there not wanting to break anything.

I did notice that there was a space in front of my oinkcode though! :) When reinstalling the package, I made sure to remove that and when I did the update it went fine.

You can remove Snort and have it clean up after itself.  The default is to "save settings" because most folks want to remove and reinstall or update the binary while keeping their existing configuration settings.  On the GLOBAL SETTINGS tab is a checkbox option to save settings when uninstalling the package.  The box is checked by default, but you can uncheck the box and when you remove Snort it will remove all traces of itself from the config.xml file.  That of course means any and all of your previous Snort configuration settings are gone.

The directories you found are being left because of a bug in the uninstall code.  That should be fixed in the latetst package version.  The only exception would be if you manually modified any files in those directory trees.

Bill