Netgate SG-1000 microFirewall

Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.


Messages - mrzaz

Pages: [1] 2 3 4 5 ... 11
1
IDS/IPS / Re: Snort OpenAppID RULES Detectors fail to download.
« on: December 10, 2017, 02:46:39 pm »
I also get problem with the APPID RULES download.

According to logs it says:
   Downloading Snort OpenAppID RULES detectors md5 file appid_rules.tar.gz.md5...
   Checking Snort OpenAppID RULES detectors md5 file...
   There is a new set of Snort OpenAppID RULES detectors posted.
   Downloading file 'appid_rules.tar.gz'...
   Done downloading rules file.
   Snort OpenAppID RULES detectors file download failed.  Bad MD5 checksum.
   Downloaded Snort OpenAppID RULES detectors file MD5: 4a919586ee271f633a04b406b1332bf9
   Expected Snort OpenAppID RULES detectors file MD5: d4539caec45fdb0484ded9de593e0dc4
   Snort OpenAppID RULES detectors file download failed.  Snort OpenAppID RULES detectors will not be updated.

And just to make sure, I manually downloaded the http://files.pfsense.org/openappid/appid_rules.tar.gz and http://files.pfsense.org/openappid/appid_rules.tar.gz.md5
and then made a manual md5 checksum of the "appid_rules.tar.gz" and compared it to the downloaded one.

DOWNLOADED:   d4539caec45fdb0484ded9de593e0dc4
MANUAL MD5:      4a919586ee271f633a04b406b1332bf9

Exactly the same as from the pfSense.  So either someone has modified the appid_rules.tar.gz after the checksum was created
OR the appid_rules.tar.gz has been updated and someone has forgot to create a new updated md5 checksum file
or possible that the the appid file has gone corrupted.

Please correct this.

The interesting part is that the appid file and the md5 file is stored at almost the same time. only 2 min apart.
http://files.pfsense.org/openappid/
appid_rules.tar.gz                                 08-Dec-2017 20:46              788480
appid_rules.tar.gz.md5                             08-Dec-2017 20:48                  33

Best regards
Dan Lundqvist
MRZAZ.COM
Stockholm, Sweden

2
IDS/IPS / Re: Snort OpenAppID RULES - Server returned error code 0
« on: December 10, 2017, 02:46:09 pm »
I've been unable to download the OpenAppID RULES for about 6 weeks due to the following error code - Server returned error code 0. All of the other rules update every day.

I've also deleted the Snort package and re-installed it and restored pfSense to a version where Snort had previously updated all rules.

Any help is much appreciated.

I also get problem with the APPID RULES download.

According to logs it says:
   Downloading Snort OpenAppID RULES detectors md5 file appid_rules.tar.gz.md5...
   Checking Snort OpenAppID RULES detectors md5 file...
   There is a new set of Snort OpenAppID RULES detectors posted.
   Downloading file 'appid_rules.tar.gz'...
   Done downloading rules file.
   Snort OpenAppID RULES detectors file download failed.  Bad MD5 checksum.
   Downloaded Snort OpenAppID RULES detectors file MD5: 4a919586ee271f633a04b406b1332bf9
   Expected Snort OpenAppID RULES detectors file MD5: d4539caec45fdb0484ded9de593e0dc4
   Snort OpenAppID RULES detectors file download failed.  Snort OpenAppID RULES detectors will not be updated.

And just to make sure, I manually downloaded the http://files.pfsense.org/openappid/appid_rules.tar.gz and http://files.pfsense.org/openappid/appid_rules.tar.gz.md5
and then made a manual md5 checksum of the "appid_rules.tar.gz" and compared it to the downloaded one.

DOWNLOADED:   d4539caec45fdb0484ded9de593e0dc4
MANUAL MD5:      4a919586ee271f633a04b406b1332bf9

Exactly the same as from the pfSense.  So either someone has modified the appid_rules.tar.gz after the checksum was created
OR the appid_rules.tar.gz has been updated and someone has forgot to create a new updated md5 checksum file
or possible that the the appid file has gone corrupted.

Please correct this.

The interesting part is that the appid file and the md5 file is stored at almost the same time. only 2 min apart.
http://files.pfsense.org/openappid/
appid_rules.tar.gz                                 08-Dec-2017 20:46              788480
appid_rules.tar.gz.md5                             08-Dec-2017 20:48                  33

Best regards
Dan Lundqvist
MRZAZ.COM
Stockholm, Sweden

3
Thanks so much for bringing this up and posting it.  Setting this up today for the first time (on 2.3.4-RELEASE-p1), I had the same hard time trying to figure out why deleting the variable or setting the value to yes/no wouldn't work.  After some Googling I found this post, set it to 0 and now I'm getting my Config+RRD data in full.

This means that I am not alone with this issue and also that the solution I found works for more than one. 
Jim, did you see this?

Best regards
Dan Lundqvist
Stockholm, Sweden

4
Yes, I can see it now. Have now started to look at it.

//Dan Lundqvist

5
Small question:
Will the presentation and presentation PDF used in the "pfSense Hangout - June 2017 - Advanced Captive Portal" session be available in the Hangout Archive soon ?
Could not find it there.   And the hangout was almost a week ago. I could not attend the hangout but would like to see it afterwards.

Best regards
Dan Lundqvist
Stockholm, Sweden

6
I can not explain it other than it did stop working at a certain date.   If that was due to upgrade of the NAS or something else I am not sure
but something happened.

BUT, if this is a local issue only at my installation, then we leave it at that. But bare in mind if you hear anyone else with same issue.

//Danne

7
Hi Jim,

It is really weird.   The problem was seen on 2.3.3_1 at the latest and as soon as I changed the script from "no" to "0" it started working and downloading RRD as it should.
"yes" and "no" defined it always exclude RRD.

I just tried again on 2.3.4 but same issue...   with yes/no it downloads both XML config without RRD and if i change script back again to 1/0 it starts working, then one contains the RRD data.
I just made a tcpdump of working and non-working session if you want?  I can send it to you offlist as it contains real config data.

//Dan Lundqvist

8
Documentation / Found a small error "Remote_Config_Backup" in Wiki.
« on: May 06, 2017, 12:18:21 pm »
Hello Jim,

I think I found a small error in the "Remote_Config_Backup" in Wiki. Or at least differences depending on what platform it is executed on.
https://doc.pfsense.org/index.php/Remote_Config_Backup

I have found that the script for taking backup and more specific the "donotbackuprrd=no" stopped working between 2017-02-01 -> 2017-03-01 timeframe
(which is the monthly backup schedule) where it stopped backup the XML with full RRD.

I started to check the script and made a change from "donotbackuprrd=no" to "donotbackuprrd=0" and then it started working again.
Please update the WIKI page with this.  Possible a note that on some platforms "=0" is needed instead of "=no".

All my backups taken with donotbackuprrd=no does NOT contain RRD data but when I changed to "donotbackuprrd=0" it started working directly.
Script running on Synology DS713+ with DSM 6.1-15047 Update 1

Code: [Select]
root@DiskStation3TB:/volume1/web_backend/tools# /usr/bin/wget --version
GNU Wget 1.15 built on linux-gnu.

+digest +https +ipv6 -iri +large-file -nls +ntlm +opie +ssl/openssl


My working script now contains:

Code: [Select]
#!/bin/ash
BACKUPDIR="/volume1/BACKUPNEW/pfsensebak/backup/daily"
USERNAME="<removed>"
PASSWORD="<removed>"
PORT="80"
SITES="x.x.x.x"
ZIP="/usr/bin/zip"
FIND="/usr/bin/find"
RMFILE="/bin/rm"
WGET="/usr/bin/wget"
BACKUPDAYS="30"
RMFILE="/bin/rm"
cd /volume1/web_backend/tools
for site in $SITES

do
       $WGET -qO- --keep-session-cookies --save-cookies /tmp/cookies.txt --no-check-certificate --timeout=10 http://$site:$PORT/diag_backup.php | grep "name='__csrf_magic'" | sed 's/.*value="\(.*\)".*/\1/' > /tmp/csrf.txt
       $WGET -qO- --keep-session-cookies --load-cookies /tmp/cookies.txt --save-cookies /tmp/cookies.txt --no-check-certificate --post-data "login=Login&usernamefld=$USERNAME&passwordfld=$PASSWORD&__csrf_magic=$(cat /tmp/csrf.txt)" --timeout=10 http://$site:$PORT/diag_backup.php  | grep "name='__csrf_magic'" | sed 's/.*value="\(.*\)".*/\1/' > /tmp/csrf2.txt
       if [ -e /tmp/csrf2.txt ]; then
       DATETMP="`date +%Y%m%d-%H%M%S`"
       FILENAME1="$BACKUPDIR/config-$site-$DATETMP.xml"
       FILENAME2="$BACKUPDIR/config-$site-$DATETMP-withrrd.xml"
                  $WGET --keep-session-cookies --load-cookies /tmp/cookies.txt --no-check-certificate --post-data "download=download&donotbackuprrd=1&__csrf_magic=$(head -n 1 /tmp/csrf2.txt)" --timeout=10 http://$site:$PORT/diag_backup.php -O $FILENAME1 > /dev/null 2>&1
                  $WGET --keep-session-cookies --load-cookies /tmp/cookies.txt --no-check-certificate --post-data "download=download&donotbackuprrd=0&__csrf_magic=$(head -n 1 /tmp/csrf2.txt)" --timeout=120 http://$site:$PORT/diag_backup.php -O $FILENAME2 > /dev/null 2>&1
                  rm -f /tmp/cookies.txt
                  rm -f /tmp/csrf.txt
                  rm -f /tmp/csrf2.txt
       $ZIP -q -9 -j $FILENAME1.zip $FILENAME1
       $ZIP -q -9 -j $FILENAME2.zip $FILENAME2
       $RMFILE $FILENAME1
       $RMFILE $FILENAME2
else
       echo "Failed to retrieve backup from $site"
fi
                                                                                                   
done
                                                                                                             
$FIND $BACKUPDIR -type f -name "*.xml.gz" -mtime +$BACKUPDAYS -exec rm {} \;
$FIND $BACKUPDIR -type f -name "*.xml.zip" -mtime +$BACKUPDAYS -exec rm {} \;



I also have a modified version as well (it requires rar executable as well):

Code: [Select]
#!/bin/ash
BACKUPDIR="/volume1/BACKUPNEW/pfsensebak/hansbuhlin/daily"
USERNAME="<removed>"
PASSWORD="<removed>"
PORT="443"
SITES="x.x.x.x"
RAR="/volume1/web_backend/tools/rar"
FIND="/usr/bin/find"
RMFILE="/bin/rm"
WGET="/usr/bin/wget"
BACKUPDAYS="30"
RMFILE="/bin/rm"
cd /volume1/web_backend/tools
/volume1/web_backend/tools/rar
for site in $SITES

do
       $WGET -qO- --keep-session-cookies --save-cookies /tmp/cookies.txt --no-check-certificate --timeout=10 https://$site:$PORT/diag_backup.php | grep "name='__csrf_magic'" | sed 's/.*value="\(.*\)".*/\1/' > /tmp/csrf.txt
       $WGET -qO- --keep-session-cookies --load-cookies /tmp/cookies.txt --save-cookies /tmp/cookies.txt --no-check-certificate --post-data "login=Login&usernamefld=$USERNAME&passwordfld=$PASSWORD&__csrf_magic=$(cat /tmp/csrf.txt)" --timeout=10 https://$site:$PORT/diag_backup.php  | grep "name='__csrf_magic'" | sed 's/.*value="\(.*\)".*/\1/' > /tmp/csrf2.txt
       if [ -e /tmp/csrf2.txt ]; then
       DATETMP="`date +%Y%m%d-%H%M%S`"
       FILENAME1="$BACKUPDIR/config-$site-$DATETMP-rarpasswordencrypted.xml"
       FILENAME2="$BACKUPDIR/config-$site-$DATETMP-withrrd-rarpasswordencrypted.xml"
                  $WGET --keep-session-cookies --load-cookies /tmp/cookies.txt --no-check-certificate --post-data "download=download&donotbackuprrd=1&__csrf_magic=$(head -n 1 /tmp/csrf2.txt)" --timeout=10 https://$site:$PORT/diag_backup.php -O $FILENAME1 > /dev/null 2>&1
                  $WGET --keep-session-cookies --load-cookies /tmp/cookies.txt --no-check-certificate --post-data "download=download&donotbackuprrd=0&__csrf_magic=$(head -n 1 /tmp/csrf2.txt)" --timeout=120 https://$site:$PORT/diag_backup.php -O $FILENAME2 > /dev/null 2>&1
                  rm -f /tmp/cookies.txt
                  rm -f /tmp/csrf.txt
                  rm -f /tmp/csrf2.txt
       $RAR a -ep -m5 -hp<replace with own password> $FILENAME1.rar $FILENAME1
       $RAR a -ep -m5 -hp<replace with own password> $FILENAME2.rar $FILENAME2
       $RMFILE $FILENAME1
       $RMFILE $FILENAME2
else
       echo "Failed to retrieve backup from $site"
fi
                                                                                                   
done
                                                                                                             
$FIND $BACKUPDIR -type f -name "*.xml.rar" -mtime +$BACKUPDAYS -exec rm {} \;

//Dan Lundqvist

9
IPv6 / Re: pfSense looses config on IPv6 interface after some time.
« on: March 02, 2017, 02:55:15 am »
OK, my bad.  Must have missed that.   Sorry for the confusion.

//Dan Lundqvist

10
IPv6 / pfSense looses config on IPv6 interface after some time.
« on: March 01, 2017, 03:44:10 pm »
Have seen a problem for a long time that has still not been resolved.
I have  Huricane IPv6 GIF and an Interface for this.
If I configure the interface with IP/mask/gateway the config is shown in the gui OK.
BUT then after a few days sometimes weeks suddenly the config is gone in the GUI but also in the XML. 
But traffic still works OK.

- Happens without reboot. Spontaniously after some time.
- Works for a few days/weeks where IPv6 config is seen as normal in GUI but then suddenly it is gone but traffic still works.
- When it occur, ping to remote site still works.  (see ping below)
- See screenshot with dashboard shoing that IPv6 traffic is working even if no config is seen in the GUI.
- Seen also in previous versions upto and including 2.3.3
- Has happened numerous times.
- Also in XML config for interfaces the data is gone.  (See example below)
- Seems like somehow pfSense screws something up and overwrites/removes some of the settings in the IPv6 interface.
  But under the hood, in the OS config, it is still configured and working.
- Never seen it on any other IPv4 interface. It is always the same IPv6 interface.

WHEN OK:
   <opt1>
      <descr><![CDATA[WANv6_TUNNELBROKER]]></descr>
      <if>gif0</if>
      <enable></enable>
      <spoofmac></spoofmac>
      <ipaddrv6>2001:470:27:dd5::2</ipaddrv6>
      <subnetv6>64</subnetv6>

      <gatewayv6>TUNNELBROKERNETGWv6</gatewayv6>
   </opt1>

WHEN FAULT OCCURS:
   <opt1>
      <descr><![CDATA[WANv6_TUNNELBROKER]]></descr>
      <if>gif0</if>
      <enable></enable>
      <spoofmac></spoofmac>
      <gatewayv6>TUNNELBROKERNETGWv6</gatewayv6>
   </opt1>


PING6(56=40+8+8 bytes) 2001:470:27:dd5::2 --> 2001:6b0:8:2::233
16 bytes from 2001:6b0:8:2::233, icmp_seq=0 hlim=57 time=2.123 ms
16 bytes from 2001:6b0:8:2::233, icmp_seq=1 hlim=57 time=6.498 ms
16 bytes from 2001:6b0:8:2::233, icmp_seq=2 hlim=57 time=2.159 ms

--- webc.sunet.se ping6 statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 2.123/3.593/6.498/2.054 ms

See attached screenshots.


This is starting to getting on my nerves as fault still occurs and now I needed to report it.

Please contact me if you need more details for the redmin bug-report or if you want me to write it there myself.

Best regards
Dan Lundqvist
MRZAZ.COM
Stockholm, Sweden

11
I have another small comment that relates to the section on page 35 "RA TAP Bridge" that is worth mentioning
is that neither Android or IOS supports TAP in the current API.  (tried Android 6.0.1 and IOS 10.2.1)

I mentioned that on slide 7.

Yep you did, I could see that now. Maybe a reminder on that page could have been done but lets just drop it
as you have already mentioned it elsewhere which was most important. :-)

//Dan Lundqvist

12
OpenVPN / OpenVPN Client Export fault in export for TAP and IOS/Android
« on: February 28, 2017, 04:32:25 pm »
Hello,

I was just adding a TAP based OpenVPN server (for the purpose of bridging LAN) on port 1195 and already have an existing OpenVPN Server on 1194
and have now found a bug in the Client Export function.

If I select the "1195" server that is TAP based and select "Inline - OpenVPN Connect (IOS and Android)" and save the file
and then try to use it, it complaints against TUN failure instead of TAP.

I then opened the file created and saw that it was missing the "dev tap" option.

Then tried "Inline - Android" with same thing.
If I select "Standard Config - Config Only" then "dev tap" is included on the first line.

Right now, TAP is not available on Android and IOS in non-rooted mode, but still I think this is a bug.  Shouldn't "dev tap" be included in the config file?

Dan Lundqvist
Stockholm, Sweden

13
I have another small comment that relates to the section on page 35 "RA TAP Bridge" that is worth mentioning
is that neither Android or IOS supports TAP in the current API.  (tried Android 6.0.1 and IOS 10.2.1)
- Tested with OpenVPN Connect 1.1.17 (build 76) on Samsung S7 Edge
- Tested with OpenVPN 1.1.1 (build 212) on iPaid Air 2 10.2.1 IOS 64bit

So it seems to only work on desktop apps.
Not meant as a complaint, but rather a observation.

//Dan Lundqvist

14
I think there is a small misstake...
The file had a space included and the upload did not handle this well and broke the link at the space instead of add URL space padding.

If I manually change the URL to http://portal.pfsense.org/webcasts/201702_-_pfSense_Hangout_-_Advanced_OpenVPN_on_pfSense%202.4.pdf
it downloads OK but the original looks like:
http://portal.pfsense.org/webcasts/201702_-_pfSense_Hangout_-_Advanced_OpenVPN_on_pfSense
And the text version looks like:
The slides can be found here: portal.pfsense.org/webcasts/201702_-... 2.4.pdf

Either rename the file and re-upload or correct it by adding %202.4.pdf at the end.

//Dan Lundqvist

15
Small question:
Will the presentation PDF used in this session be available in the Hangout Archive ? (as for other hangouts)
Could not find it there.   Usually it is included below the videofeed but not this time.

Could you make a practice to always add the PDF with the video in the archive at the same time ?

Btw. Thanks for very good and informative presentations. They are really handy and often gives tips
on things I didn't know, (or knew something about but you fill in details even further), even if I consider
myself somewhat advanced.  Keep up the very good work. :-)

Dan Lundqvist
Stockholm, Sweden

Pages: [1] 2 3 4 5 ... 11