pfSense Support Subscription

Author Topic: no bogonsv6 in tables?  (Read 3141 times)

0 Members and 1 Guest are viewing this topic.

Offline johnpoz

  • Hero Member
  • *****
  • Posts: 2884
    • View Profile
no bogonsv6 in tables?
« on: November 23, 2011, 10:01:59 am »
I don't show any entries in bogonsv6 table under diag?

I have tried toggling the setting, just they are not listed?

gitsync as of this morning, and reboot after gitsync

2.1-DEVELOPMENT (i386)
built on Fri Oct 21 12:51:56 EDT 2011
FreeBSD 8.1-RELEASE-p6

- An intelligent man is sometimes forced to be drunk to spend time with his fools.
- Have I helped you, want to say thanks?  Donate to pfsense the cost of a beer http://pfsense.org/donate.html

Offline databeestje

  • Administrator
  • Hero Member
  • *****
  • Posts: 1048
  • It just might be your luck day, if you only knew.
    • View Profile
Re: no bogonsv6 in tables?
« Reply #1 on: November 23, 2011, 01:52:44 pm »
I will need to verify. I do see we ship with a empty table but the file it fetches should hold some 50k networks.

Offline Cino

  • Hero Member
  • *****
  • Posts: 1048
    • View Profile
Re: no bogonsv6 in tables?
« Reply #2 on: November 23, 2011, 08:35:23 pm »
i manually adding it a while ago and it hit a pf-filter limit. Can't remember what it was but its the same limit that IPBlocker and pfBlocker has when large list.

Offline databeestje

  • Administrator
  • Hero Member
  • *****
  • Posts: 1048
  • It just might be your luck day, if you only knew.
    • View Profile
Re: no bogonsv6 in tables?
« Reply #3 on: November 24, 2011, 12:42:22 am »
It could be that the import is going wrong. Is there contents i. /Etc/bogonsv6 ?

Offline johnpoz

  • Hero Member
  • *****
  • Posts: 2884
    • View Profile
Re: no bogonsv6 in tables?
« Reply #4 on: November 24, 2011, 06:47:28 am »
-rw-r--r--   1 root  wheel          0 Mar  6  2011 bogonsv6

File is empty.
- An intelligent man is sometimes forced to be drunk to spend time with his fools.
- Have I helped you, want to say thanks?  Donate to pfsense the cost of a beer http://pfsense.org/donate.html

Offline databeestje

  • Administrator
  • Hero Member
  • *****
  • Posts: 1048
  • It just might be your luck day, if you only knew.
    • View Profile
Re: no bogonsv6 in tables?
« Reply #5 on: November 24, 2011, 12:44:40 pm »
Arr. That is suspect. Since my install does have a filled file you need to check if your dns works.

There is a script in /etc rc.update_bogons.sh that normally runs. Does that work?

Offline johnpoz

  • Hero Member
  • *****
  • Posts: 2884
    • View Profile
Re: no bogonsv6 in tables?
« Reply #6 on: November 24, 2011, 01:14:34 pm »
I can assure you my dns works ;)

ok I ran that script by hand, it first picked a random number of 53000 something -- that is seconds? So it was going to sleep for 14 hours?  WTF

Nov 24 13:03:23    root: rc.update_bogons.sh is sleeping for 53124

So I killed it and ran it by hand with 10 seconds  And it clearly updated the files.

[2.1-DEVELOPMENT][root@pfsense.local.lan]/etc(18): ls -la bogons*
-rw-r--r--  1 root  wheel     132 Nov 24 13:10 bogons
-rw-r--r--  1 root  wheel  761384 Nov 24 13:10 bogonsv6

Nov 24 13:10:11    root: rc.update_bogons.sh is ending the update cycle.
Nov 24 13:10:11    root: Bogons files downloaded: 49475 addresses added.
Nov 24 13:10:10    root: Bogons file downloaded: no changes.
Nov 24 13:10:08    root: rc.update_bogons.sh is beginning the update cycle.
Nov 24 13:10:08    root: rc.update_bogons.sh is starting up.

So something wrong with cron?  Why has that script not kicked off?  After I kicked it off by hand, the tables loads bogonsv6 -- and yeah its huge and does take a while to load the table ;)


- An intelligent man is sometimes forced to be drunk to spend time with his fools.
- Have I helped you, want to say thanks?  Donate to pfsense the cost of a beer http://pfsense.org/donate.html

Offline johnpoz

  • Hero Member
  • *****
  • Posts: 2884
    • View Profile
Re: no bogonsv6 in tables?
« Reply #7 on: November 30, 2011, 02:53:31 pm »
ok did an update last night to
2.1-DEVELOPMENT (i386)
built on Fri Nov 25 17:45:38 EST 2011
FreeBSD 8.1-RELEASE-p6

and now bogonsv6 is empty again?
No entries exist in this table.

looked at /etc
-rw-r--r--  1 root  wheel  132 Apr 28  2011 bogons
-rw-r--r--  1 root  wheel    0 Mar  6  2011 bogonsv6

So is there something wrong in the cron that should update these?


So I manually ran script rc.update_bogons.sh 10
Nov 30 14:50:29    admin: Bogons files downloaded: 49621 addresses added.

[2.1-DEVELOPMENT][admin@pfsense.local.lan]/etc(19): ls -la bo*
-rw-r--r--  1 root  wheel     132 Nov 30 14:50 bogons
-rw-r--r--  1 root  wheel  763666 Nov 30 14:50 bogonsv6


So what is suppose to update these, do you have to manually run the script - should something kick it off after an update?  On some sort of schedule?
- An intelligent man is sometimes forced to be drunk to spend time with his fools.
- Have I helped you, want to say thanks?  Donate to pfsense the cost of a beer http://pfsense.org/donate.html

Offline iamzam

  • Jr. Member
  • **
  • Posts: 43
    • View Profile
Re: no bogonsv6 in tables?
« Reply #8 on: January 03, 2012, 10:47:59 am »
I am running the IPv6 2.1-DEVELOPMENT (amd64) branch, and I see tons of these in the system log:

Jan  3 09:03:07 pfs root: rc.update_bogons.sh is beginning the update cycle.
Jan  3 09:03:09 pfs root: Could not download http://files.pfsense.org/mirrors/bogon-bn-nonagg.txt.md5 (md5 mismatch)
Jan  3 09:03:09 pfs root: rc.update_bogons.sh is starting up.
Jan  3 09:03:09 pfs root: rc.update_bogons.sh is sleeping for 35036
Jan  3 09:03:09 pfs root: Could not download http://files.pfsense.org/mirrors/fullbogons-ipv6.txt.md5 (md5 mismatch)
Jan  3 09:03:09 pfs root: rc.update_bogons.sh is ending the update cycle.
Jan  3 09:03:09 pfs root: rc.update_bogons.sh is starting up.
Jan  3 09:03:09 pfs root: rc.update_bogons.sh is sleeping for 37978
Jan  3 09:03:52 pfs root: rc.update_bogons.sh is beginning the update cycle.



I logged in via ssh and there are 64 "sh /etc/rc.update_bogons.sh" processes running.

I could kill all the processes and manually download the file and put it in place but if the md5 sums don't match there is possibly a bigger problem than that?

Offline jimp

  • Administrator
  • Hero Member
  • *****
  • Posts: 14931
    • View Profile
Re: no bogonsv6 in tables?
« Reply #9 on: January 03, 2012, 10:56:59 am »
Looks fine from here:

Code: [Select]
[jimp@files mirrors]$ md5 bogon-bn-nonagg.txt
MD5 (bogon-bn-nonagg.txt) = 82bf1e457c3cbb64021067e2a2230051
[jimp@files mirrors]$ cat bogon-bn-nonagg.txt.md5
MD5 (bogon-bn-nonagg.txt) = 82bf1e457c3cbb64021067e2a2230051
[jimp@files mirrors]$ md5 fullbogons-ipv6.txt
MD5 (fullbogons-ipv6.txt) = 0c56fda817b9933477c576a36da0e5b6
[jimp@files mirrors]$ cat fullbogons-ipv6.txt.md5
MD5 (fullbogons-ipv6.txt) = 0c56fda817b9933477c576a36da0e5b6
Need help fast? Commercial Support!

Co-Author of pfSense: The Definitive Guide. - Check the Doc Wiki for FAQs.

Do not PM for help!

Offline iamzam

  • Jr. Member
  • **
  • Posts: 43
    • View Profile
Re: no bogonsv6 in tables?
« Reply #10 on: January 03, 2012, 01:21:22 pm »
I'm not sure what was happening with the md5 sums not matching.  I killed all the sleeping processes (and all the sleep processes too) and i still couldn't run it manually.  I finally downloaded the http://files.pfsense.org/mirrors/fullbogons-ipv6.txt file (and removed the first and last comment lines) and put it into place at /etc/bogonsv6 but the web interface still showed the bogonsv6 table empty. 

I then went into the IPv4 bogons table and clicked the Download the latest bogon data button.  After that the bogonsv6 table was populated.

I do notice one thing with the rc.update_bogons.sh script:

in /etc/rc.update_bogons.sh there is a problem with the last part:

BOGON_MD5=`/usr/bin/fetch -q -o - "http://files.pfsense.org/mirrors/fullbogons-ipv6.txt.md5" | awk '{ print $4 }'`
ON_DISK_MD5=`md5 /tmp/bogonsv6 | awk '{ print $4 }'`
if [ "$BOGON_MD5" = "$ON_DISK_MD5" ]; then
   egrep -v "^#" /tmp/bogonsv6 > /etc/bogonsv6
   /etc/rc.conf_mount_ro
   RESULT=`/sbin/pfctl -t bogonsv6 -T replace -f /etc/bogonsv6 2>&1`
   rm /tmp/bogons
   echo "Bogons files downloaded:  $RESULT" | logger
else
   echo "Could not download http://files.pfsense.org/mirrors/fullbogons-ipv6.txt.md5 (md5 mismatch)" | logger
   # Relaunch and sleep
   sh /etc/rc.update_bogons.sh &    
fi


It writes the downloaded file (minus the '^#'  lines) to /tmp/bogonsv6 but after replacing /etc/bogonsv6 it deletes /tmp/bogons instead of /tmp/bogonsv6

It is possible, i suppose, that if the script was run multiple times simultaneously the IPv6 part would delete the /tmp/bogons file while the other script was attempting to use it to update /etc/bogons

Offline jimp

  • Administrator
  • Hero Member
  • *****
  • Posts: 14931
    • View Profile
Re: no bogonsv6 in tables?
« Reply #11 on: January 03, 2012, 02:05:25 pm »
I fixed that line in the script just now.

Not sure how you could have had so many of them sitting there unfinished though, that is a bit perplexing.
Need help fast? Commercial Support!

Co-Author of pfSense: The Definitive Guide. - Check the Doc Wiki for FAQs.

Do not PM for help!

Offline iamzam

  • Jr. Member
  • **
  • Posts: 43
    • View Profile
Re: no bogonsv6 in tables?
« Reply #12 on: January 03, 2012, 03:24:53 pm »
Thanks jimp for fixing that!

It is strange since my crontab has the rc.update_bogons.sh running once a day,

...although each time the rc.update_bogons.sh script is run, it has the initial sleep plus each section has an additional relaunch and sleep in it for a total of 4 relaunch and 5 sleeps if it has major problems...
Such as if the WAN interface is down, or some other problem such as md5 (weird i know).

maybe an exit 1 should be called after the first relaunch so that it doesn't relaunch up to 4 times/script and start a relaunch cascade!

« Last Edit: January 03, 2012, 03:28:34 pm by iamzam »