Netgate SG-1000 microFirewall

Author Topic: pfBlockerNG configuration for a newbie :)  (Read 743 times)

0 Members and 1 Guest are viewing this topic.

Offline belgiumrom

  • Newbie
  • *
  • Posts: 14
  • Karma: +0/-0
    • View Profile
pfBlockerNG configuration for a newbie :)
« on: December 30, 2017, 09:58:36 am »
Hello.

I finally managed to install and briefly configure pfBlockerNG.
It seems to work if I try to browse directly to a blocked website (see logs below) and I am getting the black page.
But when the domain shows as an ad into a webpage, it seems that it's not blocked.
Am I doing something wrong or it needs more config?

DNSBL Reject,Dec 30 09:44:16,www.steepto.com,10.10.10.40, | / | Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML like Gecko) Chrome/62.0.3202.94 Safari/537.36 OPR/49.0.2725.64
DNSBL Reject,Dec 30 09:44:16,www.steepto.com,10.10.10.40,http://www.steepto.com/ | /favicon.ico | Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML like Gecko) Chrome/62.0.3202.94 Safari/537.36 OPR/49.0.2725.64
DNSBL Reject,Dec 30 09:44:31,www.steepto.com,10.10.10.40, | /ghits | Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML like Gecko) Chrome/62.0.3202.94 Safari/537.36 OPR/49.0.2725.64
DNSBL Reject,Dec 30 09:52:05,trafficfactory.biz,10.10.10.40, | / | Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML like Gecko) Chrome/62.0.3202.94 Safari/537.36 OPR/49.0.2725.64
---
DNSBL Reject,Dec 30 10:43:25,www.iwanttodeliver.com,10.10.10.40, | / | Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML like Gecko) Chrome/62.0.3202.94 Safari/537.36 OPR/49.0.2725.64
DNSBL Reject,Dec 30 10:43:26,www.iwanttodeliver.com,10.10.10.40,http://www.iwanttodeliver.com/ | /favicon.ico | Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML like Gecko) Chrome/62.0.3202.94 Safari/537.36 OPR/49.0.2725.64

Offline belgiumrom

  • Newbie
  • *
  • Posts: 14
  • Karma: +0/-0
    • View Profile
Re: pfBlockerNG configuration for a newbie :)
« Reply #1 on: December 30, 2017, 10:31:16 am »
Some other logs:

DNSBL Reject HTTPS,Dec 30 11:08:56,aax-us-east.amazon-adsystem.com
DNSBL Reject HTTPS,Dec 30 11:08:56,aax-us-east.amazon-adsystem.com
DNSBL Reject HTTPS,Dec 30 11:08:56,aax-us-east.amazon-adsystem.com
DNSBL Reject HTTPS,Dec 30 11:08:56,aax-us-east.amazon-adsystem.com
DNSBL Reject HTTPS,Dec 30 11:08:57,aax-us-east.amazon-adsystem.com
DNSBL Reject HTTPS,Dec 30 11:08:57,aax-us-east.amazon-adsystem.com
DNSBL Reject HTTPS,Dec 30 11:08:58,aax-us-east.amazon-adsystem.com
DNSBL Reject HTTPS,Dec 30 11:08:59,s.amazon-adsystem.com
DNSBL Reject HTTPS,Dec 30 11:08:59,s.amazon-adsystem.com
DNSBL Reject HTTPS,Dec 30 11:09:17,s.amazon-adsystem.com
DNSBL Reject HTTPS,Dec 30 11:09:17,s.amazon-adsystem.com
DNSBL Reject HTTPS,Dec 30 11:10:28,cm.g.doubleclick.net
DNSBL Reject HTTPS,Dec 30 11:10:28,googleads.g.doubleclick.net
DNSBL Reject HTTPS,Dec 30 11:10:28,googleads.g.doubleclick.net
DNSBL Reject HTTPS,Dec 30 11:10:28,googleads.g.doubleclick.net
DNSBL Reject HTTPS,Dec 30 11:10:30,googleads.g.doubleclick.net
DNSBL Reject HTTPS,Dec 30 11:10:30,googleads.g.doubleclick.net
DNSBL Reject HTTPS,Dec 30 11:10:30,googleads.g.doubleclick.net
DNSBL Reject HTTPS,Dec 30 11:10:30,googleads.g.doubleclick.net
DNSBL Reject HTTPS,Dec 30 11:10:30,googleads.g.doubleclick.net
DNSBL Reject HTTPS,Dec 30 11:10:30,googleads.g.doubleclick.net
DNSBL Reject HTTPS,Dec 30 11:10:30,googleads.g.doubleclick.net
DNSBL Reject HTTPS,Dec 30 11:10:30,googleads.g.doubleclick.net
DNSBL Reject HTTPS,Dec 30 11:11:01,googleads.g.doubleclick.net
DNSBL Reject HTTPS,Dec 30 11:11:01,googleads.g.doubleclick.net
DNSBL Reject HTTPS,Dec 30 11:11:01,cm.g.doubleclick.net
DNSBL Reject HTTPS,Dec 30 11:11:01,googleads.g.doubleclick.net
DNSBL Reject HTTPS,Dec 30 11:11:01,googleads.g.doubleclick.net
DNSBL Reject HTTPS,Dec 30 11:11:01,googleads.g.doubleclick.net
DNSBL Reject HTTPS,Dec 30 11:11:01,googleads.g.doubleclick.net
DNSBL Reject HTTPS,Dec 30 11:11:01,googleads.g.doubleclick.net
DNSBL Reject HTTPS,Dec 30 11:11:01,googleads.g.doubleclick.net
DNSBL Reject HTTPS,Dec 30 11:11:01,googleads.g.doubleclick.net
DNSBL Reject HTTPS,Dec 30 11:11:01,googleads.g.doubleclick.net
DNSBL Reject HTTPS,Dec 30 11:11:01,googleads.g.doubleclick.net
DNSBL Reject HTTPS,Dec 30 11:11:01,googleads.g.doubleclick.net
DNSBL Reject HTTPS,Dec 30 11:11:28,aax-us-east.amazon-adsystem.com
DNSBL Reject HTTPS,Dec 30 11:11:28,aax-us-east.amazon-adsystem.com
DNSBL Reject HTTPS,Dec 30 11:11:28,aax-us-east.amazon-adsystem.com
DNSBL Reject HTTPS,Dec 30 11:11:28,aax-us-east.amazon-adsystem.com
DNSBL Reject HTTPS,Dec 30 11:11:30,s.amazon-adsystem.com
DNSBL Reject HTTPS,Dec 30 11:11:30,s.amazon-adsystem.com
DNSBL Reject HTTPS,Dec 30 11:15:08,googleads.g.doubleclick.net
DNSBL Reject HTTPS,Dec 30 11:15:09,googleads.g.doubleclick.net
DNSBL Reject HTTPS,Dec 30 11:15:09,cm.g.doubleclick.net
DNSBL Reject HTTPS,Dec 30 11:15:09,googleads.g.doubleclick.net
DNSBL Reject HTTPS,Dec 30 11:15:09,googleads.g.doubleclick.net
DNSBL Reject HTTPS,Dec 30 11:15:09,googleads.g.doubleclick.net
DNSBL Reject HTTPS,Dec 30 11:15:09,googleads.g.doubleclick.net
DNSBL Reject HTTPS,Dec 30 11:15:09,googleads.g.doubleclick.net
DNSBL Reject HTTPS,Dec 30 11:15:09,googleads.g.doubleclick.net
DNSBL Reject HTTPS,Dec 30 11:15:09,googleads.g.doubleclick.net
DNSBL Reject HTTPS,Dec 30 11:15:09,googleads.g.doubleclick.net
DNSBL Reject HTTPS,Dec 30 11:15:09,googleads.g.doubleclick.net
DNSBL Reject HTTPS,Dec 30 11:15:09,googleads.g.doubleclick.net
DNSBL Reject HTTPS,Dec 30 11:15:09,googleads.g.doubleclick.net
DNSBL Reject HTTPS,Dec 30 11:26:48,googleads.g.doubleclick.net

Offline BBcan177

  • Moderator
  • Hero Member
  • *****
  • Posts: 2601
  • Karma: +810/-5
    • View Profile
    • Click for Support
Re: pfBlockerNG configuration for a newbie :)
« Reply #2 on: December 30, 2017, 02:45:49 pm »
Hello.

I finally managed to install and briefly configure pfBlockerNG.
It seems to work if I try to browse directly to a blocked website (see logs below) and I am getting the black page.
But when the domain shows as an ad into a webpage, it seems that it's not blocked.
Am I doing something wrong or it needs more config?

Blocking depends on what Feeds you add to DNSBL...

You can test with from pfSense:
Code: [Select]
host -t A example.com
or in Windows:
Code: [Select]
whois example.com
If its blocked via DNSBL, it will reply with the DNSBL VIP address that you configured.

If you see an AD in the browser, right-click on it, and click Inspect.... This will show the Domain name (if its based on a remote domain) and you can see if that Domain is listed in your DNSBL Feeds:

Code: [Select]
grep "example\.com" /var/db/pfblockerng/dnsbl/*
"Experience is something you don't get until just after you need it."

 | http://pfblockerng.com | Twitter @BBcan177  | #pfBlockerNG |

Offline belgiumrom

  • Newbie
  • *
  • Posts: 14
  • Karma: +0/-0
    • View Profile
Re: pfBlockerNG configuration for a newbie :)
« Reply #3 on: December 31, 2017, 10:28:22 am »
Thanks for your answer.

The domain in discussion is www.steepto.com and it is blocked as per below (domain is listed in my feeds, 192.168.0.1 is my VIP):

[2.4.2-RELEASE][root@pfsense]/root: grep "steepto\.com" /var/db/pfblockerng/dnsbl/*
/var/db/pfblockerng/dnsbl/ad_servers.txt:local-data: "imgg.steepto.com 60 IN A 192.168.0.1"
/var/db/pfblockerng/dnsbl/easylist.txt:local-data: "steepto.com 60 IN A 192.168.0.1"

...or from a host within LAN:

user@host_name ~ $ host -t A steepto.com
steepto.com has address 192.168.0.1

user@host_name ~ $ nslookup steepto.com
Server:      10.10.10.100
Address:   10.10.10.100#53

Non-authoritative answer:
Name:   steepto.com
Address: 192.168.0.1

But when browsing to a website (http://filmehd.net) that uses this domain for ads, the ads are still showing.
The URL from "inspect element" is:

<a target="_blank" href="http://www.steepto.com/ghits/2507986/i/25265/0/src/138805866/pp/3/1?h=Aw40v9imdPgmzIBP3kQbsvmxgwyG8OaOEvVUBcSTU5V_Og2ICWIO2mvzA9MO39ny&amp;tt=Direct&amp;k=185868fcNHr4D3XaZfZ*fWCtYgDRfWCtYuwof.*fNBfgfJVffJfb*fj~fX~fLb6LbBfgfQf%3AfaHR0cDovL2ZpbG1laGQubmV0Lw%3D%3DfaHR0cDovL2ZpbG1laGQubmV0Lw%3D%3DfaHR0cDovL2ZpbG1laGQubmV0Lw%3D%3Df%3DfNBfgfJWfKfcfT*faQfQafWCtYgOhf!fTW96aWxsY%2481LjAgKFgxMTsgTGludXggeDg2XzY0K%24BBcHBsZVdlYktpdC81MzcuMzYgKEtIVE1MLCBsaWtlIEdlY2tvK%24BDaHJvbWUvNjIuMC4zMjAyLjk0IFNhZmFya%2481MzcuMzYgT1B%24LzQ5LjAuMjcyN%2442NA%3D%3DfTmV3c2ZlZWRoYW5kbGVy*FdpZGV2aW5lQ29udGVudERlY3J5cHRpb25Nb2R1bGU%3Df%3AfTGludXggeDg2XzY0fLTMwMA%3D%3Df%3DfaQfP_f!faf!f%3Bf*f*" data-hash="Aw40v9imdPgmzIBP3kQbsvmxgwyG8OaOEvVUBcSTU5V_Og2ICWIO2mvzA9MO39ny" rel="nofollow"> <div class="image-container"> <img class="mcimg" width="492" height="328" data-i="2507986" src="blob:http://filmehd.net/d658c290-651c-436d-a286-c471d02ba9ff" style=""> <!--intExchangeWagesImagePlace--> </div> [/url]

Somehow it slips through configuration.

I was wondering if there is a way to particularly block this "steepto.com" domain within "filmehd.net"... some custom configuration for this site only.

Thank you



Offline BBcan177

  • Moderator
  • Hero Member
  • *****
  • Posts: 2601
  • Karma: +810/-5
    • View Profile
    • Click for Support
Re: pfBlockerNG configuration for a newbie :)
« Reply #4 on: December 31, 2017, 10:33:40 am »
The issue is that you don't have the TLD option enabled. So as is, it's only blocking the exact domain/sub-domain that is listed in the feeds. You can see from the grep command above that "www." is not listed, so it's not blocked.   

When TLD is enabled it will check each domain that is listed in the selected feeds to determine if it should block the whole sub-domain with a Redirect Zone in the resolver. 

So you have two choices. Add the "www." domain to a DNSBL customlist to get that domain blocked. Or enable TLD.
"Experience is something you don't get until just after you need it."

 | http://pfblockerng.com | Twitter @BBcan177  | #pfBlockerNG |

Offline belgiumrom

  • Newbie
  • *
  • Posts: 14
  • Karma: +0/-0
    • View Profile
Re: pfBlockerNG configuration for a newbie :)
« Reply #5 on: December 31, 2017, 03:09:22 pm »
Thanks for your suggestions.

I enabled TLD, even added www.steepto.com into TLD blacklist, it showed up in update logs, but ads still showing on that website :(

------------------------------------------
Assembling database... completed
Executing TLD
 Blocking full TLD/Sub-Domain(s)... |www.steepto.com| completed
TLD analysis. completed
Finalizing TLD...  completed
 ----------------------------------------

[2.4.2-RELEASE][root@pfsense]/root: grep "steepto\.com" /var/db/pfblockerng/dnsbl/*
/var/db/pfblockerng/dnsbl/DNSBL_TLD.txt:www.steepto.com
/var/db/pfblockerng/dnsbl/easylist.txt:local-data: "steepto.com 60 IN A 192.168.0.1"

-----------------------------------------

Stumped...

Offline RonpfS

  • Hero Member
  • *****
  • Posts: 705
  • Karma: +96/-2
    • View Profile
Re: pfBlockerNG configuration for a newbie :)
« Reply #6 on: December 31, 2017, 05:01:52 pm »
Without TLD enabled, you add www.steepto.com to custom whitelist

With TLD Enabled it's steepto.com that is the TLD not www.steepto.com
Code: [Select]
/var/unbound/pfb_dnsbl.conf:898777:local-zone: "steepto.com" redirect local-data: "steepto.com 60 IN A 10.10.10.1"DNSBL Update gathers (and remove) all the *.steepto.com domain names from the tables and configure Outbound to answer the VIP for any request ending in .steepto.com.

So if your custom whitelist has .steepto.com it will disable blocking for all *.steepto.com domain names.

Now if you place steepto.com into the TLD exclude list, DNSBL Update gathers all the *.steepto.com as listed from the tables. You might end up with 100s of domains in the block list as TLD didn't consolidate the domain names. At this point if you put .steepto.com into the custom whitelist, this will whitelist all sub-domains.

However, you can just put www.steepto.com and/or steepto.com into the DNSBL Custom Whitelist and leave all other *.steepto.com domain names blocked.



« Last Edit: December 31, 2017, 05:23:11 pm by RonpfS »
2.3.5-RELEASE-p1 (amd64)
Intel Core2 Quad CPU Q8400 @ 2.66GHz 8GB
pfBlockerNG 2.1.2_2/Dev, suricata 4.0.1_1

Offline belgiumrom

  • Newbie
  • *
  • Posts: 14
  • Karma: +0/-0
    • View Profile
Re: pfBlockerNG configuration for a newbie :)
« Reply #7 on: January 03, 2018, 11:07:22 am »
Hello again,

Thanks for both solutions from BBcan177 and RonpfS, unfortunately none works.
I am sure I am doing something wrong...
Browsing to http://filmehd.net still serves me with ads from www.steepto.com.
Watching what happens in a terminal (tail -f /var/log/pfblockerng/dnsbl.log) when I access filmehd.net, doesn't log any ads blocked as compared with many other sites...
It looks like the domain (www.steepto.com) is whitelisted or there is a rule that ignores ads on filmehd.net completely.
Presently, steepto.com is in TLD blacklist with TLD enabled.

[2.4.2-RELEASE][root@pfsense]/root: grep "steepto\.com" /var/db/pfblockerng/dnsbl/*
/var/db/pfblockerng/dnsbl/DNSBL_TLD.txt:steepto.com
/var/db/pfblockerng/dnsbl/easy_list.txt:local-data: "steepto.com 60 IN A 192.168.0.1"

I apologize if I am being too insistent on this, it's really important for me that I block ads on that specific website (filmehd.net).




Offline RonpfS

  • Hero Member
  • *****
  • Posts: 705
  • Karma: +96/-2
    • View Profile
Re: pfBlockerNG configuration for a newbie :)
« Reply #8 on: January 03, 2018, 12:24:07 pm »
Here is my infoblock about TLD :
Quote
TLD
Enable This is an Advanced process to determine if all Sub-Domains should be blocked for each listed Domain.
Click infoblock before enabling this feature! 
This Feature is not recommended for Low-Perfomance/Low-Memory installations!
Definition: TLD -  represents the last segment of a domain name. IE: example.com (TLD = com), example.uk.com (TLD = uk.com)

The 'Unbound Resolver Reloads' can take several seconds or more to complete and may temporarily interrupt DNS Resolution until the Resolver has been fully Reloaded with the updated Domain changes. Consider updating the DNSBL Feeds 'Once per Day', if network issues arise.

When enabled and after all downloads for DNSBL Feeds have completed; TLD will process the Domains.
TLD uses a predetermined list of TLDs, to determine if the listed Domain should be configured to block all Sub-Domains.
The predetermined TLD list can be found in  /usr/local/pkg/pfblockerng/dnsbl_tld

To exclude a TLD/Domain from the TLD process, add the TLD/Domain to the TLD Exclusion. The specific Domain will be Blocked, but all other Sub-Domains will only be blocked if they are listed elsewhere. Whitelisting a Domain in the Custom Domain Whitelist can also be used to bypass TLD, however, the listed Domain will not be Blocked.

TLD Blacklist, can be used to block whole TLDs.  IE: xyz
TLD Whitelist is only used in conjunction with TLD Blacklist and is used to allow access to a Domain that is being blocked by a TLD Blacklist.

When Enabling/Disabling TLD, a Force Reload - DNSBL is required.

Once the TLD Domain limit below is exceeded, the balance of the Domains will be listed as-is. IE: Blocking only the listed Domain (Not Sub-Domains)
TLD Domain Limit Restrictions:

    < 1.0GB RAM - Max 100k Domains
    < 1.5GB RAM - Max 150k Domains
    < 2.0GB RAM - Max 200k Domains
    < 2.5GB RAM - Max 250k Domains
    < 3.0GB RAM - Max 400k Domains
    < 4.0GB RAM - Max 600k Domains
    < 5.0GB RAM - Max 1.0M Domains
    < 6.0GB RAM - Max 1.5M Domains
    < 7.0GB RAM - Max 2.5M Domains
    > 7.0GB RAM - > 2.5M Domains

Here is my infoblock about TLD Blacklist/Whitelist :
Quote
Note:
The TLD Blacklist is used to block a whole TLD (IE: pw).
The TLD Whitelist is used to allow access to the specific domain/sub-domains that is blocked by a TLD Blacklist; while blocking all others.
TLD Blacklist/Whitelist: A static zone entry is used in the DNS Resolver for this feature, therefore no Alerts will be generated.

Enter one   TLD  per line. ie: xyz
No Regex Entries and no leading/trailing 'dot' allowed!

TLD Blacklist is to use for TOP TLD  domains likes .ru .cn .pw etc. As stated no alerts are generated for these domains.

Your configuration doesn't even do that as the steepto.com isn't a Static zone. As it is now, only http://steepto.com will be blocked, not other subdomain.
If I look at my setup, all subdomains of steepto.com are blocked with a redirect zone
Code: [Select]
grep steepto.com /var/db/pfblockerng/dnsbl/*.txt /var/db/pfblockerng/dnsblorig/*.orig /var/unbound/pfb_dnsbl.conf /usr/local/pkg/pfblockerng/dnsbl_tld

/var/db/pfblockerng/dnsbl/EasyListWOE.txt:local-data: "steepto.com 60 IN A 10.10.10.1"
/var/db/pfblockerng/dnsblorig/ADs_SquidBL.orig:steepto.com
/var/db/pfblockerng/dnsblorig/ADs_hpHosts_ads.orig:127.0.0.1 imgg.steepto.com
/var/db/pfblockerng/dnsblorig/EasyListWOE.orig:||steepto.com^$third-party
/var/db/pfblockerng/dnsblorig/EasyList_French.orig:||steepto.com^$popup,third-party
/var/db/pfblockerng/dnsblorig/Malic2_JL_BD.orig:127.0.0.1 imgg.steepto.com
/var/db/pfblockerng/dnsblorig/Malic2_Quidsup_Trackers.orig:steepto.com
/var/unbound/pfb_dnsbl.conf:local-zone: "steepto.com" redirect local-data: "steepto.com 60 IN A 10.10.10.1"

So in you case if you want to block all subdomains, first remove steepto.com from the TLD Exclude list, run a Force Reload DNSBL, then see what is in pfb_dnsbl.conf, as long as it is in a table (dnsblorig) you should end up with something similar to my setup

If you don't have steepto.com in your table, you can add it to the table DNSBL Custom_List.
2.3.5-RELEASE-p1 (amd64)
Intel Core2 Quad CPU Q8400 @ 2.66GHz 8GB
pfBlockerNG 2.1.2_2/Dev, suricata 4.0.1_1

Offline belgiumrom

  • Newbie
  • *
  • Posts: 14
  • Karma: +0/-0
    • View Profile
Re: pfBlockerNG configuration for a newbie :)
« Reply #9 on: January 03, 2018, 01:43:29 pm »
RonpfS, thanks for your suggestion, will remove steepto.com from TLD blacklist, apparently I didn't understand properly the function...
Funny thing, I have similar output for a grep for this domain in my setup:

[2.4.2-RELEASE][root@pfsense]/root: grep steepto.com /var/db/pfblockerng/dnsbl/*.txt /var/db/pfblockerng/dnsblorig/*.orig /var/unbound/pfb_dnsbl.conf /usr/local/pkg/pfblockerng/dnsbl_tld
/var/db/pfblockerng/dnsbl/DNSBL_TLD.txt:steepto.com
/var/db/pfblockerng/dnsbl/easy_list.txt:local-data: "steepto.com 60 IN A 192.168.0.1"
/var/db/pfblockerng/dnsblorig/ad_servers.orig:127.0.0.1   imgg.steepto.com
/var/db/pfblockerng/dnsblorig/easy_list.orig:||steepto.com^$third-party
/var/db/pfblockerng/dnsblorig/easylist.orig:||steepto.com^$third-party
/var/db/pfblockerng/dnsblorig/hpHosts.orig:127.0.0.1   imgg.steepto.com
/var/db/pfblockerng/dnsblorig/hpHosts_ads.orig:127.0.0.1   imgg.steepto.com
/var/unbound/pfb_dnsbl.conf:local-zone: "steepto.com" redirect local-data: "steepto.com 60 IN A 192.168.0.1"

Will let you know of the results later.

Thanks again for assistance!

Offline BBcan177

  • Moderator
  • Hero Member
  • *****
  • Posts: 2601
  • Karma: +810/-5
    • View Profile
    • Click for Support
Re: pfBlockerNG configuration for a newbie :)
« Reply #10 on: January 03, 2018, 03:59:51 pm »
From your LAN device, what do you get with this command:

Code: [Select]
host -t A steepto.com
It should reply with the DNSBL VIP address of 192.168.0.1...

I assume that you are not using this address range 192.168.0.x in your network, as the DNSBL VIP needs to be in a unique IP Range...

If you browse to "www.steepto.com" or "steepto.com" it should get the 1x1 pix.

Do you have an Antivirus package installed that might be overriding the DNS requests?
« Last Edit: January 05, 2018, 09:05:00 am by BBcan177 »
"Experience is something you don't get until just after you need it."

 | http://pfblockerng.com | Twitter @BBcan177  | #pfBlockerNG |

Offline belgiumrom

  • Newbie
  • *
  • Posts: 14
  • Karma: +0/-0
    • View Profile
Re: pfBlockerNG configuration for a newbie :)
« Reply #11 on: January 03, 2018, 05:44:28 pm »
Hi again,

@RonpfS

Removing steepto.com from TLD blocking and force reloading does not do the trick...

@BBcan177

My LAN is as follows:
Pfsense as gateway/firewall/dhcp/pfblocker/squid/squidguard/dns resolver in forwarding mode. There is a clamd embedded in squid proxy though. No other antivirus inside LAN.
Another Ubuntu 16.04 server with bind9 dns server installed but this is forwarding all DNS requests from LAN to the pfsense DNS. All LAN devices are Linux mint and they are getting DNS servers automatically along with a fixed IP address from pfsense dhcp in this order: ubuntu first, pfsense second.
True, I don't use the 192.168.x.x range in my LAN, I use 10.10.x.x, that's why I set the 192.168.0.1 as VIP.

Now, I come to realise there are some weird behaviours in my LAN.
To start, if I browse directly to www.steepto.com from any device within LAN, I get the 1x1 pixel page. Up to here we're good.

Doing whois steepto.com from LAN device, I get:

user@lan_machine ~ $ whois steepto.com
   Domain Name: STEEPTO.COM
   Registry Domain ID: 1848412964_DOMAIN_COM-VRSN
   Registrar WHOIS Server: whois.godaddy.com
   Registrar URL: http://www.godaddy.com
   Updated Date: 2017-10-31T16:39:37Z
   Creation Date: 2014-02-27T11:22:59Z
   Registry Expiry Date: 2018-02-27T11:22:59Z
   Registrar: GoDaddy.com, LLC
   Registrar IANA ID: 146
   Registrar Abuse Contact Email: abuse@godaddy.com
   Registrar Abuse Contact Phone: 480-624-2505
   Domain Status: clientDeleteProhibited https://icann.org/epp#clientDeleteProhibited
   Domain Status: clientRenewProhibited https://icann.org/epp#clientRenewProhibited
   Domain Status: clientTransferProhibited https://icann.org/epp#clientTransferProhibited
   Domain Status: clientUpdateProhibited https://icann.org/epp#clientUpdateProhibited
   Name Server: NS.STEEPTO.COM
   Name Server: NS3.STEEPTO.COM
   DNSSEC: unsigned
   URL of the ICANN Whois Inaccuracy Complaint Form: https://www.icann.org/wicf/
>>> Last update of whois database: 2018-01-03T23:03:29Z <<<

For more information on Whois status codes, please visit https://icann.org/epp

NOTICE: The expiration date displayed in this record is the date the
registrar's sponsorship of the domain name registration in the registry is
currently set to expire. This date does not necessarily reflect the expiration
date of the domain name registrant's agreement with the sponsoring
registrar.  Users may consult the sponsoring registrar's Whois database to
view the registrar's reported date of expiration for this registration.

TERMS OF USE: You are not authorized to access or query our Whois
database through the use of electronic processes that are high-volume and
automated except as reasonably necessary to register domain names or
modify existing registrations; the Data in VeriSign Global Registry
Services' ("VeriSign") Whois database is provided by VeriSign for
information purposes only, and to assist persons in obtaining information
about or related to a domain name registration record. VeriSign does not
guarantee its accuracy. By submitting a Whois query, you agree to abide
by the following terms of use: You agree that you may use this Data only
for lawful purposes and that under no circumstances will you use this Data
to: (1) allow, enable, or otherwise support the transmission of mass
unsolicited, commercial advertising or solicitations via e-mail, telephone,
or facsimile; or (2) enable high volume, automated, electronic processes
that apply to VeriSign (or its computer systems). The compilation,
repackaging, dissemination or other use of this Data is expressly
prohibited without the prior written consent of VeriSign. You agree not to
use electronic processes that are automated and high-volume to access or
query the Whois database except as reasonably necessary to register
domain names or modify existing registrations. VeriSign reserves the right
to restrict your access to the Whois database in its sole discretion to ensure
operational stability.  VeriSign may restrict or terminate your access to the
Whois database for failure to abide by these terms of use. VeriSign
reserves the right to modify these terms at any time.

The Registry database contains ONLY .COM, .NET, .EDU domains and
Registrars.

Doing whois steepto.com from pfsense, I get:

[2.4.2-RELEASE][root@pfsense]/root: whois steepto.com
% IANA WHOIS server
% for more information on IANA, visit http://www.iana.org
% This query returned 1 object

refer:        whois.verisign-grs.com

domain:       COM

organisation: VeriSign Global Registry Services
address:      12061 Bluemont Way
address:      Reston Virginia 20190
address:      United States

contact:      administrative
name:         Registry Customer Service
organisation: VeriSign Global Registry Services
address:      12061 Bluemont Way
address:      Reston Virginia 20190
address:      United States
phone:        +1 703 925-6999
fax-no:       +1 703 948 3978
e-mail:       info@verisign-grs.com

contact:      technical
name:         Registry Customer Service
organisation: VeriSign Global Registry Services
address:      12061 Bluemont Way
address:      Reston Virginia 20190
address:      United States
phone:        +1 703 925-6999
fax-no:       +1 703 948 3978
e-mail:       info@verisign-grs.com

nserver:      A.GTLD-SERVERS.NET 192.5.6.30 2001:503:a83e:0:0:0:2:30
nserver:      B.GTLD-SERVERS.NET 192.33.14.30 2001:503:231d:0:0:0:2:30
nserver:      C.GTLD-SERVERS.NET 192.26.92.30 2001:503:83eb:0:0:0:0:30
nserver:      D.GTLD-SERVERS.NET 192.31.80.30 2001:500:856e:0:0:0:0:30
nserver:      E.GTLD-SERVERS.NET 192.12.94.30 2001:502:1ca1:0:0:0:0:30
nserver:      F.GTLD-SERVERS.NET 192.35.51.30 2001:503:d414:0:0:0:0:30
nserver:      G.GTLD-SERVERS.NET 192.42.93.30 2001:503:eea3:0:0:0:0:30
nserver:      H.GTLD-SERVERS.NET 192.54.112.30 2001:502:8cc:0:0:0:0:30
nserver:      I.GTLD-SERVERS.NET 192.43.172.30 2001:503:39c1:0:0:0:0:30
nserver:      J.GTLD-SERVERS.NET 192.48.79.30 2001:502:7094:0:0:0:0:30
nserver:      K.GTLD-SERVERS.NET 192.52.178.30 2001:503:d2d:0:0:0:0:30
nserver:      L.GTLD-SERVERS.NET 192.41.162.30 2001:500:d937:0:0:0:0:30
nserver:      M.GTLD-SERVERS.NET 192.55.83.30 2001:501:b1f9:0:0:0:0:30
ds-rdata:     30909 8 2 E2D3C916F6DEEAC73294E8268FB5885044A833FC5459588F4A9184CFC41A5766

whois:        whois.verisign-grs.com

status:       ACTIVE
remarks:      Registration information: http://www.verisigninc.com

created:      1985-01-01
changed:      2017-10-05
source:       IANA

   Domain Name: STEEPTO.COM
   Registry Domain ID: 1848412964_DOMAIN_COM-VRSN
   Registrar WHOIS Server: whois.godaddy.com
   Registrar URL: http://www.godaddy.com
   Updated Date: 2017-10-31T16:39:37Z
   Creation Date: 2014-02-27T11:22:59Z
   Registry Expiry Date: 2018-02-27T11:22:59Z
   Registrar: GoDaddy.com, LLC
   Registrar IANA ID: 146
   Registrar Abuse Contact Email: abuse@godaddy.com
   Registrar Abuse Contact Phone: 480-624-2505
   Domain Status: clientDeleteProhibited https://icann.org/epp#clientDeleteProhibited
   Domain Status: clientRenewProhibited https://icann.org/epp#clientRenewProhibited
   Domain Status: clientTransferProhibited https://icann.org/epp#clientTransferProhibited
   Domain Status: clientUpdateProhibited https://icann.org/epp#clientUpdateProhibited
   Name Server: NS.STEEPTO.COM
   Name Server: NS3.STEEPTO.COM
   DNSSEC: unsigned
   URL of the ICANN Whois Inaccuracy Complaint Form: https://www.icann.org/wicf/
>>> Last update of whois database: 2018-01-03T23:04:28Z <<<

Domain Name: STEEPTO.COM
Registrar URL: http://www.godaddy.com
Registrant Name: Bedigital Corporation
Registrant Organization: Bedigital Corporation
Name Server: NS.STEEPTO.COM
Name Server: NS3.STEEPTO.COM
DNSSEC: unsigned

For complete domain details go to:
http://who.godaddy.com/whoischeck.aspx?domain=STEEPTO.COM

The data contained in GoDaddy.com, LLC's WhoIs database,
while believed by the company to be reliable, is provided "as is"
with no guarantee or warranties regarding its accuracy.  This
information is provided for the sole purpose of assisting you
in obtaining information about domain name registration records.
Any use of this data for any other purpose is expressly forbidden without the prior written
permission of GoDaddy.com, LLC.  By submitting an inquiry,
you agree to these terms of usage and limitations of warranty.  In particular,
you agree not to use this data to allow, enable, or otherwise make possible,
dissemination or collection of this data, in part or in its entirety, for any
purpose, such as the transmission of unsolicited advertising and
and solicitations of any kind, including spam.  You further agree
not to use this data to enable high volume, automated or robotic electronic
processes designed to collect or compile this data for any purpose,
including mining this data for your own personal or commercial purposes.

Please note: the registrant of the domain name is specified
in the "registrant" section.  In most cases, GoDaddy.com, LLC
is not the registrant of domain names listed in this database.

But...

Doing a "dig steepto.com" from any machine within LAN, I get (please note the answering DNS is Ubuntu and the IP address returned is my VIP, 182.168.0.1):

user@lan_machine ~ $ dig steepto.com

; <<>> DiG 9.10.3-P4-Ubuntu <<>> steepto.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 8485
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 13, ADDITIONAL: 16

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;steepto.com.         IN   A

;; ANSWER SECTION:
steepto.com.      9   IN   A   192.168.0.1

;; AUTHORITY SECTION:
com.         74130   IN   NS   b.gtld-servers.net.
com.         74130   IN   NS   e.gtld-servers.net.
com.         74130   IN   NS   c.gtld-servers.net.
com.         74130   IN   NS   h.gtld-servers.net.
com.         74130   IN   NS   k.gtld-servers.net.
com.         74130   IN   NS   a.gtld-servers.net.
com.         74130   IN   NS   d.gtld-servers.net.
com.         74130   IN   NS   i.gtld-servers.net.
com.         74130   IN   NS   l.gtld-servers.net.
com.         74130   IN   NS   f.gtld-servers.net.
com.         74130   IN   NS   m.gtld-servers.net.
com.         74130   IN   NS   j.gtld-servers.net.
com.         74130   IN   NS   g.gtld-servers.net.

;; ADDITIONAL SECTION:
a.gtld-servers.net.   7499   IN   A   192.5.6.30
a.gtld-servers.net.   5006   IN   AAAA   2001:503:a83e::2:30
b.gtld-servers.net.   17421   IN   A   192.33.14.30
c.gtld-servers.net.   77756   IN   A   192.26.92.30
d.gtld-servers.net.   79   IN   A   192.31.80.30
e.gtld-servers.net.   2470   IN   A   192.12.94.30
f.gtld-servers.net.   990   IN   A   192.35.51.30
g.gtld-servers.net.   3781   IN   A   192.42.93.30
h.gtld-servers.net.   82835   IN   A   192.54.112.30
i.gtld-servers.net.   8375   IN   A   192.43.172.30
j.gtld-servers.net.   17421   IN   A   192.48.79.30
k.gtld-servers.net.   2470   IN   A   192.52.178.30
l.gtld-servers.net.   3810   IN   A   192.41.162.30
l.gtld-servers.net.   1840   IN   AAAA   2001:500:d937::30
m.gtld-servers.net.   17426   IN   A   192.55.83.30

;; Query time: 0 msec
;; SERVER: 10.10.10.100#53(10.10.10.100)
;; WHEN: Wed Jan 03 18:07:46 EST 2018
;; MSG SIZE  rcvd: 544

Doing "dig @pfsense steepto.com" (directing dig to pfsense DNS) from the same LAN machine, I get:

user@lan_machine ~ $ dig @pfsense steepto.com

; <<>> DiG 9.10.3-P4-Ubuntu <<>> @pfsense steepto.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 47814
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;steepto.com.         IN   A

;; ANSWER SECTION:
steepto.com.      60   IN   A   192.168.0.1

;; Query time: 0 msec
;; SERVER: 10.10.10.1#53(10.10.10.1)
;; WHEN: Wed Jan 03 18:08:13 EST 2018
;; MSG SIZE  rcvd: 56

Please notice again the IP address returned for steepto.com is my VIP.

Other tools are reporting the same result:

user@lan_machine ~ $ host steepto.com
steepto.com has address 192.168.0.1

user@lan_machine ~ $ nslookup steepto.com
Server:      10.10.10.100
Address:   10.10.10.100#53

Non-authoritative answer:
Name:   steepto.com
Address: 192.168.0.1

Redirecting DNS query to pfsense DNS:

user@lan_machine ~ $ nslookup steepto.com pfsense
Server:      pfsense
Address:   10.10.10.1#53

Name:   steepto.com
Address: 192.168.0.1

I read somewhere that pfblockerng does not like running together with squid/squidguard. Could this be my issue?

Thanks for your suggestions.


Offline BBcan177

  • Moderator
  • Hero Member
  • *****
  • Posts: 2601
  • Karma: +810/-5
    • View Profile
    • Click for Support
Re: pfBlockerNG configuration for a newbie :)
« Reply #12 on: January 05, 2018, 09:08:29 am »
I asked to run a "whois example.com" but I guess my brain was somewhere else when I wrote that  :o

Should run a "host -t A" or nslookup or dig or ping command to see if the Domains reply with the DNSBL VIP...

So I see from your commands that its replying with the DNSBL VIP.... so that part is working... You just need to see if the proxy or something else is causing your issue?  Do you have an AV solution that is protecting for DNS sinkhole behaviour?
"Experience is something you don't get until just after you need it."

 | http://pfblockerng.com | Twitter @BBcan177  | #pfBlockerNG |

Offline belgiumrom

  • Newbie
  • *
  • Posts: 14
  • Karma: +0/-0
    • View Profile
Re: pfBlockerNG configuration for a newbie :)
« Reply #13 on: January 06, 2018, 10:16:22 am »
LOL, yes, it sounded strange in my brain too...

Yes, there is the clamd antivirus that's embedded into squid proxy, and it's enabled but I fail to see why that would interfere with dnsbl.
Funny thing, in my endeavours of blocking this steepto.com domain, I was actually blocking it in squid access control lists too under "blacklist" but that didn't make any difference so I removed it.
To test, I disabled first clamd. No difference. Then I disabled squid proxy and squidguard. No difference as well.

...and yes, my pfblocker is working beautifully except this stubborn domain...

I am stumped again...

Offline belgiumrom

  • Newbie
  • *
  • Posts: 14
  • Karma: +0/-0
    • View Profile
Re: pfBlockerNG configuration for a newbie :)
« Reply #14 on: January 06, 2018, 10:23:43 am »
I posted a pic with all the services I have enabled in pfsense. Do you see anything that could intefere?

Also, when the lists are reloaded, I see that the hard limit of domains is overpassed, could that be an issue too?

------------------------------------------
Assembling database... completed
Executing TLD
TLD analysis....xxxxxxxxxxx completed
** TLD Domain count exceeded. [ 400000 ] All subsequent Domains listed as-is **
Finalizing TLD...  completed
 ----------------------------------------