Netgate SG-1000 microFirewall

Author Topic: Guide to filtering web content (http and https) with pfsense 2.3  (Read 91341 times)

0 Members and 1 Guest are viewing this topic.

Offline aGeekHere

  • Sr. Member
  • ****
  • Posts: 535
  • Karma: +46/-1
    • View Profile
Re: Guide to filtering web content (http and https) with pfsense 2.3
« Reply #105 on: July 04, 2017, 12:52:43 am »
firefox ie or chrome?

Quote
I use splice-all.  Don't I really need to install certs to client device ?
not when splice all

Quote
If https site is successfully blocked, what error page do I expect to see ?
The default cannot access page of the web browser.

If you enabled squid, transparent mode and SSL filtering Splice all with a cert on pfsense and you still cannot access https sites, then something is broken.
Try reinstall.
If you still have the same issue post screen shots of your setup.
Never Fear, A Geek is Here!

Offline techbee

  • Jr. Member
  • **
  • Posts: 88
  • Karma: +0/-2
    • View Profile
Re: Guide to filtering web content (http and https) with pfsense 2.3
« Reply #106 on: July 04, 2017, 01:17:42 am »
firefox ie or chrome?

Quote
I use splice-all.  Don't I really need to install certs to client device ?
not when splice all

Quote
If https site is successfully blocked, what error page do I expect to see ?
The default cannot access page of the web browser.

If you enabled squid, transparent mode and SSL filtering Splice all with a cert on pfsense and you still cannot access https sites, then something is broken.
Try reinstall.
If you still have the same issue post screen shots of your setup.

I am using chrome. most of the users are chrome.
On the other hand, the firewall rules you shown was to block port 80 and 443 right and redirect dns resolution to pfsense dns server and no more else.  That means, I don't need to redirect traffic from port 80 to 3128 and 443 to 3129 right ?

Offline aGeekHere

  • Sr. Member
  • ****
  • Posts: 535
  • Karma: +46/-1
    • View Profile
Re: Guide to filtering web content (http and https) with pfsense 2.3
« Reply #107 on: July 04, 2017, 01:32:37 am »
Quote
That means, I don't need to redirect traffic from port 80 to 3128 and 443 to 3129 right
That is what the transparent proxy is doing, remove that rule, that could be what is causing the issue.
Never Fear, A Geek is Here!

Offline techbee

  • Jr. Member
  • **
  • Posts: 88
  • Karma: +0/-2
    • View Profile
Re: Guide to filtering web content (http and https) with pfsense 2.3
« Reply #108 on: July 04, 2017, 01:44:02 am »
Quote
That means, I don't need to redirect traffic from port 80 to 3128 and 443 to 3129 right
That is what the transparent proxy is doing, remove that rule, that could be what is causing the issue.

Alright

So what I have now on my firewall rules are:
1. antilock rule on port 80 which says (allow all on tcp port 80)
2. block LAN_NET on tcp port 80 and 443
3. redirect rule for dns resolution

would that be ok ?

On the other hand, what is the default cannot access page of the web browser, is it the same like of that of http transparent proxy error page ?
« Last Edit: July 04, 2017, 02:07:32 am by techbee »

Offline techbee

  • Jr. Member
  • **
  • Posts: 88
  • Karma: +0/-2
    • View Profile
Re: Guide to filtering web content (http and https) with pfsense 2.3
« Reply #109 on: July 04, 2017, 11:37:21 pm »
aGeekHere,

My wpad now works.  I can download it from chrome browser.
I can ping and nslookup wpad now.
All squidguard group ACL and its target rules are applied to block facebook and youtube for testing purposes for now.
I HAVE NOT INSTALLED THE CA CERT ON CLIENT DEVICES.

I used http and https filtering with splice all option so that I dont need to install the cert to client devices.

I also cleared browser cache and history.

The result is I can browse http but cannot browse https.  Https sites like yahoo and google mail works though.

I have screenshot of the error for browsing https site.

Offline aGeekHere

  • Sr. Member
  • ****
  • Posts: 535
  • Karma: +46/-1
    • View Profile
Re: Guide to filtering web content (http and https) with pfsense 2.3
« Reply #110 on: July 05, 2017, 12:11:00 am »
Is squidguard blocking it by mistake? is it set to deny all?

Try allowing all in squidguard rules to test.

Post screen shots of squidguard.

Try clearing states in pfsense.

Reboot.
Never Fear, A Geek is Here!

Offline techbee

  • Jr. Member
  • **
  • Posts: 88
  • Karma: +0/-2
    • View Profile
Re: Guide to filtering web content (http and https) with pfsense 2.3
« Reply #111 on: July 05, 2017, 01:13:55 am »
Squidguard is not blocking by mistake.

I tried clearing firewall states and Rebooted, still the same situation.

Offline techbee

  • Jr. Member
  • **
  • Posts: 88
  • Karma: +0/-2
    • View Profile
Re: Guide to filtering web content (http and https) with pfsense 2.3
« Reply #112 on: July 05, 2017, 01:27:21 am »
I also have this wireshark done where I can see that it made a CONNECT method to facebook, then it has established connection, send client hello but later have RST and there is no SERVER HELLO thus no application data received.

What could I miss here.

Offline aGeekHere

  • Sr. Member
  • ****
  • Posts: 535
  • Karma: +46/-1
    • View Profile
Re: Guide to filtering web content (http and https) with pfsense 2.3
« Reply #113 on: July 05, 2017, 02:03:18 am »
lets try without the wpad and use just the transparent, set proxy direct does it work then?
If you still cannot access https sites then post you squid config.
Never Fear, A Geek is Here!

Offline techbee

  • Jr. Member
  • **
  • Posts: 88
  • Karma: +0/-2
    • View Profile
Re: Guide to filtering web content (http and https) with pfsense 2.3
« Reply #114 on: July 05, 2017, 02:08:27 am »
lets try without the wpad and use just the transparent, set proxy direct does it work then?
If you still cannot access https sites then post you squid config.


If I use only the http proxy without the ssl mitm and set browser to auto, then the http proxy filter works.

If you are saying to setup http transparent proxy and ssl mitm filtering and install cert to client device and set browser proxy, then I have not done this yet. 

Offline aGeekHere

  • Sr. Member
  • ****
  • Posts: 535
  • Karma: +46/-1
    • View Profile
Re: Guide to filtering web content (http and https) with pfsense 2.3
« Reply #115 on: July 05, 2017, 02:35:09 am »
Quote
and install cert to client device and set browser proxy, then I have not done this yet.

Just use splice all for mitm (and use the cert created in pfsense under SSL Man In the Middle Filtering\CA). 
Set browser to direct.
This will use transparent proxy for both http and https.

If this works then there is an issue with your WPAD.
If this does not work then something is broken.
Never Fear, A Geek is Here!

Offline techbee

  • Jr. Member
  • **
  • Posts: 88
  • Karma: +0/-2
    • View Profile
Re: Guide to filtering web content (http and https) with pfsense 2.3
« Reply #116 on: July 05, 2017, 02:41:06 am »
Quote
and install cert to client device and set browser proxy, then I have not done this yet.

Just use splice all for mitm (and use the cert created in pfsense under SSL Man In the Middle Filtering\CA). 
Set browser to direct.
This will use transparent proxy for both http and https.

If this works then there is an issue with your WPAD.
If this does not work then something is broken.

What do you mean for "Set browser to direct", is this the automatic detect settings ?

On the other hand, my wpad only have the following inside:

 function FindProxyForURL(url,host)
 {
 return "PROXY 192.168.1.1:3128";
 }

I am not sure what you example below means so I use the example from pfsense wiki.

function FindProxyForURL(url, host)
{
    if (isPlainHostName(host) ||
        shExpMatch(host, "*.local") ||
        isInNet(dnsResolve(host), "192.168.1.0",  "255.255.255.0"))
        return "DIRECT";
 
    return "PROXY 192.168.1.1:3128";
}
« Last Edit: July 05, 2017, 02:44:07 am by techbee »

Offline aGeekHere

  • Sr. Member
  • ****
  • Posts: 535
  • Karma: +46/-1
    • View Profile
Re: Guide to filtering web content (http and https) with pfsense 2.3
« Reply #117 on: July 05, 2017, 02:44:18 am »
yeah turn off automatic detect setting so the traffic goes direct and gets intercepted by the transparent proxy.
Never Fear, A Geek is Here!

Offline techbee

  • Jr. Member
  • **
  • Posts: 88
  • Karma: +0/-2
    • View Profile
Re: Guide to filtering web content (http and https) with pfsense 2.3
« Reply #118 on: July 05, 2017, 03:00:11 am »
yeah turn off automatic detect setting so the traffic goes direct and gets intercepted by the transparent proxy.

By turn off, you mean uncheck the automatic detect settings and others options as well ?

Offline aGeekHere

  • Sr. Member
  • ****
  • Posts: 535
  • Karma: +46/-1
    • View Profile
Re: Guide to filtering web content (http and https) with pfsense 2.3
« Reply #119 on: July 05, 2017, 03:27:08 am »
yes
Never Fear, A Geek is Here!