Netgate SG-1000 microFirewall

Author Topic: Guide to filtering web content (http and https) with pfsense 2.3  (Read 57569 times)

0 Members and 3 Guests are viewing this topic.

Offline molykule

  • Newbie
  • *
  • Posts: 15
  • Karma: +1/-0
    • View Profile
Re: Guide to filtering web content (http and https) with pfsense 2.3
« Reply #75 on: April 05, 2017, 12:50:35 pm »
Hi,

I am using the method mentioned here, through unbound. Is there a way i can let few static IP's skip the youtube safe settings in the unbound. I tried asking about segregating those ip's but then i have to use bind which i dont know about. If there is a way to skip the DMZ (i have lan, opt1, opt2 and dmz), that would help also, as i can put those static IP's on DMZ
I think it was mentioned somewhere to use port 5353 and use DNS forwader in conjunction with Unbound, but i cant find any tutorial for it. Also, i cant find any way to split the DNS i dont know if that would help. All I want is some computers can visit youtube and dont get blocked for videos. Even pfsense tutorials get blocked by it.
Any ideas as to where to look for or what to look for would help,
thanks for the great tutorial.

Molykule

Offline huuur

  • Newbie
  • *
  • Posts: 3
  • Karma: +0/-0
    • View Profile
Re: Guide to filtering web content (http and https) with pfsense 2.3
« Reply #76 on: April 10, 2017, 06:50:45 am »
Beautiful guide!
I decided to move from ipcop to pfsense just to filter some https, I followed this guide (with the updates). Everything seems to work fine before some users start complaining mainly android apps not working (play store, snapchat, whatsapp..etc) I managed to locate the blocked (using firewall log) then create a bypass rule with the IP or Port for the whole network, however other clients start having similar issues as if those apps working on different IPs for certain devices (all android)
I read this topic 4 times, reinstall from zero three times without any improvement towards this issue which begins after applying the proxy with wpad.
I appreciate any advice even if an easier approach to filter some unwanted https sites.
Thank you.

Offline aGeekHere

  • Sr. Member
  • ****
  • Posts: 522
  • Karma: +38/-1
    • View Profile
Re: Guide to filtering web content (http and https) with pfsense 2.3
« Reply #77 on: April 10, 2017, 09:59:08 pm »
On the android phone (wireless setting) try setting the proxy settings manually, instead of auto or none.
Never Fear, A Geek is Here!

Offline huuur

  • Newbie
  • *
  • Posts: 3
  • Karma: +0/-0
    • View Profile
Re: Guide to filtering web content (http and https) with pfsense 2.3
« Reply #78 on: April 11, 2017, 07:10:51 am »
On the android phone (wireless setting) try setting the proxy settings manually, instead of auto or none.

Finally, with manual proxy settings I have a stable communication with android devices.

You are the MAN!

Offline huuur

  • Newbie
  • *
  • Posts: 3
  • Karma: +0/-0
    • View Profile
Re: Guide to filtering web content (http and https) with pfsense 2.3
« Reply #79 on: May 23, 2017, 06:26:32 am »


Update Youtube safe mode
Click add under Host overrides
Host = www
Domain = youtube.com
IP =  216.239.38.120
Description = youtube
Save
NOTE: Safe search for youtube is not as advanced as google safe search, which results in a lot of safe content be filtered out.


How can we get this working with mobile devices Android and iOS that use the youtube mobile app or m.youtube.com?

I can't express how useful this guide been for me.

For the record I found the below is working for youtube mobile app safe search:

Code: [Select]
Host = youtube
Domain = googleapis.com
IP =  216.239.38.120
Description = youtube app1

Code: [Select]
Host = youtubei
Domain = googleapis.com
IP =  216.239.38.120
Description = youtube app2

and maybe..
Code: [Select]
Host = www
Domain = youtube-nocookie.com
IP =  216.239.38.120
Description = youtube nocookie

in addition to what's mentioned in Reply#60 for mobile browsers.

Offline aGeekHere

  • Sr. Member
  • ****
  • Posts: 522
  • Karma: +38/-1
    • View Profile
Re: Guide to filtering web content (http and https) with pfsense 2.3
« Reply #80 on: May 24, 2017, 04:41:47 am »
Thanks huuur, that should help others.

Though it would be good if youtube was better at filtering videos.
Never Fear, A Geek is Here!

Offline aGeekHere

  • Sr. Member
  • ****
  • Posts: 522
  • Karma: +38/-1
    • View Profile
Re: Guide to filtering web content (http and https) with pfsense 2.3
« Reply #81 on: June 08, 2017, 08:25:43 pm »
Update

You can try setting up MITM by setting the SSL/MITM Mode to splice all, that way you do not need to create a certificate for each device on the network. (you still need to create a main certificate though).
Never Fear, A Geek is Here!

Offline vielfede

  • Jr. Member
  • **
  • Posts: 90
  • Karma: +1/-0
    • View Profile
Re: Guide to filtering web content (http and https) with pfsense 2.3
« Reply #82 on: June 09, 2017, 05:47:50 am »
Update

You can try setting up MITM by setting the SSL/MITM Mode to splice all, that way you do not need to create a certificate for each device on the network. (you still need to create a main certificate though).

That works fine for me also setting Client proxy settings manually:
Proxy address/PORT= SQUID_IP 3128

And that only! Indeed if i specify different proxy settings for http/https in (client win10=> Internet Settings=>Lan settings=>Advanced=>
- http=SQUID_IP 3128
- https=SQUID_IP 3129

It does not work, as in the contrary I'd excpect....  ::)
Geek can you explain why?

Offline aGeekHere

  • Sr. Member
  • ****
  • Posts: 522
  • Karma: +38/-1
    • View Profile
Re: Guide to filtering web content (http and https) with pfsense 2.3
« Reply #83 on: June 10, 2017, 01:52:00 am »
Does automatically detect settings work?

Never Fear, A Geek is Here!

Offline vielfede

  • Jr. Member
  • **
  • Posts: 90
  • Karma: +1/-0
    • View Profile
Re: Guide to filtering web content (http and https) with pfsense 2.3
« Reply #84 on: June 14, 2017, 07:22:22 am »
Does automatically detect settings work?
Ehmm sorry, could you explain me better what you mean? As I stated I do not use WPAD, I configure proxy manually on client see picture...

Thanks.
PS. Sorry to answer late

Offline aGeekHere

  • Sr. Member
  • ****
  • Posts: 522
  • Karma: +38/-1
    • View Profile
Re: Guide to filtering web content (http and https) with pfsense 2.3
« Reply #85 on: June 19, 2017, 04:10:13 am »
if you have the transparent proxy working then you do not need to define the proxy server.
Never Fear, A Geek is Here!

Offline vielfede

  • Jr. Member
  • **
  • Posts: 90
  • Karma: +1/-0
    • View Profile
Re: Guide to filtering web content (http and https) with pfsense 2.3
« Reply #86 on: June 19, 2017, 09:37:25 am »
If I use it in transparent mode, https does not work!
Better sometimes it works, sometimes it does not! (http works always!)

If anyone managed to get http+https in splice all + transparent mode work please let me know... ;)

Offline aGeekHere

  • Sr. Member
  • ****
  • Posts: 522
  • Karma: +38/-1
    • View Profile
Re: Guide to filtering web content (http and https) with pfsense 2.3
« Reply #87 on: June 19, 2017, 06:57:33 pm »
I just enabled it mitm, added a cert, set it to splice all and its working.
Never Fear, A Geek is Here!

Offline techbee

  • Jr. Member
  • **
  • Posts: 88
  • Karma: +0/-2
    • View Profile
Re: Guide to filtering web content (http and https) with pfsense 2.3
« Reply #88 on: June 20, 2017, 09:51:01 pm »
The guide is update but confusing.  Can you please clean up the guide and have it step by step.  Its somewhat hard to follow if you are not familiar with wpad and related firewall rules. 

Thanks for the guide.

Offline aGeekHere

  • Sr. Member
  • ****
  • Posts: 522
  • Karma: +38/-1
    • View Profile
Re: Guide to filtering web content (http and https) with pfsense 2.3
« Reply #89 on: June 21, 2017, 02:33:14 am »
hi techbee, yeah after discovering a few new things the guide is a little messy now.

The transparent proxy with MITM (splice all) vs wpad should really be divided into two different choices. When I made the guide I did not know that all you need to get transparent proxy for https working is set splice all and create a cert for the router. So in fact you can now choose either using a wpad or the transparent proxy MITM or both.

when i get time I will try an clean it up a bit.


Never Fear, A Geek is Here!