pfSense Gold Subscription

Author Topic: Transparent HTTP Proxy + HTTPS/SSL Interception = No HTTPS traffic  (Read 180 times)

0 Members and 1 Guest are viewing this topic.

Offline Stewart

  • Full Member
  • ***
  • Posts: 163
  • Karma: +9/-2
    • View Profile
When I Check to Enable SSL Filtering no HTTPS traffic works.  It doesn't matter if I choose "Splice Whitelist, Bump Otherwise" or "Splice All" all HTTPS sites just time out.  No warnings, no errors, nothing.  Just timeouts.  If I disable that option, all sites immediately start working. 
/var squid/logs is giving a bunch of errors:
Code: [Select]
CPU Usage: 2.640 seconds = 1.597 user + 1.043 sys
Maximum Resident Size: 137136 KB
Page faults with physical i/o: 1
2017/03/20 16:50:01 kid1| Starting Squid Cache version 3.5.23 for amd64-portbld-freebsd10.3...
2017/03/20 16:50:01 kid1| Service Name: squid
2017/03/20 16:50:02| pinger: Initialising ICMP pinger ...
2017/03/20 16:50:12| pinger: Initialising ICMP pinger ...
2017/03/20 17:11:56 kid1| UPGRADE WARNING: URL rewriter reponded with garbage ' ldapprotover 3# No Blockssrc Celestial {        ip     192.168.1.10}# dest blk_BL_adv { domainlist blk_BL_adv/domains   urllist blk_BL_adv/urls redirect http://192.168.1.1:80/sgerror.php?ERR'. Future Squid will treat this as part of the URL.
2017/03/20 17:21:26| pinger: Initialising ICMP pinger ...
2017/03/20 17:21:37| pinger: Initialising ICMP pinger ...
2017/03/20 17:23:55 kid1| Shutdown: NTLM authentication.
2017/03/20 17:23:55 kid1| Shutdown: Negotiate authentication.
2017/03/20 17:23:55 kid1| Shutdown: Digest authentication.
2017/03/20 17:23:55 kid1| Shutdown: Basic authentication.
2017/03/20 17:24:01 kid1| Starting Squid Cache version 3.5.23 for amd64-portbld-freebsd10.3...
2017/03/20 17:24:01 kid1| Service Name: squid
2017/03/20 17:24:01| pinger: Initialising ICMP pinger ...
2017/03/20 17:24:12 kid1| WARNING: redirector #Hlpr7 exited
FATAL: The redirector helpers are crashing too rapidly, need help!

Squid Cache (Version 3.5.23): Terminated abnormally.
CPU Usage: 0.283 seconds = 0.150 user + 0.133 sys
Maximum Resident Size: 137264 KB
Page faults with physical i/o: 1
2017/03/20 17:26:01 kid1| Starting Squid Cache version 3.5.23 for amd64-portbld-freebsd10.3...
2017/03/20 17:26:01 kid1| Service Name: squid
2017/03/20 17:26:02 kid1| /var/run/squid/squid.pid: (2) No such file or directory
2017/03/20 17:26:02 kid1| WARNING: Could not write pid file
2017/03/20 17:26:02| pinger: Initialising ICMP pinger ...
2017/03/20 17:26:28 kid1| UPGRADE WARNING: URL rewriter reponded with garbage ' ldapprotover 3# No Blockssrc Celestial {        ip     192.168.1.10}# dest blk_BL_adv { domainlist blk_BL_adv/domains   urllist blk_BL_adv/urls redirect http://192.168.1.1:80/sgerror.php?ERR'. Future Squid will treat this as part of the URL.
2017/03/20 17:41:37 kid1| Starting new redirector helpers...
2017/03/20 20:38:00 kid1| WARNING: redirector #Hlpr12 exited
2017/03/20 20:38:00 kid1| WARNING: redirector #Hlpr11 exited
2017/03/20 20:38:00 kid1| WARNING: redirector #Hlpr10 exited
2017/03/20 20:38:00 kid1| WARNING: redirector #Hlpr9 exited
2017/03/20 20:38:00 kid1| WARNING: redirector #Hlpr8 exited
2017/03/20 20:38:00 kid1| WARNING: redirector #Hlpr7 exited
2017/03/20 20:38:00 kid1| WARNING: redirector #Hlpr6 exited
2017/03/20 20:38:00 kid1| WARNING: redirector #Hlpr5 exited
2017/03/20 20:38:00 kid1| WARNING: redirector #Hlpr4 exited
2017/03/20 20:38:00 kid1| WARNING: redirector #Hlpr3 exited
2017/03/20 20:38:00 kid1| WARNING: redirector #Hlpr2 exited
2017/03/20 20:38:00 kid1| WARNING: redirector #Hlpr1 exited

I'll look at this again tomorrow but can anyone give me pointers as to why HTTP traffic works but HTTPS traffic just times out?

Offline doktornotor

  • Hero Member
  • *****
  • Posts: 7955
  • Karma: +819/-214
  • Not a pfSense employee, they cannot fire me...
    • View Profile
Re: Transparent HTTP Proxy + HTTPS/SSL Interception = No HTTPS traffic
« Reply #1 on: March 22, 2017, 11:01:07 am »
Get rid of the crashing SquidGuard thing and it will work...
Do NOT PM for help!

Offline Stewart

  • Full Member
  • ***
  • Posts: 163
  • Karma: +9/-2
    • View Profile
Re: Transparent HTTP Proxy + HTTPS/SSL Interception = No HTTPS traffic
« Reply #2 on: March 22, 2017, 02:23:54 pm »
An update.  The issue was that ldap was selected but not configured.  We aren't looking to use that so I've unchecked it and it's working now, almost. 

When squidGuard block an HTTP page, I get the expected block page.
When SquidGuard blocks an HTTPS page, I get an error code:  SSL_ERROR_RX_RECORD_TOO_LONG.  I believe this is because there is an issue with sending HTTPS over HTTP or something like that.  Any idea how to fix this?

Offline doktornotor

  • Hero Member
  • *****
  • Posts: 7955
  • Karma: +819/-214
  • Not a pfSense employee, they cannot fire me...
    • View Profile
Re: Transparent HTTP Proxy + HTTPS/SSL Interception = No HTTPS traffic
« Reply #3 on: March 22, 2017, 02:30:27 pm »
The only idea about SquidGuard is that it's FUBARed and should be removed from pfSense unless someone rewrites it from scratch.
Do NOT PM for help!

Offline Stewart

  • Full Member
  • ***
  • Posts: 163
  • Karma: +9/-2
    • View Profile
The only idea about SquidGuard is that it's FUBARed and should be removed from pfSense unless someone rewrites it from scratch.

We've been using SquidGuard for a while now at various locations.  It works well on HTTP traffic as long as the databases are good.  It also seems to be blocking HTTPS properly but something about the way HTTPS and redirects doesn't allow me to display the block page.  Other than that I haven't had a problem with SquidGuard loading or blocking in a while.  What problems are you referring to?

Offline doktornotor

  • Hero Member
  • *****
  • Posts: 7955
  • Karma: +819/-214
  • Not a pfSense employee, they cannot fire me...
    • View Profile
There are numerous problems unfixed for years that you can find here and at Redmine. Not exactly sure what's the question here. The package is a piece of shit.
Do NOT PM for help!

Offline DutchSamurai

  • Jr. Member
  • **
  • Posts: 33
  • Karma: +1/-0
    • View Profile
But what is the alternative if you want to do transparent https web filtering for free/cheap?