Netgate SG-1000 microFirewall

Author Topic: [SOLVED]Help needed: bypass squid and squidGuard for iTunes, AppleStore, Android  (Read 379 times)

0 Members and 1 Guest are viewing this topic.

Offline GL

  • Newbie
  • *
  • Posts: 17
  • Karma: +0/-0
    • View Profile
Hello All,
I have been able to apply the Windows Upgrade bypass without problems in the SquidGuard.conf file, so that all my Win devices can update bypassing Squid (running with https transparent proxy and squidGuard).

I have problems with my Apple deivces which cannot access itunes, apple store nor upgrade.
The only solution found on the net is to list Apple ip addresses in the squid webgui.

Apple, however, is using Akamai network, as Microsoft, and trying to bypass squid via IP addresses is pratically impossible (Akamai network is huge). If you input some Akamai address you find via nslookup in squid bypass, the patch is working for a while before you will be redirected to some other server.

I am sure there exist a way to let Apple devices bypass squid and squidGuard via squid.conf and squidGuard.conf files, but I have not been able to reach the result. The only success for me has been to stop squid from restarting up to when I did not restore the old squid.conf file...

Is there anybody who can tell us how, what and where we should add/modify in the squid.conf and SquidGuard.conf files, as per the Wind Update bypass, to get the result ( ie bypassing squid when addressing apple web sites)?

Thanks in advance to the nice and kind guy who will give us the instructions.
I am sure that your answer will become one of the most viewed posts on this board....

Ciao,

GL
« Last Edit: December 19, 2017, 04:42:01 pm by GL »

Offline GL

  • Newbie
  • *
  • Posts: 17
  • Karma: +0/-0
    • View Profile
Re: Help needed: bypass squid and squidGuard for iTunes and AppleStore
« Reply #1 on: December 17, 2017, 12:56:09 pm »
just un update: I would like to bypass squid only when reaching itunes and AppleStore for upgrading and/or downloading new apps.
during normal navigation I would like to be proxied by squid.

If I remove SSL transparet proxy option and leave squid as http transparent proxy only, everything works fine.
« Last Edit: December 17, 2017, 02:36:33 pm by GL »

Offline slim2016

  • Newbie
  • *
  • Posts: 16
  • Karma: +1/-0
    • View Profile
Re: Help needed: bypass squid and squidGuard for iTunes and AppleStore
« Reply #2 on: December 17, 2017, 03:19:52 pm »
squid proxy server /  general /[Transparent Proxy Settings] - Bypass Proxy for These Destination IPs

That should do the trick

Offline GL

  • Newbie
  • *
  • Posts: 17
  • Karma: +0/-0
    • View Profile
Re: Help needed: bypass squid and squidGuard for iTunes and AppleStore
« Reply #3 on: December 17, 2017, 03:30:48 pm »
Hello ,
thanks for your reply but this is not the definitive solution.
apple is using akamai network and their servers ip are continuosly changing...
adding the IPs to squid works for a while, then you are back with the problem.

The IP s are also geolocated and changing country by country.

Is there a definitive set of squid directives to be added to squid.conf and squidGuard.conf to fix the problem forever?

After adding the Windows Update fixing to squidGuard.conf, I am sure this can be done also for Apple, but my knowledge of squid  is not allowing me to do that alone.

Pls, help me and the rest of pfsense supporters who are also Apple users.
thans in advance.

Ciao,
GL

Offline sichent

  • Jr. Member
  • **
  • Posts: 56
  • Karma: +9/-0
    • View Profile
Re: Help needed: bypass squid and squidGuard for iTunes and AppleStore
« Reply #4 on: December 18, 2017, 02:37:46 am »
Not a user of transparent proxy but not sure why adding these as SNI bypass should not work on the recent Squid
https://docs.diladele.com/faq/squid/sslbump_exlusions/apple_app_store.html

Offline GL

  • Newbie
  • *
  • Posts: 17
  • Karma: +0/-0
    • View Profile
Re: Help needed: bypass squid and squidGuard for iTunes and AppleStore
« Reply #5 on: December 18, 2017, 07:33:07 am »
thanks for the reply, however actually the webgui allows you to add exclusions only with IPs, and because Apple is using Akamai, servers IP are floating and they are potentially hundreds...
we need to add the exclusion in the config files, as per the Windows Update workaround, the problem is that me and most of the not advanced users do not know how.
I tried to replicate that solution using the windows update workaround, but it is not working. I am missing something...
Is there anyone able to help us?
Thanks in advance.
Ciao,
GL

Offline GL

  • Newbie
  • *
  • Posts: 17
  • Karma: +0/-0
    • View Profile
Re: [SOLVED]Help needed: bypass squid and squidGuard for iTunes and AppleStore
« Reply #6 on: December 18, 2017, 04:14:28 pm »
I found the solution: simply there is no solution.
If you use Squid as https transparent proxy, the only info not encrypted that reaches Squid is the IP of the servers, so Squid has no opportunity to read the domain names of the servers.
If you want to use Squid on https, you must use it as explicit proxy.
This is also why in pfsense webgui you can exclude Squid only based on IPs and not domain names.

Offline sichent

  • Jr. Member
  • **
  • Posts: 56
  • Karma: +9/-0
    • View Profile
Re: [SOLVED]Help needed: bypass squid and squidGuard for iTunes and AppleStore
« Reply #7 on: December 19, 2017, 03:53:04 am »
Not exactly correct; the version of Squid in pfSense is good enough to use the SNI info from the HTTPS connection being established.
The acl you need to use is ssl::server_name

For example this what we use when filtering all connections except spliced in Web Safety (does not apply to SquidGuard!!):


acl ssl_exclude_domains ssl::server_name "/opt/websafety/etc/squid/ssl/exclude/domains.conf"

ssl_bump peek step1 all
ssl_bump splice ssl_exclude_domains
ssl_bump stare step2 all
ssl_bump bump all

I am not sure where to put that in pfSense UI - but should not be very complex to find.


Offline GL

  • Newbie
  • *
  • Posts: 17
  • Karma: +0/-0
    • View Profile
Re: [SOLVED]Help needed: bypass squid and squidGuard for iTunes and AppleStore
« Reply #8 on: December 19, 2017, 07:54:56 am »
thanks for your input, I will try this we and will let everybody know the outcome.
Cross our fingers...
I think this will be a very important input for most of the "home" users.
ciao,
GL

Offline GL

  • Newbie
  • *
  • Posts: 17
  • Karma: +0/-0
    • View Profile
Re: [SOLVED]Help needed: bypass squid and squidGuard for iTunes and AppleStore
« Reply #9 on: December 19, 2017, 04:05:59 pm »
I implemented the solution and for now it seems working, I am testing it.
here a step by step guide of what i implemented.

1) Go to Services->Squid Proxy Server

2) Enable and configure HTTPS transparent proxy

3) Go to the bottom of the page, click Show Advanced Options

4) Cut and past the following text in the box "Custom Options (Before Auth)":

acl ssl_exclude_domains ssl::server_name "/usr/local/etc/squid/exclude_domains.conf"

ssl_bump peek step1 all
ssl_bump splice ssl_exclude_domains
ssl_bump stare step2 all
ssl_bump bump all

5) wait to save

6) connect with secure shell to pfsense and login

7) choose option 8 "Shell"

8) cd /usr/local/etc/squid

9) ee exclude_domains.conf

10) input the following text :

.apple.com
.mzstatic.com
.icloud.com
.dropbox.com
.microsoft.com
.oneDrive.com
.live.com
.messenger.live.com
.skype.com
.trouter.com
.login.live.com
.whatsapp.com
.whatsapp.net

11) press "esc" then press "a", again press "a"

12) go back to pfsense webgui and save your squid configuration

13) restart squid service

The configuration in exclude_domains.conf should let you use your Apple devices with iTunes and Apple store, let you use WhatsApp, sync with iCloud (pls check also allowed ports in your firewall rules), sync with OneDrive, let your skype work with https transparent proxy, allow you to update with Microsoft in Win10.

I am still testing, if someone will implement it, pls let us know the outcomes.
« Last Edit: December 19, 2017, 04:10:01 pm by GL »

Offline GL

  • Newbie
  • *
  • Posts: 17
  • Karma: +0/-0
    • View Profile
Re: [SOLVED]Help needed: bypass squid and squidGuard for iTunes and AppleStore
« Reply #10 on: December 19, 2017, 04:41:04 pm »
Adding support also for Android devices.

Add following entries to exclude_domains.conf file:

.ggpht.com
.play.googleapis.com
android.clients.google.com
www.googleapis.com
.gvt1.com

It seems working for me, up to now.

Offline GL

  • Newbie
  • *
  • Posts: 17
  • Karma: +0/-0
    • View Profile
Adding following entries to exclude_domains.conf you should bypass Netflix. I am still testing this.

.netflix.com
.llnwd.net
.edgesuite.net
.nflximg.com
.nflxvideo.net

Offline GL

  • Newbie
  • *
  • Posts: 17
  • Karma: +0/-0
    • View Profile
Up to today no main issues found.
Everything seems working well.
Thanks to sichent for his help and suggestion!