pfSense Support Subscription

Author Topic: Improve Custom refresh pattern  (Read 9861 times)

0 Members and 1 Guest are viewing this topic.

Offline aGeekHere

  • Sr. Member
  • ****
  • Posts: 525
  • Karma: +43/-1
    • View Profile
Improve Custom refresh pattern
« on: May 09, 2016, 06:10:59 am »
Hi all, been trying to get the most out of squid cache and its refresh patterns.

Please post improvements.

Updated - Added kivimart refresh patterns

Updated - Added deajan refresh patterns

Updated - Added Valeriy refresh patterns

Updated - added kikawala refresh patterns

Updated on 3 March 2017
Code: [Select]

#new refresh patterns 3
acl Windows_Update dstdomain windowsupdate.microsoft.com
acl Windows_Update dstdomain .update.microsoft.com
acl Windows_Update dstdomain download.windowsupdate.com
acl Windows_Update dstdomain www.download.windowsupdate.com
acl Windows_Update dstdomain au.download.windowsupdate.com
acl Windows_Update dstdomain bg.v4.pr.dl.ws.microsoft.com

#new refresh patterns 2
refresh_pattern -i (\.|-)(ini|def|sig|upt|mid|midi|mpg|mpeg|ram|cav|acc|alz|apk|at3|bke|arc|ass|ba|big|bik|bkf|bld|c4|cals|clipflair|cpt|daa|dmg|ddz|dpe|egg|egt|ecab|ess|esd|gho|ghs|gz|ipg|jar|lbr|lqr|lha|lz|lzo|lzma|lzx|mbw|mc.meta|mpq|nth|osz|pak|par|par2|paf|pyk|pk3|pk4|rag|sen|sitx|skb|tb|tib|uha|uue|viv|vsa|z|zoo|nrg|adf|adz|dms|dsk|d64|sdi|mds|mdx|cdi|cue|cif|c2d|daa|b6t)(\?.*)?$ 43200 100% 432000 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-no-store ignore-private ignore-auth
#end new refresh patterns 2

#new refresh patterns
refresh_pattern -i (\.|-)(mp3|m4a|aa?c3?|wm?av?|og(x|v|a|g)|ape|mka|au|aiff|zip|flac|m4(b|r)|m1v|m2(v|p)|mo(d|v)|arj|appx|lha|lzh|on2)(\?.*)?$ 43200 100% 432000 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-no-store ignore-private ignore-auth
refresh_pattern -i (\.|-)(exe|bin|(n|t)ar|acv|(r|j)ar|t?gz|(g|b)z(ip)?2?|7?z(ip)?|wm[v|a]|patch|diff|mar|vpu|inc|r(a|p)m|kom|iso|sys|[ap]sf|ms[i|u|f]|dat|msi|cab|psf|dvr-ms|ace|asx|qt|xt|esd)(\?.*)?$ 43200 100% 432000 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-no-store ignore-private ignore-auth
refresh_pattern -i (\.|-)(ico(.*)?|pn[pg]|css|(g|t)iff?|jpe?g(2|3|4)?|psd|c(d|b)r|cad|bmp|img)(\?.*)?$ 43200 100% 432000 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-no-store ignore-private ignore-auth
refresh_pattern -i (\.|-)(webm|(x-)?swf|mp(eg)?(3|4)|mpe?g(av)?|(x-)?f(l|4)v|divx?|rmvb?|mov|trp|ts|avi|m38u|wmv|wmp|m4v|mkv|asf|dv|vob|3gp?2?)(\?.*)?$ 43200 100% 432000 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-no-store ignore-private ignore-auth
refresh_pattern -i (\.|-)(docx?|xlsx?|pptx?|rtf|xml|pdf|tiff?|txt)(\?.*)?$ 43200 100% 432000 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-no-store ignore-private ignore-auth
#Website
refresh_pattern -i (\.|-)(xml|js|jsp|txt|css)(\?.*)?$ 360 40% 1440 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-no-store ignore-private ignore-auth
refresh_pattern -i .index.(html|htm)$ 0 40% 1440
refresh_pattern . 30 25% 1440
#end new refresh patterns

refresh_pattern -i \.(3gp|7z|ace|asx|avi|bin|cab|dat|deb|rpm|divx|dvr-ms)      129600 100% 129600 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.(rar|jar|gz|tgz|tar|bz2|iso|m1v|m2(v|p)|mo(d|v)|(x-|)flv) 129600 100% 129600 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.(jp(e?g|e|2)|gif|pn[pg]|bm?|tiff?|ico|swf|css|js)         129600 100% 129600 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.(mp(e?g|a|e|1|2|3|4)|mk(a|v)|ms(i|u|p))                   129600 100% 129600 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.(og(x|v|a|g)|rar|rm|r(a|p)m|snd|vob|wav)                  129600 100% 129600 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.(pp(s|t)|wax|wm(a|v)|wmx|wpl|zip|cb(r|z|t))               129600 100% 129600 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims ignore-reload

refresh_pattern ^gopher:  1440  0%  1440
refresh_pattern ^ftp:    10080 95% 43200 override-lastmod reload-into-ims

refresh_pattern -i \.(doc|pdf)$           100080 90% 43200 override-expire ignore-no-cache ignore-no-store ignore-private reload-into-ims
refresh_pattern -i \.(html|htm)$          1440   40% 40320 ignore-no-cache ignore-no-store ignore-private override-expire reload-into-ims
refresh_pattern (Release|Packages(.gz)*)$    0   20%  2880
refresh_pattern .                          180   95% 43200 override-lastmod reload-into-ims

# 1 year = 525600 mins, 1 month = 43800 mins
refresh_pattern -i (/cgi-bin/|\?)         0      0%      0
refresh_pattern \.(ico|video-stats)$ 129600 100% 129600 override-expire ignore-reload ignore-no-cache ignore-no-store ignore-private ignore-auth override-lastmod ignore-must-revalidate

refresh_pattern imeem.*\.flv$                           0     0%         0 override-lastmod override-expire
refresh_pattern \.rapidshare.*\/[0-9]*\/.*\/[^\/]* 161280    90%    161280 ignore-reload

refresh_pattern (get_video\?|videoplayback\?|videodownload\?|\.flv?) 129600 100% 129600 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims
refresh_pattern (get_video\?|videoplayback\?id|videoplayback.*id|videodownload\?|\.flv?) 129600 100% 129600 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims
refresh_pattern ^.*(utm\.gif|ads\?|rmxads\.com|ad\.z5x\.net|bh\.contextweb\.com|bstats\.adbrite\.com|a1\.interclick\.com|ad\.trafficmp\.com|ads\.cubics\.com|ad\.xtendmedia\.com|\.googlesyndication\.com|advertising\.com|yieldmanager|game-advertising\.com|pixel\.quantserve\.com|adperium\.com|doubleclick\.net|adserving\.cpxinteractive\.com|syndication\.com|media.fastclick.net).* 129600 20% 129600 ignore-no-cache ignore-no-store ignore-private override-expire ignore-reload ignore-auth ignore-must-revalidate

refresh_pattern ^.*safebrowsing.*google                                  129600 100% 129600 override-expire ignore-reload ignore-no-cache ignore-private ignore-auth ignore-must-revalidate
refresh_pattern ^http://((cbk|mt|khm|mlt)[0-9]?)\.google\.co(m|\.uk)     129600 100% 129600 override-expire ignore-reload ignore-private
refresh_pattern ytimg\.com.*\.jpg                                        129600 100% 129600 override-expire ignore-reload
refresh_pattern images\.friendster\.com.*\.(png|gif)                     129600 100% 129600 override-expire ignore-reload
refresh_pattern garena\.com                                              129600 100% 129600 override-expire reload-into-ims
refresh_pattern photobucket.*\.(jp(e?g|e|2)|tiff?|bmp|gif|png)           129600 100% 129600 override-expire ignore-reload
refresh_pattern vid\.akm\.dailymotion\.com.*\.on2\?                      129600 100% 129600 ignore-no-cache override-expire override-lastmod
refresh_pattern mediafire.com\/images.*\.(jp(e?g|e|2)|tiff?|bmp|gif|png) 129600 100% 129600 reload-into-ims override-expire ignore-private
refresh_pattern ^http:\/\/images|pics|thumbs[0-9]\.                      129600 100% 129600 reload-into-ims ignore-no-cache ignore-no-store ignore-reload override-expire
refresh_pattern ^http:\/\/www.onemanga.com.*\/                           129600 100% 129600 reload-into-ims ignore-no-cache ignore-no-store ignore-reload override-expire

# ANTI VIRUS
refresh_pattern guru.avg.com/.*\.(bin)                              43200 100% 43200  ignore-no-cache ignore-no-store ignore-reload reload-into-ims
refresh_pattern (avgate|avira).*(idx|gz)$                           43200 100% 43200  ignore-no-cache ignore-no-store ignore-reload reload-into-ims
refresh_pattern kaspersky.*\.avc$                                   43200 100% 43200  ignore-no-cache ignore-no-store ignore-reload reload-into-ims
refresh_pattern kaspersky                                           43200 100% 43200  ignore-no-cache ignore-no-store ignore-reload reload-into-ims
refresh_pattern update.nai.com/.*\.(gem|zip|mcs)                    43200 100% 43200  ignore-no-cache ignore-no-store ignore-reload reload-into-ims
refresh_pattern ^http:\/\/liveupdate.symantecliveupdate.com.*\(zip) 43200 100% 43200  ignore-no-cache ignore-no-store ignore-reload reload-into-ims
refresh_pattern -i symantecliveupdate.com/.*\.(zip|exe)             43200 100% 43200 reload-into-ims
refresh_pattern -i avast.com/.*\.(vpu|vpaa) 4320 100% 43200 reload-into-ims
refresh_pattern -i avira-update.com/.*\.* 720 100% 10800 reload-into-ims


#windows update NEW UPDATE 0.04
refresh_pattern windowsupdate.com/.*\.(cab|exe|dll|msi|psf) 10080 100% 43200 reload-into-ims
refresh_pattern update.microsoft.com/.*\.(cab|exe)                  43200 100% 129600 ignore-no-cache ignore-no-store ignore-reload reload-into-ims
refresh_pattern download.microsoft.com/.*\.(cab|exe|dll|msi|psf) 10080 100% 43200 reload-into-ims
refresh_pattern -i microsoft.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 4320 80% 43200 reload-into-ims
refresh_pattern -i windowsupdate.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 4320 80% 43200 reload-into-ims
refresh_pattern -i windows.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 4320 80% 43200 reload-into-ims
refresh_pattern ([^.]+\.)?(download|(windows)?update)\.(microsoft\.)?com/.*\.(cab|exe|msi|msp|psf) 4320 100% 43200 reload-into-ims
refresh_pattern update.microsoft.com/.*\.(cab|exe|dll|msi|psf) 10080 100% 43200 reload-into-ims
refresh_pattern windowsupdate.com/.*\.(cab|exe|dll|msi|psf) 10080 100% 43200 reload-into-ims
refresh_pattern download.microsoft.com/.*\.(cab|exe|dll|msi|psf) 10080 100% 43200 reload-into-ims
refresh_pattern www.microsoft.com/.*\.(cab|exe|dll|msi|psf) 10080 100% 43200 reload-into-ims

refresh_pattern au.download.windowsupdate.com/.*\.(cab|exe|dll|msi|psf) 4320 100% 43200 reload-into-ims
refresh_pattern bg.v4.pr.dl.ws.microsoft.com/.*\.(cab|exe|dll|msi|psf) 4320 100% 43200 reload-into-ims
refresh_pattern -i .*windowsupdate.com/.*\.(cab|exe)                     259200 100% 259200 ignore-no-store ignore-reload reload-into-ims
refresh_pattern -i .*update.microsoft.com/.*\.(cab|exe|dll|msi|psf)                  259200 100% 259200 ignore-no-store ignore-reload reload-into-ims
refresh_pattern au.download.windowsupdate.com/.*\.(cab|exe|dll|msi|psf) 4320 100% 43200 reload-into-ims
refresh_pattern bg.v4.pr.dl.ws.microsoft.com/.*\.(cab|exe|dll|msi|psf) 4320 100% 43200 reload-into-ims

#apple update
refresh_pattern -i (download|adcdownload).apple.com/.*\.(pkg|dmg) 4320 100% 43200 reload-into-ims
refresh_pattern -i appldnld\.apple\.com 129600 100% 129600 ignore-reload ignore-no-store override-expire override-lastmod ignore-must-revalidate
refresh_pattern -i phobos\.apple\.com 129600 100% 129600 ignore-reload ignore-no-store override-expire override-lastmod ignore-must-revalidate
refresh_pattern -i iosapps\.itunes\.apple\.com 129600 100% 129600 ignore-reload ignore-no-store override-expire override-lastmod ignore-must-revalidate

#images facebook
refresh_pattern ((facebook.com)|(85.131.151.39)).*\.(jpg|png|gif) 129600 100% 129600 ignore-reload override-expire ignore-no-cache ignore-no-store
refresh_pattern -i \.fbcdn.net.*\.(jpg|gif|png|swf|mp3)           129600 100% 129600 ignore-reload override-expire ignore-no-cache ignore-no-store
refresh_pattern static\.ak\.fbcdn\.net*\.(jpg|gif|png)            129600 100% 129600 ignore-reload override-expire ignore-no-cache ignore-no-store
refresh_pattern ^http:\/\/profile\.ak\.fbcdn.net*\.(jpg|gif|png)  129600 100% 129600 ignore-reload override-expire ignore-no-cache ignore-no-store

#banner IIX
refresh_pattern ^http:\/\/openx.*\.(jp(e?g|e|2)|gif|pn[pg]|swf|ico|css|tiff?) 129600 100% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache ignore-no-store
refresh_pattern ^http:\/\/ads(1|2|3).kompas.com.*\/                           43200  100% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache ignore-no-store
refresh_pattern ^http:\/\/img.ads.kompas.com.*\/                              43200  100% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache ignore-no-store
refresh_pattern .kompasimages.com.*\.(jpg|gif|png|swf)                        43200  100% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache ignore-no-store
refresh_pattern ^http:\/\/openx.kompas.com.*\/                                43200  100% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache ignore-no-store
refresh_pattern kaskus.\us.*\.(jp(e?g|e|2)|gif|png|swf)                       43200  100% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache ignore-no-store
refresh_pattern ^http:\/\/img.kaskus.us.*\.(jpg|gif|png|swf)                  43200  100% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache ignore-no-store

#IIX DOWNLOAD
refresh_pattern ^http:\/\/\.www[0-9][0-9]\.indowebster\.com\/(.*)(mp3|rar|zip|flv|wmv|3gp|mp(4|3)|exe|msi|zip) 43200 100% 129600 reload-into-ims  ignore-reload override-expire ignore-no-cache ignore-no-store  ignore-auth

refresh_pattern -i ^http://(khm?)([^/]*?)\.google\.(de|com)     129600 100% 129600 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims ignore-reload

refresh_pattern -i ^http://ecn\.t\d\.tiles\.virtualearth\.net/tiles/\w*\.jpeg     129600 100% 129600 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims ignore-reload
« Last Edit: April 09, 2017, 08:29:16 pm by aGeekHere »
Never Fear, A Geek is Here!

Offline kivimart

  • Newbie
  • *
  • Posts: 6
  • Karma: +1/-0
    • View Profile
Re: Improve Custom refresh pattern
« Reply #1 on: May 11, 2016, 02:14:47 pm »
This is my custom

# All Files
######################
refresh_pattern -i (\.|-)(mp3|m4a|aa?c3?|wm?av?|og(x|v|a|g)|ape|mka|au|aiff|zip|flac|m4(b|r)|m1v|m2(v|p)|mo(d|v)|arj|appx|lha|lzh|on2)(\?.*)?$ 43200 100% 432000 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-no-store ignore-private ignore-auth
refresh_pattern -i (\.|-)(exe|bin|(n|t)ar|acv|(r|j)ar|t?gz|(g|b)z(ip)?2?|7?z(ip)?|wm[v|a]|patch|diff|mar|vpu|inc|r(a|p)m|kom|iso|sys|[ap]sf|ms[i|u|f]|dat|msi|cab|psf|dvr-ms|ace|asx|qt|xt|esd)(\?.*)?$ 43200 100% 432000 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-no-store ignore-private ignore-auth
refresh_pattern -i (\.|-)(ico(.*)?|pn[pg]|css|(g|t)iff?|jpe?g(2|3|4)?|psd|c(d|b)r|cad|bmp|img)(\?.*)?$ 43200 100% 432000 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-no-store ignore-private ignore-auth
refresh_pattern -i (\.|-)(webm|(x-)?swf|mp(eg)?(3|4)|mpe?g(av)?|(x-)?f(l|4)v|divx?|rmvb?|mov|trp|ts|avi|m38u|wmv|wmp|m4v|mkv|asf|dv|vob|3gp?2?)(\?.*)?$ 43200 100% 432000 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-no-store ignore-private ignore-auth
refresh_pattern -i (\.|-)(docx?|xlsx?|pptx?|rtf|xml|pdf|tiff?|txt)(\?.*)?$ 43200 100% 432000 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-no-store ignore-private ignore-auth
#Website
refresh_pattern -i (\.|-)(xml|js|jsp|txt|css)(\?.*)?$ 360 40% 1440 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-no-store ignore-private ignore-auth
###################
refresh_pattern -i .index.(html|htm)$ 0 40% 1440
refresh_pattern . 30 25% 1440
« Last Edit: May 11, 2016, 03:07:21 pm by kivimart »

Offline aGeekHere

  • Sr. Member
  • ****
  • Posts: 525
  • Karma: +43/-1
    • View Profile
Re: Improve Custom refresh pattern
« Reply #2 on: May 11, 2016, 06:31:25 pm »
updated, thanks
Never Fear, A Geek is Here!

Offline aGeekHere

  • Sr. Member
  • ****
  • Posts: 525
  • Karma: +43/-1
    • View Profile
Re: Improve Custom refresh pattern
« Reply #3 on: May 14, 2016, 11:47:57 pm »
added

Code: [Select]
#new refresh patterns 2
refresh_pattern -i (\.|-)(mid|midi|mpg|mpeg|ram|cav|acc|alz|apk|at3|bke|arc|ass|ba|big|bik|bkf|bld|c4|cals|clipflair|cpt|daa|dmg|ddz|dpe|egg|egt|ecab|ess|gho|ghs|gz|ipg|jar|lbr|lqr|lha|lz|lzo|lzma|lzx|mbw|mc.meta|mpq|nth|osz|pak|par|par2|paf|pyk|pk3|pk4|rag|sen|sitx|skb|tb|tib|uha|uue|viv|vsa|z|zoo|nrg|adf|adz|dms|dsk|d64|sdi|mds|mdx|cdi|cue|cif|c2d|daa|b6t)(\?.*)?$ 43200 100% 432000 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-no-store ignore-private ignore-auth
#end new refresh patterns 2
Never Fear, A Geek is Here!

Offline aGeekHere

  • Sr. Member
  • ****
  • Posts: 525
  • Karma: +43/-1
    • View Profile
Re: Improve Custom refresh pattern
« Reply #4 on: June 01, 2016, 08:08:06 pm »
added deajan refresh patterns

Code: [Select]
refresh_pattern -i microsoft.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 4320 80% 43200 reload-into-ims
refresh_pattern -i windowsupdate.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 4320 80% 43200 reload-into-ims
refresh_pattern -i windows.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 4320 80% 43200 reload-into-ims
refresh_pattern -i avg.com/.*\.(bin) 4320 100% 43200 reload-into-ims
refresh_pattern -i symantecliveupdate.com/.*\.(zip|exe) 43200 100% 43200 reload-into-ims
refresh_pattern -i avast.com/.*\.(vpu|vpaa) 4320 100% 43200 reload-into-ims
refresh_pattern -i avira-update.com/.*\.* 720 100% 10800 reload-into-ims
refresh_pattern -i (download|adcdownload).apple.com/.*\.(pkg|dmg) 4320 100% 43200 reload-into-ims
Never Fear, A Geek is Here!

Offline jbourn1907

  • Newbie
  • *
  • Posts: 16
  • Karma: +0/-0
    • View Profile
Re: Improve Custom refresh pattern
« Reply #5 on: October 11, 2016, 07:43:27 pm »
Hi guys, where I can put this refresh_patterns in pfsense 2.3? sorry im new here. Thanks!

Offline avdh

  • Newbie
  • *
  • Posts: 2
  • Karma: +0/-0
    • View Profile
Re: Improve Custom refresh pattern
« Reply #6 on: October 11, 2016, 10:04:53 pm »
Services -> Squid Proxy Server -> Local Cache Tab - > Look at bottom of page under Custom refresh_patterns

Offline Ninno

  • Newbie
  • *
  • Posts: 11
  • Karma: +0/-0
    • View Profile
Re: Improve Custom refresh pattern
« Reply #7 on: October 13, 2016, 07:03:32 am »
Windows 10 update use also .esd files
« Last Edit: October 13, 2016, 08:22:17 am by Ninno »

Offline CardinS2U

  • Newbie
  • *
  • Posts: 15
  • Karma: +0/-0
    • View Profile
Re: Improve Custom refresh pattern
« Reply #8 on: February 12, 2017, 03:02:35 am »
thanks! good to have a decent pattern to start with.

Offline Valeriy

  • Jr. Member
  • **
  • Posts: 48
  • Karma: +7/-0
    • View Profile
Re: Improve Custom refresh pattern
« Reply #9 on: February 15, 2017, 07:09:07 am »
FYI, Squid -k parse is complaining that 'ignore-no-cache' is obsolete.. So it has no effect.

Running 2.3.3-DEVELOPMENT (amd64) built on Sat Feb 04 16:18:22 CST 2017 with Squid 3.5.23



Code: [Select]
Processing: refresh_pattern -i (\.|-)(mid|midi|mpg|mpeg|ram|cav|acc|alz|apk|at3|bke|arc|ass|ba|big|bik|bkf|bld|c4|cals|clipflair|cpt|daa|dmg|ddz|dpe|egg|egt|ecab|ess|gho|ghs|gz|ipg|jar|lbr|lqr|lha|lz|lzo|lzma|lzx|mbw|mc.meta|mpq|nth|osz|pak|par|par2|paf|pyk|pk3|pk4|rag|sen|sitx|skb|tb|tib|uha|uue|viv|vsa|z|zoo|nrg|adf|adz|dms|dsk|d64|sdi|mds|mdx|cdi|cue|cif|c2d|daa|b6t)(\?.*)?$ 43200 100% 432000 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-no-store ignore-private ignore-auth
UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
Processing: refresh_pattern -i (\.|-)(mp3|m4a|aa?c3?|wm?av?|og(x|v|a|g)|ape|mka|au|aiff|zip|flac|m4(b|r)|m1v|m2(v|p)|mo(d|v)|arj|appx|lha|lzh|on2)(\?.*)?$ 43200 100% 432000 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-no-store ignore-private ignore-auth
UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
Processing: refresh_pattern -i (\.|-)(exe|bin|(n|t)ar|acv|(r|j)ar|t?gz|(g|b)z(ip)?2?|7?z(ip)?|wm[v|a]|patch|diff|mar|vpu|inc|r(a|p)m|kom|iso|sys|[ap]sf|ms[i|u|f]|dat|msi|cab|psf|dvr-ms|ace|asx|qt|xt|esd)(\?.*)?$ 43200 100% 432000 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-no-store ignore-private ignore-auth
UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
Processing: refresh_pattern -i (\.|-)(ico(.*)?|pn[pg]|css|(g|t)iff?|jpe?g(2|3|4)?|psd|c(d|b)r|cad|bmp|img)(\?.*)?$ 43200 100% 432000 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-no-store ignore-private ignore-auth
UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
Processing: refresh_pattern -i (\.|-)(webm|(x-)?swf|mp(eg)?(3|4)|mpe?g(av)?|(x-)?f(l|4)v|divx?|rmvb?|mov|trp|ts|avi|m38u|wmv|wmp|m4v|mkv|asf|dv|vob|3gp?2?)(\?.*)?$ 43200 100% 432000 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-no-store ignore-private ignore-auth
UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
Processing: refresh_pattern -i (\.|-)(docx?|xlsx?|pptx?|rtf|xml|pdf|tiff?|txt)(\?.*)?$ 43200 100% 432000 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-no-store ignore-private ignore-auth
UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
Processing: refresh_pattern -i (\.|-)(xml|js|jsp|txt|css)(\?.*)?$ 360 40% 1440 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-no-store ignore-private ignore-auth
UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
Processing: refresh_pattern -i .index.(html|htm)$ 0 40% 1440
Processing: refresh_pattern . 30 25% 1440
Processing: refresh_pattern -i \.(3gp|7z|ace|asx|avi|bin|cab|dat|deb|rpm|divx|dvr-ms)      129600 100% 129600 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims ignore-reload
UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
Processing: refresh_pattern -i \.(rar|jar|gz|tgz|tar|bz2|iso|m1v|m2(v|p)|mo(d|v)|(x-|)flv) 129600 100% 129600 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims ignore-reload
UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
/usr/local/etc/squid/squid.conf line 52: refresh_pattern -i \.(rar|jar|gz|tgz|tar|bz2|iso|m1v|m2(v|p)|mo(d|v)|(x-|)flv) 129600 100% 129600 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims ignore-reload
refreshAddToList: Invalid regular expression '\.(rar|jar|gz|tgz|tar|bz2|iso|m1v|m2(v|p)|mo(d|v)|(x-|)flv)': empty (sub)expression
Processing: refresh_pattern -i \.(jp(e?g|e|2)|gif|pn[pg]|bm?|tiff?|ico|swf|css|js)         129600 100% 129600 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims ignore-reload
UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
Processing: refresh_pattern -i \.(mp(e?g|a|e|1|2|3|4)|mk(a|v)|ms(i|u|p))                   129600 100% 129600 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims ignore-reload
UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
Processing: refresh_pattern -i \.(og(x|v|a|g)|rar|rm|r(a|p)m|snd|vob|wav)                  129600 100% 129600 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims ignore-reload
UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
Processing: refresh_pattern -i \.(pp(s|t)|wax|wm(a|v)|wmx|wpl|zip|cb(r|z|t))               129600 100% 129600 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims ignore-reload
UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
Processing: refresh_pattern ^gopher:  1440  0%  1440
Processing: refresh_pattern ^ftp:    10080 95% 43200 override-lastmod reload-into-ims
Processing: refresh_pattern -i \.(doc|pdf)$           100080 90% 43200 override-expire ignore-no-cache ignore-no-store ignore-private reload-into-ims
UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
Processing: refresh_pattern -i \.(html|htm)$          1440   40% 40320 ignore-no-cache ignore-no-store ignore-private override-expire reload-into-ims
UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
Processing: refresh_pattern (Release|Packages(.gz)*)$    0   20%  2880
Processing: refresh_pattern .                          180   95% 43200 override-lastmod reload-into-ims
Processing: refresh_pattern -i (/cgi-bin/|\?)         0      0%      0
Processing: refresh_pattern \.(ico|video-stats)$ 129600 100% 129600 override-expire ignore-reload ignore-no-cache ignore-no-store ignore-private ignore-auth override-lastmod ignore-must-revalidate
UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
Processing: refresh_pattern imeem.*\.flv$                           0     0%         0 override-lastmod override-expire
Processing: refresh_pattern \.rapidshare.*\/[0-9]*\/.*\/[^\/]* 161280    90%    161280 ignore-reload
Processing: refresh_pattern (get_video\?|videoplayback\?|videodownload\?|\.flv?) 129600 100% 129600 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims
UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
Processing: refresh_pattern (get_video\?|videoplayback\?id|videoplayback.*id|videodownload\?|\.flv?) 129600 100% 129600 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims
UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
Processing: refresh_pattern ^.*(utm\.gif|ads\?|rmxads\.com|ad\.z5x\.net|bh\.contextweb\.com|bstats\.adbrite\.com|a1\.interclick\.com|ad\.trafficmp\.com|ads\.cubics\.com|ad\.xtendmedia\.com|\.googlesyndication\.com|advertising\.com|yieldmanager|game-advertising\.com|pixel\.quantserve\.com|adperium\.com|doubleclick\.net|adserving\.cpxinteractive\.com|syndication\.com|media.fastclick.net).* 129600 20% 129600 ignore-no-cache ignore-no-store ignore-private override-expire ignore-reload ignore-auth ignore-must-revalidate
UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
Processing: refresh_pattern ^.*safebrowsing.*google                                  129600 100% 129600 override-expire ignore-reload ignore-no-cache ignore-private ignore-auth ignore-must-revalidate
UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
Processing: refresh_pattern ^http://((cbk|mt|khm|mlt)[0-9]?)\.google\.co(m|\.uk)     129600 100% 129600 override-expire ignore-reload ignore-private
Processing: refresh_pattern ytimg\.com.*\.jpg                                        129600 100% 129600 override-expire ignore-reload
Processing: refresh_pattern images\.friendster\.com.*\.(png|gif)                     129600 100% 129600 override-expire ignore-reload
Processing: refresh_pattern garena\.com                                              129600 100% 129600 override-expire reload-into-ims
Processing: refresh_pattern photobucket.*\.(jp(e?g|e|2)|tiff?|bmp|gif|png)           129600 100% 129600 override-expire ignore-reload
Processing: refresh_pattern vid\.akm\.dailymotion\.com.*\.on2\?                      129600 100% 129600 ignore-no-cache override-expire override-lastmod
UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
Processing: refresh_pattern mediafire.com\/images.*\.(jp(e?g|e|2)|tiff?|bmp|gif|png) 129600 100% 129600 reload-into-ims override-expire ignore-private
Processing: refresh_pattern ^http:\/\/images|pics|thumbs[0-9]\.                      129600 100% 129600 reload-into-ims ignore-no-cache ignore-no-store ignore-reload override-expire
UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
Processing: refresh_pattern ^http:\/\/www.onemanga.com.*\/                           129600 100% 129600 reload-into-ims ignore-no-cache ignore-no-store ignore-reload override-expire
UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
Processing: refresh_pattern guru.avg.com/.*\.(bin)                              43200 100% 43200  ignore-no-cache ignore-no-store ignore-reload reload-into-ims
UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
Processing: refresh_pattern (avgate|avira).*(idx|gz)$                           43200 100% 43200  ignore-no-cache ignore-no-store ignore-reload reload-into-ims
UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
Processing: refresh_pattern kaspersky.*\.avc$                                   43200 100% 43200  ignore-no-cache ignore-no-store ignore-reload reload-into-ims
UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
Processing: refresh_pattern kaspersky                                           43200 100% 43200  ignore-no-cache ignore-no-store ignore-reload reload-into-ims
UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
Processing: refresh_pattern update.nai.com/.*\.(gem|zip|mcs)                    43200 100% 43200  ignore-no-cache ignore-no-store ignore-reload reload-into-ims
UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
Processing: refresh_pattern ^http:\/\/liveupdate.symantecliveupdate.com.*\(zip) 43200 100% 43200  ignore-no-cache ignore-no-store ignore-reload reload-into-ims
UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
Processing: refresh_pattern -i symantecliveupdate.com/.*\.(zip|exe)             43200 100% 43200 reload-into-ims
Processing: refresh_pattern -i avast.com/.*\.(vpu|vpaa) 4320 100% 43200 reload-into-ims
Processing: refresh_pattern -i avira-update.com/.*\.* 720 100% 10800 reload-into-ims
Processing: refresh_pattern windowsupdate.com/.*\.(cab|exe)                     43200 100% 129600 ignore-no-cache ignore-no-store ignore-reload reload-into-ims
UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
Processing: refresh_pattern update.microsoft.com/.*\.(cab|exe)                  43200 100% 129600 ignore-no-cache ignore-no-store ignore-reload reload-into-ims
UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
Processing: refresh_pattern download.microsoft.com/.*\.(cab|exe)                43200 100% 129600 ignore-no-cache ignore-no-store ignore-reload reload-into-ims
UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
Processing: refresh_pattern -i microsoft.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 4320 80% 43200 reload-into-ims
Processing: refresh_pattern -i windowsupdate.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 4320 80% 43200 reload-into-ims
Processing: refresh_pattern -i windows.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 4320 80% 43200 reload-into-ims
Processing: refresh_pattern ([^.]+\.)?(download|(windows)?update)\.(microsoft\.)?com/.*\.(cab|exe|msi|msp|psf) 4320 100% 43200 reload-into-ims
Processing: refresh_pattern update.microsoft.com/.*\.(cab|exe|dll|msi|psf) 10080 100% 43200 reload-into-ims
Processing: refresh_pattern windowsupdate.com/.*\.(cab|exe|dll|msi|psf) 10080 100% 43200 reload-into-ims
Processing: refresh_pattern download.microsoft.com/.*\.(cab|exe|dll|msi|psf) 10080 100% 43200 reload-into-ims
Processing: refresh_pattern www.microsoft.com/.*\.(cab|exe|dll|msi|psf) 10080 100% 43200 reload-into-ims
Processing: refresh_pattern au.download.windowsupdate.com/.*\.(cab|exe|dll|msi|psf) 4320 100% 43200 reload-into-ims
Processing: refresh_pattern bg.v4.pr.dl.ws.microsoft.com/.*\.(cab|exe|dll|msi|psf) 4320 100% 43200 reload-into-ims
Processing: refresh_pattern -i (download|adcdownload).apple.com/.*\.(pkg|dmg) 4320 100% 43200 reload-into-ims
Processing: refresh_pattern ((facebook.com)|(85.131.151.39)).*\.(jpg|png|gif) 129600 100% 129600 ignore-reload override-expire ignore-no-cache ignore-no-store
UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
Processing: refresh_pattern -i \.fbcdn.net.*\.(jpg|gif|png|swf|mp3)           129600 100% 129600 ignore-reload override-expire ignore-no-cache ignore-no-store
UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
Processing: refresh_pattern static\.ak\.fbcdn\.net*\.(jpg|gif|png)            129600 100% 129600 ignore-reload override-expire ignore-no-cache ignore-no-store
UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
Processing: refresh_pattern ^http:\/\/profile\.ak\.fbcdn.net*\.(jpg|gif|png)  129600 100% 129600 ignore-reload override-expire ignore-no-cache ignore-no-store
UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
Processing: refresh_pattern ^http:\/\/openx.*\.(jp(e?g|e|2)|gif|pn[pg]|swf|ico|css|tiff?) 129600 100% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache ignore-no-store
UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
Processing: refresh_pattern ^http:\/\/ads(1|2|3).kompas.com.*\/                           43200  100% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache ignore-no-store
UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
Processing: refresh_pattern ^http:\/\/img.ads.kompas.com.*\/                              43200  100% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache ignore-no-store
UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
Processing: refresh_pattern .kompasimages.com.*\.(jpg|gif|png|swf)                        43200  100% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache ignore-no-store
UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
Processing: refresh_pattern ^http:\/\/openx.kompas.com.*\/                                43200  100% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache ignore-no-store
UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
Processing: refresh_pattern kaskus.\us.*\.(jp(e?g|e|2)|gif|png|swf)                       43200  100% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache ignore-no-store
UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
Processing: refresh_pattern ^http:\/\/img.kaskus.us.*\.(jpg|gif|png|swf)                  43200  100% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache ignore-no-store
UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
Processing: refresh_pattern ^http:\/\/\.www[0-9][0-9]\.indowebster\.com\/(.*)(mp3|rar|zip|flv|wmv|3gp|mp(4|3)|exe|msi|zip) 43200 100% 129600 reload-into-ims  ignore-reload override-expire ignore-no-cache ignore-no-store  ignore-auth
UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
Processing: refresh_pattern -i ^http://(khm?)([^/]*?)\.google\.(de|com)     129600 100% 129600 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims ignore-reload
UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
/usr/local/etc/squid/squid.conf line 136: refresh_pattern -i ^http://(khm?)([^/]*?)\.google\.(de|com)     129600 100% 129600 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims ignore-reload
refreshAddToList: Invalid regular expression '^http://(khm?)([^/]*?)\.google\.(de|com)': repetition-operator operand invalid
Processing: refresh_pattern -i ^http://ecn\.t\d\.tiles\.virtualearth\.net/tiles/\w*\.jpeg     129600 100% 129600 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims ignore-reload
UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.

Offline Valeriy

  • Jr. Member
  • **
  • Posts: 48
  • Karma: +7/-0
    • View Profile
Re: Improve Custom refresh pattern
« Reply #10 on: February 15, 2017, 08:31:34 am »
Two invalid expressions.
Somebody can fix them?

/usr/local/etc/squid/squid.conf line 52: refresh_pattern -i \.(rar|jar|gz|tgz|tar|bz2|iso|m1v|m2(v|p)|mo(d|v)|(x-|)flv) 129600 100% 129600 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims ignore-reload
refreshAddToList: Invalid regular expression '\.(rar|jar|gz|tgz|tar|bz2|iso|m1v|m2(v|p)|mo(d|v)|(x-|)flv)': empty (sub)expression



  kid1| /usr/local/etc/squid/squid.conf line 136: refresh_pattern -i ^http://(khm?)([^/]*?)\.google\.(de|com)     129600 100% 129600 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims ignore-reload
 refreshAddToList: Invalid regular expression '^http://(khm?)([^/]*?)\.google\.(de|com)': repetition-operator operand invalid



Offline KOM

  • Hero Member
  • *****
  • Posts: 5506
  • Karma: +680/-23
    • View Profile
Re: Improve Custom refresh pattern
« Reply #11 on: February 15, 2017, 08:51:24 am »
For the first one, I think the problem is here:

(x-|)

There is nothing on the other side of the OR.  I don't know what file type would be handled by the extension x-.  To fix it, just remove the entire (x-|).

For the second one, I don't know for sure, but it seems like a lot of regex just to handle *.google.*.

Offline doktornotor

  • Hero Member
  • *****
  • Posts: 8553
  • Karma: +956/-278
  • Not a pfSense employee, they cannot fire me...
    • View Profile
Re: Improve Custom refresh pattern
« Reply #12 on: February 16, 2017, 06:32:45 am »
I would strongly urge everyone to ditch the pointless idea of trying to cache dynamic sites. It doesn't work in like 99% of cases, plus breaks Squid. And please ditch ALL those antivirus patterns. None of them works. The AVs are streaming updates with new sigs to clients every couple of minutes these days. There's nothing to cache.
Do NOT PM for help!

Offline saluto

  • Jr. Member
  • **
  • Posts: 44
  • Karma: +0/-0
    • View Profile
Re: Improve Custom refresh pattern
« Reply #13 on: February 16, 2017, 08:26:46 am »

So why use squid? Because most sites are changing to https.

Various web content are in videos, overloading the network.

Offline KOM

  • Hero Member
  • *****
  • Posts: 5506
  • Karma: +680/-23
    • View Profile
Re: Improve Custom refresh pattern
« Reply #14 on: February 16, 2017, 08:45:58 am »
Quote
So why use squid?

I use it as a filtering platform for squidguard.

Quote
Various web content are in videos

Caching videos is pointless unless everyone is watching the same ones.

I have seen commercial products that take squid and heavily modify it with all kinds of custom stuff, and they claim they can cache almost everything.