Netgate SG-1000 microFirewall

Author Topic: squid stable stops working and can't be restarted  (Read 1519 times)

0 Members and 1 Guest are viewing this topic.

Offline kejianshi

  • Hero Member
  • *****
  • Posts: 4995
  • Karma: +199/-43
  • Debugging...
    • View Profile
squid stable stops working and can't be restarted
« on: July 02, 2013, 01:46:17 pm »
squid stable stops working and can't be restarted.  This was really annoying.  I'm pretty sure this was happening because my squid cache settings were less than ideal causing squid to eventually crash and not even start on reboot. Un-installing the package and reinstalling didn't help either.  My fix, was to install the package, ssh to the box, go to command line and issue to clear the cache:

cd /var/squid/cache
rm -rf *

Then I rebooted the box and reloaded the squid package.   Adjusted my disk cache and ram cache from web interface.

There is also the issue that there is no clear matrix for setting up squid cache.  So far, to me it seems that physical ram (not hard drive size) will determine maximum cache size since the disk cache has to be indexed in ram and that adds up.  I'm going to try (physical ram / 2) * 25 = max disk cache

allowing 1/2 of my ram as ram cache (for me I have 3GB so half is 1.5 GB)
allowing space for 4% of my disk cash in ram, so that there should be at most for me 37.5 GB disk cache. 

I have yet to see a clear, safe formula for calculating cache allocation based on system ram (1st) and disk space (2nd) and it seems this should be a very simple, easy and clear calculation, not alchemy.  What I am 100% sure of so far is that too big disk cache will exhaust ram long before disk space.

Offline Asterix

  • Hero Member
  • *****
  • Posts: 890
  • Karma: +34/-0
    • View Profile
Re: squid stable stops working and can't be restarted
« Reply #1 on: July 02, 2013, 02:38:46 pm »
Can't add much help here.. but my experience with Squid hasn't been great. If a working install goes haywire and simple steps to resolve it does not work then basically the install config has corrupted. No matter if you see all settings are correct and you do a clean re-install of squid, it won't start. A fresh install of pfSense is the only way.

Offline Nachtfalke

  • Hero Member
  • *****
  • Posts: 2889
  • Karma: +29/-1
    • View Profile
Re: squid stable stops working and can't be restarted
« Reply #2 on: July 02, 2013, 03:09:47 pm »
What you are looking for is this:
http://wiki.squid-cache.org/SquidFaq/SquidMemory

It shows you how much memory will be used when having xx GB HDD cache and so on.
But the most important thing - you should always remember - is, that the "Memory cache size" is not a limit. It is a minimum squid can use. But squid can use much more if it needs.

So I am using 8GB RAM in my pfsense 2.0.3
having 2GB for "Memory cache size" and 40GB for HDD cache size. My system overall memory usage is 75% which means 6GB. Checking "DIAGNOSTICS --> System Activity" shows me that squid is using 4GB+ of memory.

So make sure you have enough memory or probably better lower the "Memory cache size" and the HDD cache size.

Offline kejianshi

  • Hero Member
  • *****
  • Posts: 4995
  • Karma: +199/-43
  • Debugging...
    • View Profile
Re: squid stable stops working and can't be restarted
« Reply #3 on: July 02, 2013, 03:49:18 pm »
What you just explained sounds pretty close to what I am discovering.

For your 8GB system I would have expected that allocating 2GB "Memory cache" and 40GB HD cache
would have resulted in:

4% of 40GB = 1.6GB

+

The 2GB you allocated to Memory cache

= 3.6GB minimum ram used by squid as the caches fill which is very near what you show.
I like simple formulas for determining such things.
 
Maybe 5% or even 6% would be a safer start point than 4% based on your post.

Thanks for the post.