pfSense Support Subscription

Author Topic: Squid cache performance  (Read 7162 times)

0 Members and 1 Guest are viewing this topic.

Offline al_reidy

  • Newbie
  • *
  • Posts: 20
  • Karma: +0/-0
    • View Profile
Squid cache performance
« on: September 01, 2008, 09:56:04 am »
I did a test on a monowall setup and managed to get nearly 15 megs per sec plus from the cached downloads from squid.

pfsense with a similar setup only manages to get 2-6 megs per sec if im lucky.

the router is a 1.8ghz athlon and 1.5gb ram of which 640mb is for squid. and a new 160gb sata 7200rpm disk.
also a 1000base intel chip network card.

files are software updates from Apple software updates and M$ updates so mainly about 200mb to 300mb files.

any suggestions on how to speed cache retrieval up?

Thanks in advance.

Offline Cry Havok

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 2818
  • Karma: +5/-0
  • Backup: n. What you should have done yesterday.
    • View Profile
Re: Squid cache performance
« Reply #1 on: September 01, 2008, 11:54:48 am »
First off, what version of pfSense and Squid package?

Secondly, have you reviewed the threads about people experiencing a slow squid configuration?
If you're planning on PMing me to ask me to look at a thread, or for individual support, don't.

Offline al_reidy

  • Newbie
  • *
  • Posts: 20
  • Karma: +0/-0
    • View Profile
Re: Squid cache performance
« Reply #2 on: September 02, 2008, 11:29:29 am »
i have 1.2 release with squid 2.6.18.1_04 on a fresh install of a uniprocessor solution.
looking at other forum posts i should have checked there first i appologise for that. the problems people were having on http://forum.pfsense.org/index.php/topic,7911.30.html are not fixed buy the fix below.

autoboot_delay="1"
#kern.ipc.nmbclusters="0"
hint.apic.0.disabled=1
kern.hz=100
#for squid
kern.ipc.nmbclusters="32768"
kern.maxfiles="65536"
kern.maxfilesperproc="32768"
net.inet.ip.portrange.last="65535"

although i might not have the same faults they are having. i can access the net at full speed but its retrieval from the cached files that is slow.

thanks

Online mhab12

  • Hero Member
  • *****
  • Posts: 685
  • Karma: +5/-0
    • View Profile
Re: Squid cache performance
« Reply #3 on: September 03, 2008, 09:39:52 am »
The slow squid issues appear to be corrected in versions of pfSense built on FreeBSD 6.3 and later.  Our slowness kept us down around 10kB/s...I'm not sure these are the same issue.  Perhaps someone else can test the throughput.

Offline trendchiller

  • Sr. Member
  • ****
  • Posts: 370
  • Karma: +0/-0
    • View Profile
Re: Squid cache performance
« Reply #4 on: September 03, 2008, 06:12:25 pm »
furthermore you can try to upgrade your squid package... there's the possibility to select between aufs or diskd... perhaps there are some speed changes... diskd might be your choice...

Offline al_reidy

  • Newbie
  • *
  • Posts: 20
  • Karma: +0/-0
    • View Profile
Re: Squid cache performance
« Reply #5 on: September 05, 2008, 07:33:14 am »
furthermore you can try to upgrade your squid package... there's the possibility to select between aufs or diskd... perhaps there are some speed changes... diskd might be your choice...

na,  no difference.  :'(

Offline Netview

  • Jr. Member
  • **
  • Posts: 30
  • Karma: +0/-0
    • View Profile
Re: Squid cache performance
« Reply #6 on: September 05, 2008, 12:52:09 pm »
I have upgraded squid-2.6.18-1 to squid-2.7.4 on pfsense 1.2. My version is running without any problems since 2 weeks:
Quote
# squid -v
Squid Cache: Version 2.7.STABLE4
configure options:  '--with-aufs-threads=10' '--bindir=/usr/local/sbin' '--sbindir=/usr/local/sbin' '--datadir=/usr/local/etc/squid' '--libexecdir=/usr/local/libexec/squid' '--localstatedir=/usr/local/squid' '--sysconfdir=/usr/local/etc/squid' '--enable-removal-policies=lru heap' '--disable-linux-netfilter' '--disable-linux-tproxy' '--disable-epoll' '--enable-auth=basic ntlm digest' '--enable-basic-auth-helpers=DB NCSA PAM MSNT SMB LDAP YP' '--enable-digest-auth-helpers=password ldap' '--enable-external-acl-helpers=ip_user session unix_group wbinfo_group ldap_group' '--enable-ntlm-auth-helpers=SMB' '--enable-negotiate-auth-helpers=squid_kerb_auth' '--with-pthreads' '--enable-storeio=ufs diskd null aufs coss' '--enable-delay-pools' '--enable-snmp' '--enable-ssl' '--with-openssl=/usr' '--enable-htcp' '--enable-forw-via-db' '--enable-cache-digests' '--enable-wccpv2' '--enable-referer-log' '--enable-useragent-log' '--enable-arp-acl' '--enable-pf-transparent' '--enable-follow-x-forwarded-for' '--with-large-files' '--enable-large-cache-files' '--enable-err-languages=Armenian Azerbaijani Bulgarian Catalan Czech Danish  Dutch English Estonian Finnish French German Greek  Hebrew Hungarian Italian Japanese Korean Lithuanian  Polish Portuguese Romanian Russian-1251 Russian-koi8-r  Serbian Simplify_Chinese Slovak Spanish Swedish  Traditional_Chinese Turkish' '--enable-default-err-language=English' '--prefix=/usr/local' '--mandir=/usr/local/man' '--infodir=/usr/local/info/' 'i386-portbld-freebsd6.2' 'build_alias=i386-portbld-freebsd6.2' 'host_alias=i386-portbld-freebsd6.2' 'target_alias=i386-portbld-freebsd6.2' 'CC=cc' 'CFLAGS=-O2 -fno-strict-aliasing -pipe  -I/usr/local/include  -I/usr/include' 'LDFLAGS= -L/usr/local/lib -rpath=/usr/lib:/usr/local/lib -L/usr/lib' 'CPPFLAGS='

Anybody is interested to have a upgrade-package?

Online mhab12

  • Hero Member
  • *****
  • Posts: 685
  • Karma: +5/-0
    • View Profile
Re: Squid cache performance
« Reply #7 on: September 05, 2008, 01:25:45 pm »
I know of several who would be interested...myself included.

Offline acidrop

  • Jr. Member
  • **
  • Posts: 66
  • Karma: +0/-0
    • View Profile
Re: Squid cache performance
« Reply #8 on: September 07, 2008, 04:38:17 am »
Hello!

I have the same poblem.I'm using Pfsense 1.2 + squid2.6.18.1_04 + squidguard1.2.0_1 with squid in transparent mode.
I have a dual wan setup with a 24mbit adsl line on WAN1 and a 5mbit adsl line on WAN2.Since squid doesn't support dual wan
setup all requests when squid is enabled are forwarded on WAN1.
Now if a i do a speedtest with proxy enabled i hardly get maximum 1mbit download on wan1.If i disable it i get 10mbits!
Also with proxy enabled the web browsing is very slow if i disable it it gets in normal.I have this problem since Pfsense 1.2 RC2 so it's not
something new.I've tried to adjust the throttling settings on squid with worse results so i disabled them...

Online mhab12

  • Hero Member
  • *****
  • Posts: 685
  • Karma: +5/-0
    • View Profile
Re: Squid cache performance
« Reply #9 on: September 08, 2008, 10:48:32 am »
@Acidrop

Try upgrading to a pfSense 1.2 build on FreeBSD 6.3  Search the forums for links and info about these newer builds and squid slowness.  We are getting full line-speed through squid at this point (though only dual t1/3mbps).

Offline chudy

  • Full Member
  • ***
  • Posts: 123
  • Karma: +1/-1
    • View Profile
Re: Squid cache performance
« Reply #10 on: October 03, 2008, 07:05:37 am »
I've done this with squidhead(/w storeurl feature),vmware,pfsense 1.2.

I just build an freebsd 6.3, compile squid, installed and tar. Then copy it to pfsense. Its been running for 3 month I think. Then I wipe out. I'm working now on pfsense 1.3

For performance tuning just Dig it on tuning FreeBSD you'll find it there.
The default squid provided is not tuned so you have to edit some of squid.inc