The pfSense Store

Author Topic: HFSC - acceptable # of drops?  (Read 2300 times)

0 Members and 1 Guest are viewing this topic.

Offline miles267

  • Full Member
  • ***
  • Posts: 240
    • View Profile
HFSC - acceptable # of drops?
« on: June 14, 2012, 10:22:48 pm »
Hi, I have a 28/4 Mbit connection with pfsense 2.0.1 HFSC traffic shaping enabled.

I've found that I generally maintain a 28 Mb/s (3.2 MB/s) download of a 2 GB file with PF's traffic shaper reporting 2354 pps (packets per second?).  However during the course of the file download, the traffic will appear in the correct "bulk" queue, but there will be upwards of 5000+ "drops" reported in the Status > Queue screen on this queue.  I realize that HFSC disregards "priority" so I've not populated and of the priority boxes within the HFSC shaping parameters of any queue.

I have no issue with ACK drops, but was wondering if this is typical or should I expect to see 0 drops?  I can tell that everything is going to the appropriate QoS queues now although I had hoped that "drops" in a lower priority "bulk" queue were a sign of an effective traffic shaping setup.

Any thoughts would be appreciated.  Thanks.

Offline dhatz

  • Hero Member
  • *****
  • Posts: 1002
    • View Profile
Re: HFSC - acceptable # of drops?
« Reply #1 on: June 15, 2012, 09:47:41 am »
It might not be relevant to your case, but have you enabled ECN & RED for your "bulk" queue ?

iirc you'd need to also tune net.inet.tcp.ecn.enable=1

Offline miles267

  • Full Member
  • ***
  • Posts: 240
    • View Profile
Re: HFSC - acceptable # of drops?
« Reply #2 on: June 15, 2012, 11:16:16 am »
It might not be relevant to your case, but have you enabled ECN & RED for your "bulk" queue ?

iirc you'd need to also tune net.inet.tcp.ecn.enable=1

dhatz, I had NOT previously enabled RED on that queue though ECN had already been enabled.  I've since enabled RED (not RED In/out) on this bulk queue on both my LAN and WAN interface.

Where is the net.inet.tcp.ecn.enable=1 option?

As always, I appreciate your insights.  Thanks.

Offline dhatz

  • Hero Member
  • *****
  • Posts: 1002
    • View Profile
Re: HFSC - acceptable # of drops?
« Reply #3 on: June 15, 2012, 11:26:18 am »
Edit/create file /boot/loader.conf.local and add a line with:

net.inet.tcp.ecn.enable="1"

Let us know if it makes any difference to # of packets dropped.

PS: Hmm, putting it in loader.conf.local doesn't seem to have any effect on my 2.1 test sytem, based on the output of sysctl -a If however I set it from the CLI using sysctl net.inet.tcp.ecn.enable=1 it works.
« Last Edit: June 15, 2012, 11:35:36 am by dhatz »

Offline miles267

  • Full Member
  • ***
  • Posts: 240
    • View Profile
Re: HFSC - acceptable # of drops?
« Reply #4 on: June 15, 2012, 12:00:53 pm »
OK - I've gone ahead and made this change but the drop still appear to accumulate rather steadily.  In the order of +100 drops per 5 secs (the interval of status > queue UI refreshes) during a download @ 28 MB/sec.  Wasn't sure whether drops were standard procedure and an indication that queuing was doing its job?

Offline dhatz

  • Hero Member
  • *****
  • Posts: 1002
    • View Profile
Re: HFSC - acceptable # of drops?
« Reply #5 on: June 15, 2012, 12:02:58 pm »
What is the output of sysctl -a | fgrep tcp.ecn on your router?

Offline miles267

  • Full Member
  • ***
  • Posts: 240
    • View Profile
Re: HFSC - acceptable # of drops?
« Reply #6 on: June 15, 2012, 12:30:42 pm »
What is the output of sysctl -a | fgrep tcp.ecn on your router?

Seems to return the following:

net.inet.tcp.ecn.maxretries: 1
net.inet.tcp.ecn.enable: 0

Offline dhatz

  • Hero Member
  • *****
  • Posts: 1002
    • View Profile
Re: HFSC - acceptable # of drops?
« Reply #7 on: June 15, 2012, 12:36:00 pm »
Run sysctl net.inet.tcp.ecn.enable=1 from the shell command line.

Let us know if it makes any difference to # of packets dropped.

Offline miles267

  • Full Member
  • ***
  • Posts: 240
    • View Profile
Re: HFSC - acceptable # of drops?
« Reply #8 on: June 15, 2012, 12:51:56 pm »
OK.  I've enabled per your suggestion and confirmed the flag was enabled, but it appears for that particular queue (NNTP traffic) running at 28 MB/s while downloading the drop count still appears rather steady.  Should I be expecting 10s of drops typically or 100s, even 1000s?  The drop count appears especially evident on my system with NNTP high speed downloads.  If it's helpful, attached are my current settings on a 28/4 Mbit connection (below).  The queue where I'm noticing the most drops is:

queue qNNTP on em2 bandwidth 1% qlimit 500 hfsc (  red  , ecn  )

WAN:
 altq on  em3 hfsc bandwidth 3880Kb queue {  qACK,  qDefault,  qP2P,  qBackup,  qDNS,  qNNTP,  qSSH,  qVoIP  }
 queue qACK on em3 bandwidth 30% qlimit 500 hfsc (  ecn  ,  realtime 20% ) 
 queue qDefault on em3 bandwidth 20% qlimit 1000 hfsc (  ecn  , default  ,  realtime 20% ) 
 queue qP2P on em3 bandwidth 1% qlimit 500 hfsc (  ecn  , upperlimit 95%  ) 
 queue qBackup on em3 bandwidth 5% qlimit 500 hfsc (  red  , ecn  ) 
 queue qDNS on em3 bandwidth 5% qlimit 500 hfsc (  ecn  ,  realtime 5% ) 
 queue qNNTP on em3 bandwidth 1% qlimit 500 hfsc (  red  , ecn  ) 
 queue qSSH on em3 bandwidth 5% qlimit 500 hfsc (  ecn  ) 
 queue qVoIP on em3 bandwidth 5% qlimit 500 hfsc (  ecn  ) 
 
LAN:
 altq on  em2 hfsc bandwidth 1Gb queue {  qLink,  qInternet  }
 queue qLink on em2 bandwidth 20% qlimit 500 hfsc (  ecn  , default  ) 
 queue qInternet on em2 bandwidth 28000Kb qlimit 500 hfsc (  ecn  , linkshare 28000Kb  , upperlimit 28000Kb  )  {  qACK,  qP2P,  qDefault,  qBackup,  qDNS,  qNNTP,  qSSH,  qVoIP  }
 queue qACK on em2 bandwidth 30% qlimit 500 hfsc (  ecn  ,  realtime 20% ) 
 queue qP2P on em2 bandwidth 1% qlimit 500 hfsc (  ecn  , upperlimit 95%  ) 
 queue qDefault on em2 bandwidth 20% qlimit 1000 hfsc (  ecn  ,  realtime 20% ) 
 queue qBackup on em2 bandwidth 5% qlimit 500 hfsc (  red  , ecn  ) 
 queue qDNS on em2 bandwidth 5% qlimit 500 hfsc (  ecn  ,  realtime 5% ) 
 queue qNNTP on em2 bandwidth 1% qlimit 500 hfsc (  red  , ecn  ) 
 queue qSSH on em2 bandwidth 5% qlimit 500 hfsc (  ecn  ) 
 queue qVoIP on em2 bandwidth 5% qlimit 500 hfsc (  ecn  )

Offline dhatz

  • Hero Member
  • *****
  • Posts: 1002
    • View Profile
Re: HFSC - acceptable # of drops?
« Reply #9 on: June 15, 2012, 01:00:51 pm »
Does the remote host support ECN ?

Read more http://en.wikipedia.org/wiki/Explicit_Congestion_Notification

Offline miles267

  • Full Member
  • ***
  • Posts: 240
    • View Profile
Re: HFSC - acceptable # of drops?
« Reply #10 on: June 15, 2012, 01:04:36 pm »
Does the remote host support ECN ?

Read more http://en.wikipedia.org/wiki/Explicit_Congestion_Notification

I contacted my ISP (Cox Business) and they indicated that ECN is only available on their fiber lines (not cable).  However I've not taken any measures to enabled ECN on my Windows 2008 R2 server which is the one doing the downloading.  Would there be any benefit to enabling on the WS2008R2 box via this command (from Wiki you linked me to):

netsh interface tcp set global ecncapability=enabled

Thanks again.

EDIT 6/16/12 - appears once I reduced the total # of NNTP connections to my provider from 20 to 7, I am still able to achieve full download speed without queue drops.
« Last Edit: June 16, 2012, 07:57:48 am by miles267 »