Netgate SG-1000 microFirewall

Author Topic: Traffic shaper limit throughput not stable  (Read 242 times)

0 Members and 1 Guest are viewing this topic.

Offline FlangeMonkey

  • Newbie
  • *
  • Posts: 6
  • Karma: +0/-0
    • View Profile
Traffic shaper limit throughput not stable
« on: January 31, 2018, 04:26:12 pm »
Hi Guys,

I've been trying to setup some basic limiters for testing before I move onto more advanced QoS.  I am using pfSense in a VM on ESXi.

I have simple limiters of 200Mbit and 10Mbit.  When I do a download test without limiters its 100% 230Mbit stable throughput.  When I turn on the limiter of 200Mbit down, I get approx. 100 to 170 Mbit.  Additionally, the download spikes up and down sometimes below 70Mbit. 

My processor or memory isn't constrained.  Does anyone have any idea why the limiter isn't limiting to a stable/constant 200Mbit?  I've tried all kinds of things except physical hardware and I think this could be a virtual/FreeBSD thing.

Thanks,


Offline FlangeMonkey

  • Newbie
  • *
  • Posts: 6
  • Karma: +0/-0
    • View Profile
Re: Traffic shaper limit throughput not stable
« Reply #1 on: February 01, 2018, 07:28:19 am »
Hi Guys,

Is it my understanding of what the limter are?  I'm not expecting any schedulers or algorithms on the pipe, so should I be expecting a hard limit or a scheduler of some sort to help with bufferbloat and therefore not 100% limit.

I notice when I run 'ipfw pipe list', the pipe is FIFO, but I am seeing a sched using WF2Q+, when I execute 'ipfw sched list'.

Thanks,

Offline Harvy66

  • Hero Member
  • *****
  • Posts: 2321
  • Karma: +213/-12
    • View Profile
Re: Traffic shaper limit throughput not stable
« Reply #2 on: February 01, 2018, 12:41:38 pm »
Is there a queue depth set? I don't know about limiters, but shapers default to 50 packets, which is not enough and causes dropped packets at high rates which reduces throughput.

Offline FlangeMonkey

  • Newbie
  • *
  • Posts: 6
  • Karma: +0/-0
    • View Profile
Re: Traffic shaper limit throughput not stable
« Reply #3 on: February 01, 2018, 01:13:07 pm »
Is there a queue depth set? I don't know about limiters, but shapers default to 50 packets, which is not enough and causes dropped packets at high rates which reduces throughput.

Thank you Harvy66, this appears to have resolved the issue.  I do however need to set this to 400 to get constant throughput, what would you expect this to be and can it be bad having too high a number?

Edit: so basically bufferbloat if the number is too high.  So I need to understand the settings at this point and possibly move onto Codel or something else.

Thanks,

« Last Edit: February 01, 2018, 01:21:49 pm by FlangeMonkey »

Offline Harvy66

  • Hero Member
  • *****
  • Posts: 2321
  • Karma: +213/-12
    • View Profile
Re: Traffic shaper limit throughput not stable
« Reply #4 on: February 01, 2018, 09:19:09 pm »
If you're worried about bufferbloat, you could look into setting up fq_Codel on your limiters. Otherwise, a good rule of thumb is to aim for about 10ms-50ms of buffer. How to calculate this. The average backbone packet size is about 600bytes. 600 bytes time 8 = 4800bits.

(Bandwidth*10ms)/4800bits = queue size

(200Mbit*10ms)/4800bits = 416 packets.

I guess your 400 packets is about right.

Keep an eye on this if you don't want to manually setup fq_Codel. https://redmine.pfsense.org/issues/6620   They're planning on adding it to the UI at some point. I assume it may be as easy as a check box on limiters.

Offline Lawrence Dol

  • Newbie
  • *
  • Posts: 13
  • Karma: +0/-0
    • View Profile
    • Software Monkey
Re: Traffic shaper limit throughput not stable
« Reply #5 on: February 05, 2018, 01:20:20 am »
Check out this link:

https://www.reddit.com/r/PFSENSE/comments/3e67dk/flexible_vs_fixed_limiters_troubleshooting_with/

It worked perfectly for me, including giving me top-notch VOIP while bandwidth is fully saturated with upload or download traffic, and perfectly dividing the bandwidth between multiple users.

The most salient sections from that post follow.



Fixed Limiters

These are the more commonly discussed limits from what I've seen. Fixed limits are used when a network operator wants to permit only a very specific upper bound of bandwidth to be used by an individual device, no matter what. Use cases might include public WiFi scenarios, where a network operator wants to discourage people from relying on it being a top quality connection to avoid attracting people who camp out on their network consuming maximum bandwidth all day.

Example goal: 256kbps upload limit, 1mbps download limit (enforced per device)

The configuration in Firewall > Traffic Shaper > Limiter:

    Create a new Pipe
    Name: Upload
    Bandwidth: 256kbps
    Schedule: None
    Mask: Source addresses (no need to type a number into either of the numeric field boxes in this section)
    Create a new Pipe
    Name: Download
    Bandwidth: 1mbps
    Schedule: None
    Mask: Destination addresses (no need to type a number into either of the numeric field boxes in this section)

The configuration in the applicable LAN-side firewall rule:

    Advanced > In/Out: Upload / Download



Flexible Limits

These are less common, and I didn't realize it was actually possible to do this with pfSense until I got Steve's feedback (forum discussions allude to it, but I haven't seen a correct config fully described anywhere yet). The purpose of flexible limits is to allow pfSense to enforce a total cap on user traffic and to dynamically manage the connections based on real network conditions -- allocating more bandwidth per device when the network is quiet and less bandwidth per device when many clients are chatting at the same time. In my case, I've seen users report pleasantly usable network conditions consistently even while the network link was 100% saturated -- this is a very good tool to have in your kit for overloaded Internet uplinks (in one case, I've got a large download capacity but a very small upload capacity, and the users would completely overload the upload, resulting in poor conditions for everyone until I implemented this flexible limiter).

Example goal: Provide a high quality user experience for hundreds/thousands of devices sharing a business-class cable connection with 300mbps download and 20mbps upload capacity.

The configuration in Firewall > Traffic Shaper > Limiter:

Create a new Pipe

    Name: Upload
    Bandwidth: 18mbps (put the total amount of bandwidth available here; remember to save a small amount of bandwidth for remote management, downloading packages, etc -- in this example, we're allowing 18mbps for users on a 20mbps line)
    Schedule: None
    Mask: None
    Create a new Queue under Upload
    Name: UploadQueue
    Mask: Source addresses
    Create a new Pipe
    Name: Download
    Bandwidth: 290mbps (in this example, we're allowing 290mbps for users on a 300mbps line)
    Schedule: None
    Mask: None
    Create a new Queue under Download
    Name: DownloadQueue
    Mask: Destination addresses

The configuration in the applicable LAN-side firewall rule:

    Advanced > In/Out: UploadQueue / DownloadQueue
Lawrence Dol
Perfection is the enemy of excellence.
pfSense on a recycled AMD AthlonII X3 435; 3GHz; 8 GiB