Netgate SG-1000 microFirewall

Author Topic: pfSense underperforming, high jitter + random packet loss  (Read 802 times)

0 Members and 1 Guest are viewing this topic.

Offline w0w

  • Sr. Member
  • ****
  • Posts: 558
  • Karma: +35/-6
  • kernel panic attack
    • View Profile
Re: pfSense underperforming, high jitter + random packet loss
« Reply #15 on: December 19, 2017, 10:30:40 am »
ALTQ CODELQ, NaterGator settings http://www.dslreports.com/speedtest/27005845  As you can see dslreports automatucally dropped to 18 : 6 streams.
And for the 30/30 streams we have a problem! Triple test start ended with stuck on idle latency testing with spikes (failed due to overall timeout. error:2) and at the end I've got this with 24/24 http://www.dslreports.com/speedtest/27006168
And repeat test with FQ_CODEL and 30/30 http://www.dslreports.com/speedtest/27006586
There is something broken in ALTQ CODELQ...

« Last Edit: December 19, 2017, 10:36:04 am by w0w »

Offline NaterGator

  • Newbie
  • *
  • Posts: 14
  • Karma: +0/-0
    • View Profile
Re: pfSense underperforming, high jitter + random packet loss
« Reply #16 on: December 19, 2017, 10:52:31 am »
Thanks for the extra effort and offering some level of confirmation that I'm not totally crazy. I'm not sure if this is an issue I should submit to the pfSense tracker or if this belongs upstream on FreeBSD's end.


FWIW: To reduce variables I use the preferences on the dslreports test to set fixed servers that I know are close by and a fixed number of streams.

Offline Chrismallia

  • Full Member
  • ***
  • Posts: 273
  • Karma: +20/-4
    • View Profile
Re: pfSense underperforming, high jitter + random packet loss
« Reply #17 on: December 19, 2017, 11:58:27 am »
@w0w

You get great results :) using fq_codel. The minimum  ping spike I could get was 150 something just on download, upload is fine , but I think ISP matters and also that you have a symmetrical  speed  makes a difference
« Last Edit: December 19, 2017, 12:08:24 pm by Chrismallia »

Offline w0w

  • Sr. Member
  • ****
  • Posts: 558
  • Karma: +35/-6
  • kernel panic attack
    • View Profile
Re: pfSense underperforming, high jitter + random packet loss
« Reply #18 on: December 20, 2017, 09:06:58 am »
Chrismallia, yes it's ISP, just very good ISP network at least in my location.
 NaterGator, it's FreeBSD, but I don't think anybody cares ALTQ CODELQ, you have alternative with HFSC and codel enabled queue. I think next 3-5 years we will see some progress for IPFW or ALTQ it does not matter they both need code to be rewritten from scratch, because of used 32-bit integers they both do not support modern traffic bandwidth (over 4 Gigs/sec). 

Offline NaterGator

  • Newbie
  • *
  • Posts: 14
  • Karma: +0/-0
    • View Profile
Re: pfSense underperforming, high jitter + random packet loss
« Reply #19 on: December 20, 2017, 09:30:45 am »
NaterGator, it's FreeBSD, but I don't think anybody cares ALTQ CODELQ, you have alternative with HFSC and codel enabled queue. I think next 3-5 years we will see some progress for IPFW or ALTQ it does not matter they both need code to be rewritten from scratch, because of used 32-bit integers they both do not support modern traffic bandwidth (over 4 Gigs/sec).

Hmm, I do see this issue in HFSC with and without codel enabled. What I'm saying is any altq enabled shaping at all triggers the issue.

Offline Chrismallia

  • Full Member
  • ***
  • Posts: 273
  • Karma: +20/-4
    • View Profile
Re: pfSense underperforming, high jitter + random packet loss
« Reply #20 on: December 20, 2017, 12:35:19 pm »
w0w

What is strange for me is that with no traffic shaping I get low ping spikes  on download and high ping spikes on upload, when enabling any shaping including fq  I get low ping spikes on upload but then get high ping spikes on download,  see my results in post 3 if you may, I cant understand it

Offline w0w

  • Sr. Member
  • ****
  • Posts: 558
  • Karma: +35/-6
  • kernel panic attack
    • View Profile
Re: pfSense underperforming, high jitter + random packet loss
« Reply #21 on: December 20, 2017, 12:47:39 pm »
NaterGator, I did not tested HFSC for a long time, but I can test it also, later this week. Can you provide more settings regarding queues you have used? Did you change any other settings like RED or ECN, priority, queue limit in packets?

Offline w0w

  • Sr. Member
  • ****
  • Posts: 558
  • Karma: +35/-6
  • kernel panic attack
    • View Profile
Re: pfSense underperforming, high jitter + random packet loss
« Reply #22 on: December 20, 2017, 01:00:39 pm »
w0w

What is strange for me is that with no traffic shaping I get low ping spikes  on download and high ping spikes on upload, when enabling any shaping including fq  I get low ping spikes on upload but then get high ping spikes on download,  see my results in post 3 if you may, I cant understand it
I have had some similar results, but I am not sure if it's some DSLreports bug or anything else, their server, or other problems, ex. ISP.
You should repeat your tests 10 times at least with traffic shaping and without, to make some conclusions.


Offline w0w

  • Sr. Member
  • ****
  • Posts: 558
  • Karma: +35/-6
  • kernel panic attack
    • View Profile
Re: pfSense underperforming, high jitter + random packet loss
« Reply #23 on: December 23, 2017, 03:03:52 am »
http://www.dslreports.com/speedtest/27223186 ALTQ HFSC codel enabled queue + ECN, dslreports 24/6 HD enabled
https://www.dslreports.com/speedtest/27223235 ALTQ HFSC codel enabled queue + ECN, dslreports 30/30 HD enabled twice got failed due to overall timeout. error:2
http://www.dslreports.com/speedtest/27223412 FQ_CODEL again

NaterGator did you try to set bandwidth limit for the ALQ shapers more tighten then for FQ_CODEL? I think it can be possible reason for the spikes you've got. The overhead for the ALTQ is slightly bigger so it's possible that it fails on a bandwidth limit. To make sure put the bandwidth to 50% of overall.

I can not explain why dslreports often fails to start with ALTQ CODEL and FQ_CODEL mostly never fails. May be it's by design of DSLreports or ALQ CODEL.  :D
« Last Edit: December 23, 2017, 06:14:41 am by w0w »

Offline NaterGator

  • Newbie
  • *
  • Posts: 14
  • Karma: +0/-0
    • View Profile
Re: pfSense underperforming, high jitter + random packet loss
« Reply #24 on: December 23, 2017, 01:03:31 pm »
NaterGator did you try to set bandwidth limit for the ALQ shapers more tighten then for FQ_CODEL? I think it can be possible reason for the spikes you've got. The overhead for the ALTQ is slightly bigger so it's possible that it fails on a bandwidth limit. To make sure put the bandwidth to 50% of overall.

Oh yes, I tried some absurdly low limits just to be absolutely positive I had plenty of headroom, for example:
https://www.dslreports.com/speedtest/25228491

The insane spikey buffering effect from ALTQ is definitely still there.

Offline w0w

  • Sr. Member
  • ****
  • Posts: 558
  • Karma: +35/-6
  • kernel panic attack
    • View Profile
Re: pfSense underperforming, high jitter + random packet loss
« Reply #25 on: December 23, 2017, 01:59:39 pm »
Did you try ECN on ALTQ? It is enabled by default on IPFW FQ_CODEL.

Offline NaterGator

  • Newbie
  • *
  • Posts: 14
  • Karma: +0/-0
    • View Profile
Re: pfSense underperforming, high jitter + random packet loss
« Reply #26 on: December 23, 2017, 02:03:43 pm »
Yes, I tried with and without ECN.