The pfSense Store

Author Topic: Dashboard Traffic Graphs...SPIKES  (Read 1655 times)

0 Members and 1 Guest are viewing this topic.

Offline Valeriy

  • Jr. Member
  • **
  • Posts: 48
  • Karma: +7/-0
    • View Profile
Dashboard Traffic Graphs...SPIKES
« on: February 22, 2017, 08:46:31 am »
Hi!


Is it possible to revert the traffic graphs to how it was in 2.3.2?
Now it shows unnecessary spikes even when there is none. Tried different browsers, the result is the same.
I had thought that limiters or other packages might have interfered , so I installed 2.3.3 and 2.4.b onto different VMs, ran iperf and got the same issues.

Result is the same, disregardless of:

-  different browsers, including keeping current TAB always active or inactive
- system in production or clean install (without load).
- with different time interval (1-10sec)
- different interfaces
- pfsense 2.3.3 and 2.4.0b are affected
- etc.

It shows spikes where clearly there weren't.

Verified interface load with
>systat -ifstat 1 on corresponding systems

speed remained stable ..

Nobody else experienced it?

Screenshots are attached.

« Last Edit: March 07, 2017, 07:29:53 am by Valeriy »

Offline Valeriy

  • Jr. Member
  • **
  • Posts: 48
  • Karma: +7/-0
    • View Profile
Re: Dashboard Traffic Graphs...SPIKES
« Reply #1 on: February 22, 2017, 08:51:26 am »
Sometimes it stalls for a few seconds, and shows spike for x3 higher than average b/w load. seems to be that data is gathered by browser app in specific way directly instead of getting correct interface loads from pfSense OS.

Offline jimp

  • Administrator
  • Hero Member
  • *****
  • Posts: 21404
  • Karma: +1437/-26
    • View Profile
Re: Dashboard Traffic Graphs...SPIKES
« Reply #2 on: February 24, 2017, 08:44:47 am »
Open a bug report for that on https://redmine.pfsense.org
Need help fast? Commercial Support!

Co-Author of pfSense: The Definitive Guide. - Check the Doc Wiki for FAQs.

Do not PM for help!

Offline RonpfS

  • Hero Member
  • *****
  • Posts: 684
  • Karma: +96/-2
    • View Profile
Re: Dashboard Traffic Graphs...SPIKES
« Reply #3 on: February 24, 2017, 12:42:23 pm »
I was able to reproduce the spikes in two ways :

1) in pfBlockerNG / Alerts, it takes about a minute to gather all the alerts and do DNS lookup
2) in pfBlockerNG / Update / View log until you click on End View

Both Status Traffic Graphs and Widget Traffic graph showed spikes.
There is probably other ways to reproduce the spikes.
2.3.5-RELEASE (amd64)
Intel Core2 Quad CPU Q8400 @ 2.66GHz 8GB
pfBlockerNG 2.1.2_1/Dev, suricata 4.0.1

Offline PiBa

  • Hero Member
  • *****
  • Posts: 789
  • Karma: +129/-1
  • PiBa-NL(on IRC)
    • View Profile
Re: Dashboard Traffic Graphs...SPIKES
« Reply #4 on: February 25, 2017, 10:22:26 am »
Tried to optimize trafficgraph data retrieval a bit, can you guys test if it mitigates the issue.?

https://github.com/pfsense/pfsense/pull/3585

The way the graphs call to ifstat.php to get the latest usage number doesn't seem to have changed much so doesnt explain why it would only start to happen with 2.3.3 / 2.4 so that by itself shouldn't be the issue.. It could be a different issue underneath somewhere. With pullrequest above large parts of the re-authentication code/initialization are skipped. Anyhow I hope it helps for this case as well.

Offline PiBa

  • Hero Member
  • *****
  • Posts: 789
  • Karma: +129/-1
  • PiBa-NL(on IRC)
    • View Profile
Re: Dashboard Traffic Graphs...SPIKES
« Reply #5 on: February 25, 2017, 04:00:24 pm »
Okay new patches available :).

First one same as above but now with a 2.3 link as well:
## traffic-graphs, optimize retrieval of data every x seconds ##
For 2.3: https://github.com/PiBa-NL/pfsense/commit/34e1ceabf20419a1e1d8436c03504a0d7706be5d
For 2.4: https://github.com/pfsense/pfsense/pull/3585

The second commit seems to have more effect, combined they are even better. From initial testing the 'big spikes' are gone, a few small ones seem to remain though. Not sure yet what can be done there without moving to a whole different method for retrieving the data needed..

## flash_message, commit session when done with it ##
For 2.3 https://github.com/PiBa-NL/pfsense/commit/d188e833d1c7a0756517b9e92bf41535fff9973a
For 2.4 https://github.com/pfsense/pfsense/pull/3587

I'm interested in any issues that might arise.

Offline grandrivers

  • Sr. Member
  • ****
  • Posts: 566
  • Karma: +13/-2
    • View Profile
Re: Dashboard Traffic Graphs...SPIKES
« Reply #6 on: February 25, 2017, 04:37:40 pm »
I havent tested patches yet but on graph related problems i noticed we not apparent in widget its in the status/traffic graph page where graph does not match chart I need to make a video it then becomes obvious


thread link to problem
https://forum.pfsense.org/index.php?topic=125520.0
pfsense 2.4 super micro A1SRM-2558F
C2558 8gig ECC  60gig SSD
tripple Wan dual pppoe

Offline Valeriy

  • Jr. Member
  • **
  • Posts: 48
  • Karma: +7/-0
    • View Profile
Re: Dashboard Traffic Graphs...SPIKES
« Reply #7 on: March 07, 2017, 06:45:40 am »
I am going to test the patches today.

So far I have found the following:

- seems to be phpfm process problems might have influenced the behaviour or other bottlenecks between pfsense and monitoring PC

- after Ntopng removal problem is not longer that obvious (spikes are still there but not more than 10% of current interface load average, and they do happened less frequent).

- network latency issues / monitoring PC CPU utilization 100% also trigger the issues.

Will be back after patches are tested.

Offline Valeriy

  • Jr. Member
  • **
  • Posts: 48
  • Karma: +7/-0
    • View Profile
Re: Dashboard Traffic Graphs...SPIKES
« Reply #8 on: March 07, 2017, 07:27:08 am »
Sorry for my ignorance, if somebody could quickly tell what is proper process of applying these custom patches?
...Manually editing the related files?

Offline heper

  • Hero Member
  • *****
  • Posts: 2690
  • Karma: +253/-11
    • View Profile
Re: Dashboard Traffic Graphs...SPIKES
« Reply #9 on: March 07, 2017, 07:44:50 am »
install the system_patches package / then you can copy/paste the commit url

Offline jimp

  • Administrator
  • Hero Member
  • *****
  • Posts: 21404
  • Karma: +1437/-26
    • View Profile
Re: Dashboard Traffic Graphs...SPIKES
« Reply #10 on: March 07, 2017, 07:56:13 am »
Sorry for my ignorance, if somebody could quickly tell what is proper process of applying these custom patches?
...Manually editing the related files?

1. Use the System Patches package which shows up under System > Packages once installed
2. Create a new patch entry, give it a name, paste in the URL to a commit -OR- if you want to test a pull request, enter the pull request URL with ".diff" added on. For example, to test https://github.com/pfsense/pfsense/pull/3585 enter https://github.com/pfsense/pfsense/pull/3585.diff
3. Save, fetch, apply

If there are changes on the PR and you need to update: Revert, fetch, apply
Need help fast? Commercial Support!

Co-Author of pfSense: The Definitive Guide. - Check the Doc Wiki for FAQs.

Do not PM for help!

Offline Valeriy

  • Jr. Member
  • **
  • Posts: 48
  • Karma: +7/-0
    • View Profile
Re: Dashboard Traffic Graphs...SPIKES
« Reply #11 on: March 07, 2017, 08:26:01 am »
Thanks!