Netgate SG-1000 microFirewall

Author Topic: RRD Graphs Incorrect  (Read 15219 times)

0 Members and 1 Guest are viewing this topic.

Offline mevans336

  • Full Member
  • ***
  • Posts: 215
  • Karma: +2/-0
    • View Profile
RRD Graphs Incorrect
« on: November 17, 2009, 07:03:50 am »
I've been testing my firewall at GigE speeds using iperf at 930Mbps or so. However, the RRD Graphs will not log anything above 400-450Mbps. When I utilized this same test at 100Mbps, RRD Graphed properly. I've attached a screenshot where you can see the accuracy was perfect at 100Mbps, but when I increased to a little over 900Mbps of actual throughput, the graphs don't log properly. The second screenshot is showing the iperf window on the server and you can see that all the runs are over 900Mbps.

This is 1.2.3-RC3.

Offline Evgeny

  • Hero Member
  • *****
  • Posts: 1808
  • Karma: +1/-1
    • View Profile
Re: RRD Graphs Incorrect
« Reply #1 on: November 17, 2009, 06:28:41 pm »
Is it UDP traffic?
If yes then you should see receiver's side.
Not everything you are pushing goes through firewall.

Offline mevans336

  • Full Member
  • ***
  • Posts: 215
  • Karma: +2/-0
    • View Profile
Re: RRD Graphs Incorrect
« Reply #2 on: November 17, 2009, 06:36:33 pm »
Is it UDP traffic?
If yes then you should see receiver's side.
Not everything you are pushing goes through firewall.

No, it's standard iperf TCP traffic. In this case, all traffic did go through the firewall, as the laptop was plugged directly into the LAN port of the pfSense box, which was in bridge mode. The WAN was plugged into my home network and the iperf server was on my desktop machine.

Offline Evgeny

  • Hero Member
  • *****
  • Posts: 1808
  • Karma: +1/-1
    • View Profile
Re: RRD Graphs Incorrect
« Reply #3 on: November 17, 2009, 06:44:31 pm »
Which switch "-??" do you use to regulate TCP bandwidth in iperf?

Offline mevans336

  • Full Member
  • ***
  • Posts: 215
  • Karma: +2/-0
    • View Profile
Re: RRD Graphs Incorrect
« Reply #4 on: November 17, 2009, 06:46:30 pm »
Which switch "-??" do you use to regulate TCP bandwidth in iperf?

I'm using fairly standard options:

iperf -c 192.168.1.20 -w 64000 -N

Basically, increase the TCP window to 64000 bytes and enable TCP No Delay.

Offline Evgeny

  • Hero Member
  • *****
  • Posts: 1808
  • Karma: +1/-1
    • View Profile
Re: RRD Graphs Incorrect
« Reply #5 on: November 17, 2009, 06:51:20 pm »
Did you try to utilize it with about 300Mb/s ?

Offline mevans336

  • Full Member
  • ***
  • Posts: 215
  • Karma: +2/-0
    • View Profile
Re: RRD Graphs Incorrect
« Reply #6 on: November 17, 2009, 07:04:57 pm »
Did you try to utilize it with about 300Mb/s ?

Do you mean limit iperf to 300Mbps? I believe you can only specify bandwidth when using UDP.

Offline Evgeny

  • Hero Member
  • *****
  • Posts: 1808
  • Karma: +1/-1
    • View Profile
Re: RRD Graphs Incorrect
« Reply #7 on: November 17, 2009, 07:08:58 pm »
Yes iperf.

Offline mevans336

  • Full Member
  • ***
  • Posts: 215
  • Karma: +2/-0
    • View Profile
Re: RRD Graphs Incorrect
« Reply #8 on: November 17, 2009, 07:13:44 pm »
Yes iperf.

You can only specify bandwidth with iperf when using UDP. With TCP, you can specify the amount of data to transfer or the length of time to transfer, but not the rate.

Usage: iperf [-s|-c host] [options]
       iperf [-h|--help] [-v|--version]

Client/Server:
  -f, --format    [kmKM]   format to report: Kbits, Mbits, KBytes, MBytes
  -i, --interval  #        seconds between periodic bandwidth reports
  -l, --len       #[KM]    length of buffer to read or write (default 8 KB)
  -m, --print_mss          print TCP maximum segment size (MTU - TCP/IP header)
  -o, --output    <filename> output the report or error message to this specifie
d file
  -p, --port      #        server port to listen on/connect to
  -u, --udp                use UDP rather than TCP
  -w, --window    #[KM]    TCP window size (socket buffer size)
  -B, --bind      <host>   bind to <host>, an interface or multicast address
  -C, --compatibility      for use with older versions does not sent extra msgs
  -M, --mss       #        set TCP maximum segment size (MTU - 40 bytes)
  -N, --nodelay            set TCP no delay, disabling Nagle's Algorithm
  -V, --IPv6Version        Set the domain to IPv6

Server specific:
  -s, --server             run in server mode
  -D, --daemon             run the server as a daemon
  -R, --remove             remove service in win32

Client specific:
  -b, --bandwidth #[KM]    for UDP, bandwidth to send at in bits/sec
                           (default 1 Mbit/sec, implies -u)
  -c, --client    <host>   run in client mode, connecting to <host>
  -d, --dualtest           Do a bidirectional test simultaneously
  -n, --num       #[KM]    number of bytes to transmit (instead of -t)
  -r, --tradeoff           Do a bidirectional test individually
  -t, --time      #        time in seconds to transmit for (default 10 secs)
  -F, --fileinput <name>   input the data to be transmitted from a file
  -I, --stdin              input the data to be transmitted from stdin
  -L, --listenport #       port to recieve bidirectional tests back on
  -P, --parallel  #        number of parallel client threads to run
  -T, --ttl       #        time-to-live, for multicast (default 1)

Miscellaneous:
  -h, --help               print this message and quit
  -v, --version            print version information and quit

[KM] Indicates options that support a K or M suffix for kilo- or mega-

The TCP window size option can be set by the environment variable
TCP_WINDOW_SIZE. Most other options can be set by an environment variable
IPERF_<long option name>, such as IPERF_BANDWIDTH.

Report bugs to <dast@nlanr.net>

Offline Evgeny

  • Hero Member
  • *****
  • Posts: 1808
  • Karma: +1/-1
    • View Profile
Re: RRD Graphs Incorrect
« Reply #9 on: November 17, 2009, 07:56:02 pm »
Yes, I know that is why I asked the first question.
For RRD it does not matter what kind of traffic you are passing. So, could you please transfer 300Mb/s UDP and see what you get on graph?

Offline mevans336

  • Full Member
  • ***
  • Posts: 215
  • Karma: +2/-0
    • View Profile
Re: RRD Graphs Incorrect
« Reply #10 on: November 17, 2009, 08:14:58 pm »
Yes, I know that is why I asked the first question.
For RRD it does not matter what kind of traffic you are passing. So, could you please transfer 300Mb/s UDP and see what you get on graph?

Sure, I'll give that a shot first thing tomorrow AM and report back.

Offline Evgeny

  • Hero Member
  • *****
  • Posts: 1808
  • Karma: +1/-1
    • View Profile
Re: RRD Graphs Incorrect
« Reply #11 on: November 17, 2009, 09:55:07 pm »
When you'll be doing testing please get output of below command for 3-4 minutes
Code: [Select]
/usr/bin/netstat -nbf link -I XXX -w60where XXX is the name of your interface.

Offline Evgeny

  • Hero Member
  • *****
  • Posts: 1808
  • Karma: +1/-1
    • View Profile
Re: RRD Graphs Incorrect
« Reply #12 on: November 19, 2009, 06:13:32 pm »
You are right. It seems rrd graphs do not show correct values when trafic goes above 500Mb/s.
You might wish to try the next:
1) kill your current updaterrd.sh process
2) modify script /var/db/rrd/updaterrd.sh replacing line
Code: [Select]
sleep 60with
Code: [Select]
sleep 303) start new one by
Code: [Select]
/bin/sh /var/db/rrd/updaterrd.sh &Note: your graphs will be updated two times more often than before which will create some additional cpu usage.

Offline mevans336

  • Full Member
  • ***
  • Posts: 215
  • Karma: +2/-0
    • View Profile
Re: RRD Graphs Incorrect
« Reply #13 on: November 19, 2009, 06:19:23 pm »
You are right. It seems rrd graphs do not show correct values when trafic goes above 500Mb/s.
You might wish to try the next:
1) kill your current updaterrd.sh process
2) modify script /var/db/rrd/updaterrd.sh replacing line
Code: [Select]
sleep 60with
Code: [Select]
sleep 303) start new one by
Code: [Select]
/bin/sh /var/db/rrd/updaterrd.sh &Note: your graphs will be updated two times more often than before which will create some additional cpu usage.

You're the man Eugene. I didn't have time to set everything up to test this today and it looked like it was going to be this weekend before I was going to be able to get around to it. So will lowering the sleep timer effectively increase the sampling rate of RRD? I.E. the "1 minute average" will become a "30 second average?"

Offline Evgeny

  • Hero Member
  • *****
  • Posts: 1808
  • Karma: +1/-1
    • View Profile
Re: RRD Graphs Incorrect
« Reply #14 on: November 19, 2009, 06:27:20 pm »
Unfortunately I do not the answer to this question.
I tend to think yes - it will become 30 sec average but I do not know rrd well enough to stay firm on this. More research needed.