The pfSense Store

Author Topic: Understanding throughput and ethernet  (Read 3117 times)

0 Members and 1 Guest are viewing this topic.

Offline YQ

  • Full Member
  • ***
  • Posts: 140
    • View Profile
Understanding throughput and ethernet
« on: September 25, 2012, 12:59:18 pm »
I had a little doubt regarding throughput that I have seen of many commercially available firewalls and also some of the pfsense based appliances.

So I saw one of the firewalls with around 4 gigabit ports, showing a firewall throughput of 1.5 GBPS!

How is that possible? I mean considering the fact that the link speed of that gigabit Ethernet is 1 GBPS?
Am I missing something?

Offline stephenw10

  • Hero Member
  • *****
  • Posts: 8077
    • View Profile
Re: Understanding throughput and ethernet
« Reply #1 on: September 25, 2012, 01:10:21 pm »
They often give the maximum throughput of the device as the total of all connections through it. 2x gigabit in 2x gigabit out. Also that might well be small udp packets, for example, not a practical value. Read the small print!

Steve

Offline YQ

  • Full Member
  • ***
  • Posts: 140
    • View Profile
Re: Understanding throughput and ethernet
« Reply #2 on: September 25, 2012, 01:13:38 pm »

Offline starshooter10

  • Jr. Member
  • **
  • Posts: 39
    • View Profile
Re: Understanding throughput and ethernet
« Reply #3 on: September 25, 2012, 03:25:48 pm »
ya they are saying that its able to push 2GB/s of raw bandwith, inside the network (claimed)



really the only thing I care about is IPSec Throughput, if thats higher then my connection chances are I wont cap it during normal use with several packages running...

not that they are really related... but its worked as a rule of thumb for me :P

Offline wallabybob

  • Hero Member
  • *****
  • Posts: 5262
    • View Profile
Re: Understanding throughput and ethernet
« Reply #4 on: September 25, 2012, 03:46:23 pm »
take a look at this :

http://www.hacom.net/catalog/mars-twitter-d525-pfsense-appliance
So I presume you are asking "how can the Mars Twitter get 2Gbps throughput?"

(It is common practice to use "B" for "bytes" and "b" for bits, 1B = 8b, hence your earlier question
So I saw one of the firewalls with around 4 gigabit ports, showing a firewall throughput of 1.5 GBPS!

How is that possible?
could be taken to mean "how is a throughput of 12Gbps (12 = 1.5 x 8) possible?" It is not possible on on a system with only 4 Gigabit ports.)

Imagine a box with 4 Gigabit ports and with sufficient capability so that its data throughput is limited only by the speed of the network ports. Since each port is capable of concurrently transmitting at 1Gbps and receiving at 1Gbps, the box has an "engineering throughput" of 4Gbps and a "marketing throughput" of 8Gbps. ("Engineers" count the bits as they go through the box; in their quest for bigger numbers, "marketers" count them on the way in and again on the way out.)

In practice, throughput can be highly dependent on frame size so an "engineering" statement of throughput will likely include a table of throughput for various frame sizes or a reference to a test specification while a "marketing" statement might be "based on" an "engineering" statement (for example, double the biggest number in the "engineering" statement).


« Last Edit: September 25, 2012, 03:50:09 pm by wallabybob »

Offline YQ

  • Full Member
  • ***
  • Posts: 140
    • View Profile
Re: Understanding throughput and ethernet
« Reply #5 on: September 25, 2012, 03:53:28 pm »
Quote
could be taken to mean "how is a throughput of 1.5x8 Gbps possible?" and the answer is it is not possible on on a system with only 4 Gigabit ports.)

Well sorry for not taking the B/b in mind. from what we generally notice, there is a max speed of 1Gbps, and that is what I was wondering how can they get a throughput of something more than that!


So can someone tell me what is the REAL convention? or rather the convention that is usually followed by companies like cisco?
Or I am guessing that these people just add up the max throughput of all the interfaces?

and how is throughput calculated? something like using iperf from one port to the other? ??? ??? ???

Offline stephenw10

  • Hero Member
  • *****
  • Posts: 8077
    • View Profile
Re: Understanding throughput and ethernet
« Reply #6 on: September 25, 2012, 06:05:20 pm »
B/b aside (though it's important to get right) I've always taken this to be maximum speed the box can forward packets, usually limited by the CPU. Earlier I said small packets but in fact using jumbo frames and udp will give higher results.
However, as Wallabybob said, marketing guys love to use big numbers.

Steve

Offline dreamslacker

  • Hero Member
  • *****
  • Posts: 808
    • View Profile
Re: Understanding throughput and ethernet
« Reply #7 on: September 27, 2012, 04:34:56 am »
Well sorry for not taking the B/b in mind. from what we generally notice, there is a max speed of 1Gbps, and that is what I was wondering how can they get a throughput of something more than that!


So can someone tell me what is the REAL convention? or rather the convention that is usually followed by companies like cisco?
Or I am guessing that these people just add up the max throughput of all the interfaces?

and how is throughput calculated? something like using iperf from one port to the other? ??? ??? ???

Ignoring how the throughput is tested (different testing methods will give different results), you have to understand first that throughput is the sum of all traffic flowing through the router/ firewall on all interfaces and in all directions.

It is entirely possible for a box with a 1 GBe WAN and 1GBe LAN to have a throughput of 1.5Gbit/s.  It just isn't 1.5GBit/s one-way.  It's both ways - if you were downloading large files and also uploading files at the same time.

For testing methodology, always check the fine print.  The buffer size, window size, packet size, number of simultaneous connections etc. all play a part in the results you get.

Offline YQ

  • Full Member
  • ***
  • Posts: 140
    • View Profile
Re: Understanding throughput and ethernet
« Reply #8 on: September 27, 2012, 09:28:15 am »
Quote
throughput is the sum of all traffic flowing through the router/ firewall on all interfaces and in all directions.
@dreamslacker - thank you for that info. That does solve SOME puzzles :)

any ideas on how is it tested?

So if i have four interfaces, 1,2,3,4 and I have two computers, PC1 and PC2, how is it done?

PC1 -> 1 -> PC2 (and reverse)
PC1 -> 2 -> PC2  (and reverse)
?
and then I would just add up the results and call that throughput? ???

Offline dreamslacker

  • Hero Member
  • *****
  • Posts: 808
    • View Profile
Re: Understanding throughput and ethernet
« Reply #9 on: September 27, 2012, 09:45:01 am »
If you have 4 interfaces, then you would likely need 4 computers or at least 2 nics on each of 2 computers.  Each of these NICs will be on a separate subnet from the others to ensure the routing works correctly.

eg.
Computer 1 NIC1 IP: 10.0.1.1/24
Computer 1 NIC2 IP: 10.0.2.1/24
Computer 2 NIC1 IP: 10.0.3.1/24
Computer 2 NIC2 IP: 10.0.4.1/24

You would then run 2 instances of the software (assuming bidirectional testing is enabled) on each computer.  Each instance will bind to one of the NICs on the computer and will target the other computer on a specific IP address that corresponds to the particular NIC.

So using the above example,you will have:

Instance 1 bound to 10.0.1.1 NIC on computer 1 and with remote end point as 10.0.3.1 on Computer 2
Instance 2 bound to 10.0.2.1 NIC on computer 1 and with remote end point as 10.0.4.1 on Computer 2
Instance 3 bound to 10.0.3.1 NIC on computer 2 and listening for connection
Instance 4 bound to 10.0.4.1 NIC on computer 2 and listening for connection


Of course, the appropriate interface IPs on the router/ firewall interfaces must be set and the routing and/ or firewall rules where applicable must be set to allow the traffic to flow through the router/ firewall.
The testing software should allow you to bind to one of the network adapters for a particular instance.

Offline YQ

  • Full Member
  • ***
  • Posts: 140
    • View Profile
Re: Understanding throughput and ethernet
« Reply #10 on: September 27, 2012, 09:52:22 am »
Quote
You would then run 2 instances of the software (assuming bidirectional testing is enabled)

Ill be using iperf, and it does have an option for bidirectional testing.
So I suppose whatever output I get from computer1 and computer2, I would add them up and call it a day?

The other question - do I need to start both instances of iperf at the same time?

Offline YQ

  • Full Member
  • ***
  • Posts: 140
    • View Profile
Re: Understanding throughput and ethernet
« Reply #11 on: September 28, 2012, 04:10:03 am »
And I don't know if this is a stupid question. But I would need gigabit NICs on my machine as my 1U server has 4x 1G ports.?

Offline dreamslacker

  • Hero Member
  • *****
  • Posts: 808
    • View Profile
Re: Understanding throughput and ethernet
« Reply #12 on: September 28, 2012, 04:14:52 am »
Quote
You would then run 2 instances of the software (assuming bidirectional testing is enabled)

Ill be using iperf, and it does have an option for bidirectional testing.
So I suppose whatever output I get from computer1 and computer2, I would add them up and call it a day?

The other question - do I need to start both instances of iperf at the same time?

Yes, you need both instances to start at about the same time.  It may be difficult to get both started simultaneously so a batch file call might be in order.

There is a listening mode (server/ host mode) and a initiator/ client mode for iperf.  The results will be the same on both ends so you need only activate and check the results on one computer for both instances.

The results for smaller packet sizes may not be as accurate for multi-instance due to the slight delay (even if you use a batch file) in between starting both tests.
I can't remember if it is possible to extend the test by using a large amount of data but if that is possible, then it will certain help with getting more accurate results where the total throughput for a particular dataset type would be the sum of the upload and download speeds for both instances.

Offline YQ

  • Full Member
  • ***
  • Posts: 140
    • View Profile
Re: Understanding throughput and ethernet
« Reply #13 on: September 28, 2012, 04:21:18 am »
Thank you.
On a number of forums/websites i have seen that  increasing the tcp window size (which is I guess the maximum data the receiving end can receive without sending an acknowledgement to the sender)
increased throughput significantly. Like chaning the TCP window size from 8k to 64k shows an increase of about 3x.

The questions is, in a production environment, what is the default tcp window size, and how is the tcp window size negotiated when there is a TCP transmission initiated between 2 hosts?

Offline phil.davis

  • Hero Member
  • *****
  • Posts: 2303
    • View Profile
    • International Nepal Fellowship
Re: Understanding throughput and ethernet
« Reply #14 on: September 28, 2012, 05:10:46 am »
To maximise throughput for a single data flow, you certainly do need to get a big enough window size. I've done this a few rimes in the past. For example, you have an app that gathers data continuously at node A and needs to send close to 4Mbits/second on a 4Mbps link across the world to node B. The round-trip latency is usually 300ms, but often up to 500ms. So TCP has to be willing to send up to 500ms of data before it receives an ACK for even the first packet.
At 4 Mbits/second that is 2 Mbits = 250,000 bytes of data. So the window size needs to be about 250KBytes.
When it starts, node A will dump 250KBytes of data in packets directly across the local LAN onto its gateway router. The gateway router will send this down the 4Mbps link over the next 500ms. ACKs will start coming back and node A will then send more packets as ACKs arrive.
If you make the window size too small, then the flow will stop for a bit waiting for ACKs to come back - wasted time on the link.
If you make the window size too big, then your local gateway might not appreciate the initial dumping of data into its buffers, and drop some of it!
Sorry - I didn't answer your question! The different TCP implementations will do different things by default, someone else will know what that is on FreeBSD, Linux, Windows etc.
If I helped you, then help someone else - buy someone a gift from the INF catalog http://secure.inf.org/gifts/usd/