next »

[darkguard]

Alix 2D3 throughput tests

Configuration:
PFSense 1.2.1-RC2
WAN:1.1.1.2
LAN:192.168.0.1

There is one IPERF computer on the LAN (192.168.0.199) and one on the WAN (1.1.1.1)
There is a port forward of standard port 5001 TCP/UDP for IPERF

PFSense is doing the NATting.
All links are 100Mbits Full Duplex (auto-negociated)

All tests are done with IPERF

Test:  TCP both way at the time:
-----------------------------------------------------------
[ ID] Interval       Transfer     Bandwidth
[3912]  0.0-10.0 sec  69.4 MBytes  57.9 Mbits/sec
[3916]  0.0-10.0 sec  68.5 MBytes  57.4 Mbits/sec


Test:  TCP one way (Server = WAN)
-----------------------------------------------------------
[ ID] Interval       Transfer     Bandwidth
[3876]  0.0-10.0 sec   102 MBytes  86.1 Mbits/sec


Test: TCP one way (server = LAN)
------------------------------------------------------------
[ ID] Interval       Transfer     Bandwidth
[3960]  0.0-10.1 sec  95.1 MBytes  79.2 Mbits/sec



There seems to be some trouble with PFsense and UDP packet size 1470 because without tuning the packet size parameter in IPERF I was unable to get a good communication between the 2 hosts

I changed the default 1470 bytes to 1024 bytes datagrams.

Test: UDP both way (Server = WAN)
------------------------------------------------------------
[ ID] Interval       Transfer     Bandwidth       Jitter   Lost/Total Datagrams
[3908]  0.0-10.0 sec  51.4 MBytes  42.9 Mbits/sec  0.934 ms 1462/54075 (2.7%)
[3976]  0.0-10.0 sec  56.9 MBytes  47.7 Mbits/sec  2.161 ms 3802/62050 (6.1%)

Test: UDP one way (Server = WAN)
------------------------------------------------------------
[ ID] Interval       Transfer     Bandwidth       Jitter   Lost/Total Datagrams
[3976]  0.0-10.0 sec  56.9 MBytes  47.8 Mbits/sec  0.407 ms 3755/62050 (6.1%)

=============================================================

Test: UDP both way (Server = LAN)
------------------------------------------------------------
[ ID] Interval       Transfer     Bandwidth       Jitter   Lost/Total Datagrams
[3924]  0.0-10.0 sec  65.1 MBytes  54.6 Mbits/sec  1.263 ms 1981/68685 (2.9%)
[3960]  0.0-10.0 sec  51.7 MBytes  43.3 Mbits/sec  0.666 ms 9136/62050 (15%)

Test: UDP one way (Server = LAN)
------------------------------------------------------------
[ ID] Interval       Transfer     Bandwidth       Jitter   Lost/Total Datagrams
[3960]  0.0-10.0 sec  74.9 MBytes  62.7 Mbits/sec  1.892 ms  130/76798 (0.17%)



On the udp side of things, doing multiple threads in parallel or trying with bigger datagrams tends to make things go as fast as 85 Mbits/sec


OPENVPN
One windows PC conneting to PFsense via openvpn.
With PKI infrastructure.
Note that the remote machine has a TAP adapter that is at 10Mbits/sec

OPENVPN (encryption)
Test: TCP one way (server = LAN)
Ovpn: LZO compression, TCP
------------------------------------------------------------
[ ID] Interval       Transfer     Bandwidth
[1912]  0.0-10.4 sec  1.30 MBytes  1.05 Mbits/sec

OPENVPN (encryption)
Test: TCP one way (server = LAN)
Ovpn: NO compression, TCP
------------------------------------------------------------
[ ID] Interval       Transfer     Bandwidth
[1912]  0.0-10.2 sec  12.7 MBytes  10.4 Mbits/sec

OPENVPN (encryption)
Test: TCP both way (server = LAN)
Ovpn: NO compression, TCP
------------------------------------------------------------
[ ID] Interval       Transfer     Bandwidth
[1844]  0.0-10.3 sec  6.15 MBytes  5.01 Mbits/sec
[1876]  0.0-10.5 sec  6.96 MBytes  5.54 Mbits/sec


OPENVPN (encryption)
Test: TCP one way (server = WAN)
Ovpn: NO compression, TCP
------------------------------------------------------------
[ ID] Interval       Transfer     Bandwidth
[1880]  0.0- 9.8 sec  14.6 MBytes  12.6 Mbits/sec

OPENVPN (encryption)
Test: TCP both way (server = WAN)
Ovpn: NO compression, TCP
------------------------------------------------------------
[ ID] Interval       Transfer     Bandwidth
[1824]  0.0-10.3 sec  7.20 MBytes  5.89 Mbits/sec
[1848]  0.0- 9.8 sec  6.30 MBytes  5.37 Mbits/sec

OPENVPN (encryption)
Test: UDP one way (server = WAN)
Ovpn: NO compression, TCP
------------------------------------------------------------
[ ID] Interval       Transfer     Bandwidth       Jitter   Lost/Total Datagrams
[3976]  0.0-10.2 sec  19.7 MBytes  16.3 Mbits/sec  1.143 ms 4268/24451 (17%)

OPENVPN (encryption)
Test: UDP both way (server = WAN)
Ovpn: NO compression, TCP
------------------------------------------------------------
I dont get good results in anyway, and it seems random... only one side seems to be getting packets through.


OPENVPN (encryption)
Test: UDP one way (server = WAN)
Ovpn: LZO compression, TCP
------------------------------------------------------------
[ ID] Interval       Transfer     Bandwidth       Jitter   Lost/Total Datagrams
[3976]  0.0-10.3 sec  40.2 MBytes  32.8 Mbits/sec  11.761 ms 7748/48946 (16%)

========================================================================
In OpenVPN with TCP transport,
It seems like UDP results with LZO are alot better then without... but TCP results with LZO are alot worse then without.
========================================================================



OPENVPN (encryption)
Test: TCP one way (server = WAN)
Ovpn: NO compression, UDP
------------------------------------------------------------
[ ID] Interval       Transfer     Bandwidth
[3928]  0.0-10.0 sec  16.7 MBytes  14.0 Mbits/sec


OPENVPN (encryption)
Test: TCP one way (server = LAN)
Ovpn: NO compression, UDP
------------------------------------------------------------
[ ID] Interval       Transfer     Bandwidth
[3960]  0.0-10.2 sec  15.1 MBytes  12.4 Mbits/sec

OPENVPN (encryption)
Test: TCP one way (server = WAN)
Ovpn: LZO compression, UDP
------------------------------------------------------------
[ ID] Interval       Transfer     Bandwidth
[3928]  0.0-10.0 sec  22.7 MBytes  19.1 Mbits/sec

OPENVPN (encryption)
Test: TCP both way (server = WAN)
Ovpn: NO compression, UDP
------------------------------------------------------------
[ ID] Interval       Transfer     Bandwidth
[3872]  0.0-10.1 sec  8.00 MBytes  6.62 Mbits/sec
[3896]  0.0-10.0 sec  6.95 MBytes  5.83 Mbits/sec

OPENVPN (encryption)
Test: TCP both way (server = WAN)
Ovpn: LZO compression, UDP
------------------------------------------------------------
[ ID] Interval       Transfer     Bandwidth
[3916]  0.0-10.0 sec  10.8 MBytes  9.08 Mbits/sec
[3872]  0.0-10.2 sec  11.0 MBytes  9.10 Mbits/sec


OPENVPN (encryption)
Test: TCP both way (server = LAN)
Ovpn: NO compression, UDP
------------------------------------------------------------
[ ID] Interval       Transfer     Bandwidth
[3892]  0.0-10.0 sec  4.09 MBytes  3.43 Mbits/sec
[3924]  0.0-10.2 sec  9.00 MBytes  7.39 Mbits/sec


OPENVPN (encryption)
Test: UDP one way (server = WAN)
Ovpn: NO compression, UDP
------------------------------------------------------------
[ ID] Interval       Transfer     Bandwidth       Jitter   Lost/Total Datagrams
[3976]  0.0-10.0 sec  19.6 MBytes  16.5 Mbits/sec  1.100 ms 4369/24451 (18%)

OPENVPN (encryption)
Test: UDP one way (server = WAN)
Ovpn: LZO compression, UDP
------------------------------------------------------------
[ ID] Interval       Transfer     Bandwidth       Jitter   Lost/Total Datagrams
[3976]  0.0-10.1 sec  27.2 MBytes  22.7 Mbits/sec  2.931 ms 8729/36575 (24%)


OPENVPN (encryption)
Test: UDP both way (server = WAN)
Ovpn: NO compression, UDP
------------------------------------------------------------
I don't get good results in anyway, and it seems random... only one side seems to be getting packets through.




Conclusions:
---------------------
This setup can do about 85 Mbits/sec clear text throughput...So about a 40 Mbits/sec symmetric link

OpenVPN is tricky:
My test where done with a remote adapter that is suppose to be at 10 Mbits/sec, so in that regard, the box does very well.

For transport UDP seems better then TCP.
For compression LZO does penalize jitter and delais but seems to get better results for all but TCP on a TCP OpenVPN where its catastrophic (1Mbits/s).

I would recommend not going with compression with the Alix board, except if you have a special use that could use the specific performance gains of it.

The box can normally fill about 10 Mbits/sec of bandwidth or 5 Mbits/sec symmetric connection with OVPN.

I have had some issues with UDP packets size of 1470 bytes not getting through PFsense for some reason.
I also have weird results with UDP both way on a UDP OVPN transport.

These tests are by no mean scientific, just do your own test if you doubt the results...

If I could get another BOX I could probably do better test in between two Alix Boxes...
I will probably measure the power this box use to have a better idea on how many watts in take on idle and on average load.  Ill re post later if there is some interest for that.

[Evil_Bert]

Great review.  Thanks very much for doing that and taking the time to post the results.

I'd love to know the power consumption both at idle and at full capacity.

[cmb]

This sounds like exactly what I would expect. You don't want to tunnel in TCP because of precisely what you're seeing, it will retransmit the lost OpenVPN packets, which will cause performance problems where there is loss (exacerbated in cases where you're intentionally putting extreme load on the link, which by its nature generates loss).  If the traffic being tunneled needs to be retransmitted, the TCP within the tunnel needs to handle that. If you're tunneling TCP over TCP, the retransmits are doubled, which quickly leads to poor performance.

In normal usage, tunneling over TCP doesn't cause significant performance problems unless you're pushing far more load than the connection can handle.

[drees]

I will probably measure the power this box use to have a better idea on how many watts in take on idle and on average load.  Ill re post later if there is some interest for that.

If I remember correctly, the Alix 2C3 that I tested pulled about 4w at idle peaked at less than 10w when plugged into a Kill-a-watt. I would expect yours to be similar, but would be interesting to verify. :-)

Thanks for the performance numbers. I wondered how fast our Alix could go, but we only have it connected to two WAN connections, a T1 and a DSL line (5Mbps/512Kbps) so it looks like we have plenty of room to grow with this little Alix box.

[dstroot]

Not sure where to post this information but I'd like it documented somewhere here on the boards - the ALIX platform works great UNTIL you try to use the Traffic Shaper - then they tip over...  just a warning to someone.

[cmb]

Not sure where to post this information but I'd like it documented somewhere here on the boards - the ALIX platform works great UNTIL you try to use the Traffic Shaper - then they tip over...  just a warning to someone.

I've never seen that, though I usually don't use the shaper.  Please start a new thread and provide details.

[wishy]

Hiya,

Could i check openvpn was tested using blowfish for bulk crypto? Did you try any other algorithms?

Also, how big was your firewall rulebase and state table? I assume these would make a fair difference to performance.

Steve

[netphreak]

Sorry to drag up an "old" thread, but instead of creating a new one...

Is really Alix too weak to do some traffic shaping? Would I be better off running my old PIII computer? Have been working well with traffic shaping, but draws a little too much electricity for my likings...

[drees]

Is really Alix too weak to do some traffic shaping? Would I be better off running my old PIII computer? Have been working well with traffic shaping, but draws a little too much electricity for my likings...

You're not giving us much information here (specs of current firewall, bandwidth, concurrent connections would be useful), but the ALIX is just fine for traffic shaping as long as you don't try to push too much data through it.

See the first post in this thread for what "too much" might be.

[netphreak]

I have often about 6-8 concurrent connections (+ visitors on webserver, ~1000 unique visitors a day), on a 20/1 ADSL line. Hardware: PIII@733, 512MB SDRAM, 3 x Intel GB nics.

Maybe I should ask my question different: Are the ALIX boards short on CPU power when shaping ~8 concurrent users?

[wallabybob]

I have often about 6-8 concurrent connections (+ visitors on webserver, ~1000 unique visitors a day), on a 20/1 ADSL line. Hardware: PIII@733, 512MB SDRAM, 3 x Intel GB nics.

Maybe I should ask my question different: Are the ALIX boards short on CPU power when shaping ~8 concurrent users?

The Alix boards have a 433MHz or 500MHz CPU and 128MB or 256MB RAM so you could expect them to be somewhat lower performing than your current system. If you are thinking of replacing your current system with an Alix board then a good place to begin might be to consider whether your current system has rather more than enough RAM and CPU power to handle your current load.

[drees]

The Alix box will be just fine on a 20/1 ASDL line with only 8 users.