Netgate SG-1000 microFirewall

Author Topic: 10gb routing not even close  (Read 172 times)

0 Members and 1 Guest are viewing this topic.

Offline donnydavis

  • Newbie
  • *
  • Posts: 17
  • Karma: +1/-0
    • View Profile
10gb routing not even close
« on: November 07, 2017, 05:04:43 pm »
I will start by saying how thankful I am pfsense exists. It has been a core piece of my infra for many years.

Now to my issue, which I just canít seems to shake out.

I have a c2758 based machine that I am trying to get 10gb of throughput on. However it doesnít seem to be anywhere near that. I started with a chelsio t4 card and thought that was the issue. So I got an intel x520-sr2 and a chelsio t5. Results from the three all net closely the same performance.

iperf tests from server to server on my network only yields 1.12gb at best when routing.

When testing iperf from the same machines on the same l2 I get around 7gb which is a issue all its own.

Overall I would be happy with 5-6gb but I am a far cry from that.

I have tried all of the googleable tuning guides to no end.

I also thought it could be pfsense, so I gave openbsd a spin... with similar results.

Is this server even capable of 10gb. Itís nearly the same as what you buy from netgate(should have just bought it from them).

Any ideas... any at all

Offline donnydavis

  • Newbie
  • *
  • Posts: 17
  • Karma: +1/-0
    • View Profile
Re: 10gb routing not even close
« Reply #1 on: November 07, 2017, 05:19:33 pm »
Mod, this should probably be moved to hardware

Offline mikeisfly

  • Sr. Member
  • ****
  • Posts: 466
  • Karma: +6/-0
    • View Profile
Re: 10gb routing not even close
« Reply #2 on: November 07, 2017, 06:36:57 pm »
Hmm very interesting. I have ordered two Chelsio T3 cards off eBay with CX4 Interfaces. Just waiting for them to arrive. While not exactly the same check this video out it could give you some tips on how to optimize your setup. I plan on going through a switch I will report back when I have everything up and running my wan link is 1Gb symmetric.

https://www.youtube.com/watch?v=p39mFz7ORco


My PfSense firewall in a :

Intel(R) Core(TM) i5-3470 CPU @ 3.20GHz
Current: 3200 MHz, Max: 3201 MHz
4 CPUs: 1 package(s) x 4 core(s)
AES-NI CPU Crypto: Yes (active)

with 4GB of ram. I PCIe slot I believe is gen 2 maybe 3 but I  play on putting the card in the x16 slot.


My Server is a Dual Xeon E2690 with 128 GB RAM 1 TB SSD Boot Drive / 32 (8x4) TB HDD running storage spaces with 512 GB SSD Cache. The server is running Windows server 2012 R2. First I will check switch speed then I will check routing performance I would be shocked if I can get filtering performance anywhere near 10Gb but if I can get it above 2-3 Gbps I would feel like my money was well spent.

Offline donnydavis

  • Newbie
  • *
  • Posts: 17
  • Karma: +1/-0
    • View Profile
Re: 10gb routing not even close
« Reply #3 on: November 07, 2017, 06:53:38 pm »
I have 5 of these servers
One running fedora 26 - mellanox connect x
two centos 7 - mellanox connect x
one pfsense - Chelsio T520
one openbsd - Intel X520
One freenas - Chelsio S320

All connected via fiber to an arista 10GB switch


Everything expect the pfsense and openbsd boxes will get nearly 10G (8ish)
It makes no sense why these two machines want to be weirdos about their line rate, but it gets the full line rate of a 1GB link.

Things I have tried so far
Swapping drives from known 10GB working machines to the pfsense machine
swapping SFP+ modules
new fiber cables
swapping fibre cables
moving to known working 10G ports on the switch


I am totally at a loss here.. I just can't make sense of it.

Offline donnydavis

  • Newbie
  • *
  • Posts: 17
  • Karma: +1/-0
    • View Profile
Re: 10gb routing not even close
« Reply #4 on: November 07, 2017, 06:56:21 pm »