pfSense Gold Subscription

Author Topic: X11SBA-LN4F vs A1SRi-2558F  (Read 41368 times)

0 Members and 1 Guest are viewing this topic.

Offline Engineer

  • Full Member
  • ***
  • Posts: 120
  • Karma: +24/-2
    • View Profile
Re: X11SBA-LN4F vs A1SRi-2558F
« Reply #45 on: November 08, 2015, 11:27:47 am »
Have you tried disabling MSIX and MSI? Instructions at https://doc.pfsense.org/index.php/Tuning_and_Troubleshooting_Network_Cards#MSI.2FMSIX

I tried that once before but it locked the system (tried to disable both at same time).  I'll try again to see if I it makes a difference.  Will have to wait as I have a house full of people - all using the web continuously.  I believe I would know about the Internet going down faster (with all of the crying about it) than I would know about someone breaking into my house and shooting up the place, lol.  :P

From dmesg, it would appear that the ports are using MSIX.  Will disable that first and see what happens.

Edit:  msix disabled now (confirmed via dmesg).  Will report back.  If it goes down again, will try to disable both.
« Last Edit: November 08, 2015, 12:14:36 pm by Engineer »

Offline Engineer

  • Full Member
  • ***
  • Posts: 120
  • Karma: +24/-2
    • View Profile
Re: X11SBA-LN4F vs A1SRi-2558F
« Reply #46 on: November 09, 2015, 05:30:05 pm »
Disabling MSIX didn't fix it.  Just went down again.

When I get a chance, I'll try to disable both again.

 :-[

Edit:  Just disabled both MSIX and MSI and the LAN went down.  I logged in via IPMI and the console indicated all was fine (ifconfig - lists all as active).  Tried pinging from the shell and couldn't reach anything on the LAN.  Could ping the cable modem on the WAN port just fine.

Something is borking the LAN...what....I have no idea?!?
« Last Edit: November 09, 2015, 05:46:36 pm by Engineer »

Offline Thimee

  • Newbie
  • *
  • Posts: 2
  • Karma: +0/-0
    • View Profile
Re: X11SBA-LN4F vs A1SRi-2558F
« Reply #47 on: November 09, 2015, 06:32:07 pm »
I have an X10SBA-L build:
  • SSD: 120 GB Crucial BX100
  • Case + PSU Combo: M350 and 90W PicoPSU
  • RAM: 2x4GB Crucial

I had a couple of problems that might be worth mentioning:
  • My first build had a faulty PSU/12V adaptor (wavy lines on VGA output, buzzing noise). Wouldn't stay up for more than 30 seconds, I replaced both, after which no more PSU issues. I've ordered a 12V straight cable (in theory no PicoPSU necessary) but haven't gotten around to installing it yet.
  • I had lots of trouble getting the exact RAM, at first I ordered some 1.35V/1.5V from Amazon, which passed Memtest for about 15 minutes until it failed. Bizarre. The replacements I got from Microcenter (didn't want to wait for shipping) work perfectly.
  • I added a fancy Swellder Delta TFB0412EHN PWM fan - sounds like a rocket ship on startup but afterwards the fan never turns on, system barely gets warm.

It was a bit of a dog to install 2.2.4, I had endless crashes and even after successful install it wouldn't stay up for more than a few hours before corrupting the file system. However after running the exhaustive Memtest I saw one failure, changing the RAM out with a slightly different model fixed all my issues.

Since then I had 51 days of uptime (I use pfsense on my home netrowk for firewall, VPN server, DNS server), reset this morning to upgrade to pfsense 2.2.5 which appears to be running just great.

2c
« Last Edit: November 09, 2015, 06:53:10 pm by Thimee »

Offline Thimee

  • Newbie
  • *
  • Posts: 2
  • Karma: +0/-0
    • View Profile
Re: X11SBA-LN4F vs A1SRi-2558F
« Reply #48 on: November 09, 2015, 06:37:51 pm »
My X10SBA-L build use since initial startup, I have a 150MB Comcast connection, about 10-15 concurrently connected devices:



Basically total overkill for my network...



« Last Edit: November 09, 2015, 06:53:58 pm by Thimee »

Offline Engineer

  • Full Member
  • ***
  • Posts: 120
  • Karma: +24/-2
    • View Profile
Re: X11SBA-LN4F vs A1SRi-2558F
« Reply #49 on: November 09, 2015, 07:03:16 pm »
Thimee, did your entire system crash or did your interfaces go down (and console still work)?

My console still works fine...it's just the LAN (not WAN) port that goes down and resets via Watchdog timeout.

Offline rexki

  • Newbie
  • *
  • Posts: 20
  • Karma: +2/-0
  • Independent Consultant
    • View Profile
Re: X11SBA-LN4F vs A1SRi-2558F
« Reply #50 on: November 10, 2015, 11:26:28 am »
Engineer:  Doing some double checking on SuperMicro web site:

1.  Still no BIOS update
2.  Certified 4GB ram is only one item listed: 4GB – Hynix - MEM-DR340L-HL02-SO16 - HMT451S6BFR8A-PB - H5TC4G83BFR

Specs above further detailed:
•   4GB Memory Module
•   DDR3-1600MHz
•   PC3L-12800
•   Non-ECC
•   1.35v Low Voltage LV
•   SODIMM
•   204-pin
•   Cas Latency 11
•   1Rx8 (H5TC4G83BFR  - 512MB chips each)

3.  Certified 8GB stick has the same exact component chip as the 4GB   - H5TC4G83BFR  - 512MB chips each


Your spec from page one listed as follows:
      2 x 4GB Samsung PC1600 DDR3L: $36 shipped from eBay

Found Samsung 4gb pc1600 sodimm items to get comparatives:
1.  Samsung Part number M471B5273DH0-CK0 - 1.50v
2.  Samsung Part number M471B5173DBO-YK0 - 1.35v - CL11 - ?Single sided 1Rx4?
3.  Samsung Part number M471B5173BHO-YK0 - 1.35v - CL11 - ?Single sided 1Rx4?
4.  Samsung MV-3T4G3D/US - 1.35v - CL11 - 2Rx4 Double sided  - 8 each



My questions:
1.  Do these certified specs match exactly to yours especially 1Rx8 with 512 and CAS 11?
2.  Have you run with just one chip of ram at 4GB - number one cycles solo and then number two cycled solo as well to isolate behavior?
3.  Have you swapped memory places of the existing Samsung 2 *4GB sticks?
« Last Edit: November 10, 2015, 01:37:54 pm by rexki »
pfSense 2.3.4-Release(amd64) - 31 watts Min d-power mode - 843-853 mbps across LANs -  i5-2400 3xGigE - Asus P8H61-M -All slotted Intel single NICS EM drivers -  shooting for 6 watts - to save $27/year in electricity.  In Hawaii $50 per year savings over 20 watt delta!!

Offline Engineer

  • Full Member
  • ***
  • Posts: 120
  • Karma: +24/-2
    • View Profile
Re: X11SBA-LN4F vs A1SRi-2558F
« Reply #51 on: November 10, 2015, 11:40:44 am »
I have re-seated the ram but not swapped it or run it in single channel (one stick).  Once I completed the memtest+ test, I had assumed all was good.  I suppose it could be ram but I tend to think that takes the whole ship down including the console (aka - one big crash).  I suppose it might be ram.  The ram is DDR3L 1.35V (I did make sure of that when I purchased it).

I'll see about removing a stick later.  As for the certified memory stuff, I understand the testing and stuff but there's no real reason for most of the good stuff not to run.  Just hasn't been certified....YET.  When I bought the board and ram, there were NO certified sticks listed on the SuperMicro site. ;)

Oh, and for now, I've turned the Asus RT-AC68U back from AP to Router while trying to sort this out.  Busy lately so I'll have to put it on hold a little.  Hard to even do anything as people scream faster about the Internet being down than they do if we were being robbed, lol.
« Last Edit: November 10, 2015, 11:50:32 am by Engineer »

Offline rexki

  • Newbie
  • *
  • Posts: 20
  • Karma: +2/-0
  • Independent Consultant
    • View Profile
Re: X11SBA-LN4F vs A1SRi-2558F
« Reply #52 on: November 10, 2015, 01:35:11 pm »
This is extremely interesting to me - the LAN1 LAN2 LAN3 and LAN4 ports on the back are not the same!!

Manual from SuperMicro  - page 17 block diagram  - shows:

A.   1x intel I210 is a single off the SoC on PCIe(1) and
B.  the other three are 2 stepped off PCIe(2) then to a Pericom 608GP “PI7C9X2G608GP” (PCIe2 6-Port/ 8-Lane Packet Switch, GreenPacketTM Family) – add 150ns latency to stream per spec.

My thoughts would be to move the LAN port if on the three port Pericom switch to PCIe(1) directly off the SoC.  Guessing this is lower left corner LAN1 while looking from back side or page 26 of the manual.
pfSense 2.3.4-Release(amd64) - 31 watts Min d-power mode - 843-853 mbps across LANs -  i5-2400 3xGigE - Asus P8H61-M -All slotted Intel single NICS EM drivers -  shooting for 6 watts - to save $27/year in electricity.  In Hawaii $50 per year savings over 20 watt delta!!

Offline rexki

  • Newbie
  • *
  • Posts: 20
  • Karma: +2/-0
  • Independent Consultant
    • View Profile
Re: X11SBA-LN4F vs A1SRi-2558F
« Reply #53 on: November 10, 2015, 02:02:39 pm »
Going through the BIOS option in the manual some points worth noting to the problemamtic I210:

1.  ASPM "Active State Power Management" - Auto default to disable  (Applicable to the PCI-Express buses - ala LAN1,2,3,4
2.  PCIe Speed from Auto to Gen2 since this is the spec for the I210 on PCIe(1) and (2).
3.  Onboard LAN Option ROM Type - make sure Legacy which is default.
4.  Onboard LAN1 option ROM from PXE - to Disabled - ironically the rest are off by default - my guess because of the Pericom.
5.  Network Stack from Enabled to Disabled - no PXE!
6.  Kill the AST2400 serial port since it will not be used?
7.  last resort play with turning off and on WatchDog in BIOS
pfSense 2.3.4-Release(amd64) - 31 watts Min d-power mode - 843-853 mbps across LANs -  i5-2400 3xGigE - Asus P8H61-M -All slotted Intel single NICS EM drivers -  shooting for 6 watts - to save $27/year in electricity.  In Hawaii $50 per year savings over 20 watt delta!!

Offline Engineer

  • Full Member
  • ***
  • Posts: 120
  • Karma: +24/-2
    • View Profile
Re: X11SBA-LN4F vs A1SRi-2558F
« Reply #54 on: November 10, 2015, 02:10:45 pm »
This is extremely interesting to me - the LAN1 LAN2 LAN3 and LAN4 ports on the back are not the same!!

Manual from SuperMicro  - page 17 block diagram  - shows:

A.   1x intel I210 is a single off the SoC on PCIe(1) and
B.  the other three are 2 stepped off PCIe(2) then to a Pericom 608GP “PI7C9X2G608GP” (PCIe2 6-Port/ 8-Lane Packet Switch, GreenPacketTM Family) – add 150ns latency to stream per spec.

My thoughts would be to move the LAN port if on the three port Pericom switch to PCIe(1) directly off the SoC.  Guessing this is lower left corner LAN1 while looking from back side or page 26 of the manual.

1st - I've never been able to get a manual as it has not been available.  Will check that out.

2nd - Very interesting indeed.  Maybe that's why the WAN port stays up and the LAN ports go down.

Think I'm going to switch the LAN and WAN ports for giggles and see if the LAN port goes down on igb0.

Edit:  Downloaded the manual (first time that it's been available online - thanks for the heads up) and switched the LAN and WAN ports.  If it stays up, then I'm not sure where to go.  It could be a hardware issue (with the PCIe lane switch on ports 2, 3 and 4) or it could be a software issue (not certified on FreeBSD 10 according to SuperMicro documents on page).  Could also be a BIOS issue.  If it has something to do with the three (3) ports and the internal PCIe switch, I would think the WAN port would start dropping and the LAN would stay up.  I'll report back as time goes on.
« Last Edit: November 11, 2015, 07:04:06 am by Engineer »

Offline Engineer

  • Full Member
  • ***
  • Posts: 120
  • Karma: +24/-2
    • View Profile
Re: X11SBA-LN4F vs A1SRi-2558F
« Reply #55 on: November 12, 2015, 09:14:15 pm »
Well, I swapped the ports and the WAN went down tonight.  Watchdog Timeout.

I think rexki might have found something in the PCIe switch on ports 2-4 (igb1-3).  Port 1, not on the PCIe internal motherboard switch, functions normally.  The others drop with timeouts.

Maybe it's a driver or FreeBSD issue, but since this board isn't FreeBSD certified by SuperMicro, I'm not sure they will even take a look at it.

Edit:  I've opened a technical support case with SuperMicro to see if they can come up with something on this.  Would be nice to have someone else with this board and confirm (or deny) if they have the same issues.  If not, could see what they have different (power supply, cooling (or lack of as I have now), etc). I might put a USB powered fan blowing on the board to see if it makes a difference.  The PCIe switch chip is only rated at 1.2W max and has no heatsink or other cooling.

Edit #2:  SuperMicro came back with the suggestion to install a certain version of Ubantu on it and try?!?!  WTF kind of suggestion is that, lol?  I told him that was a no go unless it's just to run some sort of test program.  Starting to wonder if the chip is either defective or getting hot (since no cooling - passive).  I have lots of small video card heatsinks and a roll of very good heatsink tape to use if necessary.  And the quest goes on....
« Last Edit: November 13, 2015, 04:47:58 pm by Engineer »

Offline Engineer

  • Full Member
  • ***
  • Posts: 120
  • Karma: +24/-2
    • View Profile
Re: X11SBA-LN4F vs A1SRi-2558F
« Reply #56 on: November 14, 2015, 07:29:23 pm »
Guys and gals, I'm giving up for now.  Went down again and I don't have the time or energy right now to try to fight this.  Can't get much from SuperMicro and don't know if it's a hardware issue with the board or if it's a FreeBSD issue (since it does it on both pfsense and opnsense).  If anyone else has this board and gets better results, let me know so I can then say it's hardware.

Offline Jailer

  • Sr. Member
  • ****
  • Posts: 394
  • Karma: +54/-2
    • View Profile
    • Bored Guy Blog
Re: X11SBA-LN4F vs A1SRi-2558F
« Reply #57 on: November 14, 2015, 08:12:43 pm »
I'd love to pick this board up and give it a try but I'm holding back based on your experience with it. If something doesn't change in the near future or someone else comes along and has success I'll likely be getting the X10SBA-L for my build.

Offline Engineer

  • Full Member
  • ***
  • Posts: 120
  • Karma: +24/-2
    • View Profile
Re: X11SBA-LN4F vs A1SRi-2558F
« Reply #58 on: November 14, 2015, 09:33:05 pm »
I'd love to pick this board up and give it a try but I'm holding back based on your experience with it. If something doesn't change in the near future or someone else comes along and has success I'll likely be getting the X10SBA-L for my build.

Really sorry Jailer.  I've left the LAN port connected to my network (change IP and turned off DHCP) and am hammering it with ping requests right now.  I don't recall it going down when I was doing a burn in test for 8 days with NO WAN connection (i.e. no data flowing through the system).  Maybe just needs a system driver update in FreeBSD since this is a VERY new system.  I hate to spend the time/money to RMA it only to find out it's the OS/driver and not the hardware.  I do have a five year warranty though so ... :)

Offline jwt

  • Administrator
  • Sr. Member
  • *****
  • Posts: 343
  • Karma: +101/-31
    • View Profile
Re: X11SBA-LN4F vs A1SRi-2558F
« Reply #59 on: November 15, 2015, 02:02:07 am »
The i210 NICs only have 4 rx/tx queues, which is fine for the 4 core SoC (http://ark.intel.com/products/87261/Intel-Pentium-Processor-N3700-2M-Cache-up-to-2_40-GHz), but you'll find that future versions of pfSense have a minimum 4 core requirement (I might make it 8, I've not decided.)

As documented here: http://www.intel.com/content/dam/www/public/us/en/documents/datasheets/i210-ethernet-controller-datasheet.pdf
, there are only 4 tx and 4 rx queues on an i210.

The SoC is significantly slower than a 4 core Rangeley (1.6GHz on the N3700, 2.4Ghz on the C2558), and this will translate into real-world performance differences.  Someone pointed out 6W .vs 15W, and this is why.

Rangeley also has better (i350 .vs i210) NICs.  https://twitter.com/gonzopancho/statuses/643443335114424320

I also don't believe in integrated graphics on a standalone networking device.