pfSense Support Subscription

Author Topic: PC Engines APU2 with 4 ethernet cards  (Read 3087 times)

0 Members and 1 Guest are viewing this topic.

Offline repne

  • Jr. Member
  • **
  • Posts: 33
  • Karma: +1/-0
  • 0xF2 0xAF
    • View Profile
PC Engines APU2 with 4 ethernet cards
« on: August 29, 2016, 06:13:30 am »
Hello.

I'm upgrading my home network with PC Engines APU2C4. It has three NIC's that will be used for specific purposes, but I find myself one card short. I noticed there are two Mini-PCIe expansion slots available, but the board also has internal USB on header J11. So it's either a Mini-PCIe NIC or a USB NIC. Since I'm not very up-to-date with driver support on BSD/pfSense I'm wondering what would be the best solution here. There won't be much traffic on this card so 10/100 should suffice.

Some Mini-PCIe cards I've found (these are all gigabit though):
  • StarTech ST1000SMPEX
  • MEC-LAN-M001
  • Commell MPX-210
  • Commell MPX-574D
  • LogicSupply 8111ME


Personally I'd go for an USB one, but I can't find any barebone USB NIC's for a custom laser-cut APU2 enclosure.

~rep

Offline jahonix

  • Hero Member
  • *****
  • Posts: 2404
  • Karma: +144/-14
  • volunteer since 2006
    • View Profile
Re: PC Engines APU2 with 4 ethernet cards
« Reply #1 on: August 29, 2016, 03:37:35 pm »
I don't know about the mini-PCIe cards at all so I won't comment on those.
However, it is not advisable to use USB NICs since most of the time they don't work well or just don't work at all.

Could you share one physical NIC with two VLANs and a managed switch or is the amount of local traffic between VLANs prohibiting this? But an APU2 board doesn't have the power to route at line-speed anyways.
Chris

The issue with IPv6 jokes is that almost no one understands them and no one is using them yet.

Offline repne

  • Jr. Member
  • **
  • Posts: 33
  • Karma: +1/-0
  • 0xF2 0xAF
    • View Profile
Re: PC Engines APU2 with 4 ethernet cards
« Reply #2 on: August 30, 2016, 02:05:53 pm »
However, it is not advisable to use USB NICs since most of the time they don't work well or just don't work at all.
Aah, thanks. That's the kind of comment I wanted to hear. I found no decent USB NIC hardware anyway, so I'm considering to go with a Mini-PCIe card. I reckon one with an Intel or at least a Realtek 8111 chipset should work fine. There are some threads online where people use these, but no reports whether they work or not. I'll guess I'll see after I order one.

I'm not sure whether a VLAN is a good solution in this case since I don't possess any managed switches. I was planning to use one NIC for WAN and the other two to physically separate my network into segments like LAN and DMZ. I need another NIC for a third isolated low-traffic segment.

rep

Offline jahonix

  • Hero Member
  • *****
  • Posts: 2404
  • Karma: +144/-14
  • volunteer since 2006
    • View Profile
Re: PC Engines APU2 with 4 ethernet cards
« Reply #3 on: August 31, 2016, 08:58:56 am »
Please report back on the mini-PCIe NIC you'll be using, interests me as well.
Chris

The issue with IPv6 jokes is that almost no one understands them and no one is using them yet.

Offline lra

  • Newbie
  • *
  • Posts: 24
  • Karma: +7/-0
    • View Profile
Re: PC Engines APU2 with 4 ethernet cards
« Reply #4 on: August 31, 2016, 10:43:38 am »
Quote from: jahonix
Could you share one physical NIC with two VLANs and a managed switch
I'm not sure whether a VLAN is a good solution in this case since I don't possess any managed switches. I was planning to use one NIC for WAN and the other two to physically separate my network into segments like LAN and DMZ. I need another NIC for a third isolated low-traffic segment.
I second the suggestion from @jahonix and use a VLAN for your 3rd isolated LAN.

You can find reasonably good 5 or 8 port web managed switches for $50 USD or less, probably about what you would pay for a mini-PCIe card, and you know it will work, no BIOS issues, no driver issues, and other potential problems.

Keep it simple to start, only connect the small web managed switch to one of your APU2 interfaces and configure a mix of the untagged APU2 interface and VLAN as untagged going out to your network.  Learning the switch will be the most effort.

If VLAN's are new to you, this is a perfect time to learn about VLAN's.

Offline repne

  • Jr. Member
  • **
  • Posts: 33
  • Karma: +1/-0
  • 0xF2 0xAF
    • View Profile
Re: PC Engines APU2 with 4 ethernet cards
« Reply #5 on: August 31, 2016, 12:06:59 pm »
Hi!

Well I've already ordered a Mini-PCIe NIC from ebay so I guess it's testing time when it arrives.

True, I'm new to VLAN's. Honestly I'm a bit scared to use them because I also plan to run some other things like traffic shaper (QoS), Suricata IPS and maybe even Radius on the APU2. I have no idea how they are going to play along, and I don't want to make the setup more complicated than it already is. How secure are VLAN's on the same physical ethernet port i.e. how easy is it to jump from one subnet to another?


Offline jahonix

  • Hero Member
  • *****
  • Posts: 2404
  • Karma: +144/-14
  • volunteer since 2006
    • View Profile
Re: PC Engines APU2 with 4 ethernet cards
« Reply #6 on: August 31, 2016, 03:21:58 pm »
How secure are VLAN's on the same physical ethernet port i.e. how easy is it to jump from one subnet to another?
There's no difference in pfSense whether it's a physical or virtual NIC. All interfaces are handled identically.

If you follow some basic VLAN rules then tagged traffic along a trunk is as secure.
- Don't mix tagged and untagged traffic on the same interface.
- Do not use VLAN ID1 for anything else but nothing. (It's default in most devices and can hardly be changed - if at all).

That's it basically.
Chris

The issue with IPv6 jokes is that almost no one understands them and no one is using them yet.

Offline repne

  • Jr. Member
  • **
  • Posts: 33
  • Karma: +1/-0
  • 0xF2 0xAF
    • View Profile
Re: PC Engines APU2 with 4 ethernet cards
« Reply #7 on: September 06, 2016, 08:49:57 am »
UPDATE: I've just received the Mini-PCIe NIC I've ordered. It's a cheap one from ebay with a realtek RTL8111E chipset. I've inserted it into the slot mPCIe 1, and the system detects it just fine. The three native interfaces on APU2 appear as igb0, igb1 and igb2 while the realtek is marked as "re0". I'll see, if I can get it running. Currently I'm having trouble accessing the interface from my PC, but it's the same story with another existing NIC on APU2, so I'm pretty sure it's a firewall issue.

EDIT: Yay, fixed it. The mPCIe wired interface now works properly. I now have 4 physical NIC's on APU2C4!
« Last Edit: September 06, 2016, 12:17:50 pm by repne »

Offline Ximulate

  • Newbie
  • *
  • Posts: 17
  • Karma: +0/-0
    • View Profile
Re: PC Engines APU2 with 4 ethernet cards
« Reply #8 on: March 04, 2017, 09:22:05 am »
I'm in the same boat, but would prefer to go the VLAN route

@Ire: Can you recommend $50 or less 5 to 8 port managed switch?

@repne: What enclouse did you use for this?

Offline ManuCH

  • Newbie
  • *
  • Posts: 8
  • Karma: +0/-0
    • View Profile
Re: PC Engines APU2 with 4 ethernet cards
« Reply #9 on: March 06, 2017, 03:50:33 am »
UPDATE: I've just received the Mini-PCIe NIC I've ordered. It's a cheap one from ebay with a realtek RTL8111E chipset. I've inserted it into the slot mPCIe 1, and the system detects it just fine. The three native interfaces on APU2 appear as igb0, igb1 and igb2 while the realtek is marked as "re0". I'll see, if I can get it running. Currently I'm having trouble accessing the interface from my PC, but it's the same story with another existing NIC on APU2, so I'm pretty sure it's a firewall issue.

EDIT: Yay, fixed it. The mPCIe wired interface now works properly. I now have 4 physical NIC's on APU2C4!

I'm curious: how did you solve the problem with the case? The apu2 comes with a 3-ethernet case. Did you drill/cut out an additional hole yourself? Or did you buy a different case?

Offline kapara

  • Hero Member
  • *****
  • Posts: 933
  • Karma: +15/-0
    • View Profile
Re: PC Engines APU2 with 4 ethernet cards
« Reply #10 on: March 06, 2017, 03:02:04 pm »
I would just go with a supermicro solution.  Great celeron and atom based solutions!
Skype ID:  Marinhd

BlueKobold

  • Guest
Re: PC Engines APU2 with 4 ethernet cards
« Reply #11 on: March 23, 2017, 10:18:07 am »
I'm in the same boat, but would prefer to go the VLAN route

@Ire: Can you recommend $50 or less 5 to 8 port managed switch?

@repne: What enclouse did you use for this?

1. Why?
2. Netgear GS105E, GS108E & GS108Tv2 (~$25, ~$35, ~$69)
3. Here is a nice one that comes with additional RJ45 breakout holes.
PC Engines APU1/APU2 case with HDD, WiFi + 2 LAN Ports - Black 35 Euro plus tax.

Here is a reported miniPCIe dual NIC that is working well with pfSense.APU2C4 with 5 LAN Ports

Offline gtj

  • Jr. Member
  • **
  • Posts: 25
  • Karma: +0/-0
    • View Profile
Re: PC Engines APU2 with 4 ethernet cards
« Reply #12 on: November 10, 2017, 08:06:05 am »
Hello everyone.

I got a new APU2C4 and have installed pfSense and running it with no issues.
I know that it might be obvious but I want to utilise all 3 NICs of the board.
In the meantime I only use igb0 as WAN and igb1 as LAN.
I want to use the third one (igb2) as LAN too but not with a different gateway and subnet mask.
In short, I want igb2 to be in the same network with LAN (igb1).
I tried to bridge these 2 interfaces but still can't get the igb2 to provide internet access to the connected devices.

Can anyone please let me know how am I going to make the 2 LAN NICs to act as one?

Any help will be much appreciated

BlueKobold

  • Guest
Re: PC Engines APU2 with 4 ethernet cards
« Reply #13 on: November 10, 2017, 09:37:10 am »
Quote
I got a new APU2C4 and have installed pfSense and running it with no issues.
Which version you have installed?

Quote
I know that it might be obvious but I want to utilise all 3 NICs of the board.
In the meantime I only use igb0 as WAN and igb1 as LAN.
I want to use the third one (igb2) as LAN too but not with a different gateway and subnet mask.
Ok you can do that with ease.

Quote
In short, I want igb2 to be in the same network with LAN (igb1).
I tried to bridge these 2 interfaces but still can't get the igb2 to provide internet access to the connected devices.
Why, bridging is often a dead end road.

Quote
Can anyone please let me know how am I going to make the 2 LAN NICs to act as one?
Configure a LAG (LACP) if you switch is supporting it right now. If not a small Netgear GS108Tv2 will
do the job for ~$70 newish or for ~$40 refurbished on ebay.
« Last Edit: November 10, 2017, 03:42:48 pm by BlueKobold »

Offline gtj

  • Jr. Member
  • **
  • Posts: 25
  • Karma: +0/-0
    • View Profile
Re: PC Engines APU2 with 4 ethernet cards
« Reply #14 on: November 10, 2017, 12:51:36 pm »
Thanks for your time and kind reply.

Eventually I got to set the 2 NICs to act as ONE LAN interface following the guide below.
I'm posting it here for future reference as well as for anyone who wants to achieve the same thing.
I couldn't imagine a simple requirement like this would need such a complex configuration. However it works now.

https://mtu.net/~engstrom/configure-pfsense-bridge-over-multiple-nics-as-lan/

Offline johnkeates

  • Sr. Member
  • ****
  • Posts: 578
  • Karma: +51/-1
    • View Profile
Re: PC Engines APU2 with 4 ethernet cards
« Reply #15 on: November 10, 2017, 02:15:11 pm »
By the way, the mini PCIe cards do suck quite often, mostly because or poor space constrained design. Most of the cards I tried are RT 8111 based, and die within a year, even fir active airflow. In half the cases it was the on-board DC-DC, in the other cases it was the network chip itself. Bit of a mixed bag so far...

Going with the VLAN option would would out best.

BlueKobold

  • Guest
Re: PC Engines APU2 with 4 ethernet cards
« Reply #16 on: November 10, 2017, 04:11:20 pm »
Quote
Most of the cards I tried are RT 8111 based,
Must not be. NISK300LAN Kit
NISK300LAN Kit with universal I/O bracket is specifically designed with NISE300 and NISE 4000/NIFE 4000 models for network connectivity expansions. It provides dual Intel® Gigabit Ethernet ports with latest I210IT controllers, which gives great network connectivity and less power consumption compared to the previous generation Intel® 82574L controllers. The dual LAN ports on NISK300LAN Kit supports WoL, PXE and teaming functions for managing network activities.


Offline jahonix

  • Hero Member
  • *****
  • Posts: 2404
  • Karma: +144/-14
  • volunteer since 2006
    • View Profile
Re: PC Engines APU2 with 4 ethernet cards
« Reply #17 on: November 10, 2017, 08:15:08 pm »
Eventually I got to set the 2 NICs to act as ONE LAN interface ...
You know that bridging two interfaces in software is far from being a switch?
Each 10$ 5-port switch is way better siuted for such a task than sending each packet through the software stack down to the kernel and back up again. A switch usually does that in hardware or at least in a dedicated ASIC or FPGA highly specialized for such a task.
And do not expect wirespeed from your bridge.

Just saying.
Chris

The issue with IPv6 jokes is that almost no one understands them and no one is using them yet.

Offline jahonix

  • Hero Member
  • *****
  • Posts: 2404
  • Karma: +144/-14
  • volunteer since 2006
    • View Profile
Re: PC Engines APU2 with 4 ethernet cards
« Reply #18 on: November 10, 2017, 08:24:12 pm »
The guide you were following misses one point and that is correct filtering.

Usually you filter on each interface individually.
There's an advanced option at System Tunables where you can set pfSense to filter on the bridge instead.
net.link.bridge.pfil_member   Set to 0 to disable filtering on the incoming and outgoing member interfaces. | default   (1)   
net.link.bridge.pfil_bridge      Set to 1 to enable filtering on the bridge interface      | default   (0)

Better than using an interface group and way easier to understand when doing changes in 7 months or so.
Chris

The issue with IPv6 jokes is that almost no one understands them and no one is using them yet.