pfSense Gold Subscription

Author Topic: Is there a limit to the number of VMXNET3 NICs I can use in VMware?  (Read 477 times)

0 Members and 1 Guest are viewing this topic.

Offline MilkmanX

  • Newbie
  • *
  • Posts: 2
  • Karma: +0/-0
    • View Profile
Hello, I'm currently attempting to install using the latest version (pfSense-CE-2.3.3-RELEASE-amd64.iso) on a virtual machine in VMware vCloud.

For the VM I"m using the following settings:
     Other Linux 64 bit
     2 GB disk
     512 MB RAM
     5 x VMXNET 3 Network Adapters

I'm doing a quick install with all default settings and everything appears to go fine.  However, following the install, I'm unable to ping any of the interfaces and DHCP, etc. doesn't appear to be working.  The console is up but doesn't show any errors that I can see.  I'm able to assign interfaces and assign IPs from the console without any errors but this doesn't resolve the issue either.

What's interesting is that if I only use 4 VMXNET NICs everything works perfectly.  I can also get it to work with 5 E1000 NICs.  This leads me to believe there's some kind of limit or some other issue I'm running in to?

Thanks!

Offline jimp

  • Administrator
  • Hero Member
  • *****
  • Posts: 20158
  • Karma: +1198/-8
    • View Profile
Re: Is there a limit to the number of VMXNET3 NICs I can use in VMware?
« Reply #1 on: March 20, 2017, 09:59:19 am »
There is no limit I know of BUT ESX does wonky things with the NICs after 4. They get interleaved in the probe order.

So you end up with:

1
5
2
3
4

or

1
5
2
6
3
7
4
8

So you have to be careful to line up the detected NICs, MAC addresses, interface assignments in pfSense and so on.
Need help fast? Commercial Support!

Co-Author of pfSense: The Definitive Guide. - Check the Doc Wiki for FAQs.

Do not PM for help!

Offline MilkmanX

  • Newbie
  • *
  • Posts: 2
  • Karma: +0/-0
    • View Profile
Re: Is there a limit to the number of VMXNET3 NICs I can use in VMware?
« Reply #2 on: March 22, 2017, 01:39:40 pm »
Yes, I have seen it do strange things but I made sure to double check the order and confirmed the mac addresses matched the interfaces in the UI.  As it stands now 5 VMX nics do not work but the identical set up with 5 e1000 nics do.  For now I've just switched to e1000 so I can continue working but I do think there's something strange going on here.

Offline tortue

  • Newbie
  • *
  • Posts: 10
  • Karma: +0/-0
    • View Profile
I'm currently running pfSense 2.3.4 under ESXi 6.5 using 6 vmxnet3 nics. No issues here.

Offline fwcheck

  • Newbie
  • *
  • Posts: 11
  • Karma: +0/-0
    • View Profile
As far as i know, the maximum number of adapters within a virtual esxi-machine is 10.
https://www.vmware.com/pdf/vsphere5/r55/vsphere-55-configuration-maximums.pdf

You can use 10 Adapters with no problems (ESXi 5.5/6.0/6.5).

Be aware that adapters might be reordered as mentioned above. As this is nasty you can either
- add 10 Adapters at start, leave unused adapters disconnected.
- use a different way (e.g. VLAN-Trunking) with one adapter

Offline tortue

  • Newbie
  • *
  • Posts: 10
  • Karma: +0/-0
    • View Profile
- use a different way (e.g. VLAN-Trunking) with one adapter

I was considering of going down that path of VLANing it. I didn't think I changed around my stuff that often but I've had to straighten out the nics 3-4 times now after about 4 months.

Anyone aware of anything in pfSense that cannot be used or is limited in some form when using VLANs vs dedicated nics? Looking specifically for pfSense items, I'm aware of the virtualization layer pieces like threaded interrupt requests, multi queues, etc.

I would hate to switch off of dedicated nics only to find something I may want to use and have to recreate what I have already.