Netgate Store

Author Topic: Change Order of Interfaces on VM  (Read 3008 times)

0 Members and 1 Guest are viewing this topic.

Offline FauxShow

  • Jr. Member
  • **
  • Posts: 32
  • Karma: +2/-0
    • View Profile
Change Order of Interfaces on VM
« on: June 25, 2015, 07:22:30 am »
Hello all, I have a pfSense 2.2.1 VM running on ESXi 5.5 U2 that I'm trying to rebuild because it won't vMotion or svMotion when it's turned on. It currently has 2 E1000 and 4 VMXNET3 adapters because it gave issues when I tried to run all six as paravirtualized.

When I start a new VM, even before I install pfSense, the order of the NICs changes if I add more than 4 VMXNET3 adapters. In Linux I could simply go to /etc/udev/rules.d/70-persistent-net.rules and change it to how it's supposed to be, and all of my FreeBSD searching points me towards a /etc/rc.conf or /etc/rc.local file that doesn't exist in pfSense. I've tested with 2.2.1 and 2.2.2, and I'll revisit today with the new 2.2.3 release.

Is there a way I can manually re-order the interfaces in pfSense?
brb; gotta reboot my pfsense

Offline johnpoz

  • Hero Member
  • *****
  • Posts: 16024
  • Karma: +1529/-221
  • Not a pfSense employee, they cannot fire me...
    • View Profile
Re: Change Order of Interfaces on VM
« Reply #1 on: June 25, 2015, 07:42:55 am »
what does it matter what order they are seen in pfsense?  Are you saying they change on reboot?

I just make sure I know what the macs are in esxi, and then assign them in pfsense on what they are connected to in esxi. 
- An intelligent man is sometimes forced to be drunk to spend time with his fools.
- Please don't PM me for personal help
- if you want to say thanks applaud or https://www.freebsdfoundation.org/donate/
1x SG-2440 2.4.3-RELEASE (work)
1x SG-3100 2.4.3-RELEASE (work)
1x SG-4860 2.4.3-RELEASE-p1 (home)

Offline FauxShow

  • Jr. Member
  • **
  • Posts: 32
  • Karma: +2/-0
    • View Profile
Re: Change Order of Interfaces on VM
« Reply #2 on: June 25, 2015, 07:52:08 am »
I started with 3 and then 4 NICs and they are in the correct order, but when I add a 5th it changes the order of them to something like 1-5-4-2-3 and it re-orders them again when I add a 6th. Changing the port-groups does fix it, but will I need to change it all again if I ever need a 7th? Plus I have VLAN interfaces that all need to be changed to bind to a different interface, so it becomes a slightly confusing and very time consuming process. I'd rather just change how the OS sees them so I can keep the same config.

I haven't done enough testing to see if it's a random order, but if it is that would also create a problem since I'm trying to setup a second firewall and sync the two.
brb; gotta reboot my pfsense

Offline FauxShow

  • Jr. Member
  • **
  • Posts: 32
  • Karma: +2/-0
    • View Profile
Re: Change Order of Interfaces on VM
« Reply #3 on: June 25, 2015, 08:18:00 am »
Also, the order is persistent through reboot, but does change if I add another NIC.
brb; gotta reboot my pfsense

Offline gjaltemba

  • Sr. Member
  • ****
  • Posts: 355
  • Karma: +25/-2
    • View Profile
Re: Change Order of Interfaces on VM
« Reply #4 on: June 25, 2015, 09:51:03 am »
It is a pain. Especially when management nic is not on vmnic0. I just note the mac address of my nics and the vSwitch that they are assigned to before shutdown and configure network as needed.

Article on how nic order is determined
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2091560

Offline johnpoz

  • Hero Member
  • *****
  • Posts: 16024
  • Karma: +1529/-221
  • Not a pfSense employee, they cannot fire me...
    • View Profile
Re: Change Order of Interfaces on VM
« Reply #5 on: June 25, 2015, 10:29:30 am »
Isn't that article how your physical nics are seen by esxi, not how virtual nics would be seen to pfsense.

so your adding new physical nics to your esxi host?

I can add some more virtual nics to my pfsense setup and see if I can duplicate what your seeing..  But I don't recall pfsense seeing anything change when I have added them in the past.. I cam currently using e1000 vs vmx3 because the vmx3 don't report speed correctly nor does the duplex report correctly when using cdp or lldp via the lavdp package.  So was seeing logs on my switch about duplex mismatch.  Went back to the e1000 and speed and duplex is reported correctly.

But I am using esxi 6, so not going to be a apples to apples test.
« Last Edit: June 25, 2015, 10:32:56 am by johnpoz »
- An intelligent man is sometimes forced to be drunk to spend time with his fools.
- Please don't PM me for personal help
- if you want to say thanks applaud or https://www.freebsdfoundation.org/donate/
1x SG-2440 2.4.3-RELEASE (work)
1x SG-3100 2.4.3-RELEASE (work)
1x SG-4860 2.4.3-RELEASE-p1 (home)

Offline gjaltemba

  • Sr. Member
  • ****
  • Posts: 355
  • Karma: +25/-2
    • View Profile
Re: Change Order of Interfaces on VM
« Reply #6 on: June 25, 2015, 11:24:20 am »
Yes. Adding physical has impact. Same on esxi 6. VMs settings remain but the adapter of the network connection may be impacted by the re-order.

Offline FauxShow

  • Jr. Member
  • **
  • Posts: 32
  • Karma: +2/-0
    • View Profile
Re: Change Order of Interfaces on VM
« Reply #7 on: June 25, 2015, 11:26:10 am »
I'm just adding virtual network cards to the VM; the hosts are blades so their hardware is reasonably fixed.
brb; gotta reboot my pfsense

Offline johnpoz

  • Hero Member
  • *****
  • Posts: 16024
  • Karma: +1529/-221
  • Not a pfSense employee, they cannot fire me...
    • View Profile
Re: Change Order of Interfaces on VM
« Reply #8 on: June 25, 2015, 12:45:50 pm »
Well when I get home I will add a bunch of e1000 and vmx3 and see what what happens with the interfaces in pfsense.. Currently mine are numbered 5 through 8 because 1 to 4 where removed, they where the vmx3 interfaces.

I would do it remotely but dont want to maybe take my network down - I use the access remotely ;)
- An intelligent man is sometimes forced to be drunk to spend time with his fools.
- Please don't PM me for personal help
- if you want to say thanks applaud or https://www.freebsdfoundation.org/donate/
1x SG-2440 2.4.3-RELEASE (work)
1x SG-3100 2.4.3-RELEASE (work)
1x SG-4860 2.4.3-RELEASE-p1 (home)

Offline agrant

  • Newbie
  • *
  • Posts: 3
  • Karma: +0/-0
    • View Profile
Re: Change Order of Interfaces on VM
« Reply #9 on: December 17, 2015, 09:33:09 pm »
I'm having this exact same issue. If more then 3 VMX3 interfaces are added in pfsense the WAN and/or other interfaces will not respond to traffic. As soon as the interfaces are removed or you switch to E1000 interfaces the issue does not persist.

There appears to be an open bug in FreeBSD for this issue https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=198406

I hope this helps anyone else that comes searching for information on this problem.

Thanks!

Offline 21hertz

  • Newbie
  • *
  • Posts: 11
  • Karma: +1/-0
  • Network engineer
    • View Profile
Re: Change Order of Interfaces on VM
« Reply #10 on: February 07, 2018, 10:03:24 am »
Still seems to be a problem. Just tried to virtualize a machine and it seems like all the VMX interfaces (8 of them) are randomized...

Will try e1000 instead now.

pfSense user for 6+ years on network with 5k+ active users.