pfSense Support Subscription

Author Topic: ESXi 6.5.0 Guest OS errors...  (Read 1701 times)

0 Members and 1 Guest are viewing this topic.

Offline johnpoz

  • Hero Member
  • *****
  • Posts: 14480
  • Karma: +1343/-200
  • Not a pfSense employee, they cannot fire me...
    • View Profile
Re: ESXi 6.5.0 Guest OS errors...
« Reply #30 on: October 11, 2017, 11:51:50 am »
"While that doesn't seem like a lot per month, over the course of a year, that's a nice savings that can go to other things :)"

Are you paying this bill?  Is this your house or a place of business?  Lets call it $50 a month savings.. Or $600 a year... How much does it cost the company when something doesn't come back up and you have people sitting at their desk not able to work?

"Switches (that's about 140w alone),"

What switches are you running that pull 140watts? You running POE?  A cisco 3850-48 doesn't even pull 140W under full load.. This doesn't sound like a home setup...
- An intelligent man is sometimes forced to be drunk to spend time with his fools.
- Please don't PM me for personal help
- if you want to say thanks applaud or https://www.freebsdfoundation.org/donate/
1x SG-2440 2.3.4_p1 (work)
1x SG-4860 2.4.2-RELEASE-p1 (home)

Offline jimp

  • Administrator
  • Hero Member
  • *****
  • Posts: 21408
  • Karma: +1437/-26
    • View Profile
Re: ESXi 6.5.0 Guest OS errors...
« Reply #31 on: October 11, 2017, 01:59:38 pm »
FYI- There was a patch for the vga/vt race crash in FreeBSD-CURRENT so be brought it in, if we end up having to rebuild 2.4.0 it may end up in the release, otherwise it will be in 2.4.1 which will be very close behind (a week or two at most).
« Last Edit: October 11, 2017, 02:04:25 pm by jimp »
Need help fast? Commercial Support!

Co-Author of pfSense: The Definitive Guide. - Check the Doc Wiki for FAQs.

Do not PM for help!

Offline zxvv

  • Newbie
  • *
  • Posts: 18
  • Karma: +0/-0
    • View Profile
Re: ESXi 6.5.0 Guest OS errors...
« Reply #32 on: October 12, 2017, 05:00:36 am »
The kern.vty=sc workaround worked for me.

I rebooted a dozen times and didn't see the panic, which is definitely an improvement.

Thanks so much for the workaround and for incorporating the fix!

Offline KOM

  • Hero Member
  • *****
  • Posts: 5415
  • Karma: +674/-22
    • View Profile
Re: ESXi 6.5.0 Guest OS errors...
« Reply #33 on: October 12, 2017, 08:53:12 am »
Quote
Smoke'm if you got'em?

This is a tangent from the original post, but you can actually get worse performance with additional cores.  The way VMware does its scheduling, it will only give a VM a time slice if there are as many cores free as the max defined, regardless of what the VM is currently using.  If you have a 12-core box and have defined pfSense as using 8 of those cores, then it's only going to get time when 8 cores are free -- even if pfSense is only using 2 for example.  Depending on load from other VMs, this can delay a VMs scheduling of CPU time in a detrimental way.

Offline jimp

  • Administrator
  • Hero Member
  • *****
  • Posts: 21408
  • Karma: +1437/-26
    • View Profile
Re: ESXi 6.5.0 Guest OS errors...
« Reply #34 on: October 12, 2017, 08:57:33 am »
The kern.vty=sc workaround worked for me.

I rebooted a dozen times and didn't see the panic, which is definitely an improvement.

Thanks so much for the workaround and for incorporating the fix!

Great news! That's a much better workaround than letting it crash and reboot.
Need help fast? Commercial Support!

Co-Author of pfSense: The Definitive Guide. - Check the Doc Wiki for FAQs.

Do not PM for help!

Offline jasonsansone

  • Newbie
  • *
  • Posts: 12
  • Karma: +1/-0
    • View Profile
    • Sansone Howell
Re: ESXi 6.5.0 Guest OS errors...
« Reply #35 on: October 12, 2017, 09:14:54 am »
Quote
Smoke'm if you got'em?

This is a tangent from the original post, but you can actually get worse performance with additional cores.  The way VMware does its scheduling, it will only give a VM a time slice if there are as many cores free as the max defined, regardless of what the VM is currently using.  If you have a 12-core box and have defined pfSense as using 8 of those cores, then it's only going to get time when 8 cores are free -- even if pfSense is only using 2 for example.  Depending on load from other VMs, this can delay a VMs scheduling of CPU time in a detrimental way.

Understand, and thank you. I have 16 cores and 48GB RAM for three VM’s. FreeNAS, an Ubuntu web server, and pfSense. The web traffic is minimal and FreeNAS is more static storage. PfSense pulls the greatest load through VPN traffic.  Really, it’s massive overkill for all guests.  Our encryption settings for a site-to-site trunk are maxed out, so I provide the CPU for OpenVPN.
« Last Edit: October 12, 2017, 09:24:21 am by jasonsansone »

Offline C0RR0SIVE

  • Jr. Member
  • **
  • Posts: 70
  • Karma: +8/-0
    • View Profile
Re: ESXi 6.5.0 Guest OS errors...
« Reply #36 on: October 12, 2017, 01:23:06 pm »
jimp, thanks for looking into the crash so much as well as reporting it :)

So, back to the original posting/question...  I guess we will have to wait for FreeBSD to fix the issue then, thankfully the original issue isn't detrimental.

Once again, thanks to all, it's much appreciated. :)

Offline zxvv

  • Newbie
  • *
  • Posts: 18
  • Karma: +0/-0
    • View Profile
Re: ESXi 6.5.0 Guest OS errors...
« Reply #37 on: October 12, 2017, 08:46:58 pm »
Another possible workaround is to set kern.vty=sc in /boot/loader.conf.local since this appears to be a race condition in the VT console, according to the FreeBSD bug report.

I did manage to make one of my VMs crash once, finally.

I opened a bug report for it here: https://redmine.pfsense.org/issues/7925


This bug can also seems to be avoided by using safe mode during install, for those that may be creating a new VM from the ISO.

Interrupt the boot with <space>, then 6 4 1 1.

As you said, to make the workaround persist after the install is complete, invoke a shell and run:

echo set kern.vty=sc >> /boot/loader.conf.local



Offline jimp

  • Administrator
  • Hero Member
  • *****
  • Posts: 21408
  • Karma: +1437/-26
    • View Profile
Re: ESXi 6.5.0 Guest OS errors...
« Reply #38 on: October 13, 2017, 07:17:02 am »
This bug can also seems to be avoided by using safe mode during install, for those that may be creating a new VM from the ISO.

Interrupt the boot with <space>, then 6 4 1 1.

As you said, to make the workaround persist after the install is complete, invoke a shell and run:

echo set kern.vty=sc >> /boot/loader.conf.local


If you are going to that trouble, drop to a loader prompt and run:

Code: [Select]
set kern.vty=sc
boot

Also you don't need "set" in /boot/loader.conf.local, just "kern.vty=sc"
Need help fast? Commercial Support!

Co-Author of pfSense: The Definitive Guide. - Check the Doc Wiki for FAQs.

Do not PM for help!

Offline El Scorcho

  • Newbie
  • *
  • Posts: 7
  • Karma: +0/-0
    • View Profile
Re: ESXi 6.5.0 Guest OS errors...
« Reply #39 on: November 20, 2017, 08:49:27 pm »
I don't think that warning really matters.  VMWare is simply not properly identifying the Guest OS but you already manually selected FreeBSD, so the drivers and hardware emulation is accurate.  The same occurs when I update to brand new releases of macOS or Ubuntu builds.

I arrived here via Google looking for the answer to this problem and wanted to comment on this because it does matter.

In ESXi you can't do snapshots (or backups via tools like Veeam) unless ESXi thinks the host is in a consistent state. As long as this message appears, ESXi thinks the host is inconsistent, so no snapshots/backups.

Offline gjaltemba

  • Sr. Member
  • ****
  • Posts: 333
  • Karma: +24/-2
    • View Profile
Re: ESXi 6.5.0 Guest OS errors...
« Reply #40 on: November 21, 2017, 09:12:58 am »

I arrived here via Google looking for the answer to this problem and wanted to comment on this because it does matter.

In ESXi you can't do snapshots (or backups via tools like Veeam) unless ESXi thinks the host is in a consistent state. As long as this message appears, ESXi thinks the host is inconsistent, so no snapshots/backups.

I just just took a snapshot of pfSense 2.4.1 CE vm in ESXI 6.5. Whatcha talking about?

Offline El Scorcho

  • Newbie
  • *
  • Posts: 7
  • Karma: +0/-0
    • View Profile
Re: ESXi 6.5.0 Guest OS errors...
« Reply #41 on: November 22, 2017, 08:48:08 am »

I arrived here via Google looking for the answer to this problem and wanted to comment on this because it does matter.

In ESXi you can't do snapshots (or backups via tools like Veeam) unless ESXi thinks the host is in a consistent state. As long as this message appears, ESXi thinks the host is inconsistent, so no snapshots/backups.

I just just took a snapshot of pfSense 2.4.1 CE vm in ESXI 6.5. Whatcha talking about?

I'm running ESXi 6.5.0 Update 1 (Build 6765664). I'm running pfSense 2.4.1-RELEASE (amd64).

ESXi reports "The configured guest OS (FreeBSD (64-bit)) for this virtual machine does not match the guest that is currently running (FreeBSD 11.1-RELEASE-p2). You should specify the correct guest OS to allow for guest-specific optimizations." for my pfSense VM.

If I attempt to take a snapshot of the pfSense VM ESXi errors out with "Failed - The operation is not allowed in the current state."

That's what I'm talking about.

Offline gjaltemba

  • Sr. Member
  • ****
  • Posts: 333
  • Karma: +24/-2
    • View Profile
Re: ESXi 6.5.0 Guest OS errors...
« Reply #42 on: November 22, 2017, 09:33:34 am »

I'm running ESXi 6.5.0 Update 1 (Build 6765664). I'm running pfSense 2.4.1-RELEASE (amd64).

ESXi reports "The configured guest OS (FreeBSD (64-bit)) for this virtual machine does not match the guest that is currently running (FreeBSD 11.1-RELEASE-p2). You should specify the correct guest OS to allow for guest-specific optimizations." for my pfSense VM.

If I attempt to take a snapshot of the pfSense VM ESXi errors out with "Failed - The operation is not allowed in the current state."

That's what I'm talking about.

How do you conclude that one has anything to do with the other? Check vmware.log for details of snapshot request.