pfSense Gold Subscription

Author Topic: Installation trouble on XEN  (Read 558 times)

0 Members and 1 Guest are viewing this topic.

Offline lloydsmart

  • Newbie
  • *
  • Posts: 8
  • Karma: +0/-0
    • View Profile
Installation trouble on XEN
« on: March 19, 2017, 02:01:57 pm »
Hi Guys, first post here so be kind!

I'm trying to install pfSense as a HVM guest on Xen, but I haven't had any success so far.

I've got the hypervisor installed, with Alpine Linux as my Dom0. No GUI - everything's console based, but I do use a graphics card to give a VGA connection to view the console.

All that's working just fine so far.

The problems start when I try to install pfSense as a HVM. I've tried both the CD ISO version and the usbstick serial version. With the serial version, I can see the system booting to a certain point, then the serial connection just disconnects. With the ISO version, I can't even get the thing to boot.

Here's the Xen configuration I've been using:

Code: [Select]
builder = "hvm"
name = "pfsense"
memory = 2048
maxmem = 2048
vcpus = 2
acpi = 1
apic = 1

vif = [ 'bridge=br0, mac=00:3e:16:02:02:99' ]

disk = [ 'phy:/dev/vg0/pfsense,xvdb,w', 'phy:/dev/sdk,xvda,w' ]
boot = 'c'

sdl = 0
vnc = 0
nographics = 1
serial = 'pty'

on_poweroff = 'destroy'
on_reboot = 'destroy'
on_crash = 'destroy'

By watching the PTY interface with Screen, I can see the system start, load the kernel, and start identifying hardware devices, but it gets no further than that. After a while, the PTY device just disappears, which is weird because the VM continues to run - I just can't access it. Any ideas what's going on?

I've tried sending the serial to TCP, too. When I do this, I can see it booting up from PuTTY on another computer, but it still cuts off in the same place.

Please help!
Many thanks.

Offline johnpoz

  • Hero Member
  • *****
  • Posts: 12675
  • Karma: +1104/-108
  • Not a pfSense employee, they cannot fire me...
    • View Profile
Re: Installation trouble on XEN
« Reply #1 on: March 20, 2017, 05:37:01 am »
And did you read the STICKY on XEN and pfsense?

https://forum.pfsense.org/index.php?topic=88467.0

IMPORTANT: Xen/KVM networking will not work using default hypervisor settings!
- An intelligent man is sometimes forced to be drunk to spend time with his fools.
- If I have helped you and want to help back, https://www.freebsdfoundation.org/donate/
- Please don't PM me for personal help, info you don't want public sure. Link to thread you would like me to look at ok, etc.
1x SG-2440 2.3.3_p1 (work)
1x 2.4.0-BETA Jun 22 03:42:40 VM running on esxi 6.5 (home)

Offline lloydsmart

  • Newbie
  • *
  • Posts: 8
  • Karma: +0/-0
    • View Profile
Re: Installation trouble on XEN
« Reply #2 on: March 20, 2017, 02:26:11 pm »
Thanks for the reply.

I did read the sticky. Unfortunately that's not the issue I'm having.

My problem is not to do with ethernet but rather the serial console cutting off during boot. I haven't even got to the stage of configuring ethernet or pinging interfaces yet. I'm just trying to get through the installer using the serial console via a pty.
« Last Edit: March 20, 2017, 03:35:21 pm by lloydsmart »

Offline lloydsmart

  • Newbie
  • *
  • Posts: 8
  • Karma: +0/-0
    • View Profile
Re: Installation trouble on XEN
« Reply #3 on: March 20, 2017, 04:06:19 pm »
Ok, so today I added
Code: [Select]
xen_platform_pci=0 to my DomU config for pfSense, and got a bit more information. This time, instead of the serial console just cutting off during boot, I got an error message. Here's a screenshot:



It looks to me like a kernel panic. Is that what it is? If so, what could be causing it?
« Last Edit: March 20, 2017, 04:15:10 pm by lloydsmart »

Offline lloydsmart

  • Newbie
  • *
  • Posts: 8
  • Karma: +0/-0
    • View Profile
Re: Installation trouble on XEN
« Reply #4 on: March 25, 2017, 07:04:35 am »
*bump*

Please? Any hints as to what I'm doing wrong here?

Offline cougarmaster

  • Jr. Member
  • **
  • Posts: 87
  • Karma: +0/-0
    • View Profile
Re: Installation trouble on XEN
« Reply #5 on: March 27, 2017, 11:35:22 am »
You should give more hardware details. Also if you use hvm you cannot use console and you need to use vnc to view it.

Offline lloydsmart

  • Newbie
  • *
  • Posts: 8
  • Karma: +0/-0
    • View Profile
Re: Installation trouble on XEN
« Reply #6 on: March 27, 2017, 04:57:54 pm »
Thanks for your reply.

The server is based on the SuperMicro X10SAE motherboard, with an Intel Xeon E3 CPU. 16GB DDR3 ECC RAM @ 1600MHz. It has Vt-x and VT-D if that's what you're wondering.

if you use hvm you cannot use console and you need to use vnc to view it.

That's not actually true. You're right that you can't use Xen's built in "xl console" command, but you don't have to resort to vnc. If your VM produces serial output (as pfSense does), you can have it output that serial interface to a virtual pty port. To do this, just include the line

Code: [Select]
serial = 'pty'
in your Xen HVM config. This will cause the machine's serial ports to be re-directed to files under /dev/pts on the host machine, which you can then view and interact with using your favourite terminal emulator. I use GNU Screen for this purpose, so to see the output from my pfSense HVM, I issue:

Code: [Select]
screen /dev/pts/1 115200
That's actually how I got the screenshot in my second post. The screenshot is of a BSD kernel panic that happened when I ran my pfSense HVM under Xen, and viewed it using Screen via the pty.

Offline cougarmaster

  • Jr. Member
  • **
  • Posts: 87
  • Karma: +0/-0
    • View Profile
Re: Installation trouble on XEN
« Reply #7 on: March 29, 2017, 12:46:56 pm »
This error usually reside with your cpu not having vt-d enabled. The bios has nothing to do with cpu. Make sure your cpu has it. I never used intel for virtualization as I mainly use amd for all my servers.

Why not try using xenserver and see if it has problems first

Offline lloydsmart

  • Newbie
  • *
  • Posts: 8
  • Karma: +0/-0
    • View Profile
Re: Installation trouble on XEN
« Reply #8 on: April 03, 2017, 02:36:34 pm »
Hmm,

The processor is an Intel Xeon E3-1230 v3 Haswell, which supports VT-d according to this page: http://ark.intel.com/products/75054/Intel-Xeon-Processor-E3-1230-v3-8M-Cache-3_30-GHz.

Is there something specific I need to do to enable it?

« Last Edit: April 03, 2017, 02:54:53 pm by lloydsmart »

Offline cougarmaster

  • Jr. Member
  • **
  • Posts: 87
  • Karma: +0/-0
    • View Profile
Re: Installation trouble on XEN
« Reply #9 on: April 04, 2017, 12:30:23 pm »
I don't think its your hardware problem maybe the way you configured your OS. I still suggest using xenserver will make your life way easier. Also you still can do pci pass thru as there are lots of tutorials. I have been running xen for many years and didn't have much problems. I usually used Debian for the base and xen. Most of the cpus I used are AMD 8350 with 16/32GB ram and raid 1. I pci pass thru 1 nic for wan to pfsense. Other vms i use are zentyal and consolidate the windows servers they have into xen. Now I mainly use freenas for all storage and iSCSI and xenserver for virtualization. The xenserver boots of a raid1 with pfsense in it and all other vm runs from iSCSI.

Offline lloydsmart

  • Newbie
  • *
  • Posts: 8
  • Karma: +0/-0
    • View Profile
Re: Installation trouble on XEN
« Reply #10 on: April 15, 2017, 11:42:59 am »
Well.. I've found the problem, and boy do I feel stupid!  :-[

Turns out I hadn't allocated enough RAM to my Dom0. Whenever I launched the pfSense HVM, the Dom0 would run out of RAM and crash xl.

Changed the allocation on the boot line from 256M to 2G, and it works beautifully.  8)