pfSense Forum

pfSense English Support => Virtualization installations and techniques => Topic started by: mortenchristensen on December 23, 2014, 02:02:42 pm

Title: Very slow traffic from other VM's through pfSense on XenServer
Post by: mortenchristensen on December 23, 2014, 02:02:42 pm
I have 2 XenServers, one with XenServer 6.2 and one with Xenserver Creedence beta 3.

Both have a pfSense 2.2 RC as router/firewall and a couple of Ubuntu Linux VM's and a windows-VM.

Traffic through both the physical xenserver-box and the virtual pfSense firewall goes at expected speeds.

But traffic from the other VM's on the same xenserver through the pfSense out on wan/internet goes very, very slow.
It goes so bad they cannot update themselve with apt-get.

When I try with iperf from a linux VM through the pfSense's WAN the speed is 3,82 KBits/sec.
The VM's and pfSense are connected with an internal single-server network (as OPT1), and tests to iperf server run on pfSense from a linux VM shows gigabit-speed.

One of the pfSense' has xen-tools installed. The other has not. I cannot se improvements with the tools installed.

One of the XenServers can get several public IP'numbers. On that I now have installed VM's with both an IPCop firewall and a Zentyal firewall.
When one of those new firewall-VMs' is default gateway for the ordinary VM's on the XenServer, their wan/internet-speed is normal.


Anobody with experience on XenServer as hypervisor, that can give me in a direction to experiment in to get traffic from VM's on the same Xenserver through pfSense up at useful performance ?
Title: Re: Very slow traffic from other VM's through pfSense on XenServer
Post by: cmb on December 24, 2014, 01:18:18 am
Try disabling hardware checksum offloading under System>Advanced, Networking. TSO and LRO should also be disabled, though they likely already are since that's the default for those.

Which type of NIC is showing up in the VM? re0, em0, xn0?
Title: Re: Very slow traffic from other VM's through pfSense on XenServer
Post by: mortenchristensen on December 24, 2014, 04:59:48 am
Sorry.

Tried to disable hardware checksum offloading. The other 2 were disabled by default.

Did not improve the problem.

NIC's in the pfSense VM are nx0 to nx3
Title: Re: Very slow traffic from other VM's through pfSense on XenServer
Post by: mortenchristensen on December 27, 2014, 05:18:46 pm
New test with a pfSense 2.1.
Here internet-traffic from other VM's on the same Xenserver is normal.

The problem seems to be new in pfSense 2.2.
Title: Re: Very slow traffic from other VM's through pfSense on XenServer
Post by: cmb on December 27, 2014, 09:55:01 pm
2.1x wouldn't have xn NICs, it's specific to that. Can you force it to e1000 NICs on 2.2 and see?
Title: Re: Very slow traffic from other VM's through pfSense on XenServer
Post by: mortenchristensen on December 28, 2014, 04:26:09 am
> 2.1x wouldn't have xn NICs, it's specific to that. Can you force it to e1000 NICs on 2.2 and see?

On my 2.1.5 the nic's are called re. Can you give me some hints on, where abd how to change the driver ?
Title: Re: Very slow traffic from other VM's through pfSense on XenServer
Post by: phadm on January 14, 2015, 08:52:58 am
Hi,

i have the same problem with RC 2.2 (XenServer 6.2, SP1016, different platforms and nics) . The problem is the offload engine. If you route traffic between virtual hosts, you get tcp retransmissions, only a few sessions survive....

You have to disable the offload function at the VIF at the XenServer.
First identify the uuid of the VIF's:

xe vm-vif-list uuid=VMUUID

And disable the offload settings:

xe vif-param-set uuid=VIFUUID other-config:ethtool-gso="off"
xe vif-param-set uuid=VIFUUID other-config:ethtool-ufo="off"
xe vif-param-set uuid=VIFUUID other-config:ethtool-tso="off"
xe vif-param-set uuid=VIFUUID other-config:ethtool-sg="off"
xe vif-param-set uuid=VIFUUID other-config:ethtool-tx="off"
xe vif-param-set uuid=VIFUUID other-config:ethtool-rx="off"

shutdown / start the VM.

And now the disadvantage, whitout offload engine the TCP throughput falls on GBIT level over the vswitch. With offload I reach over 371 MBps with fetch, download the xencenter.iso from dom0 via http, whitout 98 MBps.

So who has a better solution, bring it on !!

 
Title: Re: Very slow traffic from other VM's through pfSense on XenServer
Post by: Derelict on January 24, 2015, 02:44:09 am
This all worked for me on the test stack I use which is now all 2.2-RELEASE.  I don't really care about performance much in this application, but before I did this it was useless.  Thanks much.
Title: Re: Very slow traffic from other VM's through pfSense on XenServer
Post by: mortenchristensen on January 25, 2015, 01:35:45 pm
Quote
First identify the uuid of the VIF's:
xe vm-vif-list uuid=VMUUID

And disable the offload settings:
xe vif-param-set uuid=VIFUUID other-config:ethtool-gso="off"
xe vif-param-set uuid=VIFUUID other-config:ethtool-ufo="off"
xe vif-param-set uuid=VIFUUID other-config:ethtool-tso="off"
xe vif-param-set uuid=VIFUUID other-config:ethtool-sg="off"
xe vif-param-set uuid=VIFUUID other-config:ethtool-tx="off"
xe vif-param-set uuid=VIFUUID other-config:ethtool-rx="off"

shutdown / start the VM

Used this on both a XenServer 6.5 and a 6.2 later upgraded to 6.5. On both it has given other VM's internet-access again.

Run the xe commands on a Xenserver Private Network, so I hope the speed degrade will only occur on traffic that involves that net.
I think, both the pfSense VM and the other VM's need to be restartet to get useful speed.
Title: Re: Very slow traffic from other VM's through pfSense on XenServer
Post by: apollo13 on January 27, 2015, 06:06:08 am
You have to disable the offload function at the VIF at the XenServer.
First identify the uuid of the VIF's:

Which VIF? Local or WAN or both?

Thanks,
Florian
Title: Re: Very slow traffic from other VM's through pfSense on XenServer
Post by: Derelict on January 27, 2015, 06:18:12 am
I did it on all.
Title: Re: Very slow traffic from other VM's through pfSense on XenServer
Post by: jpenninkhof on January 29, 2015, 06:32:13 am
This helped me too. I only did this for my LAN port.

In my setup it seemed to be sufficient to execute:
xe vif-param-set uuid=VIFUUID other-config:ethtool-tx="off"
xe vif-param-set uuid=VIFUUID other-config:ethtool-rx="off"
Title: Re: Very slow traffic from other VM's through pfSense on XenServer
Post by: apollo13 on January 30, 2015, 02:58:27 am
This helped me too. I only did this for my LAN port.

In my setup it seemed to be sufficient to execute:
xe vif-param-set uuid=VIFUUID other-config:ethtool-tx="off"
xe vif-param-set uuid=VIFUUID other-config:ethtool-rx="off"

I can confirm that the LAN port should be enough. On a related note, did someone install the XenServer Tools in the VM?
Title: Re: Very slow traffic from other VM's through pfSense on XenServer
Post by: corotte on January 31, 2015, 01:45:22 pm
Hi,

updated my XenServer 6.2 to 6.5 a few day ago with my VM pfsense 2.1.5 with no issue

updated pfsense to 2.2 WITH XENTOOLS (xe-guest-utilties 6.0.2_3) and got the same issue !

installed xentool using that method http://blog.feld.me/posts/2014/07/pfsense-on-citrix-xenserver/ (http://blog.feld.me/posts/2014/07/pfsense-on-citrix-xenserver/) (Thanks feld !)

look like issue remain even with Xentools :/

anyone can confirm ?
Title: Re: Very slow traffic from other VM's through pfSense on XenServer
Post by: Derelict on January 31, 2015, 01:49:09 pm
Yes.  It's broken.
Title: Re: Very slow traffic from other VM's through pfSense on XenServer
Post by: corotte on January 31, 2015, 02:07:52 pm
damn !

but a quesiton remain ... was it working well in snapshot ? was it working well with previous version of xentool ?

in this thread
https://forum.pfsense.org/index.php?topic=86827.0 (https://forum.pfsense.org/index.php?topic=86827.0)
it look like to be an issue with xn nic ...
maybe a previous version should work ?
Title: Re: Very slow traffic from other VM's through pfSense on XenServer
Post by: Derelict on January 31, 2015, 02:12:26 pm
No.

Just disable the tx/rx like in the above until FreeBSD and/or Citrix fixes it.
Title: Re: Very slow traffic from other VM's through pfSense on XenServer
Post by: corotte on January 31, 2015, 02:57:30 pm
Ok

didi the above fix and it finally work.

Thanks folks !
Title: Re: Very slow traffic from other VM's through pfSense on XenServer
Post by: dsiminiuk on February 03, 2015, 09:43:55 pm
My Internet speed normally is 20 Mb/s down and 2 Mb/s up.

I deployed pfSense 2.2-RELEASE X64 in XenServer 6.5

Without modification, the pfSense 2.2 would only muster 5 Mb/s down, and 0.06 Mb/s up. Painful.

I applied the changes to the LAN side VIF and the upload speed went back to full 2 Mb/s. The WAN speed did not improve.

I applied the changes to the WAN side VIF and the upload speed went back up to 20 Mb/s.

Eureka!



Title: Re: Very slow traffic from other VM's through pfSense on XenServer
Post by: Andy_ on February 05, 2015, 03:19:05 am
It's just the tx-offload setting that needs to be changed, rx-offload is fixed-on.

I can confirm the problem and fix with Debian Wheezy/Xen 4.1.4 dom0.

ethtool -K ${dev} tx off in vif-bridge online did the trick.

The issue wasn't submitted to freebsd-bugs so far, now it is:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=197344
Title: Re: Very slow traffic from other VM's through pfSense on XenServer
Post by: johnkeates on February 06, 2015, 04:42:05 am
You only need to disable checksum offloading on the hypervisor side of pfSense's interface.

Any interface that does DomU-DomU communication on pfSense's side produces un-checksummed packets which get dropped by PF in BSD.

sudo ethtool -K $interface tx off

where $interface is the VIF on the Xen Dom0 side is enough. Setting TX off on the bridge forces the Dom0 to calculate ALL checksums on ALL packets no matter where the come from or where they are going. This is not a smart idea since it creates a lot of calculations where they might not be needed. So if the pfSense DomU is on vif123.0 you run: sudo ethtool -K vif123.0 tx off

This has been documented in a different thread a couple of weeks ago. This goes for ALL drivers that remove checksumming as part of their 'optimisation'. The problem is that virtual IO drivers often use shared memory for fast communication, and since shmem is not the same as a bad write where packets might get corrupted, virtual IO developers often opt to disable checksumming since the packets wont corrupt anyway. But PF in BSD drops wrongly summed packets and there you have your problem. Disabling offloading forces software-calculated checksums (which is practually the same as 'offloading' to a 'software device' :p) and fixes this.

Solutions for this issue lie NOT with Xen, virtIO, Linux, BSD or pfSense, but with documentation and the users of pfSense.

Options that could be developed:

1. Xen/VirtIO/netfront detection: display a warning about shmem adapters not checksumming and how to act on both the GUI and the Console for pfSense

2. Have an option to make BSD's PF not drop packets with wrong checksums and recalculate them instead, or just not use checksums at all
Title: Re: Very slow traffic from other VM's through pfSense on XenServer
Post by: xlot on February 10, 2015, 04:44:20 pm
Interesting - only appears to apply to virtual interfaces. 

My pfSense VM is running in xen 4.2 (Centos 6.6 dom0) and has no speed issues, but I'm using pci-passthrough to give 2 dedicated hardware NICs (off a dual-port Intel card) to pfSense for LAN/WAN  (so that DMZ/intranet are physically separate too).

Title: Re: Very slow traffic from other VM's through pfSense on XenServer
Post by: johnkeates on February 10, 2015, 06:27:36 pm
Yes, it only has to do with virtIO and not with networking in general. Hell, it's basically a simple checksumming issue but it's only a big thing since 2.2 started supporting VirtIO and after the upgrade it automatically switches over to do that. :p
Title: Re: Very slow traffic from other VM's through pfSense on XenServer
Post by: bananaboy on February 25, 2015, 12:53:22 pm
Thanks johnkeats for putting that up here. It really helped me sort this out.

One thing to note is disabling tx offload using ethtool -K does not persist across guest reboots or live migration because the dom-id and assigned vif changes, while xe vif-param-set other-config:ethtool-tx="off" does.

Is there any downside to using the vif-param-set option, or are the two basically equivalent?
Title: Re: Very slow traffic from other VM's through pfSense on XenServer
Post by: johnkeates on February 28, 2015, 08:52:34 am
Thanks johnkeats for putting that up here. It really helped me sort this out.

One thing to note is disabling tx offload using ethtool -K does not persist across guest reboots or live migration because the dom-id and assigned vif changes, while xe vif-param-set other-config:ethtool-tx="off" does.

Is there any downside to using the vif-param-set option, or are the two basically equivalent?

They are basically equivalent. ethtool is more for Xen using XenLight as a toolstack rather than XenServer (which is XAPI / XCP I believe, using xe instead of xl or xm). So if you want to persist on XenServer, use the xe command. On XenLight and other Xen's, stick the ethtool command in the vif-script of your choice so it changes the offloading settings once the vif gets attached.
Title: Re: Very slow traffic from other VM's through pfSense on XenServer
Post by: bennymundz on April 30, 2015, 03:48:37 pm
You only need to disable checksum offloading on the hypervisor side of pfSense's interface.

Any interface that does DomU-DomU communication on pfSense's side produces un-checksummed packets which get dropped by PF in BSD.

sudo ethtool -K $interface tx off

where $interface is the VIF on the Xen Dom0 side is enough. Setting TX off on the bridge forces the Dom0 to calculate ALL checksums on ALL packets no matter where the come from or where they are going. This is not a smart idea since it creates a lot of calculations where they might not be needed. So if the pfSense DomU is on vif123.0 you run: sudo ethtool -K vif123.0 tx off


Sorry noob question here,

I am using a Xen implementation on a unraid distribution, when you say Dom0 side are you talking about the VIF that is spun up with the PFsense VM ? Like when i ifconfig to list my interfaces I just don't really know how to identify the interface you are referring to.

Sorry for the noob question again
Title: Re: Very slow traffic from other VM's through pfSense on XenServer
Post by: Derelict on April 30, 2015, 04:41:02 pm
It's all here:

https://forum.pfsense.org/index.php?topic=85797.msg475906#msg475906

I recently just rebuilt my test stack and all I did was the tx and rx on every NIC which is still probably more than is necessary but it worked.
Title: Re: Very slow traffic from other VM's through pfSense on XenServer
Post by: johnkeates on May 01, 2015, 01:54:44 pm
You only need to disable checksum offloading on the hypervisor side of pfSense's interface.

Any interface that does DomU-DomU communication on pfSense's side produces un-checksummed packets which get dropped by PF in BSD.

sudo ethtool -K $interface tx off

where $interface is the VIF on the Xen Dom0 side is enough. Setting TX off on the bridge forces the Dom0 to calculate ALL checksums on ALL packets no matter where the come from or where they are going. This is not a smart idea since it creates a lot of calculations where they might not be needed. So if the pfSense DomU is on vif123.0 you run: sudo ethtool -K vif123.0 tx off


Sorry noob question here,

I am using a Xen implementation on a unraid distribution, when you say Dom0 side are you talking about the VIF that is spun up with the PFsense VM ? Like when i ifconfig to list my interfaces I just don't really know how to identify the interface you are referring to.

Sorry for the noob question again

Basically, when Xen starts a VM, the Domain ID gets appended to the VIF name. So if you start pfSense and it gets domain ID 123, the name you will see in ifconfig is something like vif123.0 for the first interface, vif123.1 for the second interface, etc. Sometimes, there are double interfaces, one with -emu on it, so you'd have vif123.0-emu as well.

So, if you are running non-enterprise Xen, you use XL or XM, and you can list your domains, like: sudo xl list. That will show you all domU's, and the ID's. Using ethtool you can then set the interface options.

You can also edit the vif-up scripts, or whatever vif-script is configured for your Xen setup, and have it do the ethtool magic when the interface is setup at domain startup.

It's all here:

https://forum.pfsense.org/index.php?topic=85797.msg475906#msg475906

I recently just rebuilt my test stack and all I did was the tx and rx on every NIC which is still probably more than is necessary but it worked.

Yes, but that usually applies to XE and not XM or XL installations :) Both are important of course, but the people using Xen sometimes don't know what they have :p so we need to know what they are using to give any useful comment :p
Title: Re: Very slow traffic from other VM's through pfSense on XenServer
Post by: bennymundz on May 04, 2015, 04:13:56 am
You only need to disable checksum offloading on the hypervisor side of pfSense's interface.

Any interface that does DomU-DomU communication on pfSense's side produces un-checksummed packets which get dropped by PF in BSD.

sudo ethtool -K $interface tx off

where $interface is the VIF on the Xen Dom0 side is enough. Setting TX off on the bridge forces the Dom0 to calculate ALL checksums on ALL packets no matter where the come from or where they are going. This is not a smart idea since it creates a lot of calculations where they might not be needed. So if the pfSense DomU is on vif123.0 you run: sudo ethtool -K vif123.0 tx off


Thank you for taking the time to explain this, i turned the TX off on the pfsense vif and all was good. Happy days
Title: Re: Very slow traffic from other VM's through pfSense on XenServer
Post by: BBMitch on May 25, 2015, 03:45:09 pm
Hello all...

Thanks for the information - sure helped us solve this but I have some more information that wasn't clear to me from all posted here.

This issue only seems to apply where Pf is communicating with hosts within the same xen host (dom0).

We use xenserver 6.2 fwiw. We have two xen dom0 - pf was natting for two services - one on dom0-a and one on dom0-b

pf itself was located on dom0-b
The dom0-a service worked perfectly after the update to 2.2.2 - the dom0-b service did not.

For people new to xenserver / for completeness, we used:
xe vm-list
#then find the uuid of your pf vm
xe vif-list vm-uuid={uuid of the vm from above}
#note the uuid of the vif - not the network you want to change!
#for each vif you can check the status:
xe vif-param-get uuid={uuid of vif} param-name=other-config
xe vif-param-set uuid={uuid of vif} other-config:ethtool-tx="off"

For what it's worth I was able to turn off tx on only the LAN interface (which nats for the dom0-b service).

I tried but did not need to keep offload off for the WAN interface which seems to get proper checksum as it leaves the dom0 through the physical nic.

Once complete you need to reboot the pf vm. the setting will persist across reboots.

Hope that helps someone else :-)

Mitch
Title: Re: Very slow traffic from other VM's through pfSense on XenServer
Post by: Gr1pen on June 02, 2015, 01:17:33 pm
I've been running pfsense 2.2 on XenServer 6.2 for a while with the mentioned offloads disabled and it's been working great. I believe since I upgraded to XenServer 6.5 (or when I upgraded to 6.5 SP1) pfsense only works as before on one specific host in the pool. I have 3 hosts in the pool and when pfsense is running on 2 of them it is very slow, but on the 3rd host it works fine.

How come..?? ???
Title: Re: Very slow traffic from other VM's through pfSense on XenServer
Post by: BBMitch on June 02, 2015, 01:25:22 pm
Without knowing your network I can only guess... but see if this makes sense.

What I found was that if the pfsense was routing traffic for vm's on other systems (outside the xen box itself) then things worked - the offload worked as expected as the offload is added at the nic as the data leaves the xen server.

When I was routing traffic that was contained by the virtual network on the same xen host, that's when it didn't work - until I disabled the offloads - you only need to disable on the paths which you see the performance issues in my opinion - but you have to think it through.

Cheers.
Title: Re: Very slow traffic from other VM's through pfSense on XenServer
Post by: Derelict on June 02, 2015, 02:34:07 pm
The stack in the diagram in my sig is all on XenServer 6.5.  Works fine as long as the checksumming is turned off.
Title: Re: Very slow traffic from other VM's through pfSense on XenServer
Post by: Gr1pen on June 03, 2015, 07:30:10 am
Well, this issue is when traffic flows from external machines through pfsense wan-interface to resources on the internal lan.

The host on where this works has different hardware (including different NIC's) than the other two hosts in the pool. So when I migrate or restarts pfsense on  host 1 or 2 I don't get through the firewall from the outside (ia its so slow that it dont work). But with pfsense on host 3 it works as expected.

Before it worked on all 3 hosts. Now the pfsense is not protected against host failure.
Title: Re: Very slow traffic from other VM's through pfSense on XenServer
Post by: johnkeates on June 03, 2015, 08:46:35 am
Well, this issue is when traffic flows from external machines through pfsense wan-interface to resources on the internal lan.

The host on where this works has different hardware (including different NIC's) than the other two hosts in the pool. So when I migrate or restarts pfsense on  host 1 or 2 I don't get through the firewall from the outside (ia its so slow that it dont work). But with pfsense on host 3 it works as expected.

Before it worked on all 3 hosts. Now the pfsense is not protected against host failure.

What are the eth specs when it's failing? And is it a live migration or a shutdown-boot migration?
If you want to protect against failure, it's better to use pfSense's failover options instead of hypervisor-based failover.
Title: Re: Very slow traffic from other VM's through pfSense on XenServer
Post by: BBMitch on June 03, 2015, 10:09:44 am
I think he was trying to do that but he perceived one pfsense to work and two others not to work.

I'll try to explain it another way... the interface (if any) which transmits traffic to machines on the same physical xen server needs to have tx check sums turned off as I noted in my post. That's the only interface affected.

If you have a pf on xen and it does not route for any hosts on the same xen box you don't see any problem.

This would affect any traffic to which check sums would be applicable (all I think?) - so it would affect carp traffic too I imagine IF your pf boxes were on the same network - if they are on different boxes the carp traffic will be fine.

Just turn off the tx check sums for all the pfsense interfaces if you don't understand what I mean - the method I described surives rebooting and only affects the pf vms you apply the changes to.

Hope that clarfies. Cheers.
Title: Re: Very slow traffic from other VM's through pfSense on XenServer
Post by: Gr1pen on June 03, 2015, 02:51:33 pm
Perhaps my explanation was not so clear. The offload settings mentioned here has been applied on all interfaces of pf from the start when I was running it on XenServer 6.2. That fixed the problem then and pf worked perfectly fine on all 3 hosts. It was like living in a Dream where the streets where paved with gold and there was free candy for everyone.

After upgrading to XS 6.5/SP1 pf only works on 1 host. It doesnt matter if I live migrate or shut down and restart on Another host. It ONLY works on "host 3".

I am only running 1 instance of pfsense and sure it may be better running 2 or more in a HA  setup, but thats not really the question here. I had a fine working setup. But not anymore. The candy is all gone and the only change is XS that has been upgraded.

In reply to johnkeates I dont know what eht spec I should look into...?
Title: Re: Very slow traffic from other VM's through pfSense on XenServer
Post by: johnkeates on June 03, 2015, 04:36:49 pm
In reply to johnkeates I dont know what eht spec I should look into...?

Use XE to get all the vif specs from the working pf hypervisor and one non-functional hypervisor, as well as ethtool parameters for both.
We're looking for other variables that might mess with the in-memory transport, because that's where VirtIO related issues seem to lie.
If you could post those 4 outputs it'd help us diagnose.
Title: Re: Very slow traffic from other VM's through pfSense on XenServer
Post by: Gr1pen on June 04, 2015, 11:15:32 am
My bad...

I noticed tht the interfaces on 2 failing XenServer hosts was reordered for some reason. Correcting this solved my problem, hence it was not related to pfsense.

I am thankful for your effort to help out and apologize for confusing you!
Title: Re: Very slow traffic from other VM's through pfSense on XenServer
Post by: johnkeates on June 04, 2015, 11:23:07 am
My bad...

I noticed tht the interfaces on 2 failing XenServer hosts was reordered for some reason. Correcting this solved my problem, hence it was not related to pfsense.

I am thankful for your effort to help out and apologize for confusing you!

Glad you got it fixed!
Title: Re: Very slow traffic from other VM's through pfSense on XenServer
Post by: viniciusferrao on July 26, 2016, 11:34:52 am
Just to keep this updated.

This problem still happens on XenServer 7.0 with pfSense 2.3.1.
Title: Re: Very slow traffic from other VM's through pfSense on XenServer
Post by: johnkeates on July 26, 2016, 12:05:10 pm
Just to keep this updated.

This problem still happens on XenServer 7.0 with pfSense 2.3.1.

Yep, until it's fixed in upstream FreeBSD it won't get fixed, ever.
Title: Re: Very slow traffic from other VM's through pfSense on XenServer
Post by: gothicman02 on March 18, 2017, 05:45:07 am
Just to keep this updated.

This problem still happens on XenServer 7.0 with pfSense 2.3.1.

Yep, until it's fixed in upstream FreeBSD it won't get fixed, ever.

Just figured I'd update this thread on these issues.  It looks like freebsd 11 is supporting dom0 support for xen, so hopefully these issues will be fixed.  I'm just getting a virtualized setup going with support ending for 32 bit here soon so I may try 2.4 of PFSense to see how it works out of the box with xen.

Here is a link to the freebsd support, though it will be experimental at this stage:

https://wiki.freebsd.org/Xen
Title: Re: Very slow traffic from other VM's through pfSense on XenServer
Post by: johnkeates on March 18, 2017, 07:33:18 am
Just to keep this updated.

This problem still happens on XenServer 7.0 with pfSense 2.3.1.

Yep, until it's fixed in upstream FreeBSD it won't get fixed, ever.

Just figured I'd update this thread on these issues.  It looks like freebsd 11 is supporting dom0 support for xen, so hopefully these issues will be fixed.  I'm just getting a virtualized setup going with support ending for 32 bit here soon so I may try 2.4 of PFSense to see how it works out of the box with xen.

Here is a link to the freebsd support, though it will be experimental at this stage:

https://wiki.freebsd.org/Xen

I suppose that could actually fix the netback/netfront problems because it will be BSD on the other end too. Interesting.
Title: Re: Very slow traffic from other VM's through pfSense on XenServer
Post by: gothicman02 on March 21, 2017, 08:06:09 pm
Just to keep this updated.

This problem still happens on XenServer 7.0 with pfSense 2.3.1.

Yep, until it's fixed in upstream FreeBSD it won't get fixed, ever.

Just figured I'd update this thread on these issues.  It looks like freebsd 11 is supporting dom0 support for xen, so hopefully these issues will be fixed.  I'm just getting a virtualized setup going with support ending for 32 bit here soon so I may try 2.4 of PFSense to see how it works out of the box with xen.

Here is a link to the freebsd support, though it will be experimental at this stage:

https://wiki.freebsd.org/Xen

I suppose that could actually fix the netback/netfront problems because it will be BSD on the other end too. Interesting.

Yes very.  Although there is still some work to do.  I got the latest 2.4 snapshot running (as of March 18th) with FreeBSD 11.0-p8 under Xenserver 7.1 with all patches, and the issues with checksum offloading still exist.  Disabling it still fixes the issue through only on the rx and tx side, but I do believe there is a slight performance drop like others have said here.  I haven't tested local file transfers yet, but I do notice a slight drop in internet bandwidth.  I'll do more testing when I got time.
Title: Re: Very slow traffic from other VM's through pfSense on XenServer
Post by: Laban on June 19, 2017, 07:19:04 am
So as I understand it, we need an upstream fix from FreeBSD for this to be magically solved once and for all. What about workarounds? Can someone summarize what steps to take so we can add it to the Wiki under Virtualization / Xen?

Out of curiosity, is it the same with other environments, like KVM or ESXI?
Title: Re: Very slow traffic from other VM's through pfSense on XenServer
Post by: johnkeates on June 19, 2017, 07:25:50 am
So as I understand it, we need an upstream fix from FreeBSD for this to be magically solved once and for all. What about workarounds? Can someone summarize what steps to take so we can add it to the Wiki under Virtualization / Xen?

Out of curiosity, is it the same with other environments, like KVM or ESXI?

I already posted all of this here: https://forum.pfsense.org/index.php?topic=88467.0 if people read the top of the forum none of the 'my virtual setup is slow' topics would exist :p