Netgate SG-1000 microFirewall

Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.


Messages - rothbard

Pages: [1]
1
OpenVPN / Re: PFSense and PIA - Slow download speed
« on: September 14, 2017, 11:54:27 pm »
Ok, increasing the buffers only delayed the dreaded buffer error, but it will pop up after having max throughput for a certain amount of time. I've finally got rid of it by starting from scratch and is now fully working without any errors.
I've got a feeling that my routes got messed up, even though they showed up fine in Diagnostics -> Routes (even with pia connected). I had a complex setup with lots of nat, tags, vlans, etc. where this test machine was behind a double nat and triple router, but that is not of relevance to the issue that has now been resolved. Another possibility might be Snort. A good reminder for me to isolate machines to the scope they're intended to be used, instead of installing all kinds of packages and last, but definitely not least, a very good reminder why pfSense rocks! :)

For anybody who has the same issues:
- Low throughput => Increase send and receive buffers with openvpn arguments/options
- Udp buffer write errors => messed up routes c.q. snort c.q. other packages => last resort, make backup of configs and start from scratch

That's it, all is good!  :D

2
OpenVPN / Re: PFSense and PIA - Slow download speed
« on: September 14, 2017, 11:37:40 am »
I've changed buffers to 1572864 (the limit seems to be 3700KB, because more openvpn will force to default values). No more buffer errors and very smooth (pfsense running as hyper-v guest with pia vpn as client and totally locked down in case gateway goes down (i.e. killswitch) + another hyper-v guest going through pfsense maxing out throughput isp) :) Also wrote a ps script which forwards pia forwarding port in my program. All is done [for now...  ;)]

EDIT:
I forgot to mention that I also changed net.inet.tcp.recvspace & net.inet.tcp.sendspace (under System -> Advanced -> System Tunables) to max 2048K (=2097152 bytes) as defined by Intel, which will also increase throughput for your entire LAN as well.
So... openvpn uses 75% of 2048K available buffer, so 512K is available at worst case scenario for the rest. Everything is working consistently without anymore errors  :)

3
OpenVPN / Re: PFSense and PIA - Slow download speed
« on: September 14, 2017, 11:03:23 am »
Alright, after doing some more tests and checking the logs, I've noticed a lot of "write UDPv4: No buffer space available (code=55)" entries in openvpn log and sometimes a large filetransfer abruptly stops. I've Google a lot and there doesn't seem to be a definitive solution, let alone the cause to resolve it in the first place (many possible scenario's and theories, yes, but not definitive answer). Instead of wasting more hours on research, I've decided to test it by trial and error and increased the buffer sizes with factor 2 to a total of 1MB for both receive and send socket buffers. With that being set, the buffer error does barely come up anymore and I've got a very stable connection without drops, even at max speed file transfers.

@mauroman33 I've rechecked your settings again and I've notcied that you didn't enable "Don't pull routes". I can advice you to enable that, otherwise you'd run in to issues later on. You can verify it by checking your logfile for entries:
NOTE: unable to redirect default gateway -- Cannot read current default gateway from system 
ROUTE: default_gateway=UNDEF 
Could not retrieve default gateway from route socket:: No such process (errno=3)


When you don't pull routes, you won't have those issues anymore and less write udp buffer errors.

Greetings!

4
OpenVPN / Re: PFSense and PIA - Slow download speed
« on: September 13, 2017, 07:27:12 pm »
Ok, after reading the doc for openvpn, openssl and especially pfsense, I assume AES-NI instruction set for is automatically used by PIA OpenVPN, regardless if you have it enabled or not at the openvpn client?

The following quote from pfsense wiki seems at the least not clear (src @ https://doc.pfsense.org/index.php/Are_cryptographic_accelerators_supported#Practical_Use):
To take advantage of acceleration in OpenVPN, choose a supported cipher such as aes-128-cbc on each end of a given tunnel, then select BSD Cryptodev Engine for Hardware Crypto.
...
Nothing needs selected for OpenVPN to utilize AES-NI. The OpenSSL engine has its own code for handling AES-NI that works well without using the BSD Cryptodev Engine.


Ok, as I've assumed (it either doesn't work for pia or it's enabled by default) before I've read it, because from observations during pia speedtesting, there was no difference in cpu load per same throughput.
It would be nice to make that clear in the openvpn client config page  8), because it's very confusing when it's enabled by default in stealth when intuition tells one you have to enable it, because you have the option to, especially when the aes-256-cbc is listed (by enabling aes-ni for intel in the hardware acceleration section of System -> Advanced -> Miscellaneous).

So then... the load is still the same whether I've enabled hw accel in miscellaneous or not... rebooted and did 3x test for each mode and the results made zero differences...
I'm not in the ability to test it with vt-d off, since the host needs continuity. Maybe other people can chime in and confirm or make my argument rebuttable as to why and how. Thank you.

As for the moment, I don't know if it's enabled by default or not and the options of hardware acceleration are just obsolete, not only for aes-ni for openvpn client, but also for miscellaneous.

5
OpenVPN / Re: PFSense and PIA - Slow download speed
« on: September 13, 2017, 05:35:42 pm »
I've had some more time to test hardware acceleration and it's not working for PIA, but is for pfSense.

In console, openssl speed -evp -aes-256-cbc:
- aes-ni hw acceleration: 0.03-0.41 seconds
- without hw accel: 2.9 - 3 seconds
Pia VPN with hw accel bsd cryptodev ... aes-256-cbc checked OR unchecked, SAME results using top in shell (and yes, I've got the 4k cert from pia):
-140mbps dl @ 35% cpu
-14mbps ul @ 15% cpu

Checking the log for pia vpn:
Data Channel Encrypt: Cipher 'AES-256-CBC' initialized with 256 bit key
WARNING: 'cipher' is used inconsistently, local='cipher AES-256-CBC', remote='cipher BF-CBC'

So, I know hardware acceleration works in pfSense, but not for PIA. This lets me conclude PIA accepts it, but overrides it with the blowfish encryption...
I'd love to see some factual data to confirm or deny my thought. The internet is full of conjecture (people enabling it, but it doesn't do anything), but no proof if it works or not as what they claim or do... Enlighten me please, thanks!

EDIT:
Ok, I tested further and checked logs with verb 4 instead of 3 and see cipher is used for data channel encrypt and decrypt, which is very good. The remaining problem is, how to get hardware acceleration working for the encrypted connection for pia... will test further, but until now, there are no differences between on and off, while it does for the host machine.

6
OpenVPN / Re: PFSense and PIA - Slow download speed
« on: September 13, 2017, 04:01:48 pm »
Here my configuration, my custom options and the speed that I reach

explicit-exit-notify 2;
ifconfig-nowarn;
tls-client;
persist-key;
persist-tun;
persist-remote-ip;
remote-cert-tls server;
auth-nocache;
keysize 256;
tls-cipher TLS-DHE-RSA-WITH-AES-256-CBC-SHA;
fast-io;
sndbuf 524288;
rcvbuf 524288

Thank you so much mauroman33! I've been pulling my hair out for the last 20 hours why pia vpn as client in pfsense always capped at 40mbps, while only utilizing 7% cpu load (openvpn process) on 2 cores @ 3.5GHz each.

I thought my pfsense box was not configured right, since a vm guest getting an ip assigned from pfsense was pulling 145mbps with the pia vpn client running on it.

I then proceeded to start from scratch by installing pfsense and directly setup pia in pfsense, tested it again with speedtest.net and again 40mbps.
Then I've googled a lot today and found many useless posts and outright wrong advice and even more posts where it was never resolved and the threads died. Then I found this thread, quickly scanned over it and found your post.
I've tried the above custom options for my pia client and it actually works! From 40mpbs to 142mbps (and only 20% load on openvpn).

How do you know about those openvpn options? Did you read their entire man? Very nice, thank you so much again :-)

I'm still left with 2 pfsense issues, which many other people also experience [and which also unresolved]
- Very slow webgui after the first hour, which will only get worse and never get better (I've tried Chrome and it's totally useless, since pages will never load; I'm on Edge now and it works as of now)
>> I did found a working solution for this though by trial and error and I haven't read it anywhere else. I've put the var on ramdisk with 256mb and browsing the webgui is now instant [for the time being]. This makes me believe the requests from chrome (or webbrowsers in general, if this applies to Edge as well, we will see, it's working now) are cached in pfsense (rather than the browser, since deleting the browser cache didn't help at all);
- cpu resource usage in pfsense != cpu resource usage in vm host (hyper-v). I.e., 100% load in pfsense (watched on console via shell with cmd 'tops') is only 7% tops in hyper-v. I've allocated 2/6 cores to pfsense with 100% reserve and limit, so you'd expect a a max 33% on the host, but nope. Vt-d and virtualizations are ofourse enabled in bios.
>> I've also spent a lot of time digging through posts, which people also never resolved and I'm too lazy to make a new thread for it. It's no priority [yet] and always have a lack of time at my hands, so be it for now.

Greetings!

I've narrowed it down to those two options (the others did top out max 60mbps dl, the two options below maxed out my throughput @ 145mbps dl):
sndbuf 524288;
rcvbuf 524288;

I've also put in mtu option, since I've noticed in the logs "WARNING: 'link-mtu' is used inconsistently, local='link-mtu 1558', remote='link-mtu 1542'", where client mtu > server mtu => fragmentation, which is bad. Quote from OpenVPN man: "OpenVPN requires that packets on the control or data channels be sent unfragmented." I've set it to 1540, which is good. Setting it to 1542 will make it drop to 1470, so keep it at 1540. As a precaution, I've also set fragment and mssfix option. Increasing the buffers really helped the throughput, which makes sense. If I were the vpn server, I'd do the same. I've also removed explicit-exit-notify, because I like the timeout over faster socket closure. If the gateway is really down, the openvpn daemon will automatically stop anyway and watchdog will restart it for me automatically with e-mail notification (which in the end, decreases the amount of e-mails I'll receive and know it's not a false alarm vs a [more] serious blackout).

My working config now (max speed and no dropouts so far):
tls-client;
tls-cipher TLS-DHE-RSA-WITH-AES-256-CBC-SHA;
remote-cert-tls server;
persist-key;
persist-tun;
persist-remote-ip;
keysize 256;
reneg-sec 0;
link-mtu 1540;
fragment 0;
mssfix 0;
fast-io;
sndbuf 524288;
rcvbuf 524288;

All is working perfectly, very happy  :)

7
OpenVPN / Re: PFSense and PIA - Slow download speed
« on: September 13, 2017, 03:08:04 pm »
Here my configuration, my custom options and the speed that I reach

explicit-exit-notify 2;
ifconfig-nowarn;
tls-client;
persist-key;
persist-tun;
persist-remote-ip;
remote-cert-tls server;
auth-nocache;
keysize 256;
tls-cipher TLS-DHE-RSA-WITH-AES-256-CBC-SHA;
fast-io;
sndbuf 524288;
rcvbuf 524288

Thank you so much mauroman33! I've been pulling my hair out for the last 20 hours why pia vpn as client in pfsense always capped at 40mbps, while only utilizing 7% cpu load (openvpn process) on 2 cores @ 3.5GHz each.

I thought my pfsense box was not configured right, since a vm guest getting an ip assigned from pfsense was pulling 145mbps with the pia vpn client running on it.

I then proceeded to start from scratch by installing pfsense and directly setup pia in pfsense, tested it again with speedtest.net and again 40mbps.
Then I've googled a lot today and found many useless posts and outright wrong advice and even more posts where it was never resolved and the threads died. Then I found this thread, quickly scanned over it and found your post.
I've tried the above custom options for my pia client and it actually works! From 40mpbs to 142mbps (and only 20% load on openvpn).

How do you know about those openvpn options? Did you read their entire man? Very nice, thank you so much again :-)

I'm still left with 2 pfsense issues, which many other people also experience [and which also unresolved]
- Very slow webgui after the first hour, which will only get worse and never get better (I've tried Chrome and it's totally useless, since pages will never load; I'm on Edge now and it works as of now)
>> I did found a working solution for this though by trial and error and I haven't read it anywhere else. I've put the var on ramdisk with 256mb and browsing the webgui is now instant [for the time being]. This makes me believe the requests from chrome (or webbrowsers in general, if this applies to Edge as well, we will see, it's working now) are cached in pfsense (rather than the browser, since deleting the browser cache didn't help at all);
- cpu resource usage in pfsense != cpu resource usage in vm host (hyper-v). I.e., 100% load in pfsense (watched on console via shell with cmd 'tops') is only 7% tops in hyper-v. I've allocated 2/6 cores to pfsense with 100% reserve and limit, so you'd expect a a max 33% on the host, but nope. Vt-d and virtualizations are ofourse enabled in bios.
>> I've also spent a lot of time digging through posts, which people also never resolved and I'm too lazy to make a new thread for it. It's no priority [yet] and always have a lack of time at my hands, so be it for now.

Greetings!

Pages: [1]