Netgate SG-1000 microFirewall

Author Topic: Pfsense 2.4 ZFS File System  (Read 10494 times)

0 Members and 1 Guest are viewing this topic.

Offline ghkrauss

  • Full Member
  • ***
  • Posts: 144
  • Karma: +3/-1
    • View Profile
Pfsense 2.4 ZFS File System
« on: October 29, 2016, 01:37:25 pm »
What will be the hardware requirements for utilizing the ZFS file system?

Offline jwt

  • Administrator
  • Sr. Member
  • *****
  • Posts: 370
  • Karma: +104/-34
    • View Profile
Re: Pfsense 2.4 ZFS File System
« Reply #1 on: October 29, 2016, 08:24:56 pm »
Same as any other FreeBSD system running ZFS.

Offline Jailer

  • Sr. Member
  • ****
  • Posts: 413
  • Karma: +54/-2
    • View Profile
    • Bored Guy Blog
Re: Pfsense 2.4 ZFS File System
« Reply #2 on: October 29, 2016, 09:39:43 pm »
Shouldn't be much more resource intensive aside from needing more memory. ZFS loooves memory and will use every bit it can grab.

It can be made to work on low memory systems with some tuning but that's far beyond the scope of my knowledge. Where's Allan Jude when you need him?  :D

Offline davros123

  • Jr. Member
  • **
  • Posts: 62
  • Karma: +3/-0
    • View Profile
Re: Pfsense 2.4 ZFS File System
« Reply #3 on: November 04, 2016, 09:16:13 pm »
Yes, it loves memory but it does not NEED it. It can and does run fine in a Gig or two.

This is a common falsehood and scares alot of people off.

ZFS will utilise available free memory and release memory as other processes need it.

People really need to stop spreading this rumour!

https://docs.oracle.com/cd/E18752_01/html/819-5461/gbgxg.html
Quote
ZFS Hardware and Software Requirements and Recommendations
Ensure that you review the following hardware and software requirements and recommendations before attempting to use the ZFS software:

Use a SPARC or x86 based system that is running at least the Solaris 10 6/06 release or later release.

The minimum amount of disk space required for a storage pool is 64 MB. The minimum disk size is 128 MB.

The minimum amount of memory needed to install a Solaris system is 768 MB. However, for good ZFS performance, use at least one GB or more of memory.

If you create a mirrored disk configuration, use multiple controllers.

Offline kpa

  • Hero Member
  • *****
  • Posts: 1233
  • Karma: +138/-6
    • View Profile
Re: Pfsense 2.4 ZFS File System
« Reply #4 on: November 04, 2016, 09:29:20 pm »
Mostly true, ZFS is not memory intensive or a resource hog any more than your average database can be depending on the setup. There's one thing I'm worried about and that's the high amount of incorrect information on the net concerning the more advanced features of ZFS such as dedup. You can expect an influx of newbies asking why their pfSense locked up hard when they enabled dedup just for the fun of it.

Offline davros123

  • Jr. Member
  • **
  • Posts: 62
  • Karma: +3/-0
    • View Profile
Re: Pfsense 2.4 ZFS File System
« Reply #5 on: November 04, 2016, 10:14:46 pm »
Yep. No need for dedup in any normal system and the memory needs are freaking INSANE for your average 10's of terrabyte storage systems. No person in their right mind would enable this if they did not know what they were doing.

The other things newb's all do is smack in a zil and L2ARC thinking they're cool...um, no...your not and at best, you'll likely slow down your system rather than speed it up.

ZFS is an amazing thing, but stupid is even more amazing.

Offline pfBasic

  • Hero Member
  • *****
  • Posts: 1021
  • Karma: +139/-22
    • View Profile
Re: Pfsense 2.4 ZFS File System
« Reply #6 on: January 24, 2017, 03:49:26 pm »
What are the advantages of ZFS in a pfsense install? My understanding is that ZFS is like an advanced software RAID file system. So wouldn't ZFS either increase storage speed, reliability or both depending on how you set it up?

I could certainly see the advantage of reliability increase in a production system, would you install multiple drives for redundancy?

Where (other than squid caching) would pfsense really benefit from storage speed increases?

I'm not looking for anything specific or in depth, I was just curious as to what some general use-cases for ZFS in pfsense would be and the very basics of how they would be implemented?


I'm new to pfsense, and love it. I'm also interested in setting up a FreeNAS server at some point in the future. I have no IT background at all but came upon both pfsense and freenas separately without knowing they were both based on FreeBSD. So now I'm interested in learning FreeBSD itself so that I can better understand and use both pfsense and FreeNAS in the future.
I've installed a FreeBSD VM and am exploring it as I read through the manual, but haven't made it to ZFS yet. ZFS seems like one of the crown jewels of FreeBSD so I'm interested to learn your thoughts on it.

Offline doktornotor

  • Hero Member
  • *****
  • Posts: 8553
  • Karma: +962/-278
  • Not a pfSense employee, they cannot fire me...
    • View Profile
Re: Pfsense 2.4 ZFS File System
« Reply #7 on: January 24, 2017, 03:54:15 pm »
What are the advantages of ZFS in a pfsense install?

Unlike UFS, it doesn't crash and burn, bricking your boxes over and over again. That's enough of an advantage for me.
Do NOT PM for help!

Offline pfBasic

  • Hero Member
  • *****
  • Posts: 1021
  • Karma: +139/-22
    • View Profile
Re: Pfsense 2.4 ZFS File System
« Reply #8 on: January 25, 2017, 12:34:42 am »
What are the advantages of ZFS in a pfsense install?

Unlike UFS, it doesn't crash and burn, bricking your boxes over and over again. That's enough of an advantage for me.

There is a decent amount of stuff out there on the web recommending against using ZFS in a single file system. Do you or anyone on this forum have any advice or knowledge to share on using ZFS in a single drive system (which I assume most pfsense installs are), and why it's better than UFS?

Offline doktornotor

  • Hero Member
  • *****
  • Posts: 8553
  • Karma: +962/-278
  • Not a pfSense employee, they cannot fire me...
    • View Profile
Re: Pfsense 2.4 ZFS File System
« Reply #9 on: January 25, 2017, 02:17:19 am »
What are the advantages of ZFS in a pfsense install?

Unlike UFS, it doesn't crash and burn, bricking your boxes over and over again. That's enough of an advantage for me.

There is a decent amount of stuff out there on the web recommending against using ZFS in a single file system. Do you or anyone on this forum have any advice or knowledge to share on using ZFS in a single drive system (which I assume most pfsense installs are), and why it's better than UFS?

Are you actually reading?  ::)

https://redmine.pfsense.org/issues/6891
https://redmine.pfsense.org/issues/6340
https://redmine.pfsense.org/issues/5592
https://redmine.pfsense.org/issues/4523

(And yes UFS is so bad I've actually written a howto on using ZFS on 2.2.x)
Do NOT PM for help!

Offline pfBasic

  • Hero Member
  • *****
  • Posts: 1021
  • Karma: +139/-22
    • View Profile
Re: Pfsense 2.4 ZFS File System
« Reply #10 on: January 25, 2017, 02:36:05 am »
Cool, thanks! So you set two copies for single disk. Good to know.

Offline chrcoluk

  • Sr. Member
  • ****
  • Posts: 407
  • Karma: +22/-50
    • View Profile
Re: Pfsense 2.4 ZFS File System
« Reply #11 on: January 25, 2017, 08:11:27 am »
whilst zfs may not be perfect, if you after reliability it's a significant improvement over ufs.

Its not been run ideally if you have just one storage device but I would still prefer it over ufs.
pfSense 2.4
Qotom Q355G4 or Braswell N3150 with Jetway mini pcie 2x intel i350 lan - 4 gig Kingston 1333 C11 DDR3L
 - 60 gig kingston ssdnow ssd - ISP Sky UK

Offline Rifter

  • Newbie
  • *
  • Posts: 16
  • Karma: +2/-0
    • View Profile
Re: Pfsense 2.4 ZFS File System
« Reply #12 on: January 31, 2017, 01:18:06 pm »
when this is released will we be able to upgrade to ZFS from 2.3.x or will it require a reinstall?

Offline kpa

  • Hero Member
  • *****
  • Posts: 1233
  • Karma: +138/-6
    • View Profile
Re: Pfsense 2.4 ZFS File System
« Reply #13 on: January 31, 2017, 01:25:51 pm »
when this is released will we be able to upgrade to ZFS from 2.3.x or will it require a reinstall?

I doubt that you'll be able to avoid a full reinstall, there just aren't any tools that would automate an UFS to ZFS conversion.

Offline pfBasic

  • Hero Member
  • *****
  • Posts: 1021
  • Karma: +139/-22
    • View Profile
Re: Pfsense 2.4 ZFS File System
« Reply #14 on: January 31, 2017, 02:07:04 pm »
Would installing pfsense (full install, not NanoBSD) using ZFS to a pair of mirrored 4GB SLC USB 2.0 thumbdrives be an extremely durable/reliable configuration?

http://www.digikey.com/product-detail/en/apacer-memory-america/APHA004GAG0CG-2T/APHA004GAG0CG-2T-ND/5871573


Certainly not cheap at all, but wouldn't take any SATA slots, would be low power and would in theory give the durability of SLC combined with benefits of ZFS installed on two drives.


EDIT: While the SLC would in theory be the most durable setup, It costs so much money that it wouldn't make sense for most users. Another option that I would think would make WAY more sense would be using USB 3.0 non-SLC flash drives.
A combo of say 2x16GB or even 4x8GB drives would satisfy just about the most paranoid user, be extremely reliable and fast at very low power draw and remain extremely cheap.


Basically I'm wondering if this general type of install will be fully supported in upcoming pfsense versions?
« Last Edit: January 31, 2017, 03:08:03 pm by pfBasic »