<sekrit>Boot CD, fire up web browser, go to /installer/ on the box, and use the web installer which can already do zfs for you</sekrit>
One re-install coming up...
I guess there's a reason it's sekrit: It doesn't work!
Seriously, there's php warnings in the 2.1 version and although both 2.1 and 2.0.1 create ZFS root systems, both of them get stuck on "mountroot>". I did a fair amount of tinkering, including noting that nether creates a /tmp, but nothing could persuade those installs to find their roots.
If anyone wants to tinker: When the zfs install is finished, you'll have to "zfs set mountpoint=/mnt tank0" and then tinker away on the new ZFS root under "/mnt" and then "zfs set mountpoint=legacy tank0" before you reboot.
I suspect I may be running afoul of the zfs cache somewhere, but that's just a guess.
One other thing to watch out for is that apparently 8.3 doesn't know how to write crash dumps to a zfs swap volume, so you'll have to stick with a swap partition for that.
Color me disappointed.
One more point: ZFS (when it's available) is wonderful in a fast paced development/test environment because the Boot Environment (BE, beadm) is basically bootable root filesystem thinly provisioned snapshots with the only restriction being available space, and they're essentially instantaneous to take!
I can't help wondering that given the essential BSD-ness of Solaris if it wouldn't be easier to implement pfSense on OpenIndiana? (http://openindiana.org/
) Sigh. ZFS: it's crack for system and network administrators. Darn you big-O!