I would like to add:
- Persistent L2ARC, because I have frequent power fails, that require a new start of L2ARC and because of L2ARC I also suspend my system at night instead of powering it off.
Hi @antony-shen, @satadru
I had a very similar experience to yours: stock installation imports existing pool manually ok, but every reboot it disappears. I submitted a bug report in Launchpad, it might be the same issue:
Yes, I can see how just adding vdevs to the pool is a way to go. However, booting from an NVME SSD that can’t be mirrored easily (for example if the mobo only has one M.2 slot) will be a common use-case, and adding additional vdevs risks complete data loss on even if those vdevs are mirrored pairs.
A workaround is to use a systemd unit file to import the pools, but I think the fact that the zpool cache is ignored is a bug and should be treated as such.
Thanks @didrocks for the pointer on Twitter about submitting a bug for this.
Agreed, I have seen very similar improvements in the ARC. I don’t use an L2ARC, but my L1ARC suddenly got fantastic at caching after the upgrade, it was pretty good before, but now it’s amazing.
No change in usage from before either. Pretty standard desktop usage plus a Windows10 VM. Hourly snapshot replication to a backup drive.
Good to see my observation confirmed with a very good graph. I noticed another improvement. I run weekly incremental backups of my lz4 compressed VMs (320 GB). I use a 1 Gbps Ethernet and it runs at 200 Mbps, due to the CPU load of >90% on the receiving Pentium 4 HT. I noticed that the link now achieves a constant 200 Mbps instead of wildly fluctuating between say 40 and 200 Mbps.
My backup configuration seems to be an invitation for maximizing problems:
- from a 2018 AMD Ryzen 3 to a 2003 Intel Pentium 4
- from 64-bits to 32-bits
- from Ubuntu 19.10 (Linux) to FreeBSD-12 (Unix)
- from 3 x SATA-3 (striped) to 2 x IDE + 1 x SATA-1 (striped)
- from 16 GB DDR4 to 1.25 GB DDR
But it just worked out of the box!! And now with zfs 0.8.1 at a constant max speed!
Hello,
On my new laptop, I want to give a try to zfs as rootfs. Installer looks pretty good, no issue at all. But, I really need, at least, to encrypt my home directory, is there a way to do that? Even boot in console to enter the passphrase to start X manually is good for me.
Thanks for you help,
Best regards
Yoann
Until there is native support in the installer (probably post-LTS), I think your best bet is to use ecryptfs directly, on your home directory: Security - eCryptfs.
this is what I currently have on my setup, but for my new one, I really want to use ZFS, there is no solution to have the home directory zfs volume encrypted?
This looks interesting :
I’ll try this !
It works very well! it ask me a password to load the key at the beginning of the boot. Then, I can login to my session with my password.
Yoann
4 Likes
Monday I will have an exiting day, since I should receive my 512GB NVME SSD from Newegg. I will put the ZFS experimental system on it, but I do NOT want to use the WHOLE DISK.
At midnight I expect the system storage to look like:
PLANNED END RESULT
The NVME SSD Silicon Power (sda) of 512GB:
- ext partition + bpool 2.1GB the standard Ubuntu stuff as generated by the install.
- rpool 37.9GB the standard Ubuntu pool, but REDUCED in size.
- freespace 15GB as fallback to restore the 55GB of current ZFS installation, if needed.
- L2ARC for a HDD datapool dpool, say 15GB (or 30GB), no ZIL/LOG since that dataset hardly uses sync writes.
- vpool 442GB as storage for my virtual machines.
The HDDs 500GB Seagate (sdc) and 1TB WD-Black (sdb):
- archives pool, unchanged 584GB at sdb4.
- Ubuntu 19.10 on ext4, 18GB at sdc5 copied from WD-Black to Seagate.
- Swap, 18GB at sdc6, to try hibernation in future.
- dpool, 870GB striped (454 + 416 GB) with one dataset with copies=2.
PREPARATION
Friday, I installed Ubuntu 19.10 minimal install on a 40 GB USB disk (IDE). The rpool ashift=12, so that is OK. The system functions as Virtualbox host and only needs system utilities and Firefox.
I booted the installation from the USB HDD. I prepared the USB system as far as possible; conky with tricks for zfs, zram/lz4; installed hddtemp and lmsensors. That systems will be moved to the NVME SSD with “sudo dd /dev/sdd /dev/sda”. It works I have tested it in Virtualbox.
Saturday is my weekly backup day, so I will incrementally backup all datasets (send/receive). I will create two additional partition image files (gnome disk utility) for my current zfs install and for the ext4 install. If something goes terribly wrong, I can restore my current ZFS systems partition. I reduce the number of snapshots for each dataset to two.
Sunday I will use the new ZFS 0.8 device remove command and remove a 320GB laptop disk from its zfs datapools.
Monday
- destroy ZFS caches and logs, export datapools;
- physically insert the NVME disk;
- physically remove the SATA-SSD (128GB) and laptop HDD (320GB);
- boot from ext4; check drive assignment sda (NVME),sdb (Seagate) and sdc (WD);
- move the installation from USB to NVME: “sudo dd /dev/sdd /dev/sda”
- remove USB; update grub and reboot from NVME;
- create other NVME partitions with gnome-disk-utility;
- create vpool datapool and datasets;
- move the virtual machines to vpool on the NVME;
- delete the old dpool and vpool partions.
Reorganize the HDD partitions as indicated and restore dpool from the backup over my 1 Gbps link.
Begin December I will try dual boot by adding Mate to the existing bpool and rpool. This time I will install Mate on the USB disk and send/receive the content of bpool/BOOT/Ubuntu; rpool/ROOT/Ubuntu and rpool/USERDATA/ to the existing rpool and bpool. Update grub and look what happens 
RESULTS till THURSDAY:
Most of the operation went as planned, however after I moved the system from USB disk to NVME disk, the system did not want to boot from NVME, even not after running update-grub. I think there is a difference in booting from SATA and NVME. So I went to plan B, and installed the system from scratch on the NVME disk.
I introduce two datasets for my virtual machines rpool/VBOX and rpool/VKVM and moved my VMs to those datasets.
I’m happy with the end result! The system is really fast and Host and VMs boot twice as fast In the past the host booted from SATA-SSD and the VMs from L2ARC and 3 striped HDDs.
I measured the disk throughput in the VMs with Crystal Disk Mark and got constantly ~900MB/s in Win 7. In the gnome-disk-utility I did get the first time ~620MB/s and the following measurements where ~1100MB/s.
I can’t yet explain those differences, but I like the disk throughput in my VMs.
I noticed, that by default Ubuntu only added a swap partition of 2 GB, so I added the /dev/sdb swap partition of the ext4 system and did give it a lower priority. so now my total swap is 18GB.
I have added 4 scripts to create a snapshot for all system datasets, to destroy those snapshots and to rollback those snapshots. The 4th script is for changing the canmount property of all system datasets, to allow updating zfs in the ext4 system too. I set everything to canmount=noauto, but I had to change it to “on” for the userdata, apt and dpkg related datasets. Look how that develops in future.
1 Like
Any chance of instructions for installing 19.10 with zfs root on arm64 using some variant of the experimental installer?
Modifications of these steps seem to work to create the root filesystem: https://github.com/zfsonlinux/zfs/wiki/Ubuntu-18.04-Root-on-ZFS
( I have this working with zfs root on the RPI 4B 4Gb.)
But one has to do an initial install and then rsync everything over to the zfs pools, and we then miss the steps for creating the hash-named root and userdata zpools, and whatever needs to be done to enable the zfs userdata folder creation (is that zsys in this context?)
We only work on the desktop support right now. Once this one is stable (or if anyone in the community is interested in starting with beforehand), we’ll probably move it to other platforms.
Right now, the HOW-to you pointed is the only way I know of for this.
The other way would be to tweak a curtin installation (a little bit similar than https://github.com/ubuntu/zsys-install/blob/master/curtin-zfs.yaml, once refreshed) and integrate this into subiquity.
1 Like
After installing ZFS on my NVME drive, I did run into problems with the connection with my backup server based on send/ssh/receive. I used, it since June for approx 1-2 hour/week to create incremental backups and it worked fine. My backup server is a Pentium 4 (32 bits) with 2 IDE HDDs and 2 laptop HDDs on SATA-1 in total 1114 GB according to FreeBSD 12-1. I’m retired and Dutch, so I throw nothing away and I reuse stuff as much as possible.
I lost compatibility, because of the new feature “large-dnode”, that had been enabled and activated by Ubuntu 19.10. The support in FreeBSD for the feature is in doubt! After trying to receive data from Ubuntu, FreeBSD activated “large-dnode” also, but it did not help. In the matrix of supported features, OpenZFS states FreeBSD does NOT support it.
With some help from Richard Laager, I restored compatibility by setting for every own dataset “dnodesize=legacy” on both systems. It took me one very long day to find the solution. I also had to restore all my data to those datasets with this property and destroy the old datasets (500GB). Those actions took another long day. I have now a lot of pain in my mouse-arm 
I know the new OpenZFS setup will improve this compatibility issue, but it will not completely solve it. There still might be a couple of months difference between implementations. So I would propose: “Do not activate features silently, that might hamper send/receive compatibility”.
I have a question too:
When do we hear, what will be implemented in Ubuntu 20.04 and when can we start testing and enjoying the first parts of it.
1 Like
Now I do run Ubuntu/ZFS from a fast NVME drive (3400/2300 MB/s), I noticed some change. In the past my reboot times of Ubuntu Virtual Machine (VM) from memory cache would be say 12 seconds, Boot times were between 25 and 50 seconds dependent on the fill grade of the SSD cache. Using the NVME drive that difference disappeared, I always boot the VM in 12 seconds, maybe the reboot from L1ARC is 1 second faster.
So it has some consequences for desktop use:
- I’ll already start reducing the max size of L1ARC, since caching does not really speed up my NVME drive on my Ryzen 3 2200G. While my HDD stored music, videos and spreadsheets, do not require caching
- Maybe part of the issue is the LZ4 decompression CPU time, maybe we should avoid compression only to speed up a NVME drive.
- Or maybe we should be able to stop caching the data of NVME drives and only cache the drive administration.
Hello there!
Thank you so much for this great initiative in bringing ZFS to desktops!
Please, I’d like to know where I could submit a feature request for ZFS’ “copies” option to be available in Ubiquity? It’s one of the most interesting features for single drive configs, but in order to be really useful, it needs to have a value specified at partition/pool creation time, otherwise it’ll only apply to newly written files after the installation.
Hi @dcominottim and welcome to the community hub,
We’re glad you enjoy ZFS on Ubuntu desktop. Could you elaborate a bit and explain why you think it’d be useful to enable “copies” at installation time on single disk setup?
1 Like
Thank you a lot for the reply, Jean!
In the context of desktop/laptop Ubuntu installs, it seems reasonable to assume that many users will have a single drive dedicated to the OS or even at all (be it a NVMe SSD, SATA SSD, or HDD). Although the full benefits of ZFS’ redundancy features, etc. would require multiple drives to offer maximum data protection, the ‘copies’ feature can be an interesting option for users who don’t have/can’t use multiple drives for redundancy purposes but still want some extra protection against bitrot and similar issues. But to have the full benefit of ‘copies’, it needs to be enabled – if desired – before the OS install, otherwise it’ll only be applied to newly written data after the install (also useful, but less so).
Just to be clear, I do not propose that the default value from ‘copies’ should be changed from ‘1’ to ‘2’ or ‘3’, but just that the installer should let the user specify ‘2’ or ‘3’ if he so desires. After all, one of the major benefits from ZFS and reasons people want to use it is due to extra data protection offered by features such as this.
1 Like
We received several requests to enable different features of ZFS for different use cases like your suggestion or encryption for example. They are all valid but at this point we cannot integrate them all in the installer especially for an LTS.
Although in order to allow everyone to customize their ZFS installation we are are looking for a way to override the default set of pool features and filesystem properties currently in the installer maybe based on a configuration file.
If some features or properties are broadly used, not currently set as default and safe to enable we can still enable them by default or add an option in Ubiquity for next point release of the LTS.
We are still looking into this and will keep you posted when something is ready to test in Focal.
Thanks for the kind and insightful reply, Jean!
Question: if I make a new install using the latest Focal daily image, is it already expected to contain the dataset configs, etc. that will be used in the final version? Naturally, I know something new might happen between now and then, but just wanted to know whether all or most ZFS/disk related stuff (in terms of partitions, datasets, etc.) is expected to remain more or less the same as in the current daily images.
Yes, unless there is a good reason to change it, the layout of the pools and datasets will remain the same as it is today.