Enhancing our ZFS support on Ubuntu 19.10 - an introduction

Project Discussion Desktop
1

This is the related thread to our introductionary blog post on our ZFS support on ubuntu 19.10

22 Likes
2

Nice work man! Question, how does full disk encryption on install work, will that be supported and similar to how it is now on ext4? Thanks!

1 Like
3

Hey Jorge :wink:

We won’t support it right away (meaning, not in 19.10), but easy full disk encryption is on our radar for the next iterations. Our goal is to make it as easy to enable as it is for ext4.

Right now, people can opt in thanks to ZFS 0.8, but they will be on their own in term of integration until we get there.

3 Likes
4

I’m new here, but I am a little surprised by the lack of excitement. I would have guessed this thread would be filled with comments by now. Count me in as excited. I appreciate all the work didrocks and others have done so far.

13 Likes
5

Hi there, any plans to backport this important patch?

1 Like
6

Hi @didrocks do you know if there are any plans to address the “higher level”/user facing parts like being able to show the disk usage in the GNOME system monitor’s File Systems pane (see https://gitlab.gnome.org/GNOME/libgtop/issues/45)?

7

Hi didrocks :grinning:

So would you say there will be full support by 20.04 release or 20.10 release?

1 Like
8

When will support be added the to the ubiquity installer?

1 Like
9

Just out of curiosity, and wanting to help, but when this is added to ubiquity (if it isn’t already there), what kind of bugs are you all looking for? any and all? Just wondering if there’s something I should focus on.

10

We plan to add support to the Ubiquity installer for 19.10. Although the deadline is very very tight because the feature freeze of 19.10 is next week (Aug 22nd)

1 Like
11

Any bug will be helpful. You can report them in launchpad against the project zsys or from the command line with the command:

$ ubuntu-bug zsys

12

There is no specific plan to add support for zfs to libgtop at the moment.

13

We won’t cherrypick and backport performance fixes like the one you mention to 19.10. So unless 0.8.2 is released on time for Eoan this fix won’t be in 19.10.

14

Will the experimental installer option show up before the end of this Aug.?

15

Unlikely. There is a MIR in progress for zsys that requires some fixes and still work to do in Ubiquity. Then a feature freeze exception to fill since we missed feature freeze that happened today.

We’ll post here when it’s ready in the installer.

16

I’m excited. I use ZFS since 18.04 and I boot from ZFS since March or so. I boot and use ZFS on my 16GB Ryzen 3 2200G, my 8GB Phenom II X4 B97, my laptop 8GB i5-2520M and my Pentium 4 with a whopping 1280MB of DDR and 3 IDE HDDs :slight_smile: However it is 32 bits so I had to use FreeBSD-12. I use incremental send/receive from Ryzen to Pentium, from 2018 to 2003 and from Ubuntu to FreeBSD and it works at 200Mbps and ~90% P4 CPU load !! My compliments for OpenZFS !!

Currently I dual boot from a SSD (Ryzen) or SSHD (laptop) datapool with a dataset per OS

  1. Can I upgrade those installations, most are currently at 19.04.
  2. My laptop runs at 18.04.3, could I wait for 18.04.4 with the Linux 5.2 and the corresponding ZFS 0.8.1? Would it still boot?
  3. Another OS still runs Linux 4.15, because the upgrade to 4.18 failed on ZFS. I will have to re-install that system. Can I use the same datapool and dataset.
  4. For dual booting I use 40-common in /etc/grub.d do I still need that?
  5. Basically I snapshot my OS once per month, if needed I re-run the updates for that month. What would change? Do I get a snapshot for each 1 MB upgrade?
1 Like
17

Some additional remarks and/or questions:

I use partitions for the datapool for 2 reasons:

  • I want the Host Systems and Virtual Machine (VM) datapool at the begin of my striped HDDs.
  • I did split my SSD in two parts 50 GB as boot datapool and 50 GB for L2ARC and LOG partitions.
  1. Can I still use partitions for datapools, cache and log devices?
  2. Up to now I avoided using UEFI and can I keep it that way?

I installed my Ubuntu 19.04 on ext4 and move that OS to ZFS afterwards, chroot into it and run the necessary commands like update-grub etc. I have no separate datasets for log, var etc, they are just folders, like in ext4. I have ONE dataset per OS and that includes home too. I have all my user data in a separate datapool and datasets and they are shared between Host and Virtual Machines, because I moved all my work to a number of specialized VMs.
8. Does that cause a problem during future upgrades with respect to log, var, home etc?

18

I have seen that the developers worry about performance. My reply would be: Don’t :slight_smile:
If you run applications, the first time it loads from L2ARC, SSD or HDD. But afterwards on a desktop your programs will operate for >90% from L1ARC and your responses will be instantaneous. I display the hit rates of L1ARC and L2ARC in conky and L1ARC hit rates are always between 90% and 93%. L2ARC is between 0% and 5% for booting VMs and first program loads in VM or Host.
I even notice it on my 7 and 10 year old PCs with a L1ARC maxed by me to 2GB. It is a little bit a joke, but my DDR2 Phenom II has 6400MB/s available for loading Firefox :slight_smile: :slight_smile:
I reboot the Xubuntu 19.10 VM and Ubuntu Mate 19.10 VM on my Ryzen-3 in ~7 seconds from L1ARC LZ4 compressed.

19

Hey! Sorry, I was away for a couple of weeks but I saw jibel anwered most questions :slight_smile: We don’t know yet if we are going to commit with full support by the LTS or not. It will really depend on the amount of feedback on 19.10 and if we can get the development as far as we want before officially stamping a 1.0 for official support :wink:

Sure, however, you won’t get the zsys specificy themselves (or you will have to manually tag and understand the dataset layout and tagging we are doing). But you will get the benefits from zfs support, including a more advanced grub menu for booting from snapshots, listing machines and so on.

The specifics of dataset layout and tagging will be in the blog post suite on ZFS which is scheduled to be published during September/October.

We (including the kernel team) won’t backport ZFS 0.8.1 to a released version, as per the Stable Release Update policy. There are patches that have been backported in 0.7 to make it work with kernel 5.2 and we won’t risk our user audience by providing a new ZFS version.

As soon as we release the ubiquity code, that will be really easy :slight_smile: and avoid you to manually create the tagging which can be error-prone when done manually.

No! If you look at eoan today, you have /etc/grub.d/10_linux_zfs which is exactly designed to detect current (and other parallel installations) of ZFS system. This is enabled by default and if the script is installed, /etc/grub.d/10_linux is disabled for ZFS systems.

On zsys, each upgrade (not in eoan, but later on) will produce a new snapshot, whatever the size of the upgrade are. Due to how ZFS manages those deltas, this should be small and compressed. Snapshots automatically made by the system will be garbaged collected automatically after a period of time (which is to be defined) and the granularity will be decreased as well.

If you want those data pools to be persistent, sure (I’m sure the blog posts will shed some lights of it). It means that you will have to handle the revert and snapshots manually on those. Basically, we tried to not tight the hands of sysadmins :slight_smile:

Sure, we default UEFI/non UEFI selection will be available as it is in ubiquity today, so you can boot in legacy “bios” mode.

It shouldn’t because we don’t force one pool. The only limitation is that what we call “system datasets” (the one handled automatically by zsys for snapshots/rollback) have to be on the same pool (apart for /boot). Persistent datasets (as told previously, handled manually) can be in any pool. User datasets (basically home datasets) can be on any pool and even splitted.

1 Like
20

As someone with 40+ Ubuntu Server VMs at work along with SmartOS + LX Ubuntu zones at work and home (just to get ZFS without all the install maintenance grief of ZOL), I have been waiting/hoping for ZFS to become a first class filesystem in Ubuntu.

This sounds like you are going even further to automate and make ZFS features more accessible and that is great. But I wish you were starting with the server installer and am kind of surprised that you went with desktop first.

I just went through an mdraid root install on a server system. It was painful and requires extra steps every update. I would really love to be able to easily install Ubuntu Server on ZFS, and be able to count on ‘apt full-upgrade’ and ‘do-release-upgrade’ to work at least as well as they do now. (I still run out of space on default-sized /boot regularly so it would be nice if that could go away.)

Anyway, I am happy you are getting it in there at all. I really hope that you are able to get install/boot/upgrade stable enough to make it into 20.04 Server. I will fire up a 19.10 Desktop and play around a bit and file any issues I find. Thanks for all your work on this!