ThinkPad X1C 6th Gen on NixOS with native ZFS at-rest encryption

Be aware: Native Encryption on ZFSonLinux is not yet flagged “stable”. The aim is for version 0.8!

With the release of the ThinkPad X1C 6th Gen I’m finally leaving the T440s era and the terrible ClickPad behind. With this switch I gain a nicer screen, charging via USB-C and 4 real ULV cores.

Usually when I get a new laptop with it comes a decision to stay with the tried and true setup I’ve run for years or to try something new again.

After years of BTRFS and becoming a fan of constant snapshotting I’ve decided to use ZFSonLinux after being very happy with ZFS on FreeBSD.

The Objective

	Current	Goal
Volume Manager	LVM	ZFS
Crypto	LUKS	ZFS
File System	BTRFS	ZFS

Embrace the rampant layering violations (or as the fans would say: telescoping)!

The Preparation

BIOS Setup

Disable Secure Boot and Enable UEFI Only boot mode.

Grab NixOS and write image to USB device

Download NixOS 18.03 and write image to USB device. I use dd and /dev/disk/by-id to minimize the chance to mess up.

dd if=‘nixos-graphical-18.03.131807.489a14add9a-x86_64-linux.iso’ \
   of=/dev/disk/by-id/usb-Samsung_Flash_Drive_FIT_XXXXXXXXXXXXXXX-0:0 bs=1M

The Install

Boot and load NixOS

It should boot up without any errors to an automatically logged in root shell.

Enable networking

I use network manager for this. Launch nmtui and configure as needed.

(Optional) enable sshd

To do the work from a remote host enable sshd, get IP and change password for root so you can ssh into the system.

systemctl start sshd
ip a
passwd

Load zfs support into running system

Use your preferred editor to edit /etc/nixos/configration.nix and add

# /etc/nixos/configration.nix
boot.supportedFilesystems = [ "zfs" ];
boot.zfs.enableUnstable = true;

after the import line within the curly braces block.

Load the configuration:

nixos-rebuild switch

Create zfs pool

zpool create -o ashift=12 -o altroot="/mnt" \
-O atime=off -O compression=lz4 -O normalization=formD \
-O encryption=aes-256-gcm -O keyformat=passphrase \
rpool /dev/disk/by-id/nvme-SAMSUNG_MZVLB512HAJQ-000L7_XXXXXXXXXXXXXX-part1

ashift=12 means use 4KiB sectors, this could even be set to ashift=13 for 8KiB sectors.
atime=off is the zfs equivalent to the mount option noatime.
compression=lz4 lz4 is extremely fast. Best choice until zstd is available in ZoL.
normalization=formD my use-case allows for utf-8 everywhere and formD is the reasonable default. See here for more.
encryption=aes-256-gcm is another reasonable choice and may become the default for ZFS’s encryption in the future.
keyformat=passphrase is the easiest to work with for now. Can be changed later with zfs change-key to implement a USB-stick to decrypt the internal harddisk.

Partition the pool

As ZFS can’t be read by Linux bootloaders, it requires a partition that can be read natively.

Clear partition table:

sgdisk --zap-all /dev/disk/by-id/nvme-SAMSUNG_MZVLB512HAJQ-000L7_XXXXXXXXXXXXXX

Create EFI System partition:

sgdisk     -n2:1M:+512M -t2:EF00 /dev/disk/by-id/nvme-SAMSUNG_MZVLB512HAJQ-000L7_XXXXXXXXXXXXXX

The ZFS partition:

sgdisk     -n1:0:0      -t1:BF01 /dev/disk/by-id/nvme-SAMSUNG_MZVLB512HAJQ-000L7_XXXXXXXXXXXXXX

Sometimes it may be necessary to re-read the partition table, in that case do a partprobe.

Create the filesystems:

zfs create -o mountpoint=none rpool
zfs create -o mountpoint=legacy rpool/root/nixos
zfs create -o mountpoint=legacy rpool/home

And vfat for the EFI System partition:

mkfs.vfat /dev/disk/by-id/nvme-SAMSUNG_MZVLB512HAJQ-000L7_XXXXXXXXXXXXXX-part2

Mount and generate NixOS configuration

mkdir -p /mnt/{boot,home}
mount -t zfs rpool/root/nixos /mnt
mount -t zfs rpool/home/
mount /dev/disk/by-id/nvme-SAMSUNG_MZVLB512HAJQ-000L7_XXXXXXXXXXXXXX-part2 /mnt/boot

nixos-generate-config --root /mnt/

You can check what it generated in /mnt/etc/hardware-configuration.nix. It ought to look like this:

#/mnt/etc/hardware-configuration.nix
[...]
  fileSystems."/" =
    { device = "rpool/root/nixos";
      fsType = "zfs";
    };
  fileSystems."/home" =
    { device = "rpool/home";
      fsType = "zfs";
    };
  fileSystems."/boot" =
    { device = "/dev/disk/by-uuid/XXXX-XXXX";
      fsType = "vfat";
    };
[...]

(Optional) Change CPU governor

At the end of the file you should see the powersave governor being selected.

The CPU governor in Linux changed a few years ago. It’s now using P-State (powersave, performance) and not CPUfreq (powersave, performance, ondemand, conservative).

Contrary to the name, it may be better to use performance than powersave because it will do the work quicker and sleep longer, rather than doing the work at a lower clock and sleeping less.

#/mnt/etc/hardware-configuration.nix
powerManagement.cpuFreqGovernor = "performance";

Configure NixOS

This is the regular NixOS configuration that is needed to have a working system. Please see the manual.

However this should be included in the /mnt/etc/configuration.nix so that zfs works properly on boot:

boot.supportedFilesystems = [ "zfs" ];
boot.loader = {
  systemd-boot.enable = true;
  efi.canTouchEfiVariables = false; # true here results in errors on my system
}
boot.zfs = {
  enableUnstable = true;
  forceImportRoot = false;
  forceImportAll = false;
}

networking.hostId = "4e98920d";

The networking.hostId can be generated using:

head -c4 /dev/urandom | od -A none -t x4

Running the new system

Initial boot

After systemd boot loader is loaded press e and add zfs_force=1 to the kernel parameters. This is needed because the NixOS live-cd had a different machine ID from the one you’re importing into now.

This is a safeguard by zfs which we need to circumvent once.

(Optional) Enable auto-snapshotting service

The primary reason for me to use a Copy-on-Write filesystem. I like having 2 days worth of snapshots that I can instantly go back to, without having the backup overhead.

# /etc/nixos/configuration.nix
services.zfs.autoSnapshot = {
  enable = true;
  frequent = 96;
  hourly = 0;
  daily = 0;
  weekly = 0;
  monthly = 0;
};

A frequent snapshot is created every 15 minutes.

I only use this for my home directory as NixOS has the system generation thing going for it, which is superior to snapshots.

zfs set com.sun:auto-snapshot=true rpool/home

Automatic scrubbing

Another reason to use CoW and it’s a very good one: It protects you from bit rot.

Because the checksumming on read only helps if you actually read the files and some files sit unused more than others, you can use scrubbing to have the whole dataset be read to identify and self-heal¹ corrupted files.

# /etc/nixos/configuration.nix
services.zfs.autoScrub = true;

By default this will run every Sunday at 02:00. It uses the systemd.time format which you can play with like this:

systemd-analyse calendar "Sun, 02:00"

Use backups

While this is out of scope of this article, I want to stress that even though ZFS is awesome you should not rely on it alone.

I use Borg to back up my laptop to my NAS (ZFS) every night and my NAS propagates the backups to rsync.net. It’s quite reasonable priced and the only other provider I would consider at this point is Blackblaze B2. As it lacks the ability to ssh and use borg directly it would complicate my setup a bit but seems quite nice from a price/performance perspective.

Acknowledgements

The credit to have ZFS on Linux be so easily configured is due to the work by the NixOS contributors. This is really nice and is a big reason to adopt this interesting Linux Distribution.

And of course big thanks to ZFSonLinux that native encryption at rest is even a thing.

In this configuration it can not self-heal, unless copies=2 or higher is set. And even then, resilience is not great. ^[return]