Speeding up freebsd-update

When using freebsd-update it can take a long time to complete. I’ve been looking at ways of speeding it up.

During the update process, it’s not uncommon for it to download 50,000+ files one at a time! Lets fix that.

The update below will run 8 copies of phttpget in parallel. xargs takes care of making sure each instance of phttpget has it’s own unique list of files.

# man xargs
...
-P maxprocs
      Parallel mode: run at most maxprocs invocations of
      utility at once.  If maxprocs is set to 0, xargs will             
      run as many processes as possible.
...
# ee /usr/sbin/freebsd-update
...

fetch_setup_verboselevel () {
         case ${VERBOSELEVEL} in
         debug)
                 QUIETREDIR="/dev/stderr"
                 QUIETFLAG=" "
                 STATSREDIR="/dev/stderr"
                 DDSTATS=".."
                 XARGST="-t"
                 NDEBUG=" "
                 ;;
         nostats)
                 QUIETREDIR=""
                 QUIETFLAG=""
                 STATSREDIR="/dev/null"
                 DDSTATS=".."
                 XARGST="-P 8" # UPDATED <<
                 NDEBUG=""   
                 ;;
         stats)
                 QUIETREDIR="/dev/null"
                 QUIETFLAG="-q"
                 STATSREDIR="/dev/stdout"
                 DDSTATS=""
                 XARGST="-P 8" # UPDATED <<
                 NDEBUG="-n"
                 ;;
         esac
 }

...

Solus Linux + OpenZFS

Solus is a great Linux distribution. Very clean and has everything I need, expect ZFS. Lets fix that.

Building zfs is also discussed here: https://github.com/zfsonlinux/zfs/wiki/Building-ZFS

You’ll need to install the development tools and some packages (Kernel modules are currently not building without libelf-devel)

Basic knowledge of how to use the terminal is assumed.

sudo eopkg install -c system.devel
sudo eopkg install git linux-current-headers libelf-devel

Lets build it. As of writing, the master branch is 0.8.0-rc2. I’m using –depth 1 so we don’t get the history or tags (optional).

git clone --depth 1 https://github.com/zfsonlinux/zfs
cd zfs
sh autogen.sh
./configure
make -s -j$(nproc)
sudo make install
sudo depmod

Or if you’ve just updated and a new kernel was installed (before rebooting) you can use (replace with correct path):

./configure --with-linux=/usr/src/linux-headers-4.20.8-110.current

While there is a script to load the kernel modules ( /usr/local/share/zfs/zfs.sh ), there does not seem to be a service script that fires it up on boot.

Create /etc/modules-load.d/zfs.conf with your favorite editor and add the following:

zavl
zunicode
spl
znvpair
zcommon
icp
zlua
options zfs zfs_autoimport_disable=0
zfs

Enable the services:

sudo systemctl enable zfs-import-cache.service
sudo systemctl enable zfs-import-scan.service
sudo systemctl enable zfs-import.target
sudo systemctl enable zfs-mount.service
sudo systemctl enable zfs-share.service
sudo systemctl enable zfs-zed.service
sudo systemctl enable zfs.target

Time to reboot and check everything worked.

lsmod | grep zfs

zfs 4087808 8
zlua 180224 1 zfs
icp 311296 1 zfs
zcommon 90112 1 zfs
znvpair 90112 2 zfs,zcommon
zunicode 335872 1 zfs
zavl 16384 1 zfs
spl 118784 5 zfs,icp,znvpair,zcommon,zavl

Caveats

  • You will have to rebuild zfs every time the kernel is updated
  • If the kernel is updated your zfs pools will not mount on the next reboot – don’t mount any critical file systems which would prevent you from booting, onto a zfs pool.
  • It takes a little longer to boot since it scans for pools and mounts them

This tutorial was performed on a clean install of Solus Budgie 3.9999 after running eopkg up and rebooting.

Discus and comment here: https://getsol.us/forums/viewtopic.php?f=11&t=13207