Author: Otto Odgaard

OpenZFS Raid-Z Expansion

A small script to test zfs raid-z expansion in Solus Linux (requires root user)

#!/bin/sh

ZPOOL="/usr/local/sbin/zpool"

truncate -s 1G FILE0
truncate -s 1G FILE1
truncate -s 1G FILE2
truncate -s 1G FILE3
truncate -s 1G FILE4

losetup /dev/loop0 FILE0
losetup /dev/loop1 FILE1
losetup /dev/loop2 FILE2
losetup /dev/loop3 FILE3
losetup /dev/loop4 FILE4

$ZPOOL create RAID5 raidz loop0 loop1 loop2 loop3

# Copy a small amount of data on to the new filesystem
# (Update if not using Solus Linux)
cp -a /usr/lib64/budgie-desktop /RAID5/test

$ZPOOL status RAID5

# Expand that vdev
$ZPOOL attach RAID5 raidz1-0 loop4

$ZPOOL status RAID5
sleep 10
$ZPOOL status RAID5

$ZPOOL destroy RAID5

losetup --detach /dev/loop0
losetup --detach /dev/loop1
losetup --detach /dev/loop2
losetup --detach /dev/loop3
losetup --detach /dev/loop4

rm FILE?

Find degrees between two points

This is based on SFML where zero degrees is right -> (+x axis)

#include <cmath>

float angleBetweenVectors(const sf::Vector2f& p1, const sf::Vector2f& p2)
{
    float angleInRadians = atan2(p1.y - p2.y, p1.x - p2.x);
    float angleInDegrees = (angleInRadians / M_PI) * 180.0;
    return angleInDegrees < 0 ? angleInDegrees + 360 : angleInDegrees;
}

When the angle goes negative, we add 360 to normalise it.

ZFS Build Script

The following shell script (zfs-build-latest.sh) will automatically download the latest OpenZFS release and compile it

Requirements:

  • Build tools (gcc, make etc)
  • autoconf
  • jq (cli json parser)
  • openssl-dev
  • kernel headers that match your installed kernel
  • curl
  • zstandard libs (if zstd support is required)
  • you will have to install other missing libs if required
sudo eopkg it -c system.devel
sudo eopkg it git linux-current-headers libelf-devel libtirpc-devel 
sudo eopkg it jq curl-devel openssl-11-devel
#!/bin/sh

curl -s https://api.github.com/repos/openzfs/zfs/releases/latest > git-release.json
tag=$(jq -r ".tag_name" git-release.json)
jq '.assets[] | select(.content_type == "application/gzip")' git-release.json >latest-asset.json

url=$(jq -r '.browser_download_url' latest-asset.json)
file=$(jq -r '.name' latest-asset.json)

echo "Latest Release: ${tag} : ${file} -> ${url}"
previousTag=""

if [ -f "previous-tag.txt" ]
then
  previousTag=$(cat previous-tag.txt)
fi

if [ "${previousTag}" = "${tag}" ]
then
  echo "Already downloaded this version, compiling..."
fi


if [ ! -f "${file}" ]
then
  echo "Downloading ${url}"
  curl -L "${url}" -O
fi

rm -rf "${tag}"
tar zxf "${file}"
cd "${tag}"

# We try to determine the latest kernel headers here, sometimes this may not work as intended.

./autogen.sh
./configure --with-linux=/usr/src/$(ls -t /usr/src/ | grep linux-headers | head -n 1)
make -j$(nproc)

cd ..
echo "${tag}" >previous-tag.txt

echo "Now you can install with the following commands:"
echo "cd ${tag} && sudo make install && sudo depmod"

Configuration guide here: http://blog.twbc.net/2019/01/07/solus-linux-openzfs/

No Recursion Required

Fetching all the records from a multi page API endpoint without recursion back to the same function. Example in JavaScript.

const getAllProducts = async () => {
  // data and options is set elsewhere in this case
  data['maxRecords'] = 250;
  let res;
  let products = [];
  let page = 1;

  do {
    res = await axios.post(url, data, options);
    products = [...products, ...res.data.pageRecords];
    count = res.data.pageRecords.length;
    data['page'] = page + 1;
  } while (count === data['maxRecords'])

  return products;
}

Style Budgie on Xubuntu

Install prerequisites

sudo apt install budgie-desktop
sudo apt-add-repository ppa:tista/adapta
sudo apt install papirus-icon-theme adapta-gtk-theme breeze-cursor-theme

Select Budgie Desktop from the top right of the login screen.

Run Budgie Desktop Settings

Select

Widgets: Adapta
Icons: Papirus
Cursors: breeze_cursors
Select Position: Bottom

OpenOffice Manual Install

  • Download deb .tar.gz file
    e.g: Apache_OpenOffice_4.1.7_Linux_x86-64_install-deb_en-GB.tar.gz
  • Extract it to a temporary directory
    tar xf Apache_OpenOffice_4.1.7_Linux_x86-64_install-deb_en-GB.tar.gz
  • Make a subdirectory to extract the deb files to and cd to it.
    mkdir tmp
    cd tmp
  • Run the following script to extract the contents of every deb file:
for file in find ../*.deb
 do
   ar -xv "$file"
   tar xf data.tar.gz
   rm -f data.tar.gz control.tar.gz debian-binary
 done
  • Move opt/openoffice4 to /opt/
    sudo mv ./opt/openoffice4 /opt/
  • Run /opt/openoffice4/program/soffice

You could now create a .desktop file and set its icon.

Solus Linux – Repo change

Swapping out the Solus repo from https to http so we can cache it in a local server.

$ eopkg lr
 Solus [active]
    https://mirrors.rit.edu/solus/packages/shannon/eopkg-index.xml.xz
$ sudo eopkg ar solus http://mirrors.rit.edu/solus/packages/shannon/eopkg-index.xml.xz
Repo solus added to system.
 Updating repository: solus
 eopkg-index.xml.xz.sha1sum     (40.0  B)100%      2.96 MB/s [00:00:00] [complete]
 eopkg-index.xml.xz             (2.3 MB)100%    328.90 KB/s [00:00:01] [complete]
 Package database updated.
$ sudo eopkg rr Solus
Repo Solus removed from system.

Speeding up freebsd-update

When using freebsd-update it can take a long time to complete. I’ve been looking at ways of speeding it up.

During the update process, it’s not uncommon for it to download 50,000+ files one at a time! Lets fix that.

The update below will run 8 copies of phttpget in parallel. xargs takes care of making sure each instance of phttpget has it’s own unique list of files.

# man xargs
...
-P maxprocs
      Parallel mode: run at most maxprocs invocations of
      utility at once.  If maxprocs is set to 0, xargs will             
      run as many processes as possible.
...
# ee /usr/sbin/freebsd-update
...

fetch_setup_verboselevel () {
         case ${VERBOSELEVEL} in
         debug)
                 QUIETREDIR="/dev/stderr"
                 QUIETFLAG=" "
                 STATSREDIR="/dev/stderr"
                 DDSTATS=".."
                 XARGST="-t"
                 NDEBUG=" "
                 ;;
         nostats)
                 QUIETREDIR=""
                 QUIETFLAG=""
                 STATSREDIR="/dev/null"
                 DDSTATS=".."
                 XARGST="-P 8" # UPDATED <<
                 NDEBUG=""   
                 ;;
         stats)
                 QUIETREDIR="/dev/null"
                 QUIETFLAG="-q"
                 STATSREDIR="/dev/stdout"
                 DDSTATS=""
                 XARGST="-P 8" # UPDATED <<
                 NDEBUG="-n"
                 ;;
         esac
 }

...

Solus Linux + OpenZFS

Update 2023-04-06: Solus is no longer being maintained and appears dead for now. Be that as it may, this article is relevant to almost any Linux distro given that you install the required prerequisite packages (openssl-dev, kernel headers and build essentials etc)

Building zfs is also discussed here: https://github.com/zfsonlinux/zfs/wiki/Building-ZFS

NOTE: zfs-on-linux 0.8.3 and previous versions are NOT compatible with Linux kernel 5.5

You’ll need to install the development tools and some packages (Kernel modules are currently not building without libelf-devel)

Basic knowledge of how to use the terminal is assumed.

Commands with eopkg are for Solus

sudo eopkg install -c system.devel
sudo eopkg install git linux-current-headers libelf-devel

Lets build it. As of writing, the master branch is 0.8.0-rc2. I’m using –depth 1 so we don’t get the history or tags (optional).

Download and Compile

Recommendation: Download the release archive and not the latest git source.

git clone --depth 1 https://github.com/zfsonlinux/zfs
cd zfs
sh autogen.sh
./configure
make -s -j$(nproc)
sudo make install
sudo depmod

Or if you’ve just updated and a new kernel was installed (before rebooting) you can use (replace with correct path):

./configure --with-linux=/usr/src/linux-headers-4.20.8-110.current

Alternative to download and install: http://blog.twbc.net/2023/04/06/zfs-build-script/

Configuration

While there is a script to load the kernel modules ( /usr/local/share/zfs/zfs.sh ), there does not seem to be a service script that fires it up on boot.

Create /etc/modules-load.d/zfs.conf with your favorite editor and add the following:

zavl
zunicode
spl
znvpair
zcommon
icp
zlua
options zfs zfs_autoimport_disable=0
zfs

Enable the services:

sudo systemctl enable zfs-import-cache.service
sudo systemctl enable zfs-import-scan.service
sudo systemctl enable zfs-import.target
sudo systemctl enable zfs-mount.service
sudo systemctl enable zfs-share.service
sudo systemctl enable zfs-zed.service
sudo systemctl enable zfs.target

Time to reboot and check everything worked.

lsmod | grep zfs

 zfs                  4087808  8
 zlua                  180224  1 zfs
 icp                   311296  1 zfs
 zcommon                90112  1 zfs
 znvpair                90112  2 zfs,zcommon
 zunicode              335872  1 zfs
 zavl                   16384  1 zfs
 spl                   118784  5 zfs,icp,znvpair,zcommon,zavl

Caveats

  • You will have to rebuild zfs every time the kernel is updated
  • If the kernel is updated your zfs pools will not mount on the next reboot – don’t mount any critical file systems which would prevent you from booting, onto a zfs pool.
  • It takes a little longer to boot since it scans for pools and mounts them

This tutorial was performed on a clean install of Solus Budgie 3.9999 after running eopkg up and rebooting.

This guide has been tested on Debian based distros like Ubuntu and PopOS

Discus and comment here: https://getsol.us/forums/viewtopic.php?f=11&t=13207 (edit: no longer exists)