Chris Dzombak

sharing preview • dzombak.com

Moving fake-hwclock to a separate partition on a read-only Raspberry Pi

My guide to running a Raspberry Pi with a read-only root filesystem has one hacky caveat: it still allows fake-hwclock to write to the filesystem every hour. I don't love this solution, and this post presents a fix.

Moving fake-hwclock to a separate partition on a read-only Raspberry Pi

• Tagged: series:pi-reliability raspberry-pi linux

Part of the Raspberry Pi Reliability series.

Both iterations of my guide to running a Raspberry Pi with a read-only root filesystem have one hacky caveat: they still allow fake-hwclock to write to the filesystem every hour. I didn’t love this approach:

So I decided, on one of my read-only Pis, to move fake-hwclock’s data to an external USB drive. This is an approach I’ve used a few times now:

Moving forward, I will try to remember to set up a dedicated partition on the SD card for fake-hwclock when I first set up a new Pi. But I can’t repartition this SD card and downsize an existing partition in place, so a cheap small USB drive will suffice.

The information in this post is, to the best of my knowledge, current as of June 2024. It should work on Raspberry Pi OS versions 11 (Bullseye) and 12 (Bookworm), at least, but I make no promises.

Though this approach carries relatively little risk compared to the overall risks you took to make your Pi read-only, remember that there's always some risk. In particular, double-check your partitioning and mkfs commands before committing to them! (See my Pi Reliability post on risk vs. benefits.)

Part 1: Set up the USB drive

Connect the USB drive to your Pi. Confirm its name via lsblk. In this case, my drive is named sda and it’s already partitioned, so I’ll be using sda1 in the following steps:

$ lsblk
sda           8:0    1 29.3G  0 disk
└─sda1        8:1    1 29.3G  0 part
mmcblk0     179:0    0 59.7G  0 disk
├─mmcblk0p1 179:1    0  256M  0 part /boot
└─mmcblk0p2 179:2    0 59.4G  0 part /

Format the drive’s single partition with ext4 via sudo mkfs.ext4 /dev/sda1. This will print the UUID of the partition, or you can get it via lsblk -f:

$ lsblk -f
sda
└─sda1      ext4     1.0          cda25d93-8705-4824-847f-d7c8c34b5289
mmcblk0
├─mmcblk0p1 vfat     FAT32 bootfs 9E81-4F92                             203.5M    20% /boot
└─mmcblk0p2 ext4     1.0   rootfs cf2895ca-6dc2-4797-8040-f76ba1508f41   51.4G     8% /

Remount the root partition as read-write. If you added the shell integration I recommended, this is as simple as running the command rw.

Create a mount point for the drive. I’m calling mine /mnt/mut, since this is for storing mutable data: sudo mkdir /mnt/mut

Add the drive to /etc/fstab, using the UUID from earlier. I added this line to my fstab:

UUID=cda25d93-8705-4824-847f-d7c8c34b5289  /mnt/mut  ext4  defaults,noatime  0  1

Mount the drive via sudo mount -a. You shouldn’t get any errors at this point; if you do, you likely got something wrong in /etc/fstab.

I made my drive world-writable and world-readable, for ease of use. This is a single-user Pi, so locking down this drive isn’t a concern for me: sudo chmod 0777 /mnt/mut

Finally, change into /mnt/mut and verify that you can write a file and read it back:

$ cd /mnt/mut
/mnt/mut$ echo "hello world" > ./test.txt
/mnt/mut$ cat ./test.txt
hello world
/mnt/mut$ rm ./test.txt

Part 2: Reconfigure fake-hwclock

Configure fake-hwclock to use a new data storage location via sudo nano /etc/default/fake-hwclock. Add the following line:

FILE=/mnt/mut/fake-hwclock.data

We need to make some changes to fake-hwclock’s cron job to make it respect this setting. Edit it via sudo nano /etc/cron.hourly/fake-hwclock. The file should look like this:

#!/bin/sh
#
# Simple cron script - save the current clock periodically
# https://www.dzombak.com/blog/2024/06/Moving-fake-hwclock-to-a-separate-partition-on-a-read-only-Raspberry-Pi.html
#

if (command -v fake-hwclock >/dev/null 2>&1) ; then
  PARAM=/etc/default/fake-hwclock
  if [ -f "$PARAM" ]; then
    set -o allexport
    . "$PARAM"
    set +o allexport
  fi

  fake-hwclock save
fi

We also need to tell systemd that fake-hwclock now depends on an additional mount point. Edit the systemd unit file via sudo systemctl edit fake-hwclock.service, and add this content:

[Unit]
RequiresMountsFor=/mnt/mut/fake-hwclock.data

Finally, we’ll remove the old fake-hwclock data to avoid any confusion and cause a clear failure if something tries to access it: sudo rm /etc/fake-hwclock.data

Time to test it! Remount the root partition as read-only. With my shell integration, this is as simple as running the command ro.

First we’ll confirm that fake-hwclock is using the new storage location. Tell systemd to reload the daemon and restart it, then check its logs to make sure it didn’t hit an error trying to write to the (read-only) root partition:

$ sudo systemctl daemon-reload
$ sudo systemctl restart fake-hwclock.service
$ sudo journalctl -u fake-hwclock.service
-- Journal begins at Tue 2024-06-25 17:17:01 EDT. --
Jun 25 17:17:01 workshop-pi fake-hwclock[149]: Tue 25 Jun 2024 09:17:01 PM UTC
Jun 26 09:40:43 workshop-pi systemd[1]: Stopping Restore / save the current clock...
Jun 26 09:40:43 workshop-pi systemd[1]: fake-hwclock.service: Succeeded.
Jun 26 09:40:43 workshop-pi systemd[1]: Stopped Restore / save the current clock.
Jun 26 09:40:43 workshop-pi systemd[1]: Starting Restore / save the current clock...
Jun 26 09:40:43 workshop-pi fake-hwclock[19944]: Wed 26 Jun 2024 01:40:43 PM UTC
Jun 26 09:40:43 workshop-pi systemd[1]: Finished Restore / save the current clock.
$ cat /mnt/mut/fake-hwclock.data
2024-06-26 13:40:43

Similarly, confirm that the cron job runs without an error and updates /mnt/mut/fake-hwclock.data with the current time:

$ sudo /etc/cron.hourly/fake-hwclock
$ cat /mnt/mut/fake-hwclock.data
2024-06-26 13:56:13

Finally, time to reboot: sudo reboot now

Once the system comes back up, check the logs for fake-hwclock to see if it encountered any errors: sudo journalctl -u fake-hwclock.service

And you’re done.

References

  1. My fork of the software is at github.com/cdzombak/BirdPi. Right now it mainly contains in-progress work to move all recording and processing to ffmpeg, allowing for e.g. white noise filters. I’m also working on moving temporary recordings to a tmpfs in memory for performance and further flash drive wear reduction. I’ll post about that once it’s done, someday. ↩