Linux and systems
This is the largest part of what we do, and the oldest. Servers, routers, storage, virtualisation, and the small boards that run the same software on a fraction of the hardware. Most of the work arrives broken, usually just before something depended on it.
What I work on
Servers and system administration
The ordinary work of keeping a machine running. Daemons, services, users, logs, updates, and the slow accumulation of things nobody designed. Where most of the hours go, and where the emergencies come from.
Routers and firmware
OpenWrt on hardware that shipped with firmware nobody will ever update again. VPN tunnels that hold up under NAT rebinding, MAC and DHCP reservations, custom builds from source when the image archive does not have your model.
Virtualisation
Proxmox clusters built for failover rather than for a diagram. KVM hosts kept current past the point the vendor cared, including driver work on kernels old enough that nothing ships for them any more.
Storage and recovery
RAID arrays, and the recovery work after one goes wrong at the wrong time. ZFS and mdadm, plus the awkward part after a boot fails where the filesystem matters more than the bootloader.
Kernel and drivers
Device support for kernels past their supported life, and daemons written to a specification rather than assembled from whatever was to hand. Low level work, where the answer is usually a patch and a rebuild.
Embedded and single board
Raspberry Pi deployments that run unattended for years, and firmware for boards with kilobytes of memory. Touch interfaces built for an ESP32 controller and proven on the hardware before handover.
Windows hosts
Sometimes the machine in the cupboard runs Windows and has to stay that way. Hyper-V migrations driven from PowerShell, and image deployment automated so it stops depending on whoever is awake.
Custom boot media
Bootable images built from the ground up when the standard installer cannot produce what the deployment needs. Also the fastest way to get a rescue environment onto a machine that will not boot.
Work done in this area
Taken from projects that shipped.
- Highly redundant Proxmox cluster
- Multi node cluster configured for failover, with the behaviour written down so the next operator can reason about it.
- Stack
- Proxmox VE, Linux, shared storage
- Linux user space daemon
- A daemon developed to a written specification and tested against it, including the behaviour when its dependencies disappear.
- Stack
- C, Linux, system architecture, shell
- RAID 5 boot failure
- AlmaLinux 9 production server that stopped booting after an array event. Rebuilt and verified, client reported the recovery as flawless.
- Stack
- AlmaLinux 9, Red Hat, mdadm
- OpenWrt on a consumer router
- TP-Link configuration including MAC address reservation and network setup, done on the router the client already owned rather than replacing it.
- Stack
- OpenWrt, TP-Link, shell
- OpenWrt on Raspberry Pi
- A setup script written for a Pi running OpenWrt, so the configuration could be reproduced rather than rediscovered.
- Stack
- OpenWrt, Raspberry Pi, shell
- Custom bootable Ubuntu 24.04 ISO
- Bootable image built from the ground up for a deployment the standard installer could not produce.
- Stack
- Linux, shell, Debian
- KVM device on kernel 4.19
- Device support written for a kernel far past its supported life, where the vendor tree had stopped. The kind of task that cannot be solved with a package install.
- Stack
- C, KVM, Linux 4.19
- GPU sharing on Hyper-V
- GPU sharing configured for virtual machines on a Hyper-V host, where the passthrough rules differ by host generation.
- Stack
- Hyper-V, Windows Server, GPU passthrough
- Hyper-V migration in PowerShell
- Virtual machine relocation on Windows 10 hosts, scripted rather than clicked through.
- Stack
- Hyper-V, PowerShell, Windows Server
- Windows 11 image deployment
- Deployment automated end to end for an open source imaging project, including the handover and the documentation the maintainers needed.
- Stack
- Windows Desktop, System Admin, Ubuntu
- SSH access on a closed image
- Root and SSH configured on a Linux image shipped without an access path, which is a longer job than it sounds.
- Stack
- Linux, shell, Debian, Red Hat, Fedora
- OpenStack virtual machine
- STACKIT OpenStack instance created from a bootable ISO, configured and verified as reachable.
- Stack
- OpenStack, Linux, Debian, Red Hat
- OwnCloud on Raspberry Pi
- Self hosted storage on a single board computer, set up and handed over. Reported as good work, delivered on time.
- Stack
- Raspberry Pi, Ubuntu, Debian, OwnCloud
- ESP32 vending controller interface
- Touch and display UI for embedded hardware with multiple languages and GIF playback, built with LVGL and EEZ Studio and proven on the device.
- Stack
- LVGL, EEZ Studio, ESP32, C
- AlmaLinux 9 RAID and boot recovery
- An expert Linux engineer brought in for a boot failure on a production host, and hired again afterwards.
- Stack
- AlmaLinux 9, Red Hat, systems engineering
What I need from you
Access to the machine, a description of what you expected to happen, and the output of whatever failed. On a broken server the error message and the last thing you changed are usually enough to start. A screenshot of the console or the raw log beats a summary of it every time, because it removes the step where a symptom gets reinterpreted.