My old btrfs boot hook—mkinitcpio-btrfs, community-maintained in its later years—shipped a kernel-version check with a configuration knob to turn it off. The final hook declares the knob under one name and tests another:
: ${BTRFS_KVER_CHECK:=true}if ${BTRFS_CHECK_KVER}; thenLook again. The declared variable is BTRFS_KVER_CHECK. The tested variable is BTRFS_CHECK_KVER—two words transposed, with no assignment in the checked tree or its recovered history.
In this hook—as in a POSIX shell running without set -u—if ${UNSET_VARIABLE}; then expands to zero words. With no command left to run, that condition is a successful no-op, exit status zero, so the branch is always taken. The version check ran unconditionally on every boot, and setting BTRFS_KVER_CHECK=false in /etc/default/btrfs_advanced—the documented, commented-out option—silently did nothing.
The git history makes it better. The transposition was born with the feature, in commit 32ddc81 (“added experimental kernel version check”)—which declared the knob default-false, experimental, opt-in. Because the tested variable was empty, the experimental check ran for everyone from day one. Seven months later, commit bac3ba0 (“enable kernel version check by default”) flipped the default to true—a change with no observable effect whatsoever, since the variable it changed was still not the variable being tested. The documented behavior and the actual behavior converged by coincidence. No fix or report appears in the checked repository or history, and I did not notice it during the project’s active maintenance.
The idiom itself is worth naming, because it’s why the bug could exist. The hook runs as #!/bin/ash inside a busybox initramfs, where true and false are commands like any other—so the natural boolean flag is a variable holding a command name, tested by running it: if ${FLAG}; then. It’s compact and portable, but without set -u—as in this hook—an unset value can leave a successful empty command. The parser accepts the program; the mistake appears only when the wrong variable expands at runtime.
The readability cleanup
Specimen two is a single cleanup commit—a737b8a, “runtime: use more reasonable logical constructs”, by Dave Reisner; cleanups elsewhere in that same series improved the hook. Its commit message explains that a && b && c chains are not the same as if a; then b; c; fi—which is correct—and in the course of removing them, it made this change:
- if ! { btrfs_ask_volatile && btrfs_set_volatile; }; then
+ if ! btrfs_ask_volatile && btrfs_set_volatile; then! { A && B; } is “not (A and B)”. ! A && B is “(not A) and B”. The braces were not noise; they were the logic.
This guards the hook’s first-boot setup. btrfs_ask_volatile succeeds when you press a key at the “Press any key to setup BTRFS rollback support…” prompt; btrfs_set_volatile walks you through converting the system and succeeds if you complete it. Original meaning: unless you both pressed the key and completed setup, disable rollback for this boot.
After the change: pressing the key short-circuits the whole condition—setup never runs, and rollback stays enabled against an __active subvolume that doesn’t exist. Letting the prompt time out—the “leave me alone” gesture—is now what drops you into the setup dialog. And completing that dialog successfully makes the condition true, which disables rollback for the boot that just set it up. The flow is inverted at both ends.
A commit about the difference between two logical forms introduced a bug that is exactly a difference between two logical forms. No revert appears in the checked history—the project was coasting, and the first-boot path is the one path existing users never exercise again. The braces had been carrying a grouping rule that the cleanup removed.
The policy that hides the evidence
The hook already practiced this in miniature—its filesystem probe attaches a redirect to the function definition, root_is_btrfs() { ...; } 2>/dev/null, so every call runs with stderr gone. Specimen three is the same policy grown to full size, in my own dotfiles. They carry twin bootstrap files, boot/bash and boot/zsh, near-identical by design. The zsh one opens with a stream policy:
# Clean stderr unless debug.
if ! ((ELLIPSIS_DEBUG)); then
exec 2>/dev/null
fiFurther down, the log helpers select a color by variable indirection:
printf "## ${!at}$format$OFF\n" "$@"${!at} is bash. In zsh it is not indirection; it is a bad substitution error—the correct spelling there is ${(P)at}. The zsh twin inherited the line from the bash twin, and every call to those helpers since has failed—behind exec 2>/dev/null, where errors go to not exist. Set ELLIPSIS_DEBUG=1 and the failure appears instantly. I wrote the policy, I wrote the bug, and the first concealed the second for years.
The same failure mode
The shell hides bugs in the places where its grammar makes wrongness mean something valid:
- Without
set -u, an unset variable in command position is not a reference error—it’s a successful empty command. Misspell a flag and you gettrue. !binds to the first pipeline, not the whole condition. Drop the braces and you get a different program that parses fine.- A silenced stderr doesn’t distinguish “noise suppressed” from “diagnostic suppressed”. Whatever breaks behind it stays broken.