Skip to content

A pidfile that is really a cgroup

linux
 3 min Encoding cgroup2 CPU limits into a --pidfile flag, so moving a PID between two cgroups becomes the whole throttle.
Published

Sep 10, 2022

The lobby supervisor throttled an idle game server by writing its PID to a different file. That is the entire technique. The trick is which file, and how the flag that names it is spelled:

--pidfile cgroup:idle?cpu.max=1000+10000

A pidfile is just a file you write a pid to, and cgroup.procs is such a file. So --pidfile accepts a comma list where any entry can carry a cgroup: scheme. When it sees one, it parses the entry as a URL, finds the cgroup2 mount by scanning /proc/self/mounts for the cgroup2 line, reads its own cgroup from /proc/self/cgroup, and creates child cgroups under that. Each URL-decoded query key names a control file, and each decoded value is written to it. Then the code rewrites the entry to <cgroup>/cgroup.procs and hands that back as the effective pidfile.

The + is the load-bearing detail. cpu.max wants two space-separated numbers, quota then period. A literal space is awkward in a flag value, but + is exactly how a URL query encodes a space—so cpu.max=1000+10000 parses back to the string 1000 10000, which is precisely what the kernel expects. idle gets a 10000µs period with a 1000µs quota: 10% of one CPU. busy gets 1000+1000, a full core.

With the cgroups built, state changes are file writes. When the server goes busy, its pid goes into busy/cgroup.procs; when it idles, into idle/cgroup.procs. Moving the pid is the transition. os.WriteFile reaches the kernel’s cgroup interface directly.

The systemd side supplies the substrate:

Environment=SNAPGS_LOBBY_PIDFILE=cgroup:main?cpu.max=max+1000,cgroup:busy?cpu.max=1000+1000,cgroup:idle?cpu.max=1000+10000
CPUQuotaPeriodSec=1ms
CPUQuota=110%
Delegate=true

Delegate=true hands the unit’s subtree to the process so it may create children at all; the first entry writes the pid to main and enables +cpu in cgroup.subtree_control so the children can carry CPU limits. CPUQuotaPeriodSec=1ms and CPUQuota=110% are a separate parent guardrail: systemd measures the whole unit’s 110% cap over that 1ms period. They do not set the child periods. The supervisor writes the children’s periods directly from the URL values—1ms for busy, 10ms for idle—along with each child’s quota.

The idle policy capped the Unity server at 10% of one CPU while keeping the process running. A player joining switched it to the busy quota without restarting the game. The cgroup path requires Linux. A supplied cgroup: entry returns an error when reading the mount table fails. If that read succeeds without a cgroup2 match, the code keeps /sys/fs/cgroup as its candidate and tries the path there. An ordinary entry does not depend on discovery succeeding, although the command still attempts the mount-table read before it inspects any entry. That ordinary-path branch has its own defect: it calls MkdirAll on the pidfile pathname, then later tries to write that same path as a file.

The code lives in the retired snap-gs lobby supervisor.