I had four Arch machines, and every one of them downloaded every package update separately. The stock answers were a shared cache exported over NFS or a full local mirror on a cron schedule. I rejected both and wrote a request-driven proxy instead: pacproxy, a Python server on CherryPy that sat between pacman and the mirrors and cached exactly what got requested, when it was requested.
I posted the source to the Arch forums. The personal Git host announced in that thread is long gone, which makes the forum post the canonical surviving record: source, design notes, caveats, one field test, and follow-up plans, all in one place. The implementation used the Python and CherryPy APIs of that era.
How it worked
The cache layout doubled as the index. A package fetched for a given repository and architecture landed under .pacproxy/<repo>/<arch>/, and beside each package sat a sidecar file holding its cache state. There was no separate database.
The sidecar held HTTP validators, stored as ready-made request headers. When the mirror supplied an ETag, the sidecar kept it as cache_dict['If-None-Match'] = etag; when it did not, the fallback was cache_dict['If-Modified-Since'] = last_mod, taken from Last-Modified. On the next request for a file already in cache, the proxy asked the mirror a conditional question instead of downloading again. A HEAD that came back 304 settled it: the response to the client became an open local file, streamed as-is, with no separate response assembly. Cache coherence was delegated to HTTP’s own validators rather than a custom synchronization protocol, which was the point of fronting the mirrors instead of copying them.
The miss path is the part I still like. gen_proxy was a generator that pulled the upstream body chunk by chunk, and for every chunk did two things: write it to the local cache file, then yield the same chunk to CherryPy—local_fd.write(bytes); yield bytes. One pass over the upstream body served the requesting client while leaving a complete cache entry behind. The first machine avoided waiting for a complete fill followed by a reread, but it was still synchronously taxed for each local write before the chunk could be yielded. The second machine hit disk.
That was the implemented core: stream-and-cache, validator sidecars, conditional revalidation, filesystem-as-index. A stranger deployed it a few months later.
The caveat list was the plan
The release post carried its own caveat list, and it was specific. The sidecar was a printed Python dictionary read back with eval(), flagged as unsafe in the same post that shipped it. There was no locking, so simultaneous requests for the same uncached package could observe inconsistent cache state; the named remedy was lock files plus timeouts—one writer, followers wait, a timeout reclaims dead locks. The cache file also occupied its final name while the generator filled it, so a disconnected first client could leave a partial entry visible to the next request. The two-line tee loop supplied no publication boundary of its own. There was no eviction, so superseded packages accumulated forever; time-since-access cleanup was proposed and not written. Transport was HTTP only. Response-header fidelity was suspect. And package matching used a literal *.pkg.tar.gz suffix.
The field test arrived a few months later, when another user put pacproxy in front of a fresh install, and two caveats promptly surfaced. Arch had moved its package format to .tar.xz in the intervening months, so the literal suffix match went first. Then installs stalled and retried even though bytes were streaming; both of us suspected the proxy was dropping response metadata, size-related headers among it. A proxied body can be byte-for-byte correct and still be an invalid proxy response, because headers are protocol state, not decoration.
Two days later the tester reported pacproxy working in AIF after the package-suffix discussion. I used it for a while too, although I no longer remember for how long.
That was as far as pacproxy got. Turning local build output into repositories, retrieving builds over SSH, choosing package views by requester, and expiring old packages all remained plans.