The xattr fell outside the commit

 2 min snap-gs tried to publish file bytes and future S3 headers through one rename, but ignored errors let the rename commit either half alone.

In the retired snap-gs service, a Go daemon wrote a match file and a separate shell process later uploaded it to S3. The intended handoff was appealing: store the future HTTP headers and routing key on the file as an extended attribute, then publish bytes and metadata together with one rename.

The idea was sound. The implementation did not make the xattr part of the commit boundary.

The writer marshals a small sync.Meta blob and sets it as user.s3sync.meta just before the atomic publish:

lock := file + ".lock"
w, err := os.Create(lock)
...
defer os.Rename(lock, file)
defer xattr.Set(lock, "user.s3sync.meta", bs)
defer w.Close()
wz := gzip.NewWriter(w)
defer wz.Close()

The sync.Meta struct matches the subset of s3sync’s S3-facing metadata schema this writer needed:

type Meta struct {
    ContentType        string            `json:"content_type,omitempty"`
    ContentDisposition string            `json:"content_disposition,omitempty"`
    ContentLanguage    string            `json:"content_language,omitempty"`
    ContentEncoding    string            `json:"content_encoding,omitempty"`
    Metadata           map[string]string `json:"metadata,omitempty"`
}

Because Go’s defers run in reverse, the intended order is visible: finish the gzip stream, close the file, set the xattr, then rename. But every deferred return value is discarded. The rename still runs after a failed gzip close, file close, or xattr write. It even runs when the function returns early after a marshal or write error. The .lock name hides work in progress, but the writer still needs to check each result before publishing it.

The later shipper reads the attribute and routes on it:

lobby=$(xattr -p user.s3sync.meta $1/${s1[i]} | jq -er .metadata.lobby)
mkdir -p $2/$SNAPGS_SYNC_LOGBUCKET/$lobby/${p%/*}
mv $1/${s1[i]} $2/$SNAPGS_SYNC_LOGBUCKET/$lobby/$p

xattr -p prints the JSON blob, jq pulls out the route, and the file moves to that path. When the attribute exists, the bytes and routing metadata share one inode. That is the useful property: no sidecar filename has to stay aligned with the payload.

The correct sequence cannot be expressed as fire-and-forget defers. Finish the encoder, close the file, set the attribute, and check every result. Rename only after those operations succeed; on any failure, remove or quarantine the .lock file. If the filesystem or copy path cannot preserve xattrs, fail closed or choose a different carrier.

With those checks, the final name would expose both complete bytes and the metadata required to route them.