Rewriting a packet in place, from inside the game
A closed-source Unity game advertised its own address to matchmaking as a length-prefixed record, 0x07, len, local:port|public:port. To route players over AWS Global Accelerator instead, I had to change what it said without changing anything about the packet around it. I used an LD_PRELOAD shim on send() to rewrite the record to the same length:
memset(&send_buf2[2], '0', len+1+len1);
memcpy(&send_buf2[2+len+1+len1-len2], snapgs_lobby_listen2, len2);
send_buf2[0] = 7;
send_buf2[1] = len+1+len1;Equal length is the whole design. send_buf1 is the original local|public record; send_buf2 is a zero-padded replacement of identical size, with the accelerator address right-aligned into it. Because the byte count never moves, the frame’s own length byte stays correct, and so do surrounding lengths derived only from that byte count. The surrounding frame does not need to be rebuilt.
For buffers at least 46 bytes long, the interposed send() scans for the exact original record—memchr to the first byte, memcmp to confirm the full record—and memcpys the replacement over it in place, then calls the real send resolved once via dlsym(RTLD_NEXT, "send"):
while ((data = memchr(data, send_buf1[0], last-data)) != NULL) {
if (memcmp(data, send_buf1, send_buf1_size) != 0) {
if (++data < last) continue;
break;
}
memcpy(data, send_buf2, send_buf1_size);
if ((data += send_buf1_size) >= last) break;
}
return send_next(fd, buf, size, flags);The in-place write crosses the first boundary. send() receives a const void *buf; the interposer casts that promise away and modifies storage owned by its caller. Read-only storage can fault, and a caller that reuses its buffer after send() observes rewritten bytes. A safe interposer copies into owned writable storage, rewrites the copy, and passes that copy to the real function.
The historical scan has two more boundary defects. Its entry guard compares size with sizeof(send_buf1), the fixed 46-byte array, rather than the runtime send_buf1_size; a shorter buffer can contain a shorter configured record and still be skipped. Once scanning starts, memchr may find the first marker byte near the tail, but the following memcmp reads send_buf1_size bytes without proving that many remain. A bounded version tests the entry size against send_buf1_size and requires last - data >= send_buf1_size before comparing. Without the latter guard, a tail marker can cause an overread.
_init has a fourth boundary defect before the scan begins. It checks each address independently against the same 9-to-21-byte range, then derives the replacement’s memcpy, separator, and colon offsets from all three lengths. Those individual checks do not prove the replacement fits the combined original field: accepted extrema can overwrite the two-byte header or underflow an unsigned separator index. A bounded constructor validates every derived offset and the relational fit before writing the replacement.
The Go supervisor turns the shim on only when a <binary>-preload.so file exists beside the executable and --listen carries all three parts—local,public,accel. Both conditions must hold; either alone is a no-op. It then passes the three addresses through the environment as SNAPGS_LOBBY_LISTEN, LISTEN1, LISTEN2, and the shim’s _init reads them, checking each independently against a min/max address length before building the buffers. That is the incomplete validation described above.
The small harness in hack/test.c includes the shim by using #include "preload.c", then replays captured packet bytes through the interposed send. It contains no assertion, comparison, or observation of either the call’s return value or its output. The hard-coded descriptor 1 is ordinarily stdout rather than a socket, so those sends can fail without changing the harness’s zero exit status. The shim is C with no clean seam for a harness, so the harness compiles the shim into itself.
The build and deployment were GNU/Linux-specific (LD_PRELOAD, _init, RTLD_NEXT), and brittle by nature: if the game ever changed its advertisement format the memcmp would simply stop matching and the rewrite would silently do nothing. It shipped with snap-gs and was retired with it. The padding-to-equal-length trick is the reusable idea, after the scan is bounded, the caller’s buffer is left alone, and the enclosing protocol’s integrity checks are known: when you cannot move the bytes around a value, make the new value the same size as the old one.