Skip to content

The two-offset contract

 2 min Reading a live JSONL file needs two positions: where reading stopped and the last complete record from which it can safely resume.
Published

Jul 26, 2026

A live writer can flush one JSONL record in several writes. A follower that treats each read as complete input may try to decode half a record, then resume in the middle of it. The reader needs to retain the unfinished tail or return to its beginning on the next pass.

The fix, from the Spice harness’s transcript reader, is to report two offsets for each pass:

Forward reads for live cursors go further and hold the unterminated tail back entirely:

if hold_partial_end and not raw.endswith(b"\n"):
    # A writer mid-flush leaves its last line unterminated. Resuming
    # after that prefix would decode it as garbage and then decode the
    # rest of the same line as more garbage, so the complete record
    # would never be seen. Leave it for the pass that finds it whole.
    handle.seek(offset)
    break

Two more rules complete the contract. Replacement is detected by filesystem identity—device and inode—not by size, because a rotated-and-recreated file can easily be larger than the old cursor position and a size check would happily read garbage from the middle of an unrelated file. Replacement (identity changed) and truncation (cursor offset beyond current file size) both reset the cursor to zero rather than guessing.

Spice uses this for live Claude and Codex transcripts. Separating bytes observed from complete records lets a read show the unfinished tail while keeping the next live read at a valid record boundary.