Let the client carry the bytes

 1 min An S3 upload proxy became unnecessary once the service could authorize an upload by returning a presigned URL.

I was trying to stream uploads through a small service to S3 without buffering each entire object. The proxy read a chunk from the client and wrote it to the outgoing connection. I asked ExAws to support a payload digest supplied by the client, because the proxy did not hold the complete body to hash before starting the request.

In the issue discussion, Ben Wilson asked why I did not give the client a presigned upload URL instead. I had missed that option.

The interface could be:

client → service: authorize this upload
service → client: presigned URL
client → S3: upload the object

The service still decided which upload to authorize. It no longer needed to carry the authorized object’s bytes through its own memory and sockets. The work moved to the client producing those bytes and the destination storing them.

I tried ExAws.S3.presigned_url for a PUT and confirmed that the existing API did what I needed. My follow-up said I would take that path and invited the maintainer to close the issue unless he still wanted the proposed stream-signing behavior.

The original chunked proxy was a reasonable thing to investigate, but it was solving a transport problem created by my choice of interface. I wanted the service to authorize storage; I had assumed that also required it to relay the data.