Stream PUT to aiofiles

Questions and Help
#1

I’ve been trying to use the sanic streaming branch (because of issues in the current streaming), but am having trouble with streaming a put request to aiofiles. Can someone tell me if I am doing this incorrectly or if this is an unresolved issue in the new branch?

The file gets created, but is always empty. I can’t see that any chunk is ever received when trying to write to aiofiles.

    @app.route("/<id:uuid>/<version:int>/<part:int>", methods=["PUT"],stream=True)
    async def put_version_part(request,id,version,part):
      versionfolder = Path(app.config['storage_folder']) / str(id) / str(version)
      tf = versionfolder / 'parts'
      filename = f"data.{part}"
      try:
        os.makedirs(tf)
      except FileExistsError:
        pass
      
      async with aiofiles.open(tf/filename, 'wb') as f:
        while True:
          chunk = await request.stream.read()
          if not chunk:
            break
          await f.write(chunk)
        f.close()
     
      return JSONResponse({'id':str(id),'version':version, 'part':part}, status=201)
#2

I took your code and ran it, and it seemed to work okay. Here is the whole file.

import asyncio
from pathlib import Path
import aiofiles
import os
from sanic import Sanic
from sanic.response import json


app = Sanic("streaming file test")


@app.route("/<id:uuid>/<version:int>/<part:int>", methods=["PUT"], stream=True)
async def put_version_part(request, id, version, part):
    versionfolder = Path(__file__).parent / str(id) / str(version)
    tf = versionfolder / "parts"
    filename = f"data.{part}"
    try:
        os.makedirs(tf)
    except FileExistsError:
        pass

    async with aiofiles.open(tf / filename, "wb") as f:
        while True:
            chunk = await request.stream.read()
            if not chunk:
                break
            await f.write(chunk)
    f.close()

    return json({"id": str(id), "version": version, "part": part}, status=201)


if __name__ == "__main__":
    app.config.REQUEST_MAX_SIZE = 10_000_000_000
    app.run(debug=True, port=9999)

image

Each of those files contain:

--910118fcf7477a3f82a241367af725fc
Content-Disposition: form-data; name="foobar"; filename="foobar"
Content-Type: application/octet-stream

SS5iq3PF6beJbaXIFvYilAJdX192MD6ZI4Z5pJvlL441oVB8FxHjCLbVXI9bcZGdELY0MNzEcJPMEz
...<MORE RANDOM STUFF>...
--910118fcf7477a3f82a241367af725fc--

I tested this on a three different branches including master and streaming.

This was executed from a terminal client using:

$ httpx http://localhost:9999/8950af86-d4cc-4958-966b-f947963d9b7c/1/3 -m PUT -f foobar ./foobar

And the response:

HTTP/1.1 201 Created
content-length: 66
content-type: application/json
connection: keep-alive
keep-alive: 5

{
    "id": "8950af86-d4cc-4958-966b-f947963d9b7c",
    "version": 1,
    "part": 2
}
#3

Thanks @ahopkins. I have it working against the master branch now, but I will see if I can figure out why it is failing for me in the streams branch. I just wanted to make sure that I wasn’t doing it incorrectly.

#4

Keep me posted. If there is something wrong on that branch, this will be really helpful to know.