Transport closed and exception experienced

I kept on getting this error on my server but I can’t figure out what it means:

ERROR sanic.root:556 | Transport closed @ ('127.0.0.1', 49784) and exception experienced during error handling

It came from the following source:

What condition will trigger this error? What exception was experienced?

@smlbiobot The way that Sanic works is by leveraging some internals in the asyncio library. When a request is made, Sanic will write data to the transport layer that will asynchronously check the layer to see if there is data to push to the client. If yes, it sends the data.

However, in this case, data has been queued up and ready to for the outgoing transfer, but it cannot be sent because the transport layer (think of it as the connection between the two servers) is not in place, or there was some unhandled exception in your code.

Do you have any more information in your logs?

Thanks for your reply.

There are not any additional logs surrounding this which is what prompted me to post here. I have never seen it in dev environment and have only seen it on the production server. That’s why I posted it here!

Then I found out about Locust and decided to try to run it against my dev environment. And then I saw the error in dev when I cranked up concurrent users to about 2k+. But there are no additional errors either — just this message over and over.

It’s very hard for me to debug because I am not sure which part of the code is causing it / which part of the application cannot handle the load. This error is not specific to a particular endpoint on the app.

What would you suggest that I do?

I experienced the same problem a few days ago with Sanic 19.3.1 and Python 3.6.7, so far it had happened only one time. Most unfortunate was that this occurred in production, in a weekend. The server was started using app.create_server, listening on HTTPS. The server was unable to handle any other requests after the error, returning status 500 ECONNRESET.

I wonder if it would be a better behavior to raise an exception and let the server die. This way at least the process manager can restart the server instead of it being stuck.

With the logging library set at logging.NOTSET, the following was the only output I got. There was no second line of logger.debug message. 02:58:52 was the time of the last-handled user request. The error handler function decorated with @app.exception([SanicException, Exception]) was not called. There were no further logs thereafter.

… [Other normal logs]
2019-07-13 02:58:52,237 [MainThread ] [INFO ] Response sent. # Logged from @app.middleware(‘response’)
2019-07-13 02:58:52,238 [MainThread ] [INFO ]
2019-07-13 02:58:52,238 [MainThread ] [INFO ] PID: 34 | Memory usage: 739.6796875MB.
2019-07-13 02:58:52,238 [MainThread ] [INFO ]
2019-07-13 02:58:52,239 [MainThread ] [INFO ] ====================================================
2019-07-13 02:58:52,239 [MainThread ] [INFO ]
2019-07-13 02:58:52,239 [MainThread ] [INFO ]
2019-07-13 03:01:34,002 [MainThread ] [ERROR] Transport closed @ (IP, NUMBER) and exception experienced during error handling
2019-07-13 03:01:34,009 [MainThread ] [INFO ]