I am having some issues running a Tensorflow model with Sanic even though it works fine with Flask. I think it has something to do with the async nature of Sanic? The specific error I get is:
2019-10-22 18:05:39.062833: E tensorflow/core/grappler/clusters/utils.cc:87] Failed to get device properties, error code: 3 2019-10-22 18:05:39.731232: F tensorflow/stream_executor/cuda/cuda_driver.cc:175] Check failed: err == cudaSuccess || err == cudaErrorInvalidValue Unexpected CUDA error: initialization error
and I feel it’s at least partially related to https://github.com/pytorch/pytorch/issues/2517
One other thing is that the Sanic app works if I set the number of workers to 1, but this isn’t ideal for me and I’d like to be able to use more than one worker. I am using Sanic’s inbuilt webserver.
I know it’s a pretty general description of the issue, but I guess I just need some ideas to try. I think maybe isolating the code that serves as the entry point for Tensorflow and sharing that resource (i.e., Tensorflow model) between workers/processes?