Home » Python » python – Tensorflow getting stuck on prediction when deploying with gunicorn-Exceptionshub

python – Tensorflow getting stuck on prediction when deploying with gunicorn-Exceptionshub

Posted by: admin February 24, 2020 Leave a comment

Questions:

I’ve built myself a flask web server, and am deploying it with Gunicorn. It works perfectly well with the default flask app.run(), but this isn’t meant for production, so up until now I’ve had my flask app running on gunicorn.

However, since adding my TensorFlow program to my site, when I go to run a prediction it runs forever, I have timeout set to like 5 minutes. I tested with a print statement before and a print statement after my predict. Second print never shows up.

Any ideas?

How to&Answers:

If you are running heavy calculations inside the route, you better change your program design.

At your route start the job on some other thread or process and return the status of the process start and ID to query its progress.

Then you call query with polling [or web sockets] your heavy task with your ID, and return its status till the process is done and you can return the results