Hi, I am currently running rasa using gunicorn(4 workers) and Nginx. As the number of concurrent users increases request failures start increasing with the message (502 Bad Gateway error).
I have also tried out the Nginx load balancer with 2 instances of 8GB RAM, 4vCPU ubuntu configuration but still getting the same error. Please help me to build a scalable application that is capable of handling atleast 50K request.
I would suggest running Sanic with the internal server. If you are looking for a complete end-to-end deployment strategy, how about putting your app in a container (https://hub.docker.com/r/sanicframework/sanic) and running it inside a Kubernetes cluster that can be scaled horizontally and placed behind a load balancer. Whether you want another nginx balancer inside your cluster would depend on your use case.
Gunicorn is a WSGI server. That means that it is incapable of handling code asynchronously. So, when you use that as your server (as opposed to Sanic’s own server, or an ASGI server), you are giving up on some of the performance benefits of asyncio. Depeneding upon your app needs and what your endpoints are doing, this may or may not matter.