Broken Pipe error with systemd

Hi,

I’m currently migrating my Sanic app from a supervisor-managed instance to systemd. I’m running the latest release (22.9.1).

Sanic starts fine, but when I run systemctl stop sanic_server I get an exception (journalctl -u server-proc).

Nov 10 10:06:35 api-server systemd[1]: Stopping API Service...
Nov 10 10:06:35 api-server sanic[63887]: [2022-11-10 10:06:35 +0100] [63887] [INFO] Stopping worker [63887]
Nov 10 10:06:35 api-server sanic[63823]: [2022-11-10 10:06:35,091] [    INFO] Received signal SIGTERM. Shutting down. (manager.py:162)
Nov 10 10:06:35 api-server sanic[63887]: [2022-11-10 10:06:35,107] [    INFO] Stopping worker [63887] (runners.py:157)
Nov 10 10:06:35 api-server sanic[63823]: [2022-11-10 10:06:35 +0100] [63823] [ERROR] Experienced exception while trying to serve
Nov 10 10:06:35 api-server sanic[63823]: Traceback (most recent call last):
Nov 10 10:06:35 api-server sanic[63823]:   File "/srv/api-server/webapps/sanic_server/.venv/lib/python3.9/site-packages/sanic/mixins/startup.py>
Nov 10 10:06:35 api-server sanic[63823]:     manager.run()
Nov 10 10:06:35 api-server sanic[63823]:   File "/srv/api-server/webapps/sanic_server/.venv/lib/python3.9/site-packages/sanic/worker/manager.py>
Nov 10 10:06:35 api-server sanic[63823]:     self.monitor()
Nov 10 10:06:35 api-server sanic[63823]:   File "/srv/api-server/webapps/sanic_server/.venv/lib/python3.9/site-packages/sanic/worker/manager.py>
Nov 10 10:06:35 api-server sanic[63823]:     if self.monitor_subscriber.poll(0.1):
Nov 10 10:06:35 api-server sanic[63823]:   File "/usr/lib/python3.9/multiprocessing/connection.py", line 262, in poll
Nov 10 10:06:35 api-server sanic[63823]:     return self._poll(timeout)
Nov 10 10:06:35 api-server sanic[63823]:   File "/usr/lib/python3.9/multiprocessing/connection.py", line 429, in _poll
Nov 10 10:06:35 api-server sanic[63823]:     r = wait([self], timeout)
Nov 10 10:06:35 api-server sanic[63823]:   File "/usr/lib/python3.9/multiprocessing/connection.py", line 936, in wait
Nov 10 10:06:35 api-server sanic[63823]:     ready = selector.select(timeout)
Nov 10 10:06:35 api-server sanic[63823]:   File "/usr/lib/python3.9/selectors.py", line 416, in select
Nov 10 10:06:35 api-server sanic[63823]:     fd_event_list = self._selector.poll(timeout)
Nov 10 10:06:35 api-server sanic[63823]:   File "/srv/api-server/webapps/sanic_server/.venv/lib/python3.9/site-packages/sanic/worker/manager.py>
Nov 10 10:06:35 api-server sanic[63823]:     self.shutdown()
Nov 10 10:06:35 api-server sanic[63823]:   File "/srv/api-server/webapps/sanic_server/.venv/lib/python3.9/site-packages/sanic/worker/manager.py>
Nov 10 10:06:35 api-server sanic[63823]:     process.terminate()
Nov 10 10:06:35 api-server sanic[63823]:   File "/srv/api-server/webapps/sanic_server/.venv/lib/python3.9/site-packages/sanic/worker/process.py>
Nov 10 10:06:35 api-server sanic[63823]:     self.set_state(ProcessState.TERMINATED, force=True)
Nov 10 10:06:35 api-server sanic[63823]:   File "/srv/api-server/webapps/sanic_server/.venv/lib/python3.9/site-packages/sanic/worker/process.py>
Nov 10 10:06:35 api-server sanic[63823]:     **self.worker_state[self.name],
Nov 10 10:06:35 api-server sanic[63823]:   File "<string>", line 2, in __getitem__
Nov 10 10:06:35 api-server sanic[63823]:   File "/usr/lib/python3.9/multiprocessing/managers.py", line 808, in _callmethod
Nov 10 10:06:35 api-server sanic[63823]:     conn.send((self._id, methodname, args, kwds))
Nov 10 10:06:35 api-server sanic[63823]:   File "/usr/lib/python3.9/multiprocessing/connection.py", line 211, in send
Nov 10 10:06:35 api-server sanic[63823]:     self._send_bytes(_ForkingPickler.dumps(obj))
Nov 10 10:06:35 api-server sanic[63823]:   File "/usr/lib/python3.9/multiprocessing/connection.py", line 416, in _send_bytes
Nov 10 10:06:35 api-server sanic[63823]:     self._send(header + buf)
Nov 10 10:06:35 api-server sanic[63823]:   File "/usr/lib/python3.9/multiprocessing/connection.py", line 373, in _send
Nov 10 10:06:35 api-server sanic[63823]:     n = write(self._handle, buf)
Nov 10 10:06:35 api-server sanic[63823]: BrokenPipeError: [Errno 32] Broken pipe

My systemd unit file looks like this:

[Unit]
Description=API Service

[Service]
User=svc-user
Group=svc-user
Restart=always
RestartSec=5
TimeoutSec=5
WorkingDirectory=/srv/api-server/webapps/sanic_server/
ExecStart=/srv/api-server/webapps/sanic_server/.venv/bin/sanic sanic_server.server.app --host [ipv6-address-redacted] --port [portnum-redacted] --fast --no-access-logs

[Install]
WantedBy=multi-user.target

The service seems to start and stop OK, but the exceptions are probably not intended. My current theory is that systemd is not sending a “nice” kill signal to the service instances, but I’m not sure if that’s correct?

That’s what it sounds like. I don’t know enough about it though.