New worker manager breaking changes help

I just upgraded Sanic to the latest version (Sanic v22.9.1), and I get errors trying to run the following:

app = Sanic("Hiking-Game-Backend")

def main():
    ...
    ...
    app.run(host=cfg["web"]["host"], port=cfg["web"]["port"],
            dev=dev, access_log=True)


if __name__ == "__main__":
    main()

Now when dev is True, I get the following error:

App instantiation must occur outside if __name__ == '__main__' block or by using an AppLoader.

I looked at the breaking changes of Sanic, but I still could not get it to work.
How do I fix this?

I think there must be something else going on in your code that is impactful other than your bare code. Using the below, it works with or without dev

from sanic import Request, Sanic, json

app = Sanic("Hiking-Game-Backend")


@app.get("/")
async def handler(request: Request):
    return json({"foo": "bar"})


def main():
    app.run(host="127.0.0.1", port=9999, dev=False, access_log=True)


if __name__ == "__main__":
    main()

Now, if you put app = Sanic(...) inside of main, then you will certainly get that error.

Some information here: Dynamic Applications | Sanic Framework

I indeed have app = Sanic() outside of main.

For clarity, here is my full server:

from tortoise.contrib.sanic import register_tortoise
from sanic import Sanic
import toml
import os
import argparse
from sanic.log import logger

from .api.user import User
from .api.location import Location, AdminLocation
from .api.checkin import CheckIn
from .api.score import Score
from .api.settings import Settings
from .api.qr import QR
from .api.auth import auth
from . import config

from .cors import add_cors_headers
from .options import setup_options


app = Sanic("Hiking-Game-Backend")


def _shutdown(code=0):
    logger.info("Shutdown server")
    exit(code)


def _load_config(path: str):
    # Init config file if required
    if not os.path.isfile(path):
        config.init(path)
        logger.info(f"Initialized configuration file {path}.")
        logger.info("Please configure and restart the server.")
        _shutdown(0)

    # Load configuration
    try:
        cfg = toml.load(path)
    except Exception:
        logger.exception("Failed to load configuration. Check configuration"
                         f"file {path}")
        _shutdown(1)

    logger.info(f"Successfully loaded configuration from {path}.")

    return cfg


def _parse_args():
    parser = argparse.ArgumentParser(
        description="Startup an instance of a consumption dashboard"
    )

    parser.add_argument(
        "-m",
        "--migrate",
        action="store_true",
        default=False,
        help="Migrates or initializes the database.",
    )

    parser.add_argument(
        "-c",
        "--config",
        type=str,
        default=config.PATH,
        help="Path to a configuration file",
    )

    return parser.parse_args()


def main():
    args = _parse_args()
    cfg = _load_config(args.config)

    # Begin with server startup sequence
    logger.info("Starting server...")

    # Connect to database
    logger.info(f"Connect to database {cfg['database']['name']}" +
                f"({cfg['database']['host']}:{cfg['database']['port']})")
    if args.migrate:
        logger.info("Database will be initialized after connecting to it.")

    db = cfg["database"]

    register_tortoise(
        app,
        db_url=f"postgres://{db['user']}:{db['password']}@{db['host']}:" +
        f"{db['port']}/{db['name']}",
        modules={"models": ["backend.database"]},
        generate_schemas=args.migrate
    )

    logger.info("Successfully connected to database.")

    app.blueprint(auth)
    app.add_route(User.as_view(), "/api/user/<user_id>/")
    app.add_route(Location.as_view(), "/api/location/")
    app.add_route(AdminLocation.as_view(), "/api/adminlocation/")
    app.add_route(CheckIn.as_view(), "/api/checkin/<uuid>")
    app.add_route(Score.as_view(), "/api/score/")
    app.add_route(Settings.as_view(), "/api/settings/")
    app.add_route(QR.as_view(), "/api/qr/")
    # Make config fields accessible from all endpoints
    app.config.cfg = cfg
    dev = cfg["general"]["dev"]
    if dev:
        logger.info("Adding CORS headers.")
        # Add OPTIONS handlers to any route that is missing it
        app.register_listener(setup_options, "before_server_start")
        # Fill in CORS headers
        app.register_middleware(add_cors_headers, "response")
    app.run(host=cfg["web"]["host"], port=cfg["web"]["port"],
            dev=True, access_log=True)


if __name__ == "__main__":
    main()

This gives the error mentioned earlier, and I am not sure about the culprit.

Anything that you want to apply to your application instance (attaching handlers, etc) should be run on each worker process. In your case, that applies to most what is in main. You should either move to a factory, or use the pattern showed above (which tbh, looks overkill for you). Perhaps it might be easiest to just run with the cli

sanic path.to.server:main --factory 

sanic backend.__main__:main --factory

and

sanic backend:main --factory

Do not seem to work for me unfortunately.

I used to run it like this:

python -m backend

This is the folder structure:

backend
├── __init__.py
├── __main__.py
├── api
│   ├── __init__.py
│   ├── auth.py
│   ├── checkin.py
│   ├── location.py
│   ├── qr.py
│   ├── score.py
│   ├── settings.py
│   ├── test.py
│   └── user.py
├── config.py
├── cors.py
├── database.py
├── options.py
├── test.py
├── tests
│   ├── __init__.py
│   ├── checkin_test.py
│   ├── conftest.py
│   ├── location_test.py
│   ├── score_test.py
│   └── utils.py
└── utils.py

Where __main__.py contains the code mentioned earlier.

Do you have a clue?

python -m sanic backend:main --factory 

Unfortunately, gives me:

$ python -m sanic backend:main --factory
Traceback (most recent call last):
  File "/home/pim/.cache/pypoetry/virtualenvs/backend-FpiVwlvq-py3.10/lib/python3.10/site-packages/sanic/worker/loader.py", line 73, in load
    app = app(self.args)
TypeError: 'NoneType' object is not callable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/pim/.cache/pypoetry/virtualenvs/backend-FpiVwlvq-py3.10/lib/python3.10/site-packages/sanic/__main__.py", line 16, in <module>
    main()
  File "/home/pim/.cache/pypoetry/virtualenvs/backend-FpiVwlvq-py3.10/lib/python3.10/site-packages/sanic/__main__.py", line 12, in main
    cli.run(args)
  File "/home/pim/.cache/pypoetry/virtualenvs/backend-FpiVwlvq-py3.10/lib/python3.10/site-packages/sanic/cli/app.py", line 96, in run
    app = self._get_app(app_loader)
  File "/home/pim/.cache/pypoetry/virtualenvs/backend-FpiVwlvq-py3.10/lib/python3.10/site-packages/sanic/cli/app.py", line 153, in _get_app
    app = app_loader.load()
  File "/home/pim/.cache/pypoetry/virtualenvs/backend-FpiVwlvq-py3.10/lib/python3.10/site-packages/sanic/worker/loader.py", line 75, in load
    app = app()
TypeError: 'NoneType' object is not callable

The problem is the factory needs a bit tweaking. Mind if I make some changes for you?

Please, thanks in advance!

# Commenting out this since it is not in my env
# from tortoise.contrib.sanic import register_tortoise
import argparse
import os
from pathlib import Path

import toml
from sanic import Sanic
from sanic.log import logger
from sanic.utils import str_to_bool

# Commenting out this since it is not in my env
# from .api.user import User
# from .api.location import Location, AdminLocation
# from .api.checkin import CheckIn
# from .api.score import Score
# from .api.settings import Settings
# from .api.qr import QR
# from .api.auth import auth
# from . import config

# from .cors import add_cors_headers
# from .options import setup_options

# Moving this into the factory
# app = Sanic("Hiking-Game-Backend")


def _shutdown(code=0):
    logger.info("Shutdown server")
    exit(code)


def _load_config(path: Path):
    # Changes made for POC

    # Commenting out since I don't have these objects
    # # Init config file if required
    # if not os.path.isfile(path):
    #     config.init(path)
    #     logger.info(f"Initialized configuration file {path}.")
    #     logger.info("Please configure and restart the server.")
    #     _shutdown(0)

    # # Load configuration
    # try:
    #     cfg = toml.load(path)
    # except Exception:
    #     logger.exception("Failed to load configuration. Check configuration"
    #                      f"file {path}")
    #     _shutdown(1)

    logger.info(f"Successfully loaded configuration from {path}.")

    # return cfg
    return {
        "general": {
            "dev": True,
        },
        "database": {
            "host": "",
            "name": "",
            "port": "",
        },
    }


def _hydrate(args):
    parser = argparse.ArgumentParser()
    parser.add_argument(
        "-m",
        "--migrate",
        # Using `str_to_bool` instead of action=store_true
        type=str_to_bool,
        default=False,
    )
    parser.add_argument(
        "-c",
        "--config",
        type=str,
        # default=config.PATH,
        default=Path(),
    )
    known, _ = parser.parse_known_args(
        args=[f"--{k}={v}" for k, v in args._get_kwargs()]
    )
    for key, value in known._get_kwargs():
        setattr(args, key, value)


def main(args):
    # This line is newly moved from global
    app = Sanic("Hiking-Game-Backend")

    # Instead of using argparse, we can just accept arbitrary arguments
    # passed to the CLI. The only downside here, is that it does not accept
    # things like store_true. So, we will fake it with _hydrate
    _hydrate(args)

    cfg = _load_config(args.config)

    # Begin with server startup sequence
    logger.info("Starting server...")

    # Connect to database
    logger.info(
        f"Connect to database {cfg['database']['name']}"
        + f"({cfg['database']['host']}:{cfg['database']['port']})"
    )
    if args.migrate:
        logger.info("Database will be initialized after connecting to it.")

    # Removing this since it is not in my env
    # db = cfg["database"]

    # register_tortoise(
    #     app,
    #     db_url=f"postgres://{db['user']}:{db['password']}@{db['host']}:" +
    #     f"{db['port']}/{db['name']}",
    #     modules={"models": ["backend.database"]},
    #     generate_schemas=args.migrate
    # )

    logger.info("Successfully connected to database.")

    # Commenting out your routes and replacing with one generic for POC
    # app.blueprint(auth)
    # app.add_route(User.as_view(), "/api/user/<user_id>/")
    # app.add_route(Location.as_view(), "/api/location/")
    # app.add_route(AdminLocation.as_view(), "/api/adminlocation/")
    # app.add_route(CheckIn.as_view(), "/api/checkin/<uuid>")
    # app.add_route(Score.as_view(), "/api/score/")
    # app.add_route(Settings.as_view(), "/api/settings/")
    # app.add_route(QR.as_view(), "/api/qr/")
    app.get("/")(lambda _: ...)

    # Make config fields accessible from all endpoints
    app.config.cfg = cfg
    dev = cfg["general"]["dev"]

    # Commenting out since I don't have these funcs
    if dev:
        logger.info("Adding CORS headers.")
    #     # Add OPTIONS handlers to any route that is missing it
    #     app.register_listener(setup_options, "before_server_start")
    #     # Fill in CORS headers
    #     app.register_middleware(add_cors_headers, "response")

    # REMOVE THIS
    # Instead, you will add them as CLI arguments
    # app.run(
    #     host=cfg["web"]["host"],
    #     port=cfg["web"]["port"],
    #     dev=True,
    #     access_log=True,
    # )

    return app

So, I messed around with it a bit. To make it work I moved app instantiation into the factory, removed app.run and then returned app. That is essentially what you will need.

The bigger change though is with your use of argparse. Sanic CLI will accept arbitrary arguments passed to it. But, unfortunately it does not give you the complete power of argparse to declare their types. I was trying to take advantage of that, not sure if this solution meets your needs or not.

You could run something like this:

python -m sanic backend.__main__:main \
    --factory \
    --config=/path/to/somewhere \
    --migrate=y \
    --port=9876 \
    --dev

Works! Definitely, could not have come up with this, thanks a lot!

Sounds a new blog article and some more help documentation are in order.