Sanic-openapi overhaul proposal

[vltr]

Hello everyone!

A while ago, during one of our chat sessions, I talked to @ahopkins regarding the current state of sanic-openapi, and, mostly, why is it somewhat “not attractive” to a lot of developers and how we could proceed to make it something actually usable.

First, the pros about sanic-openapi:

• it’s built for Sanic.

I can’t think of anything else despite that, since even I (@vltr personally speaking about how I love things to be documented and correctly structured) don’t use it in my own projects and even tried to create my own solution, which has a big pitfall (which I’ll put on the cons later).

Now, what I believe it’s the biggest con of sanic-openapi is:

• it introduces another layer (which is only descriptive) of your objects structure and it doesn’t do anything regarding to validation or data-to-object transformation, which leaves the developer to recreate these structures again (using any other library, such as marshmallow, schematics, pydantic, you name it).

I believe this is the big “hold-back” for a lot of people to actually use (and perhaps embrace) the sanic-openapi project. I know it also has some other flaws, but we can work them as well.

My first proposal to get around that, which I linked above, was to use my own ODM library (called middle), which still needs a lot of work to be considered something usable / stable in the future, but it’s in pure Python and very fast (I might dare say it’s even faster than pydantic, but that’s not the discussion to take place in here).

So, in my first proposal, I introduced another problem: not being agnostic with the serialization / ODM library

Now, I believe that what will make sanic-openapi to take a step further into being something really useful is the capability to use the serialization library of your own choosing, being marshmallow, pydantic, attrs, dataclasses, whatever you want - as long as there’s a proper API (into sanic-openapi) where you can “choose” your preferred library and making all the adaptations to correctly handle these 3rd party libraries, either to process them and output their schema into the OpenAPI format or actually process input / output, validating them and writing the end JSON - all of that being highly configurable on a lot of directions (which I have in mind right now).

With the new Sanic router and 21.X versions, I believe this would be an awesome addition to our stack.

For now, I’ll only leave the high-level details I have in mind so I can hear your thoughts. If this goes ahead, we can start thinking on the specifics

Cheers,
Richard.

[vltr]

GitHub follow up about this in here.

[ashleysommer]

@vltr

it introduces another layer (which is only descriptive) of your objects structure and it doesn’t do anything regarding to validation or data-to-object transformation, which leaves the developer to recreate these structures again

Yep, I agree this is the biggest problem with Sanic-OpenAPI currently.

@vltr

the capability to use the serialization library of your own choosing, being marshmallow, pydantic, attrs, dataclasses, whatever you want - as long as there’s a proper API (into sanic-openapi) where you can “choose” your preferred library

This sounds like a cool feature, but perhaps maybe too ambitious or too complex to implement. Which would it default to? Which would come bundled when you install Sanic-OpenAPI?

[vltr]

@ashleysommer

This sounds like a cool feature, but perhaps maybe too ambitious or too complex to implement. Which would it default to? Which would come bundled when you install Sanic-OpenAPI?

I agree it´s ambitious, but not too hard to do if we plan the API design with some libraries in mind, and perhaps have a separate implementation for each one of them - where we could provide initial snippets (and we ought to prove our point) and the community could keep it up do date - like a “driver” for each ODM.

By default, I don´t think we would choose a library. I´m in favour of letting the developer to choose.

I hope this makes sense

[ahopkins]

@vltr

By default, I don´t think we would choose a library. I´m in favour of letting the developer to choose.

The reason I have never been involved in the inner-workings of this package and only acted as a rubber stamp is becasue… I hate that it forces an additional modeling layer.

Building from the ground up is probably the right approach. I would suggest having the goal to making it released by the end of the Summer (with 21.9), and “functional” by 21.12.

My initial thoughts (and yes, it will be repetitive of what has been said):

1. Framework agnostic. Either we provide an interface for “plugging” in a framework like attrs, pydantic, marshmallow, etc, or we build them in:

   pip install sanic-new-tool[attrs] 
   

   • Sanic takes effort to be unopionated. While plugins are the rightful realm of adding this sort of decision making, object modeling is such a project specific issue that we shouldn’t make that choice. Honestly, I think that it should be easily extendable to make ORMs like SQLAlchemy, Tortoise, and GINO work the same way. FTW.
   • If we need to have a fallback/default: dataclasses or TypedDict.
2. Validation. It should have an interface for handling query param and request body (form and json) validation.
3. OAS3.
4. Backwards Support. We can keep both projects going for a time. Eventually we can deprecate sanic-openapi.
5. Docstring support. I don’t think we need to try and pull data from the docstring, parse it, and convert it to openapi data, but it should be included as descriptions.
6. Swagger UI and Redoc. I could be wrong, but these seem to be the most widely used client side libs. It should be super simple and configurable just to turn one (or both) on without any additional setup.

[vltr]

@ahopkins

Building from the ground up is probably the right approach. I would suggest having the goal to making it released by the end of the Summer (with 21.9), and “functional” by 21.12.

I think that´s a good starting point, since I think we talked in private about some of the new signaling stuff that will (or can) eventually land on Sanic that would be a huge win for what I have in mind for this new API - my head is a bit clouded yet on some fronts but I already have a good idea on what direction I would start this new lib

@ahopkins

Sanic takes effort to be unopionated. While plugins are the rightful realm of adding this sort of decision making, object modeling is such a project specific issue that we shouldn’t make that choice. Honestly, I think that it should be easily extendable to make ORMs like SQLAlchemy, Tortoise, and GINO work the same way. FTW.

You kind of took the core abstract stuff of my mind, TBH. With some effort, with this new lib, you would be able to make a completely transparent plugin between your ORM of choosing to the JSON body you received. Like I explained above, it´s not a trivial thing to try and make an example code of this (right now) but I´m certain it would end up creating a lot of new possibilities

@ahopkins

Validation. It should have an interface for handling query param and request body (form and json) validation.

That´s where I believe my mind is not settled yet on how to do it. Should we do the validation by our own means (without the developer having to choose any framework) OR should we rely on that framework of choosing to do it? The second option is what I have in mind right now, but I´m open to discuss that.

@ahopkins

OAS3.

Not only that … As I have in mind right now, someone could “extend” how the inner API works to produce OAS2 or even make it as an intermediary tool to write out Protocol Buffers definitions … I mean, it´s not the goal but probably possible (in my mind right now).

@ahopkins

Docstring support. I don’t think we need to try and pull data from the docstring, parse it, and convert it to openapi data, but it should be included as descriptions.

That would be an interesting case of (draft example):

@api.doc(sanic_openapi.FROM_DOCSTRING)
async def my_method(request):
    """This is what will actually be on this API call documentation"""
    ...

We could (and should) even have different approaches to docstring, it´s just a matter of “settling” to a convention, like PEP-257, or even others if a plugin for that is provided

@ahopkins

Swagger UI and Redoc. I could be wrong, but these seem to be the most widely used client side libs. It should be super simple and configurable just to turn one (or both) on without any additional setup.

Perhaps this?

pip install sanic-new-tool[redoc|swagger]

I think that´s all for now. Let me know your thoughts - and I mean, from all of you

[vltr]

Dropping this idea in favor of sanic-ext.