How to use redis pubsub with sanic websocket?

Questions and Help
#1

I’m trying to create a new websocket app server with sanic. It should be 2+ server running sanic and get message pubsub via redis.

Can someone guide me how to compose sanic, asyncio things, websockets and aioredis ?

Somethings that come to my mind now:

  • For each ws request, create a consumer handler & a producer handler for that ws, and then asyncio.await them
  • In the consumer handler, we subscribe to the redis channel using aioredis
  • In the producer handler, we publish the message to redis channel to broadcast and send a message to the ws instance, too ?
#2

How about something like this?

Basically… this example creates a somewhat simple echo feed on a given channel. Every socket that is opened on /foobar will register on a single Feed object. The objective here is to create only a single object that would then subscribe to the pubsub.

That feed object (when first created) kicks off the receiver task that would sort of be like a producer task. Its job is to listen to the pubsub, and then when it receives a message, fire it off to all of the registered clients.

Each client, when it connects to a feed, also has a received task created. This is like the consumer task. Its job is to listen to incoming messages from the client, and when it receives something, push it off to the pubsub.

Then, when clients unsubscribe, we want to make sure that we clean things up. Therefore once there are no clients left, the feed needs to unsubscribe and destroy itself.

This is a fairly basic example that can be made much more complex. I hope this is what you had in mind.

1 Like
How to persist websocket connection in sanic
#3

Thanks. It really helps :blush:

I created two server instance running at different port and some client, the msg is pubsub to each client smoothly.

Awesome. I’ll work on a chat pubsub example.

#4

:muscle: Awesome. Glad to hear it and I look forward to seeing the example.

Now, if only we had a good place to start collecting this stuff… :thinking:

1 Like
#5

:joy: If there only were some such place…

#6

I made a PR to the awesome-sanic list. To organize it, but also to add some examples there. I think the answer is that we really need to put some effort into the frontend.

1 Like
#7

@ahopkins:

I made a copy of your gist to my sanic websocket template repo, the chat example is being worked on.

Recently, I notice that you put a name attr on class Feed (https://github.com/greentornado/sanic-websocket-boilerplate/blob/master/server/websocket_feed/main.py#L75), so it mean every feed will share the same name ?

#8

Yes. The idea was that a feed (maybe a better term to use would be “channel”, although a single feed in my usage attaches to multiple pubsub channels) is a single connection point to the pubsub.

Multiple clients register to a feed. This reduces the potential number of subscriptions to the pubsub. Then, when a feed receives an update from the pubsub, it sends it out to as many clients as are registered on the feed.

The name is used to keep track of them in the FeedCache.

#9

So the line at https://github.com/greentornado/sanic-websocket-boilerplate/blob/master/server/websocket_feed/main.py#L75 is just for type hinting ? (name: str)

I’m new to class attr lol, since I thought it overwrited for each new feed name created

#10

yes. totally not necessary.

#11

Can you clear me for the code at:

https://github.com/greentornado/sanic-websocket-boilerplate/blob/master/server/websocket_feed/main.py#L61-L62 => for sending message to the feed pubsub ?

and https://github.com/greentornado/sanic-websocket-boilerplate/blob/master/server/websocket_feed/main.py#L134-L140 => get msg from feed pubsub and send it back to each client ?

So each websocket msg coming for client side will go to the its client instance first, then got pubsubed to the Feed => then the Feed will send to all other client ?

#12

Yes, this is the basic idea.

When a Client receives a message, it “broadcasts” it out by pushing it to the pubsub channel. That channel is being listened to by the Feed. When the feed receives that message, then it loops thru the clients and pushes the message thru the existing socket.

This is a pretty simple implementation and doesn’t include any sort of “meta” programming in the message that might include dispatching an action, ignoring certain clients, etc.

1 Like