built-in kafka support

[aryaniyaps]

I was thinking if channels could support kafka built-in. (The kafka-python client installation is optional, but we'll warn if any KafkaWebsocketConsumer is used without the package). This way, projects can start using kafka immediately and will have horizontal scalability. I was thinking that the kafka consumers and producers could be modified by these settings.

# settings.py
CHANNELS_KAFKA = {
    'cluster': 'localhost:4327',
    'consumers': {},
}

class KafkaWebsocketConsumer(WebsocketConsumer):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        try:
           # kafka support is optional
           import kafka-python
        except ImportError:
            raise AssertionError(
                'kafka-python is not installed.'
                'kafka websocket consumers will not be supported.'
            )

this way, projects can choose between the groups API or kafka as the message distribution system.
I am willing to do every bit of this integration from consumers, updating the documentation and everything.

I just wanted to ask, is this worth doing?

[carltongibson]

Happy to link to a third party package, but not a consumer I'd want to support in channels itself.

[aryaniyaps]

thanks for the quick reply, I might work on this later, thanks!