This post is written by Viraj Anchan, Full Stack Engineer at Haptik.
Haptik is a chat application that connects users to their digital personal assistants in real time. When chatting with your personal assistant, you want as little delay as possible in order to get things done quickly.
When I joined Haptik after graduating from college in May 2015, my first technical challenge was learning the messaging architecture. In July 2015, I suggested to our CTO that we shift from XMPP to MQTT. Our mobile apps (Android & iOS) and Athena (web chat tool used by our assistants) used XMPP for real-time messaging. We were facing a lot of issues with XMPP. To point out a few, XMPP lacks built-in Quality of Service (QoS). QoS is an agreement between the sender and receiver of a message regarding the guarantees of delivering a message. Since XMPP lacked the built-in QoS, we had to build our own custom solution to ensure message delivery. Along with that XMPP session is one big long XML document and every client has to use an event-driven XML parser. All in all, XMPP was proving to be a lot of overhead and maintenance for us and we needed a better more scalable solution. In January 2016, we finally decided to shift from XMPP to MQTT.
MQTT is a lightweight machine-to-machine/”Internet of things” connectivity protocol. MQTT stands for MQ Telemetry Transport. MQTT is an extremely simple and lightweight publish/subscribe messaging protocol. It is designed for constrained devices, high latency or unreliable networks. The design principles are to minimise network bandwidth and device resource requirements whilst also attempting to ensure reliability and some degree of assurance of delivery. These principles make MQTT ideal for IOT and mobile apps where bandwidth and battery power are at a premium.
5 reasons why we shifted from XMPP to MQTT
1. Less overhead and lightweight
2. Supports QoS (fire and forget, at least once and exactly once)
3. Low keep-alive traffic
4. Pub/Sub Mechanism
5. Low power usage
MQTT provides 3 types of QoS for delivering messages:
QoS 0 (fire and forget) – The message is delivered at most once. QoS 0 messages are not stored. Therefore QoS 0 messages are lost if client disconnects. In QoS 0, delivery of message is not acknowledged. It is the fastest mode of transfer.
QoS 1 (at least once) – The message is delivered at least once. Messages might be delivered multiple times if sender does not receive an acknowledgement from the receiver. The messages must be stored at the sender’s end, until sender receives a confirmation from the receiver.
QoS 2 (Exactly once) – The message is delivered exactly once. QoS 2 ensures that the message is received exactly once by the receiver. The messages must be stored at the sender’s end, until sender receives a confirmation from the receiver. It is the most reliable mode of transfer. It is also the slowest mode of transfer since it uses an advanced acknowledgement sequence as compared to QoS 1.
MQTT uses a pattern called publish/subscribe. Multiple clients connect to the MQTT broker. Clients can either publish or subscribe to a topic. Topics are used by the broker to decide who will receive a message. The broker and MQTT act as a simple, common interface for everything to connect to.
Our MQTT server is powered by Mosquitto. Mosquitto is an open source message broker that implements the MQTT. It is written in C. Mosquitto is easy to install and deploy. It supports TLS, Websockets and provides authentication either via username/password, pre-shared keys or TLS client certificates. It also supports ACL. Using ACL, you can configure which clients can access which topics. We decided to use Paho Python library for the backend. Our backend maintains an MQTT connection and routes messages through our chat pipeline.
Installation (mosquitto on ubuntu)
sudo apt-get install mosquitto
Installation (paho-python)
pip install paho-python
Here is a simple client that subscribes to a topic (sports/cricket/india) and prints out the resulting messages.
<i><span style="font-weight: 400;">import paho.mqtt.client as mqtt</span></i> |
<i><span style="font-weight: 400;"># The callback for when the client receives a CONNACK response from the server.</span></i><i><span style="font-weight: 400;">def on_connect(client, userdata, flags, rc):</span></i><i><span style="font-weight: 400;"> print("Connected with result code " + str(rc))</span></i><i><span style="font-weight: 400;"> client.subscribe("sports/cricket/india")</span></i> |
<i><span style="font-weight: 400;"># The callback for when a PUBLISH message is received from the server.</span></i><i><span style="font-weight: 400;">def on_message(client, userdata, msg):</span></i><i><span style="font-weight: 400;"> print(msg.topic +" " + str(msg.payload))</span></i><i><span style="font-weight: 400;"> client = mqtt.Client()</span></i><i></i> |
<i><span style="font-weight: 400;">client.on_connect = on_connect</span></i><i><span style="font-weight: 400;">client.on_message = on_message</span></i> |
<i><span style="font-weight: 400;">client.connect("localhost", 1883, 60)</span></i> |
<i><span style="font-weight: 400;">client.loop_forever() </span></i> |
Here is a simple client that publishes to a topic (sports/cricket/india)
<i><span style="font-weight: 400;">import paho.mqtt.client as mqtt</span></i> |
<i><span style="font-weight: 400;">client = mqtt.Client()</span></i><i><span style="font-weight: 400;">client.connect("localhost", 1883, 60)</span></i><i><span style="font-weight: 400;">client.publish("sports/cricket/india", "India wins Asia Cup 2016!")</span></i><i><span style="font-weight: 400;">client.loop(2)</span></i> |
MQTT has helped to make our application lightweight and ensure real-time reliable message delivery. MQTT is an amazing protocol which has lots applications in mobile, IOT and M2M communications. If you want a lightweight and reliable messaging protocol, then you should definitely consider MQTT.
Wish to be a part of the amazing things we build? Look no further! Reach out to us at hello@haptik.co
