ØMQ - An Introduction
1 like545 views
The document introduces the ZeroMQ messaging library. It describes how ZeroMQ aims to provide zero latency, zero administration, and zero cost messaging. It then discusses the different socket types like REQ/REP, DEALER, and ROUTER that can be used together to build asynchronous client-server communication patterns. The document also covers lower-level details like how messages are formatted and how identities are used to identify message sources.
![Context ctx = ZMQ.context(1);
Socket socket = ctx.socket(ZMQ.REP);
socket.bind("tcp://*:5555");
while (true) {
byte[] req = socket.recv();
socket.send(req);
}
REQ - REP
A BASIC EXAMPLE (2)](https://image.slidesharecdn.com/zeromq-171124163023/85/OMQ-An-Introduction-5-320.jpg)
![Context ctx = ZMQ.context(1);
Socket socket = ctx.socket(ZMQ.REQ);
socket.connect("tcp://localhost:5555");
socket.send("hello".getBytes());
byte[] rep = socket.recv();
REQ - REP
A BASIC EXAMPLE (3)](https://image.slidesharecdn.com/zeromq-171124163023/85/OMQ-An-Introduction-6-320.jpg)
![Each ZeroMQ socket type pair has its own rules
related to sending and receiving data. Different type
pairs allow developers to achieve different result.
The API to send and receive, though, are independent
from the socket type.
byte[] data = socket.recv();
socket.send(data);
Sending vs Receiving
HOW DO THEY COMMUNICATE?](https://image.slidesharecdn.com/zeromq-171124163023/85/OMQ-An-Introduction-10-320.jpg)
![ZeroMQ packets can contain more than one Frame
each one following the [length | content] format.
Here is the default message envelope.
Frames & Envelops
CAN WE SEND DIFFERENT PARTS IN THE SAME MESSAGE?](https://image.slidesharecdn.com/zeromq-171124163023/85/OMQ-An-Introduction-12-320.jpg)
![A ZMsg is convenient Java Class which hides away the
complexity of decoding and the message envelope.
ZMsg msg = ZMsg.recvMsg(socket);
msg.send(socket);
Frames are managed as a stack and they can be
pushed/popped in/from the message.
Byte[] frame= msg.pop();
msg.push(frame);
ZMsg
IS THERE SOME HELP IN MANAGING COMPLEXITY?](https://image.slidesharecdn.com/zeromq-171124163023/85/OMQ-An-Introduction-13-320.jpg)
ØMQ - An Introduction
- 1. MILANO 1863 POLITECNICO ØMQ - ZeroMQ AN INTRODUCTION Carlo Bernaschina – [email protected]
- 2. “We took a normal TCP socket, injected it with a mix of radioactive isotopes stolen from a secret Soviet atomic research project, bombarded it with 1950-era cosmic rays, and put it into the hands of a drug-addled comic book author with a badly-disguised fetish for bulging muscles clad in spandex. Yes, ZeroMQ sockets are the world-saving superheroes of the networking world.” History HOW IT BEGAN
- 3. The original Ø meant: • Zero Broker • Zero Latency (as close as possible) With time it also started meaning: • Zero Administration • Zero Cost • Zero Waste The Real History WHAT Ø STANDS FOR
- 4. Let’s build a simple lock-stepped Client Server communication. Lock-Stepped Communication: Each end of the communication sends a message in turn REQ - REP A BASIC EXAMPLE
- 5. Context ctx = ZMQ.context(1); Socket socket = ctx.socket(ZMQ.REP); socket.bind("tcp://*:5555"); while (true) { byte[] req = socket.recv(); socket.send(req); } REQ - REP A BASIC EXAMPLE (2)
- 6. Context ctx = ZMQ.context(1); Socket socket = ctx.socket(ZMQ.REQ); socket.connect("tcp://localhost:5555"); socket.send("hello".getBytes()); byte[] rep = socket.recv(); REQ - REP A BASIC EXAMPLE (3)
- 7. In ZeroMQ all the IO operations are done in the background by a thread pool. A Context groups a set of sockets which share the same pool. ZMQ.Context ctx = ZMQ.context(1); The parameter of the ZMQ.context Factory is the number of threads active in the thread-pool. (1 is a good default for most of the cases) Context HOW IS IO HANDLED?
- 8. In ZeroMQ Sockets are logical endpoints of a communication system. They abstract different physical connections and protocols. Socket socket = ctx.socket(ZMQ.REP); They can be of different types: Socket HOW TO SEND AND RECEIVE DATA? • PUB • SUB • REQ • REP • ROUTER • DEALER • PUSH • PULL • PAIR
- 9. ZeroMQ socket types are agnostic w.r.t. the endpoint responsible of initiating the communication, any socket type can be a TCP Server or Client. One socket should bind to one (or more) address. socket.bind("tcp://*:5555"); The other should connect to it (or them). socket.connect("tcp://localhost:5555"); General role of thumb the endpoint which is “static” should bind the “dynamic” ones should connect. Binding vs Connecting WHO IS THE SERVER?
- 10. Each ZeroMQ socket type pair has its own rules related to sending and receiving data. Different type pairs allow developers to achieve different result. The API to send and receive, though, are independent from the socket type. byte[] data = socket.recv(); socket.send(data); Sending vs Receiving HOW DO THEY COMMUNICATE?
- 11. ZeroMQ is a message based system even though it can use stream based transport protocol like TCP. The message on the wire follows a simple format: • Length • Content Packet Format HOW ARE THE DATA OF THE WIRE?
- 12. ZeroMQ packets can contain more than one Frame each one following the [length | content] format. Here is the default message envelope. Frames & Envelops CAN WE SEND DIFFERENT PARTS IN THE SAME MESSAGE?
- 13. A ZMsg is convenient Java Class which hides away the complexity of decoding and the message envelope. ZMsg msg = ZMsg.recvMsg(socket); msg.send(socket); Frames are managed as a stack and they can be pushed/popped in/from the message. Byte[] frame= msg.pop(); msg.push(frame); ZMsg IS THERE SOME HELP IN MANAGING COMPLEXITY?
- 14. ZeroMQ sockets identify each connected endpoint with a unique identifier called Identity. The message envelope can be extended to contain one (or more) Identities. They are stored before the “empty delimiter frame”. Identifies HOW CAN WE IDENTIFY THE ORIGIN OF A PACKET?
- 15. We will focus on Request Response pairs. The Valid Combinations are the following: • REQ to REP • DEALER to REP • REQ to ROUTER • DEALER to ROUTER • DEALER to DEALER • ROUTER to ROUTER Valid Socket Combinations HOW CAN WE PUT BUILDING BLOCKS TOGETHER?
- 16. REQ Sockets are synchronous. After connection/binding they can just send a message, trying to receive in this state will produce an exception. Once one (and only one) message has been sent, they can just receive a message, trying to send in this state will produce an exception. Once one (and only one) message has been received, the sockets go back to its initial state. REQ REQUEST AND WAIT
- 17. REP Sockets are synchronous. After connection/binding they can just receive a message, trying to send in this state will produce an exception. The Identity of the source endpoint is stored into an internal state and not exposed to user code. Once one (and only one) message has been received, the sockets can just send a message, which will be sent to the stored Identity. Once one (and only one) message has been sent, the sockets go back to its initial state. REP WAIT AND REPLY
- 18. DEALER Sockets are asynchronous. After connection/binding they can send as many messages as necessary. Any number of messages can also be received, no identity though is attached to them. DEALER REQUEST AND FORGET
- 19. ROUTER Sockets are asynchronous. After connection/binding they can receive many messages. Each message is enriched with and extra frame containing the Identity of the source endpoint. These sockets can send messages just to other endpoints from which they received messages. The messages which are going to be sent MUST contain an initial frame with the Identity of the target endpoint. ROUTER ENREACH THE MESSAGE
- 20. DEALER – ROUTER Socket Pairs can be used to orchestrate a classic Client - Server communication channels with one (or more) clients and one servers. Once the DEALER sends a message it can start waiting for answers. The single server architecture guarantees the origin of the message. DEALER - ROUTER ASYNCHRONOUS CLIENT SERVER COMMUNICATION Once the ROUTER receives a message it can store the Identity and use it to forward the message back to the correct endpoint when the answer is ready. Meanwhile it can go on waiting for other requests.
- 21. Download Source Code https://github.com/zeromq/jeromq Download the JAR file http://central.maven.org/maven2/org/zeromq/jeromq JeroMQ A PURE JAVA IMPLEMENTATION
- 22. Let’s open our IDEs DEMO TIME PRACTICE NEVER HURTS