zeromq anatomy & jeromq
TRANSCRIPT
![Page 1: Zeromq anatomy & jeromq](https://reader030.vdocuments.pub/reader030/viewer/2022020110/55a782811a28ab2b3e8b47f6/html5/thumbnails/1.jpg)
ZeroMQAnatomy & JeroMQ
![Page 2: Zeromq anatomy & jeromq](https://reader030.vdocuments.pub/reader030/viewer/2022020110/55a782811a28ab2b3e8b47f6/html5/thumbnails/2.jpg)
ZeroMQ
MQ 가 아니무니다..
TCP on Steroids
![Page 3: Zeromq anatomy & jeromq](https://reader030.vdocuments.pub/reader030/viewer/2022020110/55a782811a28ab2b3e8b47f6/html5/thumbnails/3.jpg)
ZMQ Socket
ROUTER DEALER XPUB XSUB PUSH
PULLREP REQ PUB SUB
![Page 4: Zeromq anatomy & jeromq](https://reader030.vdocuments.pub/reader030/viewer/2022020110/55a782811a28ab2b3e8b47f6/html5/thumbnails/4.jpg)
ZeroMQ AnatomyR w
![Page 5: Zeromq anatomy & jeromq](https://reader030.vdocuments.pub/reader030/viewer/2022020110/55a782811a28ab2b3e8b47f6/html5/thumbnails/5.jpg)
io_thread_t
ZMQ Socket
Mailbox
Signalerypipe_t
<cmd_t>
Session
Socket
Session
Socket
poller
rfd
fd
ypipe_t <msg_t> inpipe
ypipe_t <msg_t> outpipe
ZMQSocket
ZMQSocket
ZMQSocket
ZMQSocket
SessionSessionSession
Context
Mailbox
r
![Page 6: Zeromq anatomy & jeromq](https://reader030.vdocuments.pub/reader030/viewer/2022020110/55a782811a28ab2b3e8b47f6/html5/thumbnails/6.jpg)
Msgunion { ... struct { unsigned char data [29]; unsigned char size; unsigned char type; unsigned char flags; } vsm; struct { content_t *content; unsigned char unused [29 + 1 - sizeof (content_t*)]; unsigned char type; unsigned char flags; } lmsg;
DATA S T F
T F
S DATA
vsm
lmsg
![Page 7: Zeromq anatomy & jeromq](https://reader030.vdocuments.pub/reader030/viewer/2022020110/55a782811a28ab2b3e8b47f6/html5/thumbnails/7.jpg)
writer thread
reader thread
YQueue
yqueue_t <msg_t, 256> yqueue_t <command_t, 16>
struct chunk_t { T values [N]; chunk_t *prev; chunk_t *next; };
atomic_ptr_t <chunk_t> spare_chunk
...begin
back
• Only single thread can read from begin_chunk• Only single thread can write to yqueue_t <msg_t> to back chunk• Only single thread can write to yqueue_t <command_t> at a time
![Page 8: Zeromq anatomy & jeromq](https://reader030.vdocuments.pub/reader030/viewer/2022020110/55a782811a28ab2b3e8b47f6/html5/thumbnails/8.jpg)
YPipe
atomic_ptr_t <T> c
Reader
Writer
T* r
T* wT* f
bool flush () { if ( c.cas ( w, f ) != w ) { c.set ( f ) ; w = f ; return false ; // reader is sleeping } w = f ; return true ; }
bool check_read () { if ( queue.front () != r ) return true ;
r = c.cas ( queue.front () , NULL ) ; if ( queue.front () == r ) return false ; // nothing to read
return true; }
read() => queue<T>.pop
write() => queue<T>.push
r w f
![Page 9: Zeromq anatomy & jeromq](https://reader030.vdocuments.pub/reader030/viewer/2022020110/55a782811a28ab2b3e8b47f6/html5/thumbnails/9.jpg)
Signalersocketpair ( AF_UNIX, SOCK_STREAM, 0, sv[2] )
int w = sv[0]; // writer fd
int r = sv[1]; // reader fd, Used at IOThread.poll
Reader
recv () { :: recv ( r , dummy[1] );}wait ( timeout ) { poll ( r , timeout );}
Writersend () { :: send ( w, dummy[1] );}
![Page 10: Zeromq anatomy & jeromq](https://reader030.vdocuments.pub/reader030/viewer/2022020110/55a782811a28ab2b3e8b47f6/html5/thumbnails/10.jpg)
Mailboxypipe_t <command_t, 16> cpipe
signaler_t signaler
Reader
Writer
recv (timeout) {signaler.wait (timeout)
return cpipe.read ()}
send (cmd) {lock ()cpipe.write (cmd)cpipe.flush ()unlock ()
signaler.send ()}
![Page 11: Zeromq anatomy & jeromq](https://reader030.vdocuments.pub/reader030/viewer/2022020110/55a782811a28ab2b3e8b47f6/html5/thumbnails/11.jpg)
ZMQ SocketZMQ Socket
mailbox_t
signaler_typipe_t
<cmd_t>
Session
Socket
Session
Socketr
fd
fd
ypipe_t <msg_t> inpipe
ypipe_t <msg_t> outpipe
• Session is create when connect / accept• Worker thread owns ZMQ socket• IOThread owns Sessions
• recv msg from socket and en-queue inpipe• recv msg from outpipe and send to socket
• zsocket.send en-queue to outpipe• zsocket.recv de-queue from inpipe if any
![Page 12: Zeromq anatomy & jeromq](https://reader030.vdocuments.pub/reader030/viewer/2022020110/55a782811a28ab2b3e8b47f6/html5/thumbnails/12.jpg)
main/worker
IOThreadio_thread
ZMQ Socket
mailbox
Session
Socket
poller
ZMQSocket
ZMQSocket
ZMQSocket
ZMQSocket
Session
mailbox
zsocket.write(msg)outpipe.write(msg)activate_read
session.read_activatedengine.activate_out engine.out_event
SessionSessionSession
Stream Engine
engine.in_eventoutpipe.write(msg)
activate_readzsocket.read_activatedmsg = inpipe.read
![Page 13: Zeromq anatomy & jeromq](https://reader030.vdocuments.pub/reader030/viewer/2022020110/55a782811a28ab2b3e8b47f6/html5/thumbnails/13.jpg)
in_event () { // read from socket size = 8192 // *** buffer = decoder . get_buffer (size) ::recv (buffer, size) decoder . process_buffer(buffer) session . flush() }out_event () { // write to socket size = 8192 // *** buffer = encoder . get_data (size) ::send (buffer, size)}
msg = parse (buffer) session.write (msg)
Stream Engine
msg = session.read ()buffer.fill (msg)
![Page 14: Zeromq anatomy & jeromq](https://reader030.vdocuments.pub/reader030/viewer/2022020110/55a782811a28ab2b3e8b47f6/html5/thumbnails/14.jpg)
Summary
inline
lock-free
well-defined threading model
less system calls
• malloc / free• bulk socket recv / send• balance copy overhead
![Page 15: Zeromq anatomy & jeromq](https://reader030.vdocuments.pub/reader030/viewer/2022020110/55a782811a28ab2b3e8b47f6/html5/thumbnails/15.jpg)
JeroMQ - jeromq.org
Hand-made
Compatible with ZeroMQ-3
tcp:// inproc://
ipc:// pgm://
same dev experience
95%~ performance
![Page 16: Zeromq anatomy & jeromq](https://reader030.vdocuments.pub/reader030/viewer/2022020110/55a782811a28ab2b3e8b47f6/html5/thumbnails/16.jpg)
JeroMQ - diffJava NIO
SelectableChannel(Direct) ByteBuffer
SignalerPipe.SourceChannel / Pipe.SinkChannelMixed with counter
YQueue - circular linked listPlain Socket ProxyPersistence - Kafka alike
MappedByteBufferZeroCopy
![Page 17: Zeromq anatomy & jeromq](https://reader030.vdocuments.pub/reader030/viewer/2022020110/55a782811a28ab2b3e8b47f6/html5/thumbnails/17.jpg)
Future
Zero Persistence Queue
http://github.com/miniway/zper
Faster than Kafka
50% Less code
Configurable topology
Java 7 - AsynchronousChannel
![Page 18: Zeromq anatomy & jeromq](https://reader030.vdocuments.pub/reader030/viewer/2022020110/55a782811a28ab2b3e8b47f6/html5/thumbnails/18.jpg)
Conclusion
Why reinvent the wheel
“The only limitation is your imagination and sobriety”
![Page 19: Zeromq anatomy & jeromq](https://reader030.vdocuments.pub/reader030/viewer/2022020110/55a782811a28ab2b3e8b47f6/html5/thumbnails/19.jpg)
Q/A