lindsay distributedgeventzmq

8/2/2019 Lindsay DistributedGeventZmq

1/154

Distributed Systems

with ZeroMQ and gevent

JeffLindsay@progrium


2/154

Why distributed systems?

Harness more CPUs and resources

Run faster in parallel

Tolerance of individual failures

Better separation of concerns


3/154

Most web apps evolve intodistributed systems


4/154


5/154


6/154


7/154


8/154


9/154


10/154

OpenStack


11/154

Web

API

Client

Amazon AWS

TwiML

Provider

Provider

Provider


12/154

ZeroMQ + geventTwo powerful and misunderstood tools


13/154

ConcurrencyHeart of Distributed Systems


14/154

Distributed computing

is just another flavor oflocal concurrency


15/154

Multithreading

Distributed system

Shared Memory

Thre ad Thre adThread

Shared Database

App AppApp


16/154

Concurrency models

Execution model

Defines the computational unit

Communication model

Means of sharing and coordination


17/154

Concurrency models

Traditional multithreadingOS threads

Shared memory, locks, etcAsync or Evented I/OI/O loop + callback chainsShared memory, futures

Actor modelShared nothing processesBuilt-in messaging


18/154

Examples

ErlangActor model

ScalaActor modelGo

Channels, Goroutines

Everything else (Ruby, Python, PHP, Perl, C/C++, Java)ThreadingEvented


19/154

Normally, the networking of

distributed systems is tacked on to the

local concurrency model.

Erlang is special.

MQ, RPC, REST, ...


20/154

Why not always use Erlang?


21/154


Half reasons

Weird/ugly language

Limited library ecosystem

VM requires operational expertise

Functional programming isnt mainstream


22/154


Half reasons

Weird/ugly language

Limited library ecosystem

VM requires operational expertise

Functional programming isnt mainstream

Biggest reason

Its not always the right tool for the job


23/154

Web

API

Client

Amazon AWS

TwiML

Provider

Provider

Provider


24/154

Service Oriented

ArchitectureMultiple languages

Heterogeneous cluster


25/154

RPC


26/154

Client / server

RPC


27/154

Client / serverMapping to functions

RPC


28/154

Client / serverMapping to functionsMessage serialization

RPC


29/154

Client / serverMapping to functionsMessage serialization

RPC

Poor abstraction of what you really want


30/154

What you want are tools to help you get distributed

actor model concurrency like Erlang ... without Erlang.Even better if they're decoupled and optional.


31/154

Rarely will you build an application as

part of a distributed system that doesnot also need local concurrency.


32/154

Communication model

How do we unify communications in local concurrencyand distributed systems across languages?


33/154

Execution model

How do we get Erlang-style local concurrency withoutinterfering with the language's idiomatic paradigm?


34/154

ZeroMQCommunication model


35/154

Misconceptions


36/154

Misconceptions

Its just another MQ, right?


37/154

Misconceptions

Its just another MQ, right?Not really.


38/154

Misconceptions



39/154

Misconceptions


Oh, its just sockets, right?


40/154

Misconceptions


Oh, its just sockets, right?Not really.


41/154

Misconceptions




42/154

Misconceptions



Wait, isnt messaging a solved problem?


43/154

Misconceptions



Wait, isnt messaging a solved problem?*sigh* ... maybe.


44/154

Regular Sockets


45/154

Regular Sockets

Point to point


46/154

Regular Sockets

Point to pointStream of bytes


47/154

Regular Sockets

Point to pointStream of bytesBuffering


48/154

Regular Sockets

Point to pointStream of bytesBufferingStandard API


49/154

Regular Sockets

Point to pointStream of bytesBufferingStandard API

TCP/IP or UDP, IPC


50/154

Messaging


51/154

Messaging

Messages are atomic


52/154

Messaging

Messages are atomic


53/154

Messaging

Messages are atomic


54/154

Messaging

Messages are atomic


55/154

Messaging

Messages are atomic Messages can be routed


56/154

Messaging



57/154

Messaging



58/154

Messaging


Messages may sit around


59/154

Messaging




60/154

Messaging




61/154

Messaging




62/154

Messaging




63/154

Messaging




64/154

Messaging


Messages may sit around Messages are delivered


65/154

Messaging


Messages may sit around Messages are delivered


66/154

Rise of the Big MQ


67/154

App

App

AppApp

Reliable

Message Broker

PersistentQueues


68/154

App

App

AppApp


69/154

AMQP

ProducerConsumer

MQ


70/154

AMQP

ProducerConsumer

MQ

Exchange Queue

BindingX


71/154

AMQP

ProducerConsumer

MQ

Exchange Queue

X


72/154

AMQP Recipes


73/154

AMQP Recipes

Work queuesDistributing tasks among workers


74/154

AMQP Recipes


Publish/SubscribeSending to many consumers at once

X


75/154

AMQP Recipes



X

RoutingReceiving messages selectively

X

foo

bar

baz


76/154

AMQP Recipes



X

RPCRemote procedure call implementation

RoutingReceiving messages selectively

X

foo

bar

baz


77/154

Drawbacks of Big MQ

Lots of complexity

Queues are heavyweight

HA is a challenge

Poor primitives


78/154

Enter ZeroMQFloat like a butterfly, sting like a bee


79/154

Echo in Python

Server Client

import zmq

context = zmq.Context()

socket = context.socket(zmq.REP)socket.bind("tcp://127.0.0.1:5000")

whileTrue:

msg = socket.recv()

print"Received", msg

socket.send(msg)

1

2

34

5

6

7

8

9

import zmq


socket = context.socket(zmq.REQ)socket.connect("tcp://127.0.0.1:5000")

for i inrange(10):

msg ="msg %s"% i

socket.send(msg)

print"Sending", msg

reply = socket.recv()

1

2

34

5

6

7

8

9

10


80/154

Echo in Ruby

Server Client

require"zmq"

context =ZMQ::Context.new(1)

socket = context.socket(ZMQ::REP)socket.bind("tcp://127.0.0.1:5000")

loopdo

msg = socket.recv

puts "Received #{msg}"

socket.send(msg)

end

1

2

34

5

6

7

8

9

10

require"zmq"

context =ZMQ::Context.new(1)

socket = context.socket(ZMQ::REQ)socket.connect("tcp://127.0.0.1:5000")

(0...10).each do |i|

msg ="msg #{i}"

socket.send(msg)

puts "Sending #{msg}"

reply = socket.recv

end

1

2

34

5

6

7

8

9

10

11


81/154

Echo in PHP

Server Client

12

34567891011

12

34567891011

12


82/154

Bindings

ActionScript, Ada, Bash, Basic, C, Chicken

Scheme, Common Lisp, C#, C++, D, Erlang,

F#, Go, Guile, Haskell, Haxe, Java, JavaScript,Lua, Node.js, Objective-C, Objective Caml,

ooc, Perl, PHP, Python, Racket, REBOL,

Red, Ruby, Smalltalk


83/154

Plumbing


84/154

Plumbing


85/154

Plumbing


86/154

Plumbing


87/154

Plumbing


88/154

Plumbing

inprocipc

tcpmulticast


89/154

Plumbing

inprocipc

tcpmulticast

socket.bind("tcp://localhost:5560")socket.bind("ipc:///tmp/this-socket")socket.connect("tcp://10.0.0.100:9000")socket.connect("ipc:///tmp/another-socket")socket.connect("inproc://another-socket")


90/154

Plumbing

inprocipc

tcpmulticast



91/154

Plumbing

inprocipc

tcpmulticast



92/154

Message Patterns


93/154

Message Patterns

Request-Reply

REQ REP


94/154

Message Patterns

Request-Reply

REQ REP

REP

REP


95/154

Message Patterns

Request-Reply

REQ REP

REP

REP


96/154

Message Patterns

Request-Reply

REQ REP

REP

REP


97/154

Message Patterns

Request-Reply

REQ REP

REP

REP


98/154

Message Patterns

Request-Reply Publish-Subscribe

REQ REP

REP

REP

PUB

SUB

SUB

SUB

P


99/154

Message Patterns


Push-Pull (Pipelining)

REQ REP

REP

REP

PUB

SUB

SUB

SUB

PUSH

PULL

PULL

PULL

M P


100/154

Message Patterns



REQ REP

REP

REP

PUB

SUB

SUB

SUB

PUSH

PULL

PULL

PULL

M P


101/154

Message Patterns



REQ REP

REP

REP

PUB

SUB

SUB

SUB

PUSH

PULL

PULL

PULL

M P


102/154

Message Patterns



REQ REP

REP

REP

PUB

SUB

SUB

SUB

PUSH

PULL

PULL

PULL

M P


103/154

Message Patterns


Push-Pull (Pipelining) Pair

REQ REP

REP

REP

PUB

SUB

SUB

SUB

PUSH

PULL

PULL

PULL

PAIR PAIR

D i


104/154

Devices

Queue Forwarder Streamer

Design architectures around devices.

D i


105/154

Devices



REQ REP

D i


106/154

Devices



PUB SUB

D i


107/154

Devices



PUSH PULL

P f


108/154

Performance

P f


109/154

Performance

Orders of magnitude faster than most MQs

P f


110/154

Performance


Higher throughput than raw sockets

P f


111/154

Performance


Higher throughput than raw socketsIntelligent message batching

P f


112/154

Performance


Higher throughput than raw socketsIntelligent message batching

Edge case optimizations


113/154

Concurrency?"Come for the messaging, stay for the easy concurrency"

Hintjens Law of Concurrency


114/154


e = mc2

E is effort, the pain that it takes

M is mass, the size of the code

C is conflict, when C threads collide



115/154




116/154




117/154


e=mc2, for c=1

ZeroMQ:

Z MQ


118/154

ZeroMQ

EasyCheapFastExpressive

Messaging toolkit for concurrency and distributed systems.

... familiar socket API

... lightweight queues in a library

... higher throughput than raw TCP

... maps to your architecture


119/154

geventExecution model


120/154

Threading vs EventedEvented seems to be preferred for scalable I/O applications

Evented Stack


121/154

Evented Stack

Non-blocking Code

Flow Control

I/O Abstract ion

Reactor

Event PollerI/OLoop

deflookup(country, search_term):main_d = defer.Deferred()

def first step():

1234


122/154

deffirst_step():query ="http://www.google.%s/search?q=%s"% (country,search_term)d = getPage(query)d.addCallback(second_step, country)d.addErrback(failure, country)

defsecond_step(content, country):m = re.search('


123/154

gevent

Reactor / Event Poller

Greenlets Monkey patching

Regular Python


124/154

Green threadsThreads implemented in user space (VM, library)


125/154

Monkey patchingsocket, ssl, threading, time


126/154

Twisted


127/154

Twisted~400 modules


128/154

gevent25 modules

Performance


129/154

Performance

http://nichol.as

Performance
http://nichol.as/http://nichol.as/


130/154

Performance

http://nichol.as

Performance


131/154

Performance

http://nichol.as

Building a Networking App


132/154

Building a Networking App

#===

# 1. Basic gevent TCP server

from gevent.server import StreamServer

defhandle_tcp(socket, address):

print'new tcp connection!' whileTrue:

socket.send('hello\n')

gevent.sleep(1)

tcp_server = StreamServer(('127.0.0.1', 1234), handle_tcp)

tcp_server.serve_forever()

1

2

3

4

5

6

78

9

10

11

12

13

#===

# 2. Basic gevent TCP server and WSGI server1

2

3


133/154

from gevent.pywsgi import WSGIServer

from gevent.server import StreamServer

defhandle_http(env, start_response):

start_response('200 OK', [('Content-Type', 'text/html')])

print'new http request!'

return ["hello world"]

defhandle_tcp(socket, address):

print'new tcp connection!'

whileTrue:

socket.send('hello\n')

gevent.sleep(1)

tcp_server = StreamServer(('127.0.0.1', 1234), handle_tcp)

tcp_server.start()

http_server = WSGIServer(('127.0.0.1', 8080), handle_http)

http_server.serve_forever()

3

4

5

67

8

9

10

11

1213

14

15

16

17

18

1920

21

22

from gevent.pywsgi import WSGIServerfrom gevent.server import StreamServerfrom gevent.socket import create_connection

1234


134/154

defhandle_http(env, start_response):start_response('200 OK', [('Content-Type', 'text/html')])



defhandle_tcp(socket, address): print'new tcp connection!' whileTrue:

socket.send('hello\n')gevent.sleep(1)

defclient_connect(address):

sockfile = create_connection(address).makefile() whileTrue:

line = sockfile.readline() # returns None on EOF if line isnotNone: print"


135/154










136/154

ZeroMQ in gevent?


137/154

from gevent import spawn

from gevent_zeromq import zmq

1

2

3


138/154


defserve():socket = context.socket(zmq.REP)

socket.bind("tcp://localhost:5559")

whileTrue:

message = socket.recv()

print"Received request: ", message

socket.send("World")

server = spawn(serve)

defclient():

socket = context.socket(zmq.REQ)

socket.connect("tcp://localhost:5559")

for request inrange(10):socket.send("Hello")

message = socket.recv()

print"Received reply ", request, "[", message, "]"

spawn(client).join()

3

4

5

67

8

9

10

11

12

1314

15

16

17

18

1920

21

22

23

24


139/154

Actor model?Easy to implement, in whole or in part,

optionally with ZeroMQ


140/154

What is gevent missing?


141/154




142/154


Documentation



143/154


Documentation

Application framework


144/154

gserviceApplication framework for gevent

from gevent.pywsgi import WSGIServerfrom gevent.server import StreamServerfrom gevent.socket import create_connection

d f h dl htt ( t t )

12345


145/154










146/154

from gservice.core import Service

defhandle_http(env, start_response):

start_response('200 OK', [('Content-Type', 'text/html')]) print'new http request!' return ["hello world"]



defclient_connect(address):sockfile = create_connection(address).makefile()

whileTrue:line = sockfile.readline() # returns None on EOF

if line isnotNone: print"


147/154

http_port = Setting( http_port )tcp_port = Setting('tcp_port')connect_address = Setting('connect_address')

def__init__(self): self.add_service(WSGIServer(('127.0.0.1', self.http_port), self.handle_http)) self.add_service(StreamServer(('127.0.0.1', self.tcp_port), self.handle_tcp)) self.add_service(TcpClient(self.connect_address, self.client_connect))

defclient_connect(self, address):sockfile = create_connection(address).makefile()

whileTrue:



148/154

pidfile = example.pid

logfile ='example.log'http_port =8080

tcp_port =1234connect_address = ('127.0.0.1', 1234)

defservice():

from example import MyApplication return MyApplication()

3

45

67

89

1011

# Run in the foregroundgservice -C example.conf.py

# Start service as daemongservice -C example.conf.py start

# Control service

gservice -C example.conf.py restartgservice -C example.conf.py reload

gservice -C example.conf.py stop

# Run with overriding configurationgservice -C example.conf.py -X 'http_port = 7070'


149/154

Generalizinggevent proves a model that can be implemented in almost

any language that can implement an evented stack

gevent


150/154

gevent

Easy

SmallFastCompatible

Futuristic evented platform for network applications.

... just normal Python

... only 25 modules

... top performing server

... works with most libraries


151/154

RaidenLightning fast, scalable messaging

https://github.com/progrium/raiden

Concurrency models
https://github.com/progrium/raidenhttps://github.com/progrium/raiden


152/154

y

Traditional multithreading

Async or Evented I/O

Actor model

Conclusion


153/154

Two very simple, but very powerful tools

for distributed / concurrent systems


154/154

Thanks@progrium

lindsay distributedgeventzmq

Documents