kamaelia protocol walkthrough

A walk through building & stacking protocols using Kamaelia

Michael SparksOctober 2008

The aim of this document is to guide youthrough building a system where: ● A user connects to a server and● Over a secure connection,● Receives a sequence of JSON encode-able objects

The aim of this document is to guide youthrough building a system where: ● A user connects to a server and● Over a secure connection,● Receives a sequence of JSON encode-able objects

This doc also uses speech bubbles like this to make “aside” comments regarding particlar

panels. You can skip these safely.

ServerCore

First of all, we start off with the ServerCorecomponent. The hole is for fitting a protocolhandler factory. eg a class, or function.

ServerCore

First of all, we start off with the ServerCorecomponent. The hole is for fitting a protocolhandler factory. eg a class, or function.

As an aside, this hole is pretty crucial. Without filling this hole, this component is pretty useless.

So we call this sort of component a CHASSIS

So, we provide a protocol factory.

ServerCore

«factory»protocol

ServerCore(protocol=protocol, port=2345)

ServerCore

«factory»protocol

ServerCore(protocol=protocol, port=2345)


In retrospect, “protocol” as the name of an example protocol handler is a bad idea, so let's

change this slightly.

ServerCore

«factory»stackedjson

ServerCore(protocol=stackedjson, port=2345)


ServerCore


ServerCore( .... )

We then activate it, which causes it to start asubcomponent.

ServerCore( .... ).activate()

We then activate it. It starts TCPServersubcomponent, which listens for connections.

ServerCore

TCPServer


A client then connects to the TCPServer whichcreates a ConnectedSocketAdapter to handlethe mechanics of the connection.

ServerCore

TCPServer


ConnectedSocketAdapter

TCPServer tells ServerCore it has done this.ServerCore in response calls the protocol factoryto create a handler component...

ServerCore

TCPServer



stackedjson

...which is wired up. The protocol handler justreceives bytes and sends bytes, which are sent tothe client via the ConnectedSocketAdapter

ServerCore

TCPServer



stackedjson

There, the key thing in creating a server, is tocreate an appropriate protocol handlercomponent.

ServerCore

TCPServer



stackedjson

So, let's look inside this component

stackedjson

So, let's look inside this component

stackedjson

def protocol(*args,**argd): return Pipeline( PeriodicWakeup(message="NEXT", interval=1), Chooser(messages), MarshallJSON(), Encrypter(), DataChunker(), )

So, let's look inside this component.Clearly this is what actually handles theconnection.

stackedjson

def protocol(*args,**argd): return Pipeline( PeriodicWakeup(message="NEXT", interval=1), Chooser(messages), MarshallJSON(), Encrypter(), DataChunker(), )

Let's expand that...

Pipeline( PeriodicWakeup(message="NEXT", interval=1), Chooser(messages), MarshallJSON(), Encrypter(), DataChunker(), )

PeriodicWakeup

Chooser

MarshallJSON

Encrypter

DataChunker

We see this is a uni-directional protocol – assoon as this pipeline starts up. Encrypted JSONdata chunks get sent, ignoring client data.

Direction of data flow

PeriodicWakeup

Chooser

MarshallJSON

Encrypter

DataChunker

So, given the ServerCore infrastructure meanswe can more or less ignore the network, whatdoes the client side pipeline look like?

PeriodicWakeup

Chooser

MarshallJSON

Encrypter

DataChunker

ConsoleEchoer

DeMarshallJSON

Decrypter

DataDeChunker

TCPClient

Direction of data flow

Each component in the server, that is part of theuser's protocol, has a matching component inthe client.

PeriodicWakeup

Chooser

MarshallJSON

Encrypter

DataChunker

ConsoleEchoer

DeMarshallJSON

Decrypter

DataDeChunker

TCPClient

Fundamentally, this protocol allows one side ofit to securely throw dictionary objects to clientswhich can then do something with them

MarshallJSON

Encrypter

DataChunker

DeMarshallJSON

Decrypter

DataDeChunker

TCPClient

PeriodicWakeup

Chooser ConsoleEchoer

To understand what a client sees, we just look atthese three components. The 2 highlightedserver side components are the core behaviour.

MarshallJSON

Encrypter

DataChunker

DeMarshallJSON

Decrypter

DataDeChunker

TCPClient

PeriodicWakeup

Chooser ConsoleEchoer

Let's rearrange these, showing the logicalpipeline that drives what the user sees.

PeriodicWakeup

Chooser

ConsoleEchoer

First of all the PeriodicWakeupComponentsends out the message “NEXT” once per second.

PeriodicWakeup( message="NEXT", interval=1)

PeriodicWakeup

Chooser

ConsoleEchoer

These “NEXT” messages control the Chooser.The chooser steps through a list of messages,and when told “NEXT”, sends the next message.

Messages = [ {"hello": "world" }, {"hello": [1,2,3] }, {"world": [1,2,3] }, {"world": {"game":"over"} },]*10

Chooser(messages)

PeriodicWakeup

Chooser

ConsoleEchoer

PeriodicWakeup

Chooser

ConsoleEchoer

These “NEXT” messages control the Chooser.The chooser steps through a list of messages,and when told “NEXT”, sends the next message.

Messages = [ {"hello": "world" }, {"hello": [1,2,3] }, {"world": [1,2,3] }, {"world": {"game":"over"} },]*10

Chooser(messages)

It's worth noting that the Chooser can back backwards as well, just to the start and end etc. For more

details look at the docs :)

These messages arrive intact at the client - inthis case a ConsoleEchoer. However it could besomething more interesting – eg a game system.

ConsoleEchoer(use_repr=True)

PeriodicWakeup

Chooser

ConsoleEchoer

Going back to the overview slide, this diagramshows the various protocols which are stackedon top of each other.

PeriodicWakeup

Chooser

MarshallJSON

Encrypter

DataChunker

ConsoleEchoer

DeMarshallJSON

Decrypter

DataDeChunker

TCPClient

We have our core application protocol, stackedon top of a secure serialising communicationschannel. This truly is at the application level.

PeriodicWakeup

Chooser

MarshallJSON

Encrypter

DataChunker

ConsoleEchoer

DeMarshallJSON

Decrypter

DataDeChunker

TCPClient

We have our serialisation protocol. Clearly thisis a message oriented protocol – it requires thelower layer to preserve boundaries.

PeriodicWakeup

Chooser

MarshallJSON

Encrypter

DataChunker

ConsoleEchoer

DeMarshallJSON

Decrypter

DataDeChunker

TCPClient

We have a secure transmission protocol.This is admittedly simplistic – it assumespredistributed keys. However it gives a flavour

PeriodicWakeup

Chooser

MarshallJSON

Encrypter

DataChunker

ConsoleEchoer

DeMarshallJSON

Decrypter

DataDeChunker

TCPClient

We then have our chunking protocol, whichpreserves message boundaries – necessarybecause TCP does not guarantee to do so.

PeriodicWakeup

Chooser

MarshallJSON

Encrypter

DataChunker

ConsoleEchoer

DeMarshallJSON

Decrypter

DataDeChunker

TCPClient

So let's go back and see where these things fitback into the system.

PeriodicWakeup

Chooser

MarshallJSON

Encrypter

DataChunker

ConsoleEchoer

DeMarshallJSON

Decrypter

DataDeChunker

TCPClient

So this is what we had ...

ServerCore

TCPServer



stackedjson

The client really looks like this...Also, this is a one way protocol

ServerCore

TCPServer



stackedjson

The client really looks like this & this is a oneway protocol. The protocol itself looks like this.And that's how those parts fit together.

ServerCore

TCPServer



So, finally, for completeness, we'll include thecode.

ServerCore

TCPServer



So, finally, for completeness, we'll include thecode.

Though, for code, this is a better location: http://tinyurl.com/4k3df2

Bunch of imports...import cjsonimport Axon

from Kamaelia.Chassis.Pipeline import Pipelinefrom Kamaelia.Util.Chooser import Chooserfrom Kamaelia.Util.Console import ConsoleEchoer

from Kamaelia.Internet.TCPClient import TCPClientfrom Kamaelia.Chassis.ConnectedServer import ServerCore

from Kamaelia.Protocol.Framing import DataChunker, DataDeChunkerfrom Kamaelia.Apps.Grey.PeriodicWakeup import PeriodicWakeup

from Crypto.Cipher import DES

Define the messages being slung to clientsmessages = [ {"hello": "world" }, {"hello": [1,2,3] }, {"world": [1,2,3] }, {"world": {"game":"over"} }, ]*10

Then the JSON De/Marshalling componentsclass MarshallJSON(Axon.Component.component): def main(self): while not self.dataReady("control"): for j in self.Inbox("inbox"): j_encoded = cjson.encode(j) self.send(j_encoded, "outbox") if not self.anyReady(): self.pause() yield 1

class DeMarshallJSON(Axon.Component.component): def main(self): while not self.dataReady("control"): for j in self.Inbox("inbox"): j_decoded = cjson.decode(j) self.send(j_decoded, "outbox") if not self.anyReady(): self.pause() yield 1

Define our encoding layer using normal code...1class Encoder(object): """Null encoder/base encoder - returns the same string for encode/decode""" def __init__(self, key, **argd): super(Encoder, self).__init__(**argd) self.__dict__.update(argd) self.key = key def encode(self, some_string): return some_string def decode(self, some_string): return some_string

Define our encoding layer using normal code...2class DES_CRYPT(Encoder): def __init__(self, key, **argd): super(DES_CRYPT, self).__init__(key, **argd) self.key = self.pad_eight(key)[:8] self.obj = obj=DES.new(self.key, DES.MODE_ECB)

def encode(self, some_string): padded = self.pad_eight(some_string) encrypted = self.obj.encrypt(padded) return encrypted

def decode(self, some_string): padded = self.obj.decrypt(some_string) decoded = self.unpad(padded) return decoded

# ... continued

Define our encoding layer using normal code...3# class DES_CRYPT(Encoder): # ... continued from before

def pad_eight(self, some_string): X = len(some_string) if X % 8 != 0: pad_needed = 8-X % 8 else: pad_needed = 8 pad_needed = 8-(X % 8) PAD = pad_needed * chr(pad_needed) return some_string+PAD

def unpad(self, some_string): x = ord(some_string[-1]) return some_string[:-x]

Create Encryption components using thisclass Encrypter(Axon.Component.component): key = "ABCD" def main(self): crypter = DES_CRYPT(self.key) while not self.dataReady("control"): for j in self.Inbox("inbox"): j_encoded = crypter.encode(j) self.send(j_encoded, "outbox") if not self.anyReady(): self.pause() yield 1

Note, this is not a serious protocol, it is an illustration. This form of pre-shared key would be a bad idea, but the pattern of usage will be useful.

Create Decryption components using thisclass Decrypter(Axon.Component.component): key = "ABCD" def main(self): crypter = DES_CRYPT(self.key) while not self.dataReady("control"): for j in self.Inbox("inbox"): j_decoded = crypter.decode(j) self.send(j_decoded, "outbox") if not self.anyReady(): self.pause() yield 1

Note, this is not a serious protocol, it is an illustration. This form of pre-shared key would be a bad idea, but the pattern of usage will be useful.

Create the protocol handler factory & Clientdef stackedjson(*args,**argd): return Pipeline( PeriodicWakeup(message="NEXT", interval=1), Chooser(messages), MarshallJSON(), Encrypter(), # Encrypt on the way out DataChunker(), )

def json_client_prefab(ip, port): return Pipeline( TCPClient(ip, port=port), DataDeChunker(), Decrypter(), # Decrypt on the way in DeMarshallJSON(), ConsoleEchoer(use_repr=True) )

And finally start the server and point aclient at the server.ServerCore(protocol=protocol, port=2345).activate()json_client_prefab("127.0.0.1", 2345).run()

ServerCore

TCPServer



kamaelia protocol walkthrough

Technology