riak at kivra

RIAK at KIVRABip Thelin@bipthelin

BIPCTO & Co-founder

OO / SQL-backgroundOpen Source / Tomcat Committer

Scalable platforms / HostingBeer Homebrewer / Judge

THELIN

API / Delivery serviceContent compositionContent creation Digital mailbox

ResponsiveWeb

iOS App

x

b

a

d

c

q

r

g

z

z

t u

i

e

e

p

y

f

State machineInternal, External & Time Events

Hard to test / debugError proneHardcoded flowNo overview

We outgrew

our system

QUOTE:

24 / 7 / 365

Distributable / ScALABLE

TESTABLE / OPERATIONAL

SECURE / Stable

Marriage is a wonderful institution, but who wants to live in an institution?

Our PhilosophyKeep it simple, StupidDo one thing and do it wellSeparation of concernRight tool for the jobHasty generalization

RIAK + ErlangLightweightDistributedScalableResilient to failure

Backend service

RIAK

REST API

Frontend Service

qz

x y

qz

x y

Internal ServicesExternal Services

Right datastorefor the job

3 x Riak Clusters

ONEBACKENDMany

(s)t o r u l e t h e m a l l

LeveldbBitcaskRiak-erlang-client (PB)JSON

RIAKstack

OAUTH2Authorization layer

on top of Riak

LAZYUPGRADE

-­‐record(user,  {        password                  =  <<"l0ln00b">,        name                          =  <<"noname">>,        i_primary_email    =  <<"ih@ck.se">,        i_primary_phone    =  <<"461337">>,        i_ssn                        =  <<"0123456789">>,        l_tenant                  =  {<<"B">>,  <<"K">>}}).

Get object

Upgrade object

RIAK

Check version

[1...n]

LOGGING

1. MetricsAnything you'd want toplot on a graph

2. (Un)structuredloggingLog data

you'dwant toprocess

3. Applicationlogging Error,

Notice,Warning, etc

Lager Logglygithub.com/kivra/lager_loggly.git

Lager Sysloggithub.com/basho/lager_syslog

~197 SLOC ~23 SLOC

Lager{lager,  [

{handlers,  [            {lager_console_backend,  info},                    {lager_loggly_backend,  [<<"Riak">>,

                         info,  5,  3,                      "https://<Loggly  URI>"]},

                   {lager_loggly_backend,  [<<"Riak">>,                        error,  5,  3,                        "https://<Loggly  URI>"]},

                   {lager_file_backend,  [                         {"/path/error.log",  error,  10485760,  "$D0",  5},                              {"/path/console.log",  info,  10485760,  "$D0",  5}                    ]}

]},          ...]},

DEPLOYMENTCentral configuration

Infrastructure as CodeBootstrap/Upgrade

Data modelingConflict resolution2i slowno M/R, Links, Search, etc..Key generationCaching

DESIGNDECISIONS

andTRADEOFFS

Riak CoreData encryptionSearchSession backend

What’s up?

Thank You!Bip Thelin

bip@kivra.com@bipthelin

https://www.github.com/kivra