Erlang/OTP PubSub

@yamionpgumi R&D 15 Python 5Erlang 6

PubSub

Erlang/OTP

Web

/

/

Request

ResponseClientServer

/

Request

Server

Client

HTTPServer ()10

Server

Client

SYNSYN+ACKACK Connection TCP Connection

TCP Connection

Connection

Server

Client

HTTPTCP

Response Connection

Server

Client

() 15OK

()

Windows Server ()

Pub/Sub

Pub/Sub(/)Subscribe () Publish () Sub Topic Pub Topic Pub Sub

Subscribe

Broker

Topic ATopic BABCD

Broker

Topic ATopic B

ABCD

Broker

Topic A

PublishTopic B

ABCD

Broker

Topic ATopic B

ABC

PublishD

Room Room /RoomSub 0 Topic Publish Pub Topic Subscribe

ELB

App

Job

MQ

Batch

log

PubSub

bit

TCP TLV TLV4 bytes

1bit1632Type ()Length ()SenderTimestamp () ReceiverTimesamp () Payload ()

Payload TLV 11632SectionTypeSectionLengthValue

11632TypeLengthSenderTimestampReceiverTimesampSectionTypeSectionLengthValueSectionTypeSectionLengthValue

RTT

1bit1632Type ()Length ()SenderTimestamp () ReceiverTimesamp () Payload ()

ClientServerTimerTimer1

1000SenderReceiver10

Timer0

300ms1300SenderReceiver1300301

1300321321RTT20ms421100ms1300100

SenderReceiver4211400

14201420RTT20ms

52115201520RTT20msSenderReceiver5211500

200ms1300200

TCPNATTCP(

UDPTCPUDPTCP

or (ex.

App

PubSub

1.game start

2.IP:Port, Token, Topic

3.Token

4.Token

5.OK6.OK

Sphinx(& packetdiag)End to End

Erlang/OTP

Erlang/OTP (1986)1998

OS

Let it crash

ErlangLeague of Legendshttps://engineering.riotgames.com/news/chat-service-architecture-serversCall of Dutyhttps://en.wikipedia.org/wiki/Demonwarehttp://www.erlang-factory.com/upload/presentations/395/ErlangandFirst-PersonShooters.pdfWhatsApphttp://www.erlang-factory.com/upload/presentations/558/efsf2012-whatsapp-scaling.pdfGame of Warhttps://erlangcentral.org/senior-erlang-engineer-machine-zone/

tcpsessionsession_udpErlangVM system processlinklink

tcpsessionsession_udp system process

session_supsessions_suparkps_sup

session_udp_supErlangVM

udp_sup

gen_udp

simple_one_for_oneone_for_allone_for_oneone_for_oneone_for_one

tcpsessionsession_udp system process

session_supsessions_suparkps_sup

session_udp_supErlangVM

udp_sup

gen_udp

simple_one_for_oneone_for_allone_for_oneone_for_oneone_for_one

session Down

tcpsessionsession_udp system process

session_supsessions_suparkps_sup

session_udp_supErlangVM

udp_sup

gen_udp

simple_one_for_oneone_for_allone_for_oneone_for_oneone_for_one

session_sup session_udp_sup session_udp_sup session_udp

tcpsessionsession_udp system process

session_supsessions_suparkps_sup

session_udp_supErlangVM

udp_sup

gen_udp

simple_one_for_oneone_for_allone_for_oneone_for_oneone_for_one

session_udp session_udp_sup

tcpsessionsession_udp system process

session_supsessions_suparkps_sup

session_udp_supErlangVM

udp_sup

gen_udp

simple_one_for_oneone_for_allone_for_oneone_for_oneone_for_one

session_sup

tcpsessionsession_udp system process

session_supsessions_suparkps_sup

session_udp_supErlangVM

udp_sup

gen_udp

simple_one_for_oneone_for_allone_for_oneone_for_oneone_for_one

link tcp

LocustPython Python m4.large t2.medium

Locust Slaves

PubSub

Locust Master

m4.2xlargem4.xlargeCPU: 8RAM: 32GBCPU: 4RAM: 16GBx8

PubSub

x7

1 msg 8msg

PubSub

PubSub

PubSub

user/room8message/user/sec0.2waittime[ms]5000rooms10010002000300035003750() users800800016000240002800030000() message/sec 12801280025600384004480048000() message/sec12701278325560383284478721054(Server)CPU [%]66.5360.4404.7560.9635.8444.4(Server)RAM [%]0.62.43.76.27.216.5RTT Med [ms]125121934RTT Avg [ms]14111214271260RTT Max [ms]44497535653662798End to End Med [ms]1381839460End to End Avg [ms]2121321503017End to End Max [ms]74773875410031315100979

user/room8message/user/sec1waittime[ms]1000rooms1004008009001000() users8003200640072008000() message/sec 640025600512005760064000() message/sec636025527481955324025154(Server)CPU [%]275.8618.3709.3774.7766.7(Server)RAM [%]0.51.12.7311.2RTT Med [ms]115125RTT Avg [ms]1072440494RTT Max [ms]4548491110034295End to End Med [ms]2113680310End to End Avg [ms]101354120970End to End Max [ms]554550981292344120

user/room8message/user/sec15waittime[ms]66.6666666666667rooms1050708090100() users80400560640720800() message/sec 96004800067200768008640096000() message/sec90964604758600607646353926390(Server)CPU [%]229.8694.7776.6775.6787.9786.2(Server)RAM [%]0.40.50.60.70.78RTT Med [ms]3827101524RTT Avg [ms]3014151723194RTT Max [ms]45679312252220686End to End Med [ms]191924334673End to End Avg [ms]1821273750293End to End Max [ms]618631657718123139470

UDPTCP1/5V5/s 1/s

fprof sendport_command gen_udp/tcp:send http://qiita.com/mururu/items/9b77e49b5b8a2815ceb610/s

user/room8message/user/sec0.2waittime[ms]5000rooms10020003750625075008750() users8001600030000500006000070000() message/sec 128025600480008000096000112000() message/sec12812563948124787429455597731(Server)CPU [%]46.9312.2426.8625723.2718(Server)RAM [%]26.89.81417.318.7

user/room8message/user/sec1waittime[ms]1000rooms1004008001000120014001600() users80032006400800096001120012800() message/sec 64002560051200640007680089600102400() message/sec6356254725120263890759128814399033(Server)CPU [%]139.4269.1425500.3625.8699.2765.3(Server)RAM [%]23.6555.25.26.6

user/room8message/user/sec15waittime[ms]66.6666666666667rooms1050708090100110120130140() users8040056064072080088096010401120() message/sec 96004800067200768008640096000105600115200124800134400() message/sec8912459676464874035832119158199596106775112889116739(Server)CPU [%]161.5374.2489.7535.8582.1632.1666.4703.7726.9784.7(Server)RAM [%]2.12.32.32.32.32.21.82.633.4

App

PubSub

Private TopicPrivate Topic

matching startlistmatching

Private Topic

battle starter

HW

App

PubSub

PubSub

PubSub

1. battle info2. MySQL3. IP:Port, Token, Topic4.

App

PubSub

PubSub

PubSub

1. Down2. 3. 4. 5. Down6.

xApp

Room WorkerRoom Subscribe

App

PubSub

Battle Topic

6.battle info

2.battle startroom worker3.room create4.subscribe5. ok7. IP:Port, Token, Topic8. subscribe

One More Thing

ArkPS Server

OSS

https://github.com/arkps/()