Serverless Node.js

서버 없이 돌아가는 Node.js

공진기 차장 (한국 IBM)Bluemix Technical Evangelist

Long Term Technology Trends

Increasing Automation Increasing Abstraction

time

Increasing Automation Increasing Abstraction

Technology Trends Infrastructure

on premise bare metal virtual machines

containers

Technology Trends Programming

Increasing Automation Increasing Abstraction

assembly FORTRAN Managed Languages

Frameworks

Technology Trends Platform as a Service

Increasing Automation Increasing Abstraction

Technology Trends Serverless

Increasing Automation Increasing Abstraction

a cloud-native platform for

short-running, stateless computation and

event-driven applications which

scales up and down instantly and automatically and

charges for actual usage at a millisecond granularity

event handlersevents

What is Serverless?

Microservices

Mobile Backends

IoT

Modest Stream Processing

Bots, ML Inferencing

Serverless is good for short-running stateless event-driven

Serverless is not good for long-running stateful number crunching

Databases

Deep Learning Training

Spark/Hadoop Analytics

Heavy-Duty Stream Analytics

Numerical Simulationf(x)

What is Serverless good for?

Service integration (e.g. Cognitive: ObjectStore + Watson)

Video Streaming

Why is Serverless attractive?

On-prem VMs Containers Serverless

Time to provision

Weeks-months Minutes Seconds-

Minutes Milliseconds

Utilization Low High Higher Highest

Charging granularity CapEx Hours Minutes Blocks of

milliseconds

short answer It makes development

and operations faster, cheaper, easier.

Dark Vision

https://www.youtube.com/watch?v=1teIMpkI_Sg

1. User uploads video from mobile device 2. Cloud (Watson services) analyze frames to detect

products, objects, faces, text 3. Use results to deliver improved

navigation, recommendations, search

Requirements Technologies

RASMonitoring, Replication,

Service Discovery, Routing, Logging, Message Queues

Bursty Loads Autoscaling, Load Balancing

SecurityContainers, Version Management,

Software Scans, Encryption, Updates

Mobile Backend (REST) Web Server, Web Frameworks

Integration with Services API Discovery and Management, VPN/VPC

Agility DevOps, Continuous Integration, A/B Testing, Active Deploy

Dark Vision

conventional microservice architecture

Developer Responsibilities

business logic service integrations

IaaS: programmatic PaaS: declarative

containers middleware autoscaling load balancing service discovery fault tolerance logging messaging security patches

Serverless Architecture

Developer Responsibilities business logic service integrations

(declarative) REST API definition

(declarative)

Platform Responsibilities containers middleware autoscaling load balancing service discovery fault tolerance logging messaging security patches

Dark Vision

What is OpenWhisk?

What is OpenWhisk?

IBM’s Bluemix serverless offering

What is OpenWhisk? an open-source project on github

What is OpenWhisk? a high-level serverless programming model

Trigger

Rule

Action

Package

What is OpenWhisk? a high-level serverless programming model

Trigger

Rule

Action

Package

language support to encapsulate, share, extend code

first-class event-driven programming

constructs

first-class functions compose via sequences

docker containers as

actions

all constructs first-class — powerful extensible

language

A Action: a stateless function (event handler)

Action: javascriptA

function main(params) { console.log(“Hello “ + params.name); return { msg: “Goodbye “ + params.name) }; }

Action: sequenceA

:=

Trigger: a class of events (feed)

T Aevent event handler

R Rule: a mapping from a Trigger to an Action

Rule

PackagereadwritedocChanged

languageIdtranslate

forecast

posttopic

post myActionmyFeed

/dave/myPackage

Open Source Third PartyBuild and Share

your Own!

Package: a shared collection of Actions and Feeds

What is OpenWhisk under the hood?

http://fordmustanglover.blogspot.com/

• Basic Runtime

• Meta-programming

github.com openwhisk/openwhisk

coreruntime

CLIpackagessecurityfeatures

persistent store

loggingmonitoring billing

authentication

Edge VMEdge

VM

Edge VM

Edge VMEdge

VM

Master VM

controllerEdge VMEdge

VM

Slave VM

invoker

• microservices deployed in docker containers • open-source system middleware • NoSQL (CouchDB) persistence

action containeraction

containeraction containeraction

containeraction containeraction

containeraction containeraction container

Why ?

Why ?

controller

invoker

Deploying and managing traditional microservices1

Slave VM

Why ?

Lightweight isolated execution environment for arbitrary user code

action container

action containeraction container

action container

action containeraction container

action container

action container

2

Why ?

Portable description of arbitrary binary user actions (Dockerfile)

Docker file

3

% wsk action invoke hello

in 8 easy steps

Step 1. Entering the system

Edge VMEdge

VM

Edge VM

Edge VMEdge

VM

Master VM

controller

Why

POST /api/v1/namespaces/myNamespace/actions/myAction

? • SSL termination • Load Balancing • Blue/Green continuous delivery

Master VM

controller

Step 2. Handle the request

Master VM

kafka SDK

couchDBSDK

sprayDSL

loadbalancer

consulSDK

data modelsauthcaching

Why scala ?

Step 2. Handle the request

• original prototype node.js: abandoned and rewrote • static typing

• makes refactoring much easier • whole classes of bugs went away

• nice concurrency features (actors/futures) • kafka libraries more stable on JVM

actors

controller

Step 3. Authentication + Authorization

scala

kafka SDK

couchDB SDK

spray DSL

load balancer

consul SDK

data models

authcaching

external auth

• Cloudant: hosted CouchDB • plug-in structure for custom

authentication module

actors

controller

Authorization: Basic QWxhZGRpbjpPcGVuU2VzYW1l

Step 4. Get the action

scala

kafka SDK

couchDB SDK

spray DSL

load balancer

consul SDK

data models

authcaching

• check resource limits • actions stored as documents in CouchDB

• binaries as objects (attachments)

actors

controller

Step 5. Looking for a home

scala

kafka SDK

couchDB SDK

spray DSL

load balancer

consul SDK

data models

authcaching

controller

Load balancer: find a slave to execute Slave health, load stored in consul

• Sequentially consistent KV store • Replication, Fault Tolerance • Health Check / Monitoring utilities

Why ?

actors

Step 6. Get in line!

scala

kafka SDK

couchDB SDK

spray DSL

load balancer

consul SDK

data models

authcaching

invoker

Why ? • High throughput fault-tolerant queues • Point-to-point messages via topics

• explicit load balancing

Post request to execute to queue in

actors

Master VM

Master VM Slave VM

controller

Slave VM

invoker

Step7. Get to Work!

Slave VM

Step 7. Get to work!

scala

kafka SDK

couchDB SDK

docker utilities

container pool

consul SDK

data models

caching

invoker

bound to user action

• each user action gets it own container (isolation) • containers may be reused • container pool allocates and garbage collects containers

stem cell

actors

User action containers

invoker

containerpool

cold start stem cell container

docker run

HTTP POST /init

HTTP POST /run

warm container

Step 8. Store the results.

scala

kafka SDK

couchDB SDK

docker utilities

container pool

consul SDK

data models

caching

invoker

action container

HTTPResponse

logs on filesystem

actors

median ~45 ms latency end-to-end (unloaded system, hello world)

Future directions

What’s coming next?

What are you going to do next?

What are you going to do next?