docker 對傳統 devops 工具鏈的衝擊 (docker's impact on traditional devops toolchain)

William YehArchitect @ Gogolook

2015-12-10

Albert Camus

1. How to recreate your system2. How to safely change your system3. When something has gone wrong

Do you have basic infrastructure? (2015-11-18)http://www.robustperception.io/do-you-have-basic-infrastructure/

public network

1. How to recreate your system2. How to safely change your system3. When something has gone wrong

private network

CDN LB API servers DB servers

1.1 - 1.2 -

1. How to recreate your system2. How to safely change your system3. When something has gone wrong

private network

CDN LB API servers DB servers

public network

.1 -

.2 -

1. How to recreate your system2. How to safely change your system3. When something has gone wrong

private network

CDN LB API servers DB servers

public network

detection recovery diagnosis

public network

1. How to recreate your system2. How to safely change your system3. When something has gone wrong

private network

CDN LB API servers DB servers

1.1 -

1.2 -

hardware

bare metal

OS

runtime

app

VM

hardware

OS

runtime

app

hypervisor

VM

hardware

OS

runtime

app

hypervisor

VM

hardware

OS

runtime

app

hypervisor

Docker

hardware

OS

runtime

app

hypervisor

container

Docker

hardware

OS

runtime

app

hypervisor

container

We know:• Container is faster than VM• Container is lightweight• Dockerized app anywhere…

What if…• An OS runs only containers?

Docker

hardware

OS

runtime

app

hypervisor

container CoreOSRancherOSRed Hat AtomicVMware PhotonSnappy Ubuntu CoreWindows Nano Server

Docker

hardware

OS

runtime

app

hypervisor

container

Container per VM

hardware

OS

runtime

app

hypervisor

container

Container per VM

hardware

OS

runtime

app

hypervisor

container Intel Clear Linuxhttp://www.ithome.com.tw/news/96119

Hyperhttps://hyper.sh/

Container per VM

hardware

OS

runtime

app

hypervisor

container

Unikernel

hardware

library OS

app

hypervisor

unikernelapp stack

Unikernel

hardware

library OS

app

hypervisor

unikernelapp stack

Unikernels are constructed by using “library operating systems,” from which the developer selects only the minimal set of services required for an application to run.

Unikernel

hardware

library OS

app

hypervisor

unikernelapp stack

MirageOShttps://mirage.io/

Boxfusehttps://boxfuse.com/

ClickOSCliveHaLVMLINGRump KernelsOSv

Unikernel image

Immutable infrastructure

public network

1. How to recreate your system2. How to safely change your system3. When something has gone wrong

private network

CDN ELB API servers DB servers

1.1 -

1.2 -

service consolidation

resource isolation

native app

VM

container per VM

unikernel

Docker

public network

1. How to recreate your system2. How to safely change your system3. When something has gone wrong

private network

CDN LB API servers DB servers

1.1 -

1.2 -

public network

private network

API servers

JDK

app server (Tomcat, …)

app (jar, war, …)

Node.js runtime

npm

app source

Python runtime

pip

uWSGI/Gunicorn

app source

public network

private network

DB servers

JDK

Elasticsearch

JDK

CassandraMongoDBMySQL

dependenciesdependencies

public network

private network

LB

HAProxynginx

AWS ELB dependenciesdependencies

JDK

app server (Tomcat, …)

app (jar, war, …)

Node.js runtime

npm

app source

Python runtime

pip

uWSGI/Gunicorn

app source

JDK

Elasticsearch

JDK

CassandraMongoDBMySQL

dependenciesdependencies

HAProxynginx

AWS ELB dependenciesdependencies

JDK

app server (Tomcat, …)

app (jar, war, …)

Node.js runtime

npm

app source

Python runtime

pip

uWSGI/Gunicorn

app source

JDK

Elasticsearch

JDK

CassandraMongoDBMySQL

dependenciesdependencies

HAProxynginx

AWS ELB dependenciesdependencies

DevOps tools• Configuration management• Build system• Deployment pipeline• Continuous integration

JDK

app server (Tomcat, …)

app (jar, war, …)

Node.js runtime

npm

app source

Python runtime

pip

uWSGI/Gunicorn

app source

JDK

Elasticsearch

JDK

CassandraMongoDBMySQL

dependenciesdependencies

HAProxynginx

AWS ELB dependenciesdependencies

Docker

runtime

app

image

immutable image versioned image dev/prod parity

Dockerfiledocker builddocker pushdocker pull

public network

1. How to recreate your system2. How to safely change your system3. When something has gone wrong

private network

CDN ELB API servers DB servers

1.1 -

1.2 -

uniform

Docker

Config management will only be used to install Docker, an orchestration system, configure PAM/SSH auth, and tune OS sysctl values.

traditional DevOps toolchain

tedious

… Basically anything not having to do with app deployment.https://blog.containership.io/containers-vs-config-management-e64cbb744a94

public network

1. How to recreate your system2. How to safely change your system3. When something has gone wrong

private network

CDN LB API servers DB servers

1.1 -

1.2 -

public network

private network

API servers

As the number of machines grows…how to ensure better allocation?

pets cattle

pets cattle

naming?

pets cattle

dispensable?

pets cattlenaming?

dispensable?

pets cattle

PaaSHadoop MapReduceAWS LambdaAWS KinesisGoogle Dataflow

public network

private network

API servers

As the number of machines grows…how to ensure better allocation?

Traditional app

OS

runtime

app

tight interaction

hostresources

pets

https://prezi.com/e7sdy9rdujgp

public network

private network

API servers

better mobilitycattle

public network

private network

API servers

even better mobilitycontainer

cattleindependent of

underlying machines

cattle

Docker Swarm Mesos Kubernetes

allocation, orchestration

public network

1. How to recreate your system2. How to safely change your system3. When something has gone wrong

private network

CDN ELB API servers DB servers

1.1 -

1.2 -

Docker SwarmMesosKubernetes

Config management will only be used to install Docker, an orchestration system…

traditional DevOps toolchain

… Don’t assume too much about underlying infrastructure.

cattlepets

cattle

Long Running Services • Aurora• Marathon• Singularity • SSSP

Batch Scheduling • Chronos• Jenkins• JobServer

Big Data Processing • Cray Chapel• Dpark• Exelixi• Hadoop• Hama• MPI• Spark• Storm

Data Storage • Cassandra• Elasticsearch• Hypertable

Mesos framework

cattle

Mesos

1. How to recreate your system2. How to safely change your system3. When something has gone wrong

.1 -

.2 -

private network

CDN LB API servers DB servers

public network

private network

LB API servers DB servers

• Rolling upgrade• Blue/green deployment• Canary deployment

Traditional app

• in-place update • immutable infra

public network

private network

API servers

immutable imagescontainer

cattleindependent of

underlying machines

cattle

Docker Swarm Mesos Kubernetes

allocation, orchestration

kubectl rolling-update my-nginx --image=nginx:1.9.1

• Rolling upgrade

• Canary deployment• use label

Kubernetes

Mesos

http://blog.qubit.com/opensourcing-bamboo-automated-mesos-marathon-load-balancing http://www.slideshare.net/johnadowns/making-developers-happier-with-mesos-docker-and-marathon

Universal Control Plane

https://www.docker.com/universal-control-plane

public network

1. How to recreate your system2. How to safely change your system3. When something has gone wrong

private network

CDN ELB API servers DB servers

.1 -

.2 -

Docker SwarmMesosKubernetes

traditional DevOps toolchain

Consider the benefits: • immutable infrastructure • automated allocation • automated orchestration

cattlepets

private network

CDN LB API servers DB servers

Key Takeaways

1. How to recreate your system2. How to safely change your system3. When something has gone wrong

Do you have basic infrastructure? (2015-11-18)http://www.robustperception.io/do-you-have-basic-infrastructure/

service consolidation

resource isolation

native app

VM

container per VM

unikernel

Docker

Immutable infrastructure

uniform

Docker

Config management will only be used to install Docker, an orchestration system… anything other than app deployment.

traditional DevOps toolchain

tedious

Docker SwarmMesosKubernetes

traditional DevOps toolchain

… Don’t assume too much about underlying infrastructure.

cattlepets

cattle

Docker Swarm Mesos Kubernetes

allocation, orchestration

http://s

end.wtf/

docker20

15