from docker to moby and back. what changed ?

http://strikr.in/ CC BY NC-SA 4.0

Docker to Moby Project and back

[email protected]

What changed ?


Why this talk ?

● Docker architecture● Understand the Container landscape● Eco-system dynamics● Cloud vendor losing leverage● Moby in the game● Game of Open Standard thrones● Tactical Solutions approach

– Power user is the System builder●


Goal of this talk

● What should you do to succeed with containers in a post-Docker world ?


Application

container Services

Operating System

OS Services

container Runtime

container Engine


Image credits: Docker Inc.

Docker flow centric view


Docker API centric view


Pull image from registry

docker pull index.docker.io/alpine:3.6


Docker schematic

4


● Properties of cloud native systems– Container Packaged– Dynamically managed– Micro-services oriented


● Goals to Achieve– Standardized interfaces between

subsystems.– A standard systems architecture describing

the relationship between parts– At least one standard reference

implementation of each sub-system.– Extensible architecture that end users can

extend, replace or change behavior in every layer of the stack for their purposes.


● Container packaged– Running applications and processes in

software containers as an isolated unit of application deployment, and as a mechanism to achieve high levels of resource isolation.

– Benefit● Improves overall developer experience● code and component reuse● simplify operations for cloud native app


● Dynamically managed– actively scheduled and actively managed by

a central orchestrating process.– Benefits

● Improve machine efficiency and resource utilization

● Reduce the cost associated with maintenance and operations


● Micro-services oriented– Loosely coupled with dependencies explicitly

described (ie. service end-points)– Benefits

● Increase the overall agility and maintainability of applications.


Virtualization vs Containerization

● VM world– Hypervisor

● Container world– Container engine


Solutions Approach

● Immutable infrastructure is the goal

● Containers First● Data volume

containers● Resilient Micro-

services● Fine-tuned runtime to

support it

● Scripted automated● Pipelines● DevOps

– coInt– coDep– coMon– coSec– coCmp

Gold standard: It's your runtime with your artifact that you deploy to any 'cloud' vendor.


containerD


containerD

● Core container runtime● The daemon that controls runC


ContainerD

● Architecture– designed to be embedded into a larger

system, rather than being used directly by developers or end-users.

● daemon– exposes gRPC API over a local UNIX socket.


containerD

● API design– low-level one designed for higher layers to

wrap and extend. ● CLI

– a barebone CLI (ctr) designed for development and debugging purpose.

● interface with runC– uses runC to run containers according to the

OCI specification.


the promise of containerD 1.0

● Container execution and supervision● Image distribution● Network Interfaces Management● Local storage● Native plumbing level API● Full OCI support, including the extended OCI

image specification

Windows – Linux parity


ContainerD with the ecosystem


Container engine split


runC

● universal runtime for OS Containers● CLI tool for spawning and running containers

according to the OCI specification.


runC

● a CLI tool for spawning and running containers according to the OCI specification.

● runC– Depends on runtime-spec repo– Supports Linux platform only– Must be built with Go 1.6+– Executes build tags for features– Linux kernel 4.3+– Uses 'vndr' for dependency management


RunC for container lifecycle

cd /mycontainer

runc create mycontainerid

# view the container is created and in the "created" state

runc list

# start the process inside the container

runc start mycontainerid

# after 5 seconds view that the container has exited and is now in the stopped state

runc list

# now delete the container

runc delete mycontainerid


Rootless containers

● runc has the ability to run containers without root privileges. This is referred to as rootless

● some parameters need to be passed to runc in order to run rootless containers.

●


Rootless containers● mkdir ~/mycontainer

● cd ~/mycontainer

● mkdir rootfs

● docker export $(docker create busybox) | tar -C rootfs -xvf -

● runc spec –rootless

● runc --root /tmp/runc run mycontainerid


moby

● Move away from monolithic docker● an open framework to assemble specialized

container systems.●


moby

● Tactical componentization● Support ecosystem


Container vs Distro building


Moby as it stands today

● https://github.com/moby/moby/issues/32871● Move the monolith

https://github.com/moby/moby/pull/33022● Discussions at

https://forums.mobyproject.org/t/topic-find-a-good-and-non-confusing-home-for-the-remaining-monolith/37/2●

https://github.com/moby/moby/issues/32871

https://github.com/moby/moby/pull/33022

https://forums.mobyproject.org/t/topic-find-a-good-and-non-confusing-home-for-the-remaining-monolith/37/2


Moby code org .. issues

● we have the code of the legacy "docker engine" (a monolith to be split out in multiple components) at the root and it's very confusing.

● api– cannot be moved yet, because it's used

externally● client

– cannot be moved yet, because it's used externally


Moby code org

● Moby– moby tool

● Monolith– the code where "docker engine" lives, to be

split out and eventually will disappear● Pkg

– cannot be moved yet, because it's used externally

● Vendor– vendoring


Infrastructure changes

● OCI specs● OCI Image spec● OCI Runtime spec● Storage● Networking●


Docker needs a file system


Security


filesystem performance


What is Device Mapper ?


Device Mapper and LVM


Device mapper and Userspace


Device mapper thin provisioning


How docker uses thin pool


Docker images


#15629● Docker with devicemapper driver and dm.thinpooldev lead to

data loss

● https://github.com/moby/moby/issues/15629● Steps to reproduce

– Create lvm thin pool using lvcreate or lvconvert

– Pass lvm thin pool for exclusive use by docker

– Run docker daemon with devicemapper driver and dm.thinpooldev

– Import volume to the docker or create new container

– Try to extend or make any operation on lvm thin pool using lvm tools like lvextend thin data

● Issue: Only one entity can create thin devices in pool. Either lvm or docker.

https://github.com/moby/moby/issues/15629


Solution

● configure direct-lvm mode for production● https://docs.docker.com/v1.10/engine/userguide/storagedriver/device-mapper-driver/#configure-direct-lvm-mode-for-production● Steps

https://docs.docker.com/v1.10/engine/userguide/storagedriver/device-mapper-driver/#configure-direct-lvm-mode-for-production


Networking

● Overlay networking


Docker networking


Container networking

● Two competing standards– Container Network Model (CNM) – docker– Container Network Interface (CNI) - CoreOS

● IPAM (IP address management) driver– Offload network responsibility/assignment– Avoid IP conflict and container routing issues– Enable dynamic, fan-like IPAM approaches– Operator visibility into container cloud


CNI model


CNM model


CNM interfacing approach


Real network setup.


Notary

● Based on The Update Framework (TUF)● publishers can sign their content offline using

keys kept highly secure● Software update systems are

– Application updaters– Library package managers– System package managers

● TUF is a spec and library for secure software update systems


Notary


Multiple Docker kits


SwarmKit

● Swarmkit modelled after containerD– SwarmD– SwarmCtl

● Protobuf3 with grpc over HTTP/2.0● Swarmkit masters and Raft leaders are mutual

exclusion● Master promotion /demotion can be done on

any node manually


Infrakit


VPNKit


DataKit


HyperKit


LinuxKit


Container landscape


Pause …

from docker to moby and back. what changed ?

Education