framework agnostic discovery
TRANSCRIPT
![Page 1: Framework Agnostic Discovery](https://reader031.vdocuments.pub/reader031/viewer/2022022413/58edaa671a28ab1d568b46ed/html5/thumbnails/1.jpg)
FRAMEWORK-AGNOSTICDISCOVERY
| Product Manager, | Tim Gross Joyent @0x74696d
![Page 2: Framework Agnostic Discovery](https://reader031.vdocuments.pub/reader031/viewer/2022022413/58edaa671a28ab1d568b46ed/html5/thumbnails/2.jpg)
CONTAINER-NATIVE?Containers are a first class citizen.
Each container is an equal peer on the network.
Discovery should be framework-agnostic.
![Page 3: Framework Agnostic Discovery](https://reader031.vdocuments.pub/reader031/viewer/2022022413/58edaa671a28ab1d568b46ed/html5/thumbnails/3.jpg)
REMEMBER: YOUR MISSION ISNOT "MANAGE VMs."
Your mission is what your application does for yourorganization.
Infrastructure (undifferentiated heavy lifting) is incidental costand incidental complexity.
Application containers make the full promise of cloudcomputing possible...
but require new ways of working.
![Page 4: Framework Agnostic Discovery](https://reader031.vdocuments.pub/reader031/viewer/2022022413/58edaa671a28ab1d568b46ed/html5/thumbnails/4.jpg)
Triton Elastic Container Service
Run Linux containers securelyon bare-metal in public cloud
Or run on-premise (it's opensource!)
![Page 5: Framework Agnostic Discovery](https://reader031.vdocuments.pub/reader031/viewer/2022022413/58edaa671a28ab1d568b46ed/html5/thumbnails/5.jpg)
Director of DevOps
![Page 6: Framework Agnostic Discovery](https://reader031.vdocuments.pub/reader031/viewer/2022022413/58edaa671a28ab1d568b46ed/html5/thumbnails/6.jpg)
... Docker in production since Oct 2013
![Page 7: Framework Agnostic Discovery](https://reader031.vdocuments.pub/reader031/viewer/2022022413/58edaa671a28ab1d568b46ed/html5/thumbnails/7.jpg)
WHAT DOCKER SOLVED FOR US:Human-and-machine-readable build documentation.
No more "works on my machine."
Fix dependency isolation.
Interface-based approach to application deployment.
Deployments are fast!
DevOps kool-aid for everyone!
![Page 8: Framework Agnostic Discovery](https://reader031.vdocuments.pub/reader031/viewer/2022022413/58edaa671a28ab1d568b46ed/html5/thumbnails/8.jpg)
OK, WHAT'S WRONG?
![Page 9: Framework Agnostic Discovery](https://reader031.vdocuments.pub/reader031/viewer/2022022413/58edaa671a28ab1d568b46ed/html5/thumbnails/9.jpg)
![Page 10: Framework Agnostic Discovery](https://reader031.vdocuments.pub/reader031/viewer/2022022413/58edaa671a28ab1d568b46ed/html5/thumbnails/10.jpg)
![Page 11: Framework Agnostic Discovery](https://reader031.vdocuments.pub/reader031/viewer/2022022413/58edaa671a28ab1d568b46ed/html5/thumbnails/11.jpg)
![Page 12: Framework Agnostic Discovery](https://reader031.vdocuments.pub/reader031/viewer/2022022413/58edaa671a28ab1d568b46ed/html5/thumbnails/12.jpg)
NAT
![Page 13: Framework Agnostic Discovery](https://reader031.vdocuments.pub/reader031/viewer/2022022413/58edaa671a28ab1d568b46ed/html5/thumbnails/13.jpg)
NATDocker's use of bridging and NAT noticeably increases the
transmit path length; vhost-net is fairly efficient at transmittingbut has high overhead on the receive side... In real network-
intensive workloads, we expect such CPU overhead to reduceoverall performance.
IBM Research Report: An Updated Performance Comparisonof Virtual Machines and Linux Containers
![Page 14: Framework Agnostic Discovery](https://reader031.vdocuments.pub/reader031/viewer/2022022413/58edaa671a28ab1d568b46ed/html5/thumbnails/14.jpg)
CAN WE AVOID NAT?--host networking
port conflicts
port mapping at LB
![Page 15: Framework Agnostic Discovery](https://reader031.vdocuments.pub/reader031/viewer/2022022413/58edaa671a28ab1d568b46ed/html5/thumbnails/15.jpg)
CAN WE AVOID NAT?Bridge (not --bridge)
networking
![Page 16: Framework Agnostic Discovery](https://reader031.vdocuments.pub/reader031/viewer/2022022413/58edaa671a28ab1d568b46ed/html5/thumbnails/16.jpg)
Can get IP per container
May need 2nd NIC
Scaling w/ subnet per host
![Page 17: Framework Agnostic Discovery](https://reader031.vdocuments.pub/reader031/viewer/2022022413/58edaa671a28ab1d568b46ed/html5/thumbnails/17.jpg)
DNSSimple discovery! But...
Can't address individual hosts behind a record.*
No health checking.*
TTL caching.
![Page 18: Framework Agnostic Discovery](https://reader031.vdocuments.pub/reader031/viewer/2022022413/58edaa671a28ab1d568b46ed/html5/thumbnails/18.jpg)
![Page 19: Framework Agnostic Discovery](https://reader031.vdocuments.pub/reader031/viewer/2022022413/58edaa671a28ab1d568b46ed/html5/thumbnails/19.jpg)
![Page 20: Framework Agnostic Discovery](https://reader031.vdocuments.pub/reader031/viewer/2022022413/58edaa671a28ab1d568b46ed/html5/thumbnails/20.jpg)
NETWORKING STILL SUCKS!Containers don't have their own NIC on the data center
network
Pass through proxy for all outbound requests
All packets go through NAT or port forwarding
![Page 21: Framework Agnostic Discovery](https://reader031.vdocuments.pub/reader031/viewer/2022022413/58edaa671a28ab1d568b46ed/html5/thumbnails/21.jpg)
THE CONTAINER-NATIVEALTERNATIVE?
Cut the cruft!
Push responsibility of the application topology away from thenetwork infrastructure and into the application itself where it
belongs.
![Page 22: Framework Agnostic Discovery](https://reader031.vdocuments.pub/reader031/viewer/2022022413/58edaa671a28ab1d568b46ed/html5/thumbnails/22.jpg)
RESPONSIBILITIES OF ACONTAINER
Registration
Self-introspection
Heartbeats
Look for change
Respond to change
![Page 23: Framework Agnostic Discovery](https://reader031.vdocuments.pub/reader031/viewer/2022022413/58edaa671a28ab1d568b46ed/html5/thumbnails/23.jpg)
![Page 24: Framework Agnostic Discovery](https://reader031.vdocuments.pub/reader031/viewer/2022022413/58edaa671a28ab1d568b46ed/html5/thumbnails/24.jpg)
![Page 25: Framework Agnostic Discovery](https://reader031.vdocuments.pub/reader031/viewer/2022022413/58edaa671a28ab1d568b46ed/html5/thumbnails/25.jpg)
NO SIDECARSSidecar needs to reach intoapplication container
Unsuited for multi-tenantsecurity
Deployment of sidecar bound todeployment of app
![Page 26: Framework Agnostic Discovery](https://reader031.vdocuments.pub/reader031/viewer/2022022413/58edaa671a28ab1d568b46ed/html5/thumbnails/26.jpg)
APPLICATION-AWARE HEALTHCHECKS
No packaging tooling into another service
App container lifecycle separate from discovery service
Respond quickly to changes
![Page 27: Framework Agnostic Discovery](https://reader031.vdocuments.pub/reader031/viewer/2022022413/58edaa671a28ab1d568b46ed/html5/thumbnails/27.jpg)
LEGACY PRE-CONTAINER APPSRegistration: wrap start of app in a shell script
Self-introspection: self-test?
Heartbeats: um...
Look for change: ???
Respond to change: profit?
![Page 29: Framework Agnostic Discovery](https://reader031.vdocuments.pub/reader031/viewer/2022022413/58edaa671a28ab1d568b46ed/html5/thumbnails/29.jpg)
CONTAINERBUDDY:A shim to help make existing apps container-native
Registration: registers to Consul on startup
Self-introspection: execute user-defined health check
Heartbeats: send health status w/ TTL to Consul
Look for change: poll Consul for changes
Respond to change: execute user-defined responsebehavior
![Page 30: Framework Agnostic Discovery](https://reader031.vdocuments.pub/reader031/viewer/2022022413/58edaa671a28ab1d568b46ed/html5/thumbnails/30.jpg)
![Page 31: Framework Agnostic Discovery](https://reader031.vdocuments.pub/reader031/viewer/2022022413/58edaa671a28ab1d568b46ed/html5/thumbnails/31.jpg)
NO SUPERVISIONContainerbuddy is PID1
Returns exit code of shimmed processback to Docker Engine (or Triton) and
dies
Attaches stdout/stderr from app tostdout/stderr of container
![Page 32: Framework Agnostic Discovery](https://reader031.vdocuments.pub/reader031/viewer/2022022413/58edaa671a28ab1d568b46ed/html5/thumbnails/32.jpg)
{ "consul": "consul:8500", "services": [ { "name": "nginx", "port": 80, "health": "/usr/bin/curl --fail -s http://localhost/health", "poll": 10, "ttl": 25 } ], "backends": [ { "name": "app", "poll": 7, "onChange": "/opt/containerbuddy/reload-nginx.sh" } ]}
![Page 33: Framework Agnostic Discovery](https://reader031.vdocuments.pub/reader031/viewer/2022022413/58edaa671a28ab1d568b46ed/html5/thumbnails/33.jpg)
$ cat ./nginx/opt/containerbuddy/reload-nginx.sh
# fetch latest virtualhost template from Consul k/vcurl -s --fail consul:8500/v1/kv/nginx/template?raw \ > /tmp/virtualhost.ctmpl
# render virtualhost template using values from Consul and reload Nginxconsul-template \ -once \ -consul consul:8500 \ -template \ "/tmp/virtualhost.ctmpl:/etc/nginx/conf.d/default.conf:nginx -s reload"
![Page 34: Framework Agnostic Discovery](https://reader031.vdocuments.pub/reader031/viewer/2022022413/58edaa671a28ab1d568b46ed/html5/thumbnails/34.jpg)
$ less ./nginx/default.ctmpl
# for each service, create a backend{{range services}}upstream {{.Name}} { # write the health service address:port pairs for this backend {{range service .Name}} server {{.Address}}:{{.Port}}; {{end}}}{{end}}
![Page 35: Framework Agnostic Discovery](https://reader031.vdocuments.pub/reader031/viewer/2022022413/58edaa671a28ab1d568b46ed/html5/thumbnails/35.jpg)
server { listen 80; server_name _;
# need ngx_http_stub_status_module compiled-in location /health { stub_status on; allow 127.0.0.1; deny all; }
{{range services}} location /{{.Name}}/ { proxy_pass http://{{.Name}}/; proxy_redirect off; } {{end}}}
![Page 36: Framework Agnostic Discovery](https://reader031.vdocuments.pub/reader031/viewer/2022022413/58edaa671a28ab1d568b46ed/html5/thumbnails/36.jpg)
nginx: image: 0x74696d/containerbuddy-demo-nginx mem_limit: 512m ports: - 80 links: - consul:consul restart: always environment: - CONTAINERBUDDY=file:///opt/containerbuddy/nginx.json command: > /opt/containerbuddy/containerbuddy nginx -g "daemon off;"
![Page 37: Framework Agnostic Discovery](https://reader031.vdocuments.pub/reader031/viewer/2022022413/58edaa671a28ab1d568b46ed/html5/thumbnails/37.jpg)
echo 'Starting Consul.'docker-compose -p example up -d consul
# get network info from consul. alternately we can push this into# a DNS A-record to bootstrap the clusterCONSUL_IP=$(docker inspect example_consul_1 \ | json -a NetworkSettings.IPAddress)
echo "Writing template values to Consul at ${CONSUL_IP}"curl --fail -s -X PUT --data-binary @./nginx/default.ctmpl \ http://${CONSUL_IP}:8500/v1/kv/nginx/template
echo 'Opening consul console'open http://${CONSUL_IP}:8500/ui
![Page 38: Framework Agnostic Discovery](https://reader031.vdocuments.pub/reader031/viewer/2022022413/58edaa671a28ab1d568b46ed/html5/thumbnails/38.jpg)
![Page 39: Framework Agnostic Discovery](https://reader031.vdocuments.pub/reader031/viewer/2022022413/58edaa671a28ab1d568b46ed/html5/thumbnails/39.jpg)
Starting application servers and Nginxexample_consul_1 is up-to-dateCreating example_nginx_1...Creating example_app_1...Waiting for Nginx at 72.2.115.34:80 to pick up initial configuration....................Opening web page... the page will reload every 5 seconds with any updates.Try scaling up the app!docker-compose -p example scale app=3
![Page 40: Framework Agnostic Discovery](https://reader031.vdocuments.pub/reader031/viewer/2022022413/58edaa671a28ab1d568b46ed/html5/thumbnails/40.jpg)
echo 'Starting application servers and Nginx'docker-compose -p example up -d
# get network info from Nginx and poll it for livenessNGINX_IP=$(docker inspect example_nginx_1 \ | json -a NetworkSettings.IPAddress)echo "Waiting for Nginx at ${NGINX_IP} to pick up initial configuration."while : do sleep 1 curl -s --fail -o /dev/null "http://${NGINX_IP}/app/" && break echo -ne .doneechoecho 'Opening web page... the page will reload every 5 seconds'echo 'with any updates.'open http://${NGINX_IP}/app/
![Page 41: Framework Agnostic Discovery](https://reader031.vdocuments.pub/reader031/viewer/2022022413/58edaa671a28ab1d568b46ed/html5/thumbnails/41.jpg)
![Page 42: Framework Agnostic Discovery](https://reader031.vdocuments.pub/reader031/viewer/2022022413/58edaa671a28ab1d568b46ed/html5/thumbnails/42.jpg)
DOES IT BLEND SCALE?$ docker-compose -p example scale app=3Creating and starting 2... doneCreating and starting 3... done
![Page 43: Framework Agnostic Discovery](https://reader031.vdocuments.pub/reader031/viewer/2022022413/58edaa671a28ab1d568b46ed/html5/thumbnails/43.jpg)
![Page 44: Framework Agnostic Discovery](https://reader031.vdocuments.pub/reader031/viewer/2022022413/58edaa671a28ab1d568b46ed/html5/thumbnails/44.jpg)
![Page 45: Framework Agnostic Discovery](https://reader031.vdocuments.pub/reader031/viewer/2022022413/58edaa671a28ab1d568b46ed/html5/thumbnails/45.jpg)
![Page 46: Framework Agnostic Discovery](https://reader031.vdocuments.pub/reader031/viewer/2022022413/58edaa671a28ab1d568b46ed/html5/thumbnails/46.jpg)
The Old Way The Container-Native WayExtra network hop fromLB or local proxy
Direct container-to-containercommmunication
NAT Containers have their own IPDNS TTL Topology changes propogate
immediatelyHealth checks in the LB Applications report their own
healthTwo build & orchestrationpipelines
Focus on your app alone
VMs Secure multi-tenant bare-metal