kubernetes úklid mezi kontejnery tomáš kukrál @tomkukral ... · chain prerouting (policy accept...
TRANSCRIPT
![Page 1: Kubernetes úklid mezi kontejnery Tomáš Kukrál @tomkukral ... · Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes) target prot source destination KUBE-PORTALS-CONTAINER all 0.0.0.0/0](https://reader034.vdocuments.pub/reader034/viewer/2022042307/5ed394e57ec1ed47a70d65e3/html5/thumbnails/1.jpg)
Kubernetes úklid mezi kontejnery
Tomáš Kukrál@tomkukral
ICT FIT ČVUT & tech@SU
LinuxDays 2015
![Page 2: Kubernetes úklid mezi kontejnery Tomáš Kukrál @tomkukral ... · Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes) target prot source destination KUBE-PORTALS-CONTAINER all 0.0.0.0/0](https://reader034.vdocuments.pub/reader034/viewer/2022042307/5ed394e57ec1ed47a70d65e3/html5/thumbnails/2.jpg)
Cluster / cloud s kontejnery:
1. služby na fyzických strojích2. virtualizované služby3. orchestrace více strojů
NO more PETS!
Kde běží? Kolikrát? Opravdu? Neběží!
![Page 3: Kubernetes úklid mezi kontejnery Tomáš Kukrál @tomkukral ... · Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes) target prot source destination KUBE-PORTALS-CONTAINER all 0.0.0.0/0](https://reader034.vdocuments.pub/reader034/viewer/2022042307/5ed394e57ec1ed47a70d65e3/html5/thumbnails/3.jpg)
![Page 4: Kubernetes úklid mezi kontejnery Tomáš Kukrál @tomkukral ... · Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes) target prot source destination KUBE-PORTALS-CONTAINER all 0.0.0.0/0](https://reader034.vdocuments.pub/reader034/viewer/2022042307/5ed394e57ec1ed47a70d65e3/html5/thumbnails/4.jpg)
Problémy a požadavky:
orchestracesíťováníúložiště
![Page 5: Kubernetes úklid mezi kontejnery Tomáš Kukrál @tomkukral ... · Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes) target prot source destination KUBE-PORTALS-CONTAINER all 0.0.0.0/0](https://reader034.vdocuments.pub/reader034/viewer/2022042307/5ed394e57ec1ed47a70d65e3/html5/thumbnails/5.jpg)
Kubernetes
![Page 6: Kubernetes úklid mezi kontejnery Tomáš Kukrál @tomkukral ... · Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes) target prot source destination KUBE-PORTALS-CONTAINER all 0.0.0.0/0](https://reader034.vdocuments.pub/reader034/viewer/2022042307/5ed394e57ec1ed47a70d65e3/html5/thumbnails/6.jpg)
![Page 7: Kubernetes úklid mezi kontejnery Tomáš Kukrál @tomkukral ... · Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes) target prot source destination KUBE-PORTALS-CONTAINER all 0.0.0.0/0](https://reader034.vdocuments.pub/reader034/viewer/2022042307/5ed394e57ec1ed47a70d65e3/html5/thumbnails/7.jpg)
![Page 8: Kubernetes úklid mezi kontejnery Tomáš Kukrál @tomkukral ... · Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes) target prot source destination KUBE-PORTALS-CONTAINER all 0.0.0.0/0](https://reader034.vdocuments.pub/reader034/viewer/2022042307/5ed394e57ec1ed47a70d65e3/html5/thumbnails/8.jpg)
Kube - základní pojmy:
node podrc - replication-controllersvc - servicepv - persistent volumepvc - pv claim
![Page 9: Kubernetes úklid mezi kontejnery Tomáš Kukrál @tomkukral ... · Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes) target prot source destination KUBE-PORTALS-CONTAINER all 0.0.0.0/0](https://reader034.vdocuments.pub/reader034/viewer/2022042307/5ed394e57ec1ed47a70d65e3/html5/thumbnails/9.jpg)
Kube - architektura:
etcd
apiservercontroller-managerschedulerproxykubelet
labels
kubectl
![Page 10: Kubernetes úklid mezi kontejnery Tomáš Kukrál @tomkukral ... · Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes) target prot source destination KUBE-PORTALS-CONTAINER all 0.0.0.0/0](https://reader034.vdocuments.pub/reader034/viewer/2022042307/5ed394e57ec1ed47a70d65e3/html5/thumbnails/10.jpg)
etcd:
z projektu CoreOSkey-value úložištědistribuované konzistentní
sledování změn, TTL
Raft algoritmus
API + HTTP
3+ stroje (majority)
static || discovery initial-cluster
curl -s 127.0.0.1:4001/v2/keys/registry | json_pp
github.com/coreos/etcd
![Page 11: Kubernetes úklid mezi kontejnery Tomáš Kukrál @tomkukral ... · Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes) target prot source destination KUBE-PORTALS-CONTAINER all 0.0.0.0/0](https://reader034.vdocuments.pub/reader034/viewer/2022042307/5ed394e57ec1ed47a70d65e3/html5/thumbnails/11.jpg)
apiserver:spravuje ostatní součástivalidaceREST operaceudržuje stav - neprovádí akce
--service-cluster-ip-range--bind-address--etcd-servers
![Page 12: Kubernetes úklid mezi kontejnery Tomáš Kukrál @tomkukral ... · Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes) target prot source destination KUBE-PORTALS-CONTAINER all 0.0.0.0/0](https://reader034.vdocuments.pub/reader034/viewer/2022042307/5ed394e57ec1ed47a70d65e3/html5/thumbnails/12.jpg)
kubelet:agent spuštěný na nodepracuje s kontejneryvytváření, zastavovánísprávce storage
problémy s kontejnerizací (RBD plugin)RBD v kontejneru a sdílení /var/lib/kubelet/
--api_servers--max_pods--enable-server
![Page 13: Kubernetes úklid mezi kontejnery Tomáš Kukrál @tomkukral ... · Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes) target prot source destination KUBE-PORTALS-CONTAINER all 0.0.0.0/0](https://reader034.vdocuments.pub/reader034/viewer/2022042307/5ed394e57ec1ed47a70d65e3/html5/thumbnails/13.jpg)
controller-manager:reguluje stav systémuprovádí změnykonvergovaný stav
--master--node-monitor-grace-period--pod-eviction-timeout
![Page 14: Kubernetes úklid mezi kontejnery Tomáš Kukrál @tomkukral ... · Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes) target prot source destination KUBE-PORTALS-CONTAINER all 0.0.0.0/0](https://reader034.vdocuments.pub/reader034/viewer/2022042307/5ed394e57ec1ed47a70d65e3/html5/thumbnails/14.jpg)
scheduler:umístění kontejnerů, podů, pv, ...
--master
![Page 15: Kubernetes úklid mezi kontejnery Tomáš Kukrál @tomkukral ... · Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes) target prot source destination KUBE-PORTALS-CONTAINER all 0.0.0.0/0](https://reader034.vdocuments.pub/reader034/viewer/2022042307/5ed394e57ec1ed47a70d65e3/html5/thumbnails/15.jpg)
proxy:--master--bind-address
![Page 16: Kubernetes úklid mezi kontejnery Tomáš Kukrál @tomkukral ... · Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes) target prot source destination KUBE-PORTALS-CONTAINER all 0.0.0.0/0](https://reader034.vdocuments.pub/reader034/viewer/2022042307/5ed394e57ec1ed47a70d65e3/html5/thumbnails/16.jpg)
Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes) target prot source destination KUBE-PORTALS-CONTAINER all 0.0.0.0/0 0.0.0.0/0
Chain KUBE-PORTALS-CONTAINER (1 references) target prot source destination DNAT tcp 0.0.0.0/0 172.18.0.1 /* default/kubernetes: */ tcp dpt:443 to:172.17.0.1:59333 DNAT tcp 0.0.0.0/0 172.18.0.2 /* default/linuxdays-mysql:mysql */ tcp dpt:3306 to:172.17.0.1:60546 DNAT tcp 0.0.0.0/0 172.18.0.3 /* default/linuxdays-web:http */ tcp dpt:80 to:172.17.0.1:55715
![Page 17: Kubernetes úklid mezi kontejnery Tomáš Kukrál @tomkukral ... · Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes) target prot source destination KUBE-PORTALS-CONTAINER all 0.0.0.0/0](https://reader034.vdocuments.pub/reader034/viewer/2022042307/5ed394e57ec1ed47a70d65e3/html5/thumbnails/17.jpg)
![Page 18: Kubernetes úklid mezi kontejnery Tomáš Kukrál @tomkukral ... · Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes) target prot source destination KUBE-PORTALS-CONTAINER all 0.0.0.0/0](https://reader034.vdocuments.pub/reader034/viewer/2022042307/5ed394e57ec1ed47a70d65e3/html5/thumbnails/18.jpg)
Souvislosti mezi objekty?
labels!
![Page 19: Kubernetes úklid mezi kontejnery Tomáš Kukrál @tomkukral ... · Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes) target prot source destination KUBE-PORTALS-CONTAINER all 0.0.0.0/0](https://reader034.vdocuments.pub/reader034/viewer/2022042307/5ed394e57ec1ed47a70d65e3/html5/thumbnails/19.jpg)
kubectl:kubectl [flags] kubectl [commands]
get, describe, create, delete, scale, stop, expose, label
nástroj k ovládání clusteru
kubectl get po kubectl create -f souborkubectl scale rc arc --replicas=10kubectl get no,rc,svc
![Page 20: Kubernetes úklid mezi kontejnery Tomáš Kukrál @tomkukral ... · Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes) target prot source destination KUBE-PORTALS-CONTAINER all 0.0.0.0/0](https://reader034.vdocuments.pub/reader034/viewer/2022042307/5ed394e57ec1ed47a70d65e3/html5/thumbnails/20.jpg)
Kube - postup nasazení:
kubernetest.shetcd.sh
tomkukral/gentoo-hyperkubetomkukral/gentoo-etcd
Ansible role: kube-common, kube-proxy, kube-minion
![Page 21: Kubernetes úklid mezi kontejnery Tomáš Kukrál @tomkukral ... · Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes) target prot source destination KUBE-PORTALS-CONTAINER all 0.0.0.0/0](https://reader034.vdocuments.pub/reader034/viewer/2022042307/5ed394e57ec1ed47a70d65e3/html5/thumbnails/21.jpg)
Síťování:
pod - pod node - storage
Flannel? Weave? OpenVPN? BIRD!
![Page 22: Kubernetes úklid mezi kontejnery Tomáš Kukrál @tomkukral ... · Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes) target prot source destination KUBE-PORTALS-CONTAINER all 0.0.0.0/0](https://reader034.vdocuments.pub/reader034/viewer/2022042307/5ed394e57ec1ed47a70d65e3/html5/thumbnails/22.jpg)
nodes: 10.38.2.0/24
pods: 172.17.{node}.0/24 - dbr0služby: 172.18.0.0/24
OSPF area 0:nody propagují síť na "dbr0"
![Page 23: Kubernetes úklid mezi kontejnery Tomáš Kukrál @tomkukral ... · Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes) target prot source destination KUBE-PORTALS-CONTAINER all 0.0.0.0/0](https://reader034.vdocuments.pub/reader034/viewer/2022042307/5ed394e57ec1ed47a70d65e3/html5/thumbnails/23.jpg)
![Page 24: Kubernetes úklid mezi kontejnery Tomáš Kukrál @tomkukral ... · Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes) target prot source destination KUBE-PORTALS-CONTAINER all 0.0.0.0/0](https://reader034.vdocuments.pub/reader034/viewer/2022042307/5ed394e57ec1ed47a70d65e3/html5/thumbnails/24.jpg)
Storage:
PODy jsou postradatelné
pluginy: emptyDir, hostPath, gitReponfs, iscsigcePersistentDisk, awsElasticBlockStoreglusterfs, rbdcephfs
používáme RBD plugin:/dev/rbd0 on
/var/lib/kubelet/plugins/kubernetes.io/rbd/rbd/kube-image-pv07/dev/rbd0 on
/var/lib/kubelet/pods/d2f26023c/volumes/kubernetes.io~rbd/pv07
RBD plugin v kontejneru je problém
![Page 25: Kubernetes úklid mezi kontejnery Tomáš Kukrál @tomkukral ... · Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes) target prot source destination KUBE-PORTALS-CONTAINER all 0.0.0.0/0](https://reader034.vdocuments.pub/reader034/viewer/2022042307/5ed394e57ec1ed47a70d65e3/html5/thumbnails/25.jpg)
Kubernetes na ICT FIT ČVUT:
i-{01..02} - Supermicro H8DGT-HFi-master - VM v OpenNebule
kc-{01..05} - HP ProLiant DL380g5 pro Ceph3x MON + 15x OSDkube pool pro RBD
![Page 26: Kubernetes úklid mezi kontejnery Tomáš Kukrál @tomkukral ... · Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes) target prot source destination KUBE-PORTALS-CONTAINER all 0.0.0.0/0](https://reader034.vdocuments.pub/reader034/viewer/2022042307/5ed394e57ec1ed47a70d65e3/html5/thumbnails/26.jpg)
![Page 27: Kubernetes úklid mezi kontejnery Tomáš Kukrál @tomkukral ... · Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes) target prot source destination KUBE-PORTALS-CONTAINER all 0.0.0.0/0](https://reader034.vdocuments.pub/reader034/viewer/2022042307/5ed394e57ec1ed47a70d65e3/html5/thumbnails/27.jpg)
![Page 28: Kubernetes úklid mezi kontejnery Tomáš Kukrál @tomkukral ... · Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes) target prot source destination KUBE-PORTALS-CONTAINER all 0.0.0.0/0](https://reader034.vdocuments.pub/reader034/viewer/2022042307/5ed394e57ec1ed47a70d65e3/html5/thumbnails/28.jpg)
Ukázka - web LinuxDays.cz
![Page 30: Kubernetes úklid mezi kontejnery Tomáš Kukrál @tomkukral ... · Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes) target prot source destination KUBE-PORTALS-CONTAINER all 0.0.0.0/0](https://reader034.vdocuments.pub/reader034/viewer/2022042307/5ed394e57ec1ed47a70d65e3/html5/thumbnails/30.jpg)
BA