develop qnap nas app by docker
TRANSCRIPT
Develop QNAP NAS Appby Docker
Terry Chen 陳彥村
Engineer @ QNAP
Who am I?
https://tw.linkedin.com/in/seterrychen
https://github.com/seterrychen
https://twitter.com/seterrychen
● Backend engineer : Java, Python, Linux
● QNAP Container Station/Linux Station
Outline
● How to develop QNAP NAS App● Base knowledge of Docker● Create Docker App on QNAP NAS in QPKG format
NAS － Network attached storage
From : http://www.slideshare.net/QNAP_Inc/general-tech-trainingstorageabcqnap20140302compressed-copy
Network
QNAP AppCenter
● Customize your QNAP NAS with Apps
QPKG
● QNAP package, like deb(Debian/Ubuntu),rpm(Red Hat/CentOS)
● QNAP tool-chain:
https://sourceforge.net/projects/qosgpl/files/QNAP%20NAS%20Tool%20Chains/
● QPKG format:
[/share/CACHEDEV3_DATA/.qpkg/QDK/tester] # ls
arm-x09/ arm-x31/ config/ package_routines shared/x86_64/ arm-x19/ arm-x41/ icons/qpkg.cfg x86/ x86_ce53xx/
Package application by QDK/QDK2
● QDKhttp://www.qnap.com.tw/event/dev/cht/p_qdk.php
● QDK2
https://github.com/qnap-dev/qdk2
● QDK2 with Docker
https://github.com/fcwu/docker-qdk2
QDK
http://download.qnap.com/dev/QDK_Quick_Start_Guide_v4_eng.pdf
QDK2
● https://github.com/qnap-dev/qdk2
● Debian/Ubuntu base tool
● Support various source
Docker review
What is Docker?
● Docker allows you to package an applicationwith all of its dependencies into a standardizedunit for software development.
OthersApp
image
Images center - Docker Hub
How to ship application by Docker
892607a8d00b 0 B
0b10aa9d5e8a 0 B
f4e6d72dbf27 22 B
9c6eed7ce0e9 57.66 MB
2eda4505e518 0 B
2a6ec6bae792 0 B
32f2a4cccab8 0 B
70e9a6907f10 125.1 MB
nginx:1.10
ImageLayer
$ docker pull nginx:1.10
Docker storage (version: 1.9.1)
vagrant@test:~$ sudo docker infoContainers: 0Images: 2Server Version: 1.9.1Storage Driver: aufs Root Dir: /var/lib/docker/aufs Backing Filesystem: extfs………
892607a8d00b 0 B
0b10aa9d5e8a 0 B
f4e6d72dbf27 22 B
9c6eed7ce0e9 57.66 MB
2eda4505e518 0 B
2a6ec6bae792 0 B
32f2a4cccab8 0 B
70e9a6907f10 125.1 MB
Nginx:1.10
vagrant@test:~$ sudo ls -l /var/lib/docker/aufs/difftotal 32drwxr-xr-x 2 root root 4096 May 1 07:40 0b10aa9d5e8a3dd8de176539bd...drwxr-xr-x 2 root root 4096 May 1 07:39 2a6ec6bae79235ea27adcceff6...drwxr-xr-x 2 root root 4096 May 1 07:39 2eda4505e518d08147e4fab915...drwxr-xr-x 2 root root 4096 May 1 07:15 32f2a4cccab87ff519e17c9cf4...drwxr-xr-x 21 root root 4096 May 1 07:15 70e9a6907f1068b3e08a9e6357...drwxr-xr-x 2 root root 4096 May 1 07:40 892607a8d00ba2ad196a7d8a43...drwxr-xr-x 7 root root 4096 May 1 07:40 9c6eed7ce0e9c7f8ab8e95d2f7...drwxr-xr-x 3 root root 4096 May 1 07:40 f4e6d72dbf272d7e2ba0ef48f7...
Docker storage (version: 1.9.1)
vagrant@test:~$ sudo docker infoContainers: 0Images: 2Server Version: 1.9.1Storage Driver: aufs Root Dir: /var/lib/docker/aufs Backing Filesystem: extfs………
892607a8d00b 0 B
0b10aa9d5e8a 0 B
f4e6d72dbf27 22 B
9c6eed7ce0e9 57.66 MB
2eda4505e518 0 B
2a6ec6bae792 0 B
32f2a4cccab8 0 B
70e9a6907f10 125.1 MB
Nginx:1.10
root@test:/var/lib/docker/aufs/diff/70e9a6907f1068b3e08a9e...# lsbin boot dev etc home lib lib64 media mnt optproc root run sbin srv sys tmp usr var
root@test:/var/lib/docker/aufs/diff/9c6eed7ce0e9c7f8ab8e95...# lsetc lib tmp usr var
AUFS – Another Union File System
From : https://docs.docker.com/engine/userguide/storagedriver/aufs-driver/
Run container (version: 1.9.1)$ docker run -d -p 80:80 nginx:1.10
DockerClient
DockerDaemon
Send Request
892607a8d00b 0 B
0b10aa9d5e8a 0 B
f4e6d72dbf27 22 B
9c6eed7ce0e9 57.66 MB
2eda4505e518 0 B
2a6ec6bae792 0 B
32f2a4cccab8 0 B
70e9a6907f10 125.1 MB
Read/Writer Container Layer
Nginx:1.10
Send Response
Image read only
Simplified diagram
copy-on-write strategy
892607a8d00b 0 B
0b10aa9d5e8a 0 B
f4e6d72dbf27 22 B
9c6eed7ce0e9 57.66 MB
2eda4505e518 0 B
2a6ec6bae792 0 B
32f2a4cccab8 0 B
70e9a6907f10 125.1 MB
Read/Writer
Image read onlyNginx:1.10
Read/Writer
Read/Writer
Container-1
Container-2
Container-3
Basic isolation of Linux
● chroot – change root directory
root@test:/var/lib/docker/aufs/diff/70e9a6907f1068b3e08a9e...# lsbin boot dev etc home lib lib64 media mnt optproc root run sbin srv sys tmp usr var
root@test:/var/lib/docker/aufs/diff/70e9a6907f1068b3e08a9e...# chroot . bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)root@test:/# ls -albin boot dev etc home lib lib64 media mnt optproc root run sbin srv sys tmp usr var
root@test:/# ls -al ../bin boot dev etc home lib lib64 media mnt optproc root run sbin srv sys tmp usr var
Basic isolation of Linux
● chroot – change root directory
Filesystem Isolation
root@test:/# mount -t proc proc /proc
vagrant@test:~$ ps -efroot 2192 906 0 02:54 ? 00:00:00 docker-proxy -proto ...root 2215 906 0 02:54 ? 00:00:00 docker-proxy -proto .........vagrant 2275 1915 0 03:03 pts/5 00:00:00 ps -ef
# using chrootroot@test:/# ps -efroot 2192 906 0 02:54 ? 00:00:00 docker-proxy -proto ...root 2215 906 0 02:54 ? 00:00:00 docker-proxy -proto .........root 2277 2269 0 03:03 ? 00:00:00 ps -ef
How to isolate by Docker
● Linux Kernel feature:– Cgroups
– Namespaces● Mount namespaces● UTS namespaces● IPC namespaces● PID namespaces● Network namespaces● User namespaces
Stop container
From : http://dockone.io/article/783 From:https://twitter.com/turnoff_us/status/710736561676132352/photo/1
Docker on Linux
From : https://en.wikipedia.org/wiki/Docker_(software)
libcontainer
libvirt systemd-nspawnLXC
cgroups namespaces
SELinux
AppArmor
Netfilter
Netlink
capabilities
Docker
Linux kernel
Dockerize App by Dockerfile
From : https://github.com/nginxinc/docker-nginx/blob/41aa13f7d2c24407e483c40fb1e8b33e73462ff1/mainline/jessie/Dockerfile
Build and Ship
● Build– docker build -t your_account/app_name .
● Ship– docker login --username=yourname
– docker push your_account/app_name
Start/Stop script#!/bin/sh...
case "$1" in start) ENABLED=$(/sbin/getcfg $QPKG_NAME Enable -u -d FALSE -f $CONF) if [ "$ENABLED" != "TRUE" ]; then echo "$QPKG_NAME is disabled." exit 1 fi : ADD START ACTIONS HERE ;;
stop) : ADD STOP ACTIONS HERE ;;
restart) $0 stop $0 start ;;
*) echo "Usage: $0 {start|stop|restart}" exit 1esac
exit 0
Example: ghost
● Just a blogging platform https://ghost.org
Build qpkg$ git clone https://github.com/fcwu/docker-qdk2.git
$ cd docker-qdk2
$ docker run -it --rm \
-v ${PWD}/example:/example \
dorowu/qdk2 bash -c "cd /example; make"
$ ls -l example/ghost/build/ghost_0.7.4_x86_64.qpkg
-rw-r--r-- 1 u u 24242 Dec 30 13:10 example/ghost/build/ghost_0.7.4_x86_64.qpkg
Container Station API
● Version 1.6 - qbus post com.qnap.dqpkg/qpkg '{"qpkg": "online-document", "action": "install"}'
- qbus post com.qnap.dqpkg/qpkg '{"qpkg": "online-document", "action": "start"}'
- qbus post com.qnap.dqpkg/qpkg '{"qpkg": "online-document", "action": "stop"}'
- qbus post com.qnap.dqpkg/qpkg '{"qpkg": "online-document", "action": "remove"}'
docker-compose
● docker-compose.yml of Online-documentapp: image: dorowu/online-document.amd64:0.8.111 ports: - "127.0.0.1:10001:5000" volumes: - /share:/share - /mnt/rf/cd:/mnt/rf/cd - /var/run/qbus.sock:/var/run/qbus.sock
● Read more: https://docs.docker.com/compose/
Apache config
● online-document.confProxyPass /online-document/ws ws://127.0.0.1:10001/online-document/ws retry=0
ProxyPass /online-document/ http://127.0.0.1:10001/online-document/ retry=0
Create Issues/Pull Request
● https://github.com/fcwu/docker-qdk2
● https://github.com/qnap-dev/container-station-api
How to debug (In my case)
From : http://9gag.com/gag/anYZ9Eo/my-code-works-but-i-don-t-know-why
How to Learn (In my case)
“Try to learn something about everything andeverything about something.”
Thomas Huxley Quotes
Read more : 陈天：如何快速掌握一门技术？ (http://blog.jobbole.com/77712/)
Thank you