automated infrastructure

Kaunas Java User Group, 2014 1

Automated InfrastructureLaurynas Tretjakovaslaurynas.tretjakovas@gmail.com

Kaunas Java User Group, 2014 2

Agenda• Manual Infrastructure vs Automated Infrastructure• Puppet• Docker• Vagrant• Demo• Q & A

Kaunas Java User Group, 2014 3

Manual vs Automated Automated infrastructure has the following benefits:• Version control the configuration• Reusable• Cheaper for the organization• Provision a fleet of nodes with a single click• Works well with continious integration and delivery• New servers on demand – scalability• Smaller chance for human error• Is fun, motivates and creates a sense of accomplishment

Kaunas Java User Group, 2014 4

Kaunas Java User Group, 2014 5

Puppet• Open source configuration management utility• Ensures that the node is in a defined state• Ruby DSL• Cross-platform• Agents can fetch configuration form the master• Enterprise edition is free if managing 10 or less nodes• Lots of modules in Puppet Forge

Kaunas Java User Group, 2014 6

Puppet• Puppet configuration files are called manifests

• Manifests do not apply actions from top to bottom, they use relationships to determine the order, in which the resources should be set up

Kaunas Java User Group, 2014 7

Puppet

Kaunas Java User Group, 2014 8

Kaunas Java User Group, 2014 9

Kaunas Java User Group, 2014 10

Kaunas Java User Group, 2014 11

Kaunas Java User Group, 2014 12

Puppet• Do not forget to define dependencies• Test your configuration with CI, use puppet parser validate and puppet-lint• Write module tests• Use puppet librarian to update and install modules• Manage node configuration with Hiera• Write classes and in the main fail only include the classes

Kaunas Java User Group, 2014 13

Kaunas Java User Group, 2014 14

Docker• An open source platform for developers and system administrators to build, ship, and run distributed applications

• “Dockerized” apps are portable and can run on most popular Linux distributions

• On other OS, use boot2docker VM – a lightweight Linux distribution made specifically to run Docker containers

• System administrators can use Docker to standardize development environments and abstract away differences in different OS distributions

• Docker hub contains thousands of apps

Kaunas Java User Group, 2014 15

Kaunas Java User Group, 2014 16

Docker Images Docker image is a read-only template. An image could contain an Ubuntu operating system with Tomcat and your web application installed. Images are used to create Docker containers. Docker provides a simple way to build new images or update existing images, or you can download Docker images that other people have already created. Docker images are the build component of Docker

Kaunas Java User Group, 2014 17

Docker Registries Docker registries hold images. These are public or private stores from which you upload or download images. The public Docker registry is called Docker Hub. It provides a huge collection of existing images for your use. These can be images you create yourself or you can use images that others have previously created. Docker registries are the distribution component of Docker

Kaunas Java User Group, 2014 18

Docker Containers Docker containers are similar to a directory. A Docker container holds everything that is needed for an application to run. Each container is created from a Docker image. Docker containers can be run, started, stopped, moved, and deleted. Each container is an isolated and secure application platform. Docker containers are the run component of Docker

Kaunas Java User Group, 2014 19

Docker Images• Image name pattern – vendor/imageName:tag• Each image starts from a base image, for example a base Ubuntu image• Each image consists of a series of layers that are combined into a single image, every change to an image is a new layer, which can be saved as a new image• Docker images can be built by hand or by using a Dockerfile with a set of instructions

Kaunas Java User Group, 2014 20

Docker Containers• Running a command from an image creates a container. After that command exits, container stops• One container should only do a single thing, don‘t run an ssh service• Upstart does not work in containers• To share data between containers create data volumes and mount them into containers

Kaunas Java User Group, 2014 21

Docker Containers• Containers get an IP from a pool, there is no way to explicity assign an ip• Use –p flag to forward ports when starting a container• To access one container from the other, you have to link them: docker run -d --name web --link db:db training/webapp• Linked container‘s IP is available from the hosts file

Kaunas Java User Group, 2014 22

Ubuntu with Oracle JDK

Kaunas Java User Group, 2014 23

Tomcat 8

Kaunas Java User Group, 2014 24

Web Application

Kaunas Java User Group, 2014 25

Docker• Group common commands into a single statement, so only one layer is created• Install with --no-install-recommends• Don‘t overwrite existing tags, use new ones each time so you can fall back easily• Use automated builds in the hub• Avoid private registries as Docker Hub server files over a cdn

Kaunas Java User Group, 2014 26

Kaunas Java User Group, 2014 27

Vagrant• Create VMs locally for development

• Portable and version controlled configuration

• Replicate production environment

• Every team member has the exact same setup

• No more “works on my machine”

• A place to test your automated infrastructure scripts

• Has plugin API

• Provisioning with Puppet, Chef, Ansible and more

Kaunas Java User Group, 2014 28

Vagrant• Install Virtualbox or other VM provider• Install Vagrant• Create Vagrantfile with configuration• Base boxes are available from vagrant cloud• vagrant up

Kaunas Java User Group, 2014 29

Vagrant• Vagrant can automatically build and run docker images for you• You do not save much from this, but it can spin up a boot2docker VM if your host OS does not support docker

Kaunas Java User Group, 2014 30

Vagrant

Kaunas Java User Group, 2014 31

Vagrant

Kaunas Java User Group, 2014 32

Vagrant• Use shell provisioning to install and run Puppet• Use NFS file shares on Linux hosts• Do not write to shared folders, it is slow• On Windows hosts, even reading from shared folders can be slow, so avoid doing that as much as you can

Kaunas Java User Group, 2014 33

Demo