deploying to production 50+ times a day - calgary agile users group 2015

DEPLOYING TO PRODUCTION 50+

TIMES A DAY

1

Stuart Charlton

SOFTWARE IS EATING THE WORLD

2

AMAZON.COM FACTS• ~ $90 Billion annual revenue

• May 2011 Stats

• Mean time between deployments: 11.6s

• Max deployments in a single hour: 1,079

• Mean hosts receiving a deploy: 10,000

• Max hosts receiving a deploy: 30,000

3

http://amazon.com

WALMART FACTS

• $480 billion annual revenue

• @WalmartLabs is the SF Bay Area Software Company driving R&D

• 100,000 core OpenStack cloud cluster

• Many open source projects

4

SOFTWARE IS CHANGING INDUSTRIES

5

$3.5B valuation Financial Services

$3.5B valuation Travel & Hospitality

$19.5B valuation Transportation

$3.2B Acquisition by Google Home Automation

$20B valuation Entertainment

$26B Evaluation Automotive

EVERY COMPANY NEEDS TO BUILD GREAT

SOFTWARE

6

Francisco Gonzalez, CEO

“Banks need to take on Amazon and Google or die. The shift to digital requires a complete overhaul of banks technology…it is a matter of survival.”

WHAT MAKES THESE COMPANIES TICK?

7

CLOUD PLATFORM

AGILE

NEW KNOWLEDGE CHANGES THE COURSE

OF HISTORY• Better Technology

• Better Tactics

• Better Strategy

8

TAKING AGILE & LEAN THINKING

END-TO-END

9

$Busin

ess

Devel

opm

ent

QA

Ope

ratio

nsAn

alytic

sMission: Deliver Value

TAKING AGILE & LEAN THINKING

END-TO-END

10

$Busin

ess

Devel

opm

ent

QA

Ope

ratio

nsAn

alytic

sTimeline: Months

ELIMINATE WASTE: WAITING,

WORK IN PROGRESS

11

$Busin

ess

Devel

opm

ent

& Q

A

Ope

ratio

nsAn

alytic

sTimeline: Weeks-Months

ELIMINATE WASTE: HAND-OFFS,

“NOT MY PROBLEM”

12

$Busin

ess

Devel

oper

s & Q

A

&Ope

rato

rs

Analy

tics

Timeline: Days-Weeks

BUILDING A DEVOPS CULTURE

• Professional movement

• Sharing culture (end-to-end value)

• “You build it, you run it”

• Anti-fragile, continuous improvement

• Automation and metrics driven

13

THE BEGINNING: 2001

14

No Smart Phones

But most software ships on CDs

People are excited about the internet

No Cloud Computing

2001

15

Companies like Google and Amazon were inventing new ways of computing

to solve problems no one had ever dealt with before

AUTOMATION ENABLES PRODUCTIVITY &

QUALITY

16

EVERYBODY WANTS TO GO FAST

17

BUT GOING FAST IS NOT ENOUGH

18

have to go safe

“OPERATIONS IS THE

19

PLATFORMS, ARCHITECTURES &

PROCESSES PROTECT US

20

MONOLITHIC ARCHITECTURE

21

Relational Database

Data Access

Service

HTML JavaScript MVC

Service

Monolithic ApplicationBrowser

MONOLITHIC ARCHITECTURES

• Modularity Dependent Upon Language / Frameworks

• Change Cycles Tightly Coupled

• Inefficient Scaling

• Can Be Intimidating to New Developers

• Obstacle to Scaling Development

• Requires Long-Term Commitment to Technical Stack 22

MICROSERVICES

23

…HTTP

HTTP

HTTP

HTTP

HTTP

HTTP

AMQPAMQP

Relational DB

Key/Value Store

Graph DB

MICROSERVICES• Services Oriented Architecture AND Services-

Oriented Delivery

• Modularity Based on Component Services

• Enable Frequent Deploys

• Efficient Scaling

• Individual Components Less Intimidating to New Developers

• Enables Scaling of Development 24

CONWAY’S LAW

25

Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the

organization's communication structure.

Melvyn Conway, 1967

http://martinfowler.com/articles/microservices.html#OrganizedAroundBusinessCapabilities

SPAN SILOS WITH MICROSERVICES

26

Data Access

Service

HTML JavaScript MVC

Service

UI S

pecialists

Middlew

are S

pecialists

DB

As

Business C

apability

Business C

apability

Business C

apability

Siloed Functional Teams

http://martinfowler.com/articles/microservices.html#OrganizedAroundBusinessCapabilities

Siloed Application Architectures

Cross- functional Teams

Microservice Architectures

PARTITIONING

• By Noun (e.g. product info service)

• By Verb (e.g. shipping service)

• Single Responsibility Principle(http://programmer.97things.oreilly.com/wiki/index.php/The_Single_Responsibility_Principle)

27

http://programmer.97things.oreilly.com/wiki/index.php/The_Single_Responsibility_Principle

YOU MUST BE THIS TALL •RAPID PROVISIONING

•BASIC MONITORING

•RAPID APPLICATION DEPLOYMENT

•DEVOPS CULTURE

28

http://martinfowler.com/bliki/MicroservicePrerequisites.html

https://www.flickr.com/photos/gusset/3723961589

IT’S GONNA TAKE A PLATFORM…

29

PLATFORM CAPABILITIES

• Environment Provisioning

• On-Demand/Automatic Scaling

• Failover/Resilience

• Routing/Load Balancing

• Data Service Operations

• Monitoring 30

•Distributed/Versioned Config • Service Registration/Discovery • Routing/Load Balancing • Service Integration • Fault Tolerance • Asynchronous Messaging

OPEN PLATFORMS

• Cloud Foundry

• Kubernetes

• Docker

• Mesosphere

31

Life as an Operator

Developer Operator

32

Install runtime and container

Deploy to dev

33


Install services (db, messaging, hadoop, …)

Deploy to dev

34



Setup load-balancing, SSL termination and dynamic routing

Deploy to dev

35



Setup / config High Availability

Deploy to dev


36




Setup APM

Deploy to dev


37




Setup APM

Setup log streaming

Deploy to dev


38




Setup APM

Setup log streaming

Deploy to dev

App Deployed


39



Setup load-balancing and dynamic routing


Setup APM

Setup log streaming

Deploy to dev

Deploy to Test

Deploy to QA

Promote to production

Scale up

Update app

Scale down

Redeploy

Unbind service

Bind service

Restart server

Change route

Scale Platform

Create service

Delete App

Blue-green deployment

Check logsEscale memory

Update app server

Change JDK

Upgrade app

40



Setup load-balancing and dynamic routing


Setup APM

Setup log streaming

Deploy to dev

Deploy to Test

Deploy to QA

Promote to production

Scale up

Update app

Scale down

Redeploy

Unbind service

Bind service

Restart server

Change route

Scale Platform

Create service

Delete App

Blue-green deployment

Check logsEscale memory

Update app server

Change JDK

Upgrade app

41

Deploy to dev

Life as a Developer or Operator with a Platform

42

Deploy to dev

> cf pushCloud

ControllerHealth

Manager

NATS (message bus)

RouterApp Deployed

Using a Platform

43

WHAT MAKES THESE COMPANIES TICK?

44

CLOUD PLATFORM

AGILE

AGILE

• Taken truly end-to-end

• Collaborative QA+Operations with Dev

• Continuous Delivery

• Adopting a DevOps Culture

• Adopting Automation at Scale, Responsibly

45

PLATFORM

• Conway’s Law: Team = Product

• Microservices

• Adopting an Open Platform to Get Started

46

GET THE BOOKS

47

WITH THANKS

48

This presentation includes content by:

Andrew Clay Shafer Matt Stine Fred Melo

+ many others @ Pivotal