getting release management right for sql server

@_AlexYates_#SqlSatIceland

Getting Release Management right for SQL Server Alex Yates


Sérstakar þakkir til styrktaraðila

SQL SATURDAY | #602 | REYKJAVIK 2017


DLM Consultant

workingwithdevs.com

[email protected]

Alex Yates

@_AlexYates_


@_AlexYates_ | #SqlSatIceland


AgileScrumLeanDevOpsIterativeContinuous Delivery

Developers

http://commons.wikimedia.org/wiki/File:Athlete_at_starting_block.jpg


MonitoringDeployment

IntegrityPerformance

DBAs



source:%20http://commons.wikimedia.org/wiki/File:Mocking_Bird_Argument.jpg


Farm Credit Services of America (FCSA)

• 100 person IT team, 14 sub-teams

• Database version control inconsistent

• Deployment process manual

• Delivery was slow and unreliable



• Growing test suite

• Delivery faster and more reliable by catching issues early

• Standardised source control process

• Automated builds/deploys from source control


The deployment pipeline


Databases are hard

• Schema changes vs existing data• Reference data vs production data• Teamwork and testing• Database drift (change outside

process, e.g. production hot-fixes)


The deployment pipeline


AgendaVersion control

Model vs migrationsContinuous integration

Automating deploymentsRelease management

Drift


Version Control


Version control - LMGTFY

http://lmgtfy.com/?q=version+control

http://lmgtfy.com/?q=version+control





Drift


There’s more than one way to skin a cat



There’s more than one way to skin automatea cat



V1 V2


V1 V2

Migrations-based solutions


V1 V2

Model-based solutions


“There's nothing more reliable than keeping track of exactly the scripts you intend to run, and running them, without trying to compare model and guess.”



“There's nothing more reliable than keeping track of exactly the scripts you intend to run, and running them, without trying to compare model and guess.”

Paul Stovell, built Octopus Deploy

http://docs.octopusdeploy.com/display/OD/SQL+Server+databases


http://docs.octopusdeploy.com/display/OD/SQL+Server+databases


“As soon as you have multiple changes on a single aspect of an object, ordering and the ability to detect which change needs to be made gets very complicated.”



“As soon as you have multiple changes on a single aspect of an object, ordering and the ability to detect which change needs to be made gets very complicated.”

Gert Drapers, built DataDude

https://blogs.msdn.microsoft.com/gertd/2009/06/05/declarative-database-development/


https://blogs.msdn.microsoft.com/gertd/2009/06/05/declarative-database-development/


Model vs migrations

http://workingwithdevs.com/delivering-databases-migrations-vs-state/

http://workingwithdevs.com/delivering-databases-migrations-vs-state/


Model Easier (less control) Better for

sprocs/functions Better for

large/distributed teams Better for frequent

changes Better for dependency

nightmares Drift: rolled back Better for development

Migrations More control

(harder/needs discipline) Better for data

migrations Better for small teams Better for infrequent

changes Better for simple data

stores Drift: ignored Better for automation

VS


DLMConsultants.com/model-vs-mig


Demo





Drift


Continuous Integration


What is CI?


What is CI?

Continuous Integration (CI) is a development practice that requires developers to integrate code into a shared repository several times a day. Each check-in is then verified by an automated build, allowing teams to detect problems early.

www.thoughtworks.com/continuous-integration

http://www.thoughtworks.com/continuous-integration


http://www.jamesshore.com/Blog/Continuous-Integration-on-a-Dollar-a-Day.html




Build servers


VCS Build Test Sync

Upgrade scriptDatabase package

Test results

The automated build process


Builds/Tests are run on CI agents

Builds/Tests are managed on

CI server

CI server

How do build servers work?


DLMConsultants.com/ci


Demo





Drift


What is CD?

Continuous Delivery (CD) is the natural extension of Continuous Integration: an approach in which teams ensure that every change to the system is releasable, and that we can release any version at the push of a button. Continuous Delivery aims to make releases boring, so we can deliver frequently and get fast feedback on what users care about.

www.thoughtworks.com/continuous-delivery


Deploying databases, a history lesson

SOURCE TARGET

Process:Manual scripting


Deploying databases, a history lessonProcess:Database comparison tool

SOURCE TARGET



SOURCE TARGET

Process:Generating the script is separate from manually executing the script



SOURCE

Problem:Human error

Doh!

HumanError



SOURCE

Problem:Drift (unscheduled or unmanaged changes)

DriftDrift


What do you really need to deploy with confidence?

Replacing a single deployment script with a credible release



1. Source model




1. Source model

2. Target model




1. Source model

2. Target model

3. Upgrade script




1. Source model

2. Target model

3. Upgrade script

4. Change report

SELECT ContactsID,ContactFullName,Address1,CountryCode,



How should deployments work?

1. Does target match target model?2. Run script3. Does target match source model?

TARGET


What should happen if anything goes wrong?

1. If target doesn’t match target model (or source model) – error and abort

2. If script fails – roll back transaction

3. If target doesn’t match source model after script execution – do not roll back but send warning, then roll-forward


Demo





Drift


Release Management


Release Management Tools


How do release management tools work?


Deployments are run on RM agents

Deployments are managed on

RM server

RM server

How do release management tools work?

Builds/Tests are run on CI agents

Builds/Testsare managed on

CI server

CI server

Just like a CI server!


RM server

Test

Staging

Production

AppDB

AppDB

AppDB

Servers, environments, machines and agents


RM server

Server RM software responsible for managing your deployments

AppDB

AppDB

AppDB

Servers, environments, machines and agentsTest

Staging

Production


RM server

App DB

App DB

App

Machines physical or virtual boxes

that you wish to deploy to

AppDB

AppDB

AppDB


Staging

Production


Environments groups of machines that you will deploy to at the

same time

RM server

App DB

App DB

App DB

AppDB

AppDB

AppDB


Staging

Production


RM server

Agentsrelease management software that executes deployments on

machines

AppDB

AppDB

AppDB


Staging

Production


RM server

AppDB

AppDB

AppDB

How database deployments should workTest

Staging

Production


RM server

AppDB

AppDB

AppDB

DB server doesn’t actually need an agent


Staging

Production


RM server

AppDB

AppDB

AppDB

Install DB deployment tool and an RM agent on a machine

(wherever you like)


Staging

Production


RM server

AppDB

AppDB

AppDB

RM server sends instruction to RM agent


Staging

Production


RM server

AppDB

AppDB

AppDB

RM agent calls DB deployment tool on its

machine


Staging

Production


RM server

AppDB

AppDB

AppDB

DB deployment tool talks to target databases


Staging

Production


RM server

AppDB

AppDB

AppDB

Some people install an agent and DB deployment tool on the

RM server for simplicity


Staging

Production


RM server

AppDB

If environments are isolated by firewalls you may need several

DB deployment tool installations

Production

DB

DB

Test

Staging

App

App

How database deployments should work


In summary• Install the release management server on a test server or

build server etc• Install agents that will deploy applications on the application

servers themselves• Install agents that will deploy databases on a machine that

has access to your target databases• Install DB deployment tool on the same machine as the

agent that you will use to deploy your databases


Demo





Drift


“When making changes directly on production, you are making a decision that the delay due to poor cycle time is more expensive than the risk of making a mistake.”


“When making changes directly on production, you are making a decision that the delay due to poor cycle time is more expensive than the risk of making a mistake.”

Alex Yates, built this slide


The bad stuff:

• Accidental roll-backs (model)

• Failed deployments (migrations)

• Environment inconsistency (migrations)


DriftStrategies to help

Improve cycle time Strict security policies Monitor drift (play with DDL

triggers) Redgate DLM Dashboard

(free)https://www.simple-talk.com/sql/database-administration/database-deployment-the-bits-database-version-drift/

https://www.simple-talk.com/sql/database-administration/database-deployment-the-bits-database-version-drift/


DriftStrategies to help

Improve cycle time Strict security policies Monitor drift (play with DDL

triggers) Redgate DLM Dashboard

(free)http://www.red-gate.com/products/dlm/dlm-dashboard/

http://www.red-gate.com/products/dlm/dlm-dashboard/


Demo





Drift


• Standardised VCS, CI and RM processes• FCSA delivering much more efficiently

• FCSA process model for sister organisations

• The team all have excellent CV’s



• Standardised VCS, CI and RM processes• FCSA delivering much more efficiently

• FCSA process model for sister organisations

• The team all have excellent CV’s


http://www.codeaperture.io/2016/09/13/how-redgate-helped-define-our-process/



Siloed sparrows suck at DLM.


source:%20http://commons.wikimedia.org/wiki/File:Mocking_Bird_Argument.jpg


Heated hippos are closed minded.



Model Easier (less control) Better for

sprocs/functions Better for

large/distributed teams Better for frequent

changes Better for dependency

nightmares Drift: rolled back Better for development

Migrations More control

(harder/needs discipline) Better for data

migrations Better for small teams Better for infrequent

changes Better for simple data

stores Drift: ignored Better for automation

VS

Clever people consider options …


… use appropriate tools …


… and make pain on his smug face!


This stuff makes a big difference.



http://www.red-gate.com/sql-source-control/

http://www.red-gate.com/sql-source-control/


DLMConsultants.com/rm


Takk fyrir að koma á SQL Saturday IcelandSérstakar þakkir til styrktaraðila

SQL SATURDAY | #602 | REYKJAVIK 2017

http://www.sqlsaturday.com/602/Sessions/SessionEvaluation.aspx

http://www.sqlsaturday.com/602/Sessions/SessionEvaluation.aspx


Questions?

Links: DLMConsultants.com/rmContact me: [email protected]


Image sourcesAuthor Source Information

Chiltepinster Wikimedia Commons Mocking Bird Argument.jpg – Wikimedia Commons. This file is licensed under the Creative Commons Attribution-Share Alike 3.0 Unported license. Source on Wikimedia Commons: “Own work”

Bit Boy Flickr The elephant in the room – Flickr. This file is licensed under the Creative Commons Attribution 2.0 Generic license.

Nils Rinaldi Flickr Hippo fight 2/3 – Flickr. This file is licensed under the Creative Commons Attribution 2.0 Generic license.

My own collection Taken by/property of Alex Yates Kitten, “There’s more than one way to skin a cat!”

Memegenerator.net Memegenerator.net I don’t always edit database. Content designed to be shared and delivered with credit to memegenerator.net.

Ctrl.Alt.Design ctrla.lt Social Media share icons

http://commons.wikimedia.org/wiki/File:Mocking_Bird_Argument.jpg

http://commons.wikimedia.org/wiki/File:Mocking_Bird_Argument.jpg

http://en.wikipedia.org/wiki/en:Creative_Commons

http://creativecommons.org/licenses/by-sa/3.0/deed.en

http://creativecommons.org/licenses/by-sa/3.0/deed.en

https://www.flickr.com/photos/bitboy/

https://www.flickr.com/photos/44124323641@N01/246805948

https://creativecommons.org/licenses/by/2.0/


https://www.flickr.com/photos/nilsrinaldi/

https://www.flickr.com/photos/nilsrinaldi/5158417206/in/photolist-8RQeHG-6q2btE-sa1Pq8-aoA8XG-8RQeGE-g2Jpns-kKuki6-qKVjKK-8RQeKW-oA3fvn-7SzhUB-9xGtsp-pJKvSt-kNUtzR-rmjcY7-e9rWjm-cs3KtE-4UFehR-o2N1kG-7ncr52-s6oCP-rdumpt-6CGxgo-ceMb-dz7Fq8-26AYfc-9jAfuP-owxCHJ-o8nbm1-7p7Nxk-jRQQNa-9Agi8x-4ivCVG-8zmib1-33AMcN-fiRUKK-9UZwUm-dQCLYP-bek6oK-9NqsTq-pvjgYp-qFK1Lq-7QzWQc-6bmRTg-61bqE-bvEySt-bjT3jY-5YPGPg-7ERG57-7ERrg7





http://memegenerator.net/

http://memegenerator.net/instance/59793617

http://ctrla.lt/social-media-icons-modern-metro-download/

http://ctrla.lt/social-media-icons-modern-metro-download/

getting release management right for sql server

Technology