best practices running sql server on aws

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

John Chang ( 張書源 )Technology Evangelist, AWS

March 2017

SQL Server 在 AWS 的最佳實踐

What to Expect from the Session

• Microsoft SQL Server deployment options on AWS

• Understanding licensing options• Best practices:

• SQL Server on Amazon EC2• Amazon RDS for SQL Server

AWS 現況

約 130 億美元(過去12 個月，截止到2016 Q3)

55%成長(2015 Q3 v.s. 2016 Q3)

數百萬每月活躍用戶

AWS 全球基礎設施

16 區域

42可用區域

新地理區域巴黎寧夏

AWS Global Infrastructure

RegionsGeographic locationsConsists of at least two Availability Zones (AZs)

Availability ZonesClusters of data centersIsolated from failures in other Availability Zones

Availability Zones (AZs)

At least 2 AZs per region.Examples:

• US East (N. Virginia)• us-east-1a• us-east-1b• us-east-1c• us-east-1d• us-east-1e

• Asia Pacific (Tokyo)• ap-northeast-1a• ap-northeast-1b• ap-northeast-1c

Note: Conceptual drawing only. The number of Availability Zones (AZ) may vary.

US East (VA)

AZ - A AZ - B

AZ - C AZ - D

AZ - E

Asia Pacific (Tokyo)

AZ - A AZ - B

AZ - C

Achieving High Availability Using Multi-AZ

Availability Zone - A

Availability Zone - B

Availability Zone - C

Region

AWS Taiwan Customers

AWS 大數據分析服務

Amazon EMR

Amazon Elasticsearch

Amazon Kinesis

Amazon Redshift

Amazon Quicksight

Amazon Machine Learning

Hadoop, Spark, HBase, Hive, Presto, Mahout, Pig, Zeppelin

Elasticsearch 即時串流資料資料倉儲商業智慧機器學習

AWS 大數據分析服務

Amazon EMR

Amazon Elasticsearch

Amazon Kinesis

Amazon Redshift

Amazon Quicksight

Amazon Machine Learning

Hadoop, Spark, HBase, Hive, Presto, Mahout, Pig, Zeppelin

Elasticsearch 即時串流資料資料倉儲商業智慧機器學習

Amazon Athena

使用標準的SQL語法分析儲存在 Amazon

S3的資料

Architecture

Availability Zone

Private SubnetPublic Subnet

Availability Zone


Remote Users

SampleMicrosoft

Architecture

Virtual Private Gateway

Corporate Office

IISApp

IISWeb

IISApp

IISWeb

VPN

AWS Direct Connect

InternetGateway

RDGW

VPC NATGateway

RDGW

VPC NATGateway

AWS Directory Service

AWS Directory Service

MS SQL

MS SQL

Always On Availability

Group

VPC Endpoint Amazon S3

Auto Scaling

Secure remote administration architecture

Availability Zone

Gateway Security Group Web Security Group


Accept TCP Port 443 from Admin IP

Accept traffic from Gateway SG

AWS Administrator

Corporate Data Center

WEB2

TCP 443 WEB1RDGW

Requires one connection:• Connect to the RD Gateway, and the gateway proxies the RDP or PowerShell connection to the back-

end instance.

Deploying SQL Server on AWSChoosing the Best Option for Your Needs

Choose the Best Option for Your Needs

Choose the Best Option for Your Needs

ü Managed physicalinfrastructure

ü Managed OS installation

ü Managed scalingü OS-level control

ü Managed physicalinfrastructure

ü Managed DB installation and backups

ü Managed OS and patching

ü Managed high availability and scaling

Your Responsibility

v App optimization, tuning

v Deploymentv Monitoringv High availabilityv Backupsv DB & OS patching

v App optimization, tuning

v Deploymentv Monitoring

Amazon RDS for SQL ServerConsider RDS firstFocus on:

• Business value tasks• High-level tuning tasks• Schema optimization

No in-house database expertise

Which Option Is Right for You?

SQL Server on Amazon EC2Need control over:

• DB instance & OS• Backups, Replication• Clustering• sysadmin role

Use options not in Amazon RDS

SQL Server Features at a Glance

* Self-installed

Amazon RDS for SQL Server SQL Server on Amazon EC2

Versions Supported: 2008 R2, 2012, 2014, 2016 2005*, 2008*, 2008 R2, 2012, 2014, 2016

Editions Supported: Express, Web, Standard, Enterprise

High Availability: Self-managed; AlwaysOn, Mirror, Log ShipAWS-managed

Encrypted storage using AWS KMS (all editions); TDE supportEncryption:

Authentication: Windows & SQL authentication

Maintenance plans & third-party toolsManaged automated backupsBackups:

Self-managedAutomatic software patchingMaintenance:

License Included• Available for Amazon RDS• Use an Amazon Machine Image

(AMI) that includes SQL Server for use on Amazon EC2

• Licensing cost included in the hourly cost of the EC2 instance or RDS DB instance

• Available for Web, Standard, and Enterprise editions

Licensing Options for SQL Server on AWS

Bring Your Own License• Amazon RDS and Amazon EC2 in

default tenancy require License Mobility through Software Assurance

• Can use per-core or per-socket licenses with Amazon EC2 Dedicated Hosts without License Mobility benefit

• License Mobility requires benefit verification with Microsoft

SQL Server on Amazon EC2Best Practices

SQL Server Best Practices on Amazon EC2

• Getting the most out of AWS storage options• Configure tempdb with multiple files on instance

storage (or fast Amazon EBS storage if instance storage is unavailable)

• Availability Zones and AlwaysOn Availability Groups: achieving both HA and DR with just two servers

• Failover cluster instances: I Get By With a Little Help From My Friends

• Instant file initialization

Amazon Elastic Compute Cloud (EC2)

Resizable compute capacityComplete control of your computing resourcesReduces the time required to obtain and boot new server instances to minutesAmazon

EC2

Instances and AMIs

Select an AMI based on:RegionOperating systemArchitecture (32-bit or 64-bit)Launch permissionsStorage for the root device

AMI

Instances

Instance

Launch instances of any

typeHost computer

Host computer

Amazon EC2 Instances

OS, Applications, & Configuration

AMI

Running or Stopped VM

Instances

AZ

VPC

Region

EBS

S3

EBS Snapshots S3 Buckets

EBS EBS EBS EBS EBS

AZ

Instances Instances

Amazon EBS vs. Amazon EC2 Instance Store

Amazon EBS• Data stored on an Amazon EBS volume can persist

independently of the life of the instance.• Storage is persistent.

Amazon EC2 Instance Store• Data stored on a local instance store persists only as long as the

instance is alive.• Storage is ephemeral.

AMI Types - Storage for the Root Device

Characteristic Amazon EBS-Backed Amazon Instance Store-BackedBoot time Usually < 1 minute Usually < 5 minutes

Size limit 16 TiB 10 GiB

Data persistence

The root volume is deleted when the instance terminates. Data on any other Amazon EBS volumes persists after instance termination.

Data on any instance store volumes persists only during the life of the instance.

Charges Instance usage, Amazon EBS volume usage, and storing your AMI as an Amazon EBS snapshot.

Instance usage and storing your AMI in Amazon S3.

Stopped state Can be stopped. Cannot be stopped.

Instance Lifecycle

AMI

pendingLaunch

runningrebootingReboot

Start

terminated

shutting-down

Terminate

Terminate

EBS-backed instances only

Stopstopping stopped

Choosing the Right Amazon EC2 Instance

EC2 instance types are optimized for different use cases and come in multiple sizes. This allows you to optimally scale resources to your workload requirements.AWS uses Intel® Xeon® processors for EC2 instances, providing customers with high performance and value.Consider the following when choosing your instances: Core count, memory size, storage size and type, network performance, and CPU technologies.Hurry Up and Go Idle - A larger compute instance can save you time and money, therefore paying more per hour for a shorter amount of time can be less expensive.

Amazon EBS LifecycleVast amounts of unused space Create

Call CreateVolume1 GB to 16 TB

AttachCall AttachVolume to affiliate with one Amazon EC2 instance

Attached and

In Use

• Format from Amazon EC2 instance OS

• Mount formatted drive

CreateSnapshotSnapshot to

Amazon S3Detach

Call DetachVolume

Deleted

Call DeleteVolume

Amazon EBS and Amazon S3

Amazon EBS Amazon S3

Paradigm Block storage with file system Object store

Performance Very fast FastRedundancy Across multiple servers in an

Availability ZoneAcross multiple facilities in a

RegionSecurity EBS Encryption – Data volumes

and SnapshotsEncryption

Access from the Internet?

No (1) Yes (2)

Typical use case It is a disk drive Online storage

(1) Accessible from the Internet if mounted to server and set up as FTP, etc.(2) Only with proper credentials, unless ACLs are world-readable

Amazon Elastic Block Storage

What is Amazon Elastic Block Storage (EBS)?• Network-attached block storage• Available for all instance types• Many instance types support EBS optimization

– dedicated channel for network storage I/O, eliminating contention with regular I/O

• Some instance types are EBS optimized, others offer it as an option

Amazon EBS Volume Types

Volume Type

GeneralPurpose: GP2

ProvisionedIOPS: PIOPS/IO1

Throughput Optimized: ST1

Cold HDD: SC1

Technology: SSD SSD Magnetic Magnetic

Sizes: 1 GiB – 16 TiB 4 GiB – 16 TiB 500 GiB – 16TiB 500 GiB – 16 TiB

Max. IOPS: 10,000 20,000 500 250

Max. Throughput:

160 MiB/sec 320 MiB/sec 500 MiB/sec 250 MiB/sec

Properties: 3 IOPS/1 GB, burstable up to 3000 IOPS for max 1 TiB volumes

Consistent provisioned performance, up to 50 IOPS/GB

Optimized for throughput, and sequential read/write workloads, baseline per TiB throughput, with burst capability

Amazon EC2 Instance Storage

What is instance storage?• Some instance types come with direct attached

disk-based storage• Included in the hourly cost• Data on instance storage does not persist a user-

initiated instance stop/start or hardware failure• Must be allocated at launch• Fast disk I/O without going over the network

Storage Performance for EC2 SQL Server

Consider IOPS and throughput needed by your workload

• Enable EBS optimization on instance• Create a single volume for data and

logs• Format with 64K allocation unit size• Match total EBS IOPS and throughput

to instance type• Stripe EBS PIOPS volumes for more

than 20,000 IOPS

Example volume layout:

C:\ Boot on General Purpose SSDD:\ Data and log files on PIOPS

single or striped setE:\ Backups on ST1 or SC1Z:\ Tempdb on instance storage (if available)

Configuring tempdb on Instance Storage

Move tembdb files to instance-storage-backed drives:1

2

ALTER DATABASE tempdbMODIFY FILE (NAME = tempdev, FILENAME = 'Z:\tempdb.mdf'); GOALTER DATABASE tempdbMODIFY FILE (NAME = templog, FILENAME = 'Z:\templog.mdf'); GO

Modify startup to grant service account access:icacls Z:\ /grant "NT SERVICE\MSSQLSERVER”:(OI)(CI)(F)

More tempdb Optimization Options

Consider:• Using multiple tempdb files (1:1 mapping with CPUs, up to 8)• Striping multiple instance storage disks together for higher I/O• Changing SQL Server service startup to Automatic (Delayed Start)

to allow instance storage to provision• Scripting/automating configuration on instance boot

Striping solution by consulting partner IFM Ltd.http://www.ifm.net.nz/cookbooks/amazon-sql-tempdb/index.html

SQL Server HA & DR on Amazon EC2

Use multiple Availability Zones• Instance-level and AZ-level failure tolerance• Synchronous replication

Options• Enterprise Edition: AlwaysOn Availability Groups• Standard Edition: Failover cluster instances using

partner block-level replication solution

Amazon Virtual Private Cloud (VPC)

Provision a private, isolated virtual network on the AWS cloud.Have complete control over your virtual networking environment.

AmazonVPC

VPCs and Subnets

A subnet defines a range of IP addresses in your VPC.You can launch AWS resources into a subnet that you select.A private subnet should be used for resources that won’t be accessible over the Internet.A public subnet should be used for resources that will be accessed over the Internet.Each subnet must reside entirely within one Availability Zone and cannot span zones.

Amazon VPC Example

Availability Zone A

Virtual Private Cloud

AWS Cloud

Public Subnet

Internet

Virtual Private Cloud

Availability Zone B

Private Subnet

Availability Zone C

VPN Only Subnet

DB Server DB Server

App Server

DB Server DB Server

DB Server

Web Server Web Server

NAT

Customer Network

R

Security in Your VPC

Security groupsNetwork access control lists (ACLs)

Subnet10.0.0.0/24

Internet GatewayVPN Gateway

VPC Router10.0.0.0/16

Security Group Security Group

Security Group

Network ACL Network ACL

Routing Table Routing Table

Instance Instance Instance Instance

Subnet10.0.1.0/24

VPN Connections

VPN Connectivity option Description

AWS Hardware VPN You can create an IPsec, hardware VPN connection between your VPC and your remote network.

AWS Direct Connect AWS Direct Connect provides a dedicated private connection from a remote network to your VPC.

AWS VPN CloudHubYou can create multiple AWS hardware VPN connections via your VPC to enable communications between various remote networks.

Software VPNYou can create a VPN connection to your remote network by using an Amazon EC2 instance in your VPC that’s running a software VPN appliance.

Multi-AZ AlwaysOn Availability Group

Availability Zone 1

Private Subnet

EC2Primary Replica

Availability Zone 2

Private Subnet

EC2Secondary

Replica

AWS Region

Synchronous CommitAutomatic Failover

Multi-region AlwaysOn Availability Group

Availability Zone 1

Private Subnet

EC2Primary Replica

Primary: 10.0.2.100WSFC: 10.0.2.101AG Listener: 10.0.2.102

AWS Region A

Availability Zone 2

Private Subnet

EC2Secondary

Replica


Availability Zone 1

Private Subnet

EC2Secondary

Replica


AWS Region B

Elastic IP Elastic IP

VPN

Synchronous CommitAutomatic Failover

Asynchronous CommitManual Failover

Failover Cluster Instance

Amazon EBS Amazon EBS

Availability Zone 1

Private Subnet

EC2Primary

Node

Availability Zone 2

Private Subnet

EC2Secondary

Node

AWS Region

Data Replication

SoftNAS / SIOS

SQL Server Instant File Initialization

What is database file initialization?• Normally, database and log files are initialized

to overwrite leftover disk data • File initialization causes some DB operations to

take longer• Instant database file initialization reclaims

unused disk space without zeroing it out

Instant File Initialization Security Concerns

• Deleted content is overwritten only when new data is written to file• Deleted content might be accessible by an unauthorized principal• Disclosure threat is reduced while the DB file is attached to the SQL

Server instance

Mitigations:• Apply restrictive discretionary ACLs on data files and backup files• Disable instant file initialization

SQL Server 2016 Install Time

Enabling Instant Database File Initialization

Post-Install or Other VersionsGrant Perform volume maintenance tasks to SQL Server service account

1. Open the Local Security Policy app,2. From Local Policy, choose User

Rights Assignment.3. Double-click Perform volume

maintenance tasks.4. Choose Add User or Group.

https://msdn.microsoft.com/en-us/library/ms175935.aspx

Amazon RDS for SQL ServerBest Practices

Amazon RDS for SQL Server Best Practices

• Moving/migrating data from Amazon RDS• Leveraging SQL Server’s native .bak

backup and restore• Using highly available SQL Server

deployments in Amazon RDS• Managing SQL Server storage and I/O

performance• Leveraging existing Active Directory with

Amazon RDS for SQL Server

Moving Data In and Out of RDS for SQL Server

.BAK File Import and ExportLeverages SQL Server’s native backup functionality

AWS Database Migration ServiceMinimize downtime during migrations, migrate between different DB platforms, Schema Conversion Tool

AWS MarketplaceThird-party data import and export tools and solutions

1

3

4

Microsoft SQL Server Database Publishing Wizard, Import/ExportExport to T-SQL files, load using sqlcmd

2

.bak File Import and Export Prerequisites

RDS for SQL Server DB Instance✓

S3 Bucket (to store .bak files)✓

DB Option Group enabling SQLSERVER_BACKUP_RESTORE✓

SSMS or other client to connect to DB instance and execute the stored procedures

✓

Using .bak File Import and Export

/* Restoring from backup file */exec msdb.dbo.rds_restore_database@restore_db_name='your database name'@s3_arn_to_restore_from='arn:aws:s3:::<bucket>/<file path>';

/* Exporting to backup file */exec msdb.dbo.rds_backup_database@source_db_name='your database name', @s3_arn_to_backup_to='arn:aws:s3:::<bucket>/<file path>', @overwrite_S3_backup_file=1;

/* Check job status */exec msdb.dbo.rds_task_status;

High Availability in RDS for SQL Server

Amazon RDS for SQL Server Multi-AZ• Principal and secondary DB nodes in

different Availability Zones • Leverages SQL Server DB mirroring• Automatic failover (typically, 1–2 minutes)• Always run production workloads in

Multi-AZ mode

Amazon RDS Multi-AZ in-Depth

Failure scenarios mitigated:• Loss of availability in primary AZ• Loss of network connectivity to principal DB node• Compute unit or storage failure on principal DB node

Failover process:

Consider:• Implementing retry logic at the application layer—trigger manual failover to test• Impact on mirroring of changing heavy workloads (for example, index rebuilds)

Mirroring stopped

Address apply debt

Promote to master

Change DNS endpoint

Provision new

secondary

Storage I/O Performance

Amazon RDS Amazon EC2Type Size Performance Size Performance Burst Capacity Pricing Model

Magnetic Storage

20 GiB–1 TiB ~100 IOPS 1 GiB–1 TiB ~ 100 IOPS Yes, several hundred IOPS

Allocated storage; I/O operations

General Purpose (SSD)

20 GiB–4 TiB(min. 100 GiBrecommended)

3 IOPS/GiB 1 GiB–16 TiB 3 IOPS/GiB for volumes 1 TiB or less, up to 10,000 IOPS for larger volumes

Yes, up to 3000 IOPS per volume, subject to credits (< 1 TiB in size)

Allocated storage

Provisioned IOPS(SSD)

100 GiB–4 TiB(min. 200 GiB for Standard edition and up)

Up to max. 20,000 IOPS

4 GiB–16 TiB Up to 20,000 IOPS

No, fixed allocation

Allocated storage; provisionedIOPS

Storage I/O Performance Planning

Amazon RDS storage throughputdepends on DB instance class

I/O requests sizes: Provisioned IOPS can handle I/O up to 256 KB in sizeI/Os larger than 32 KB consume multiple IOPS

Maximum storage IOPS: 20,000Capacity for concurrent I/O–optimize latency

1

3

2

Average queue depth: I/O requests waiting to be serviced~5 outstanding I/O op/1000 IOPS provisioned

4

Existing Active Directory Integration

• Windows Authentication support provided by AWS Directory Service Microsoft AD directory

• RDS DB instance joined to the directory operated domain

• Integrate with existing AD deployment using a forest trust

• Configure inbound trust on the external forest + outbound trust in the directory

• Configure conditional forwarders for the two domains

Thank you!

Useful Resources

Microsoft SQL Server on AWShttps://aws.amazon.com/windows/products/sql/

Deploying SQL Server on AWS (whitepaper)https://d0.awsstatic.com/whitepapers/RDS/Deploying_SQLServer_on_AWS.pdf

Amazon RDS for SQL Server Supported Featureshttp://amzn.to/2dHsNEU

Implementing Microsoft Windows Server Failover Clustering and SQL Server AlwaysOn Availability Groups in the AWS Cloudhttp://amzn.to/2cQTD1h