how to build scalable websites in the cloud
TRANSCRIPT
1
How to Build
Scalable Web Applications
in the Cloud
Reference Architectures and Best Practices
December 14, 2010
2
Your Panel Today
Presenting:
• Michael Crandell, CEO
• Brian Adler, Sr. Professional Services Architect
Q&A:
• Jarryd Hensel, Account Manager
Please use the questions window to ask questions any time!
3
Agenda
• What’s, why’s, when’s, and how’s of scalable web apps
• Comparison of traditional and cloud resource models
• Reference architecture for a scalable web application
• Best practices for each tier of the reference architecture
• Summary & conclusions
Please use the questions window to ask questions any time!
4
Scalable Web Application
• What?
• An application built on an architecture that can adapt to changing
conditions
5
Scalable Web Application
• What?
• An application layered on an architecture that can adapt to changing
conditions
• Why?
• Traffic and load patterns are unpredictable• Viral or flash-mob events can result in very dynamic conditions
• Availability and Reliability• Application must be distributed to increase likelihood of end-user accessibility
• Overprovision• Under-utilized resources == wasted $$$
• Underprovision• Missed opportunities – users unable to access your site/product
• Don’t be a victim of your own success
6
Pre-Cloud Provisioning
7
Cloud-enabled Provisioning
8
Reference Architecture
9
Load Balancing Tier
10
Load Balancing
• HAProxy + Apache
• Can handle SSL termination on the load balancer
• Connection statistics available via socket connection and status web page
• Each instance can handle a specific amount of traffic with no ramp-up time
• Each instance can only handle a specific amount of traffic
• Addition of load balancers is possible, but requires DNS modifications
11
Load Balancing
• HAProxy + Apache
• Can handle SSL termination on the load balancer
• Connection statistics available via socket connection and status web page
• Each instance can handle a specific amount of traffic with no ramp-up time
• Each instance can only handle a specific amount of traffic
• Addition of load balancers is possible, but requires DNS modifications
• Elastic Load Balancer (ELB)
• SSL termination is now supported
• Can scale to handle large amounts of traffic, but can be slow to ramp-up• Options do exist for “pre-warming” the ELB
• Only need one – it will scale to accommodate traffic load
• Essentially a load balancing appliance• No visibility into inner-workings and/or connection rates, statistics, failures, etc.
• (RightScale has a technical white paper on load balancing solutions that is
available at www.RightScale.com/whitepapers)
12
Load Balancing
• Load Balancer + Application server
• Possible, and good for test and dev
• Not a best practice for a production environment• Traffic spikes can cause instance to perform both load balancing and application
functions…poorly
13
Load Balancing
• Load Balancer + Application server
• Possible, and good for test and dev
• Not a best practice for a production environment• Traffic spikes can cause instance to perform both load balancing and application
functions…poorly
• Recommendation: Minimum of two load balancers
• Each load balancer should be in a different availability zone (AZ) to
increase reliability and availability
• RightScale testing has shown that m1.large is a good choice for load
balancers• Due to 100K-120K packet-per-second limit, larger instances do not provide much gain in
throughput
• Roughly 5K responses/second can be handled by m1.large
• With the 5K threshold in mind, select the number of load balancers required to handle your
peak traffic
14
Application Server Tier
• Puts the “scalable” in a scalable application
• True autoscaling a must in any dynamic/unpredictable
environment
15
Application Server Tier
• Autoscaling
• Fully automated server launch based on autoscaling triggers
• No manual intervention (can be challenging in certain environments, i.e.
Windows)
• Download and install application code from common repository to ensure
identical configuration of all servers in the tier
16
Application Server Tier
• Autoscaling
• Fully automated server launch based on autoscaling triggers
• No manual intervention (can be challenging in certain environments, i.e.
Windows)
• Download and install application code from common repository to ensure
identical configuration of all servers in the tier
• Triggers
• Common• CPU idle
• Free memory
• System load
• Custom• Web server connections
• Application-specific metrics
17
Application Server Tier
• When to scale?
• Conservatively. Both up and down
18
Application Server Tier
• When to scale?
• Conservatively. Both up and down
• Up
• Allow adequate lead time for new servers to become operational
• Before system is negatively impacted
• Look for trends in activity and react early
• Worst that can happen: Charged for an extra instance hour
19
Application Server Tier
• When to scale?
• Conservatively. Both up and down
• Up
• Allow adequate lead time for new servers to become operational
• Before system is negatively impacted
• Look for trends in activity and react early
• Worst that can happen: Charged for an extra instance hour
• Down
• When system has been underutilized for a consistent, consecutive period
of time
• Scale down fewer servers than in a scale-up event
• Again, only downside is an extra hour of instance charge
• Better safe than sorry
20
Application Server Tier
• Array considerations
21
Application Server Tier
• Array considerations
• Weight the array across all availability zones (not regions)• Increases reliability of application
• NOTE: Traffic within an AZ on private IPs is free. Traffic between AZs incurs a per-gigabyte
charge
• Traffic between regions is charged at public Internet rates
22
Application Server Tier
• Array considerations
• Weight the array across all availability zones (not regions)• Increases reliability of application
• NOTE: Traffic within an AZ on private IPs is free. Traffic between AZs incurs a per-gigabyte
charge
• Traffic between regions is charged at public Internet rates
• Set minimums and maximums appropriately• Minimum can assist in cost savings in times of low usage
• Maximum can limit overall cost exposure
23
Application Server Tier
• Array considerations
• Weight the array across all availability zones (not regions)• Increases reliability of application
• NOTE: Traffic within an AZ on private IPs is free. Traffic between AZs incurs a per-gigabyte
charge
• Traffic between regions is charged at public Internet rates
• Set minimums and maximums appropriately• Minimum can assist in cost savings in times of low usage
• Maximum can limit overall cost exposure
• Instance size• m1.large is typically a good choice for array-based servers in a production environment
• m1.smalls (and even micro instances) can be used in test and development environments
• Every application is different, so run load tests and benchmarks to find the optimal solution for
your environment
24
Application Server Tier
• Array considerations
• Weight the array across all availability zones (not regions)• Increases reliability of application
• NOTE: Traffic within an AZ on private IPs is free. Traffic between AZs incurs a per-gigabyte
charge
• Traffic between regions is charged at public Internet rates
• Set minimums and maximums appropriately• Minimum can assist in cost savings in times of low usage
• Maximum can limit overall cost exposure
• Instance size• m1.large is typically a good choice for array-based servers in a production environment
• m1.smalls (and even micro instances) can be used in test and development environments
• Every application is different, so run load tests and benchmarks to find the optimal solution for
your environment
• Code Deployment• Updated code can be pushed to all servers in an array via a single click of a button
25
Caching Tier
• Caching can dramatically decrease the load on the database
• Particularly in read-intensive applications
• Costs of caching
• Application complexity/modification
• Additional instance hours to support the cache
26
Caching Tier
• Best practice is to have a separate, dedicated caching tier
• Caching can be implemented on each application server• Prevents the use of a distributed cache
• Local cache should only be used by the co-resident application server
• Application complexities
• Database performance degradation
27
Caching Tier
• Best practice is to have a separate, dedicated caching tier
• Caching can be implemented on each application server• Prevents the use of a distributed cache
• Local cache should only be used by the co-resident application server
• Application complexities
• Database performance degradation
• Instance Size and Count
• Determine memory caching footprint• Select instance size and count that spreads the load over several servers
• Prevents loss of entire cache if a single instance fails
• Distribute caching servers across AZs for reliability
• Overprovision if possible
• Provide capacity for system to grow to fully utilize cache (budget permitting)
28
Caching Tier
• Best practice is to have a separate, dedicated caching tier
• Caching can be implemented on each application server• Prevents the use of a distributed cache
• Local cache should only be used by the co-resident application server
• Application complexities
• Database performance degradation
• Instance Size and Count
• Determine memory caching footprint• Select instance size and count that spreads the load over several servers
• Prevents loss of entire cache if a single instance fails
• Distribute caching servers across AZs for reliability
• Overprovision if possible
• Provide capacity for system to grow to fully utilize cache (budget permitting)
• Manually scaling caching servers is possible but non-trivial
• Involves application and database performance degradation
• Time To Lives (TTLs)• Always set to expire
29
Caching Tier
• Write-intensive applications
• Don’t see as large a performance gain as read-intensive apps
30
Caching Tier
• Write-intensive applications
• Don’t see as large a performance gain as read-intensive apps
• Third-party providers
• Vendor solutions exist that allow dynamic memcached scaling
31
Database Tier
• Numerous database architecture options exist
• No “one size fits all” solution, so testing and benchmarking are
critical to determine proper configuration
32
Database Tier
• Masters and Slave(s)
• Multiple Slaves if budget permits
• Distribute Master and Slave(s) across AZs
• Always use same instance size for Master and Slaves
33
Database Tier
• Masters and Slave(s)
• Multiple Slaves if budget permits
• Distribute Master and Slave(s) across AZs
• Always use same instance size for Master and Slaves
• Data Storage
• EBS volumes for data store
• Never use ephemeral storage for persistent data
• Back up Master and Slaves frequently
• Upload snapshots to S3 or some other persistent, redundant storage
34
Database Tier
• Masters and Slave(s)
• Multiple Slaves if budget permits
• Distribute Master and Slave(s) across AZs
• Always use same instance size for Master and Slaves
• Data Storage
• EBS volumes for data store
• Never use ephemeral storage for persistent data
• Back up Master and Slaves frequently
• Upload snapshots to S3 or some other persistent, redundant storage
• Instance Size
• Varies greatly based on the nature of the application and site traffic
• Load testing and benchmarking can assist in identifying a reasonable
initial size
35
Database Tier
• Relational Database Service (RDS)
• Database Appliance• No access to instance (no visibility into CPU utilization, memory usage, slow-query logs, etc.)
• Requires scheduled downtime• Announcement of multi-AZ functionality in May 2010 allows 24/7 operation
• Read replicas announced in October 2010
36
Database Scaling
37
Database Scaling
• Vertical
• “Grow” or “shrink” a database server from one instance size to another
38
Database Scaling
• Vertical
• “Grow” or “shrink” a database server from one instance size to another
• Horizontal
• Add additional servers to spread the database load
39
Database Vertical/Horizontal Scaling
RightScale Makes Vertical or Horizontal Scaling Easier
SmallType A
SmallType B
SmallType B
SmallType B
SmallType A
SmallType A
LargeType A
LargeType B
HorizontalScaling
VerticalScaling
More servers of same types
Same quantity of larger servers
40
Horizontal Database Scaling
• Addition of read Slaves
• Effective for read-intensive applications
• Only writes need to access the master
• Replication lag to slaves must be considered
41
Horizontal Database Scaling
• Addition of read Slaves
• Effective for read-intensive applications
• Only writes need to access the master
• Replication lag to slaves must be considered
• Effective mechanism is to use MySQL Proxy
42
Horizontal Database Scaling
• Sharding
• Concept is to partition the database into distinct, non-overlapping pieces
• “Horizontal slicing” of the database tables into groups of rows
• Forethought required in setting up shards since cross-shard joins are
resource intensive
43
Horizontal Database Scaling
Before Sharding
44
Horizontal Database Scaling
After Sharding
45
Horizontal Database Scaling
• Master-Master
• Two (or more) Master DBs
• Any Master can modify any database object
• Replication lag can result in database inconsistencies
• Poorly-designed applications can cause data object collisions and leave
databases in indeterminate state
• Not a recommended best practice, nor supported by RightScale
46
Horizontal Database Scaling
• NoSQL solutions
• Many options exist –
SimpleDB, Cassandra, Membase, CouchDB, MongoDB, Riak, etc.
• Basically a Key/Value store
• No complex operations between data objects (no relational operations)
• Multiple nodes can be used to implement a distributed data store• Coordinated backup and recovery can be challenging
• Some RightScale customers are beginning to use some NoSQL solutions
in specific use cases.
47
So What’s Best?
48
So What’s Best?
• As is typical in many technology discussions…
49
So What’s Best?
• As is typical in many technology discussions…
“It depends”
50
So What’s Best?
• As is typical in many technology discussions…
• Many scalable environments share some common underlying
architecture concepts
“It depends”
51
So What’s Best?
• As is typical in many technology discussions…
• Many scalable environments share some common underlying
architecture concepts
• Every application is different. As such, there is no “one size fits
all”
“It depends”
52
So What’s Best?
• As is typical in many technology discussions…
• Many scalable environments share some common underlying
architecture concepts
• Every application is different. As such, there is no “one size fits
all”
• But…
“It depends”
53
Mix-n-Match Pre-Built Components
Operating Systems
Windows 2003/2008
CentOS
Ubuntu
SUSE
Language/Framework
PHP
Ruby/Rails
Java/Tomcat
.NET
Python/Django
Database
MySQL
SQL Server
MySQL Proxy
dbShards
Membase
Web Server
IIS
Apache
Nginx
Lighttpd
Load Balancing
HAProxy
Elastic Load Balancer
Zeus
aiCache
Automation
Chef
Ruby
Bash
PowerShell
54
Have a project, but need some help?
Contact us: [email protected] or (866) 720-0208
Ready to get started?
Sign up for our Free Edition: www.RightScale.com/Free
Call us for a VIP trial of our paid editions
Need to learn more?
Scalable Web Apps white paper – Coming Soon!
TCO calculator: www.RightScale.com/tco-calculator
Webinar archive: www.RightScale.com/webinars
White papers: www.RightScale.com/whitepapers
Q&A / Getting Started
55
Thank You!