8/14/2019 Rails Conf

1/38

Utilizing Amazon S3 and EC2

in Rails

Jonathan Weiss, Peritor Wissensmanagement GmbH

RailsConf Europe Berlin, 2007

8/14/2019 Rails Conf

2/38

2

Starting Point

One machine:

Apache

Ruby / Rails

MySQL

8/14/2019 Rails Conf

3/38

3

Worst Case Populariy

8/14/2019 Rails Conf

4/38

4

A Difficult Path

8/14/2019 Rails Conf

5/38

5

Problem: Backup

High availability

Redundancy

Very big data sets

8/14/2019 Rails Conf

6/38

6

Problem: File System

Important files have to be accessed by many servers

NFS / Samba not practical

8/14/2019 Rails Conf

7/38

7

Problem: Spontaneous Traffic

8/14/2019 Rails Conf

8/38

8

Problem: Load Fluctuation

8/14/2019 Rails Conf

9/38

9

Dont reinvent the wheel!

8/14/2019 Rails Conf

10/38

10

Amazon Web Services

Simple Storage Service - S3

Redundant Data Store

$0,15 per GB data per month

$0,10 - $0,20 per GB traffic

Virtual server per hour

$0,10 per CPU hour

$0,10 - $0,20 per GB traffic

Elastic Compute Cloud - EC2

.

E-Commerce S..

8/14/2019 Rails Conf

11/38

11

S3 - Simple Storage Service

Redundant storage - as much as you like

max. 5 GB per object

Organized in Buckets

Web Service API for uploads

Downloads via

Web Service

HTTP / HTTPS

BitTorrent

8/14/2019 Rails Conf

12/38

12

S3 - Buckets

Unique over all S3

Contains many

key-value-metadata tupel

Cannot contain other buckets!

Key can contain /

MyBucket_name

S3

foto_5.jpg

backups/januar.zip

fotos/2007/001.png

www.peritor.com

site/screen.css

site/script.js

FreeBSD-6.2.iso

8/14/2019 Rails Conf

13/38

8/14/2019 Rails Conf

14/38

14

EC2 - Elastic Compute Cloud

Based on XEN virtualization

On demand virtual servers - controlled with Web Service API

1,7 GHz

1,75 GB RAM

160 GB local file system (not persistent!)

Use your favorite Linux distro (Linux 2.6.16),

Amazon Machine Images (AMI) are stored on S3

ACLs for hosts/ports access control

8/14/2019 Rails Conf

15/38

8/14/2019 Rails Conf

16/38

16

amazon-ec2 gem

Setup

Usage

8/14/2019 Rails Conf

17/38

17

And now?

How does this solvemy problems?

8/14/2019 Rails Conf

18/38

18

S3 - Backup

s3sync.rb

Brackup

Jungle Disk

S3 FUSE

s3DAV

Duplicity

S3Browser

Firefox S3 Organizer extension

8/14/2019 Rails Conf

19/38

8/14/2019 Rails Conf

20/38

20

S3 Asset Host

8/14/2019 Rails Conf

21/38

21

S3 Asset Host

Setup DNS

Rails configuration

8/14/2019 Rails Conf

22/38

22

S3 Asset Host

welcome.rhtml template

Output

8/14/2019 Rails Conf

23/38

23

S3 - AuthenticatedUser Data

8/14/2019 Rails Conf

24/38

8/14/2019 Rails Conf

25/38

25

attachment_fu Rails plugin

Upload

Download

8/14/2019 Rails Conf

26/38

26

On-Demand Computing with EC2

Time based, e.g. with cron

8/14/2019 Rails Conf

27/38

8/14/2019 Rails Conf

28/38

28

On-Demand Computing with EC2

8/14/2019 Rails Conf

29/38

29

EC2 for extra capacity

8/14/2019 Rails Conf

30/38

30

Load Balancer / Proxy

Example mod_proxy_blancer:

Talks to multiple backend servers

(Mongrel)

Central Proxy/Load-Balancerconfiguration that has knowledge

about nodes

Typically proxy restart on config

change

8/14/2019 Rails Conf

31/38

31

Swiftiply

Swiftiply Proxy:

Multiple backend clients have a

persistent connection to the backend

port

Proxy listens on cluster port forrequests and forwards them

No re-configuration

8/14/2019 Rails Conf

32/38

8/14/2019 Rails Conf

33/38

33

Swiftiplied Mongrel

Gem plugin that patches Mongrel

Transforms Mongrel in Swiftiply client

Experimental

Start

or

8/14/2019 Rails Conf

34/38

34

EC2 on Demand before Swiftiply

8/14/2019 Rails Conf

35/38

35

EC2 on Demand with Swiftiply

8/14/2019 Rails Conf

36/38

36

EC2 on Demand with Swiftiply

8/14/2019 Rails Conf

37/38

37

Ressources

Amazon Web Services

http://aws.amazon.com

Swiftiply

http://swiftiply.swiftcore.org

Attachment_fuhttp://svn.techno-weenie.net/projects/plugins/attachment_fu/

Elastic Rails

http://www.elasticrails.com

Capazon

http://capazon.rubyforge.org

RightScale

http://www.rightscale.com

8/14/2019 Rails Conf

38/38

38

Peritor Wissensmanagement GmbH

Lenbachstrae 212157 Berlin

Internet: www.peritor.comEmail: info@peritor.com

Peritor Wissensmanagement GmbH - All Rights Reserved

Telefon: +49 (0)30 69 40 11 94Telefax: +49 (0)30 69 40 11 95