san basics

Course ObjectiveCourse Objective

Business requirement for a SANBusiness requirement for a SAN What is SANWhat is SAN DAS, NAS, SAN ConceptsDAS, NAS, SAN Concepts Disk Technologies RAIDDisk Technologies RAID Fiber Channel ProtocolFiber Channel Protocol FC SAN BasicsFC SAN Basics

Business needs Business needs

24x7 Operations24x7 Operations Data miningData mining E-commerceE-commerce

Why business needs the above?Why business needs the above?

Fast / High performanceFast / High performance High CapacityHigh Capacity Cost EffectiveCost Effective SecuritySecurity

A Few Storage Basics….A Few Storage Basics…. Where will data finally end up? Where will data finally end up? How will it get there?How will it get there? What will it pass through?What will it pass through?

Storage typesStorage types

Single Disk DriveSingle Disk Drive JBOD (Just Bunch Of Disk)JBOD (Just Bunch Of Disk) VolumeVolume Storage Array (RAID)Storage Array (RAID) DAS (Direct Attached Storage) DAS (Direct Attached Storage) NAS (Network Attached Storage)NAS (Network Attached Storage) SAN (Storage Area Network) SAN (Storage Area Network)

Direct Attached Storage Direct Attached Storage (Internal)(Internal)

Computer System

CPU

Memory

Bus

I/O - RAID Controller

Disk Drives


Computer System

CPU

Memory

Bus


Disk Drives

12345

John Smith512-555-1212

1424 Main Street

Data


Computer System

CPU

Memory

Bus


Disk Drives

12345


1424 Main Street

DAS with internal controller DAS with internal controller and external storageand external storage

CPU

Memory

Bus


Computer System

Disk Drives

Disk Drives

Disk Drives

Disk Enclosure

12345


1424 Main Street

Comparing Internal and Comparing Internal and External StorageExternal Storage

Internal Storage

Server

Storage

RAID controllers and disk drives are internal to the server

SCSI, ATA, or SATA protocol between controller and disks

SCSI Bus w/ external storage

Server

RAID Controller

Storage

RAID Controller

Disk Drives

RAID controller is internal

SCSI or SATA protocol between controller and disks

Disk drives are external

Disk Drives

DAS with external controller DAS with external controller and external storageand external storage

Computer System

CPU

Memory

Bus

HBA

RAIDController

Storage System

Disk Drives

Disk Drives

Disk Drives

Disk Enclosure12345


1424 Main Street

DAS over Fibre ChannelDAS over Fibre ChannelServer

HBA

RAID Controller

Disk Drive are external

Disk Drives

RAID Controller

HBA is internal

Fibre Channel protocol between HBAs and external RAID controller

External SAN Array

I/O TransferI/O Transfer RAID ControllerRAID Controller

Contains the “S.M.A.R.T” Contains the “S.M.A.R.T”

((Self-Monitoring, Analysis and Reporting Self-Monitoring, Analysis and Reporting Technology)Technology)

Determines how the data will be written Determines how the data will be written (striping, mirroring, RAID 5, RAID 6, RAID 10 (striping, mirroring, RAID 5, RAID 6, RAID 10 and 01)and 01)

Host Bus Adapter (HBA)Host Bus Adapter (HBA) Simply transfers the data to the RAID Simply transfers the data to the RAID

controller. controller. Doesn’t do any RAID or striping calculations. Doesn’t do any RAID or striping calculations. Required for external storage.Required for external storage.

Direct Attached Storage – Direct Attached Storage – (DAS)(DAS)

Traditional way of implementing storageTraditional way of implementing storage Storage is managed by a single hostStorage is managed by a single host Block Level AccessBlock Level Access Other hosts must access the storage through that single host Other hosts must access the storage through that single host

over the LANover the LAN

DAS: Strengths & LimitationsDAS: Strengths & Limitations

InexpensiveInexpensive Familiar TechnologyFamiliar Technology No retrainingNo retraining

FlexibilityFlexibility Storage must be located close Storage must be located close

to the serverto the server ScalabilityScalability

SCSI bus limitationsSCSI bus limitations Distance LimitationsDistance Limitations

Physical SCSI Bus limitationsPhysical SCSI Bus limitations

Strengths Limitations

NAS: What is it?NAS: What is it?

Network Attached StorageNetwork Attached Storage File Level AccessFile Level Access Utilizes a TCP/IP network to “share” Utilizes a TCP/IP network to “share”

datadata uses file sharing protocols like Unix uses file sharing protocols like Unix

NFS and Windows CIFSNFS and Windows CIFS

Networked Attached Networked Attached StorageStorage

NAS Server

Storage

Server has a Network Interface Card

No RAID Controller or HBA in the server

Public or Private Ethernet network

RAID Controller

Disk Drives

All data converted to file protocol for transmission (may slow down database transactions)

Server

NIC NIC

Network Attached Storage Network Attached Storage (NAS)(NAS)

Provides access to storage over networkProvides access to storage over network NAS devices contain a thin server that provides NAS devices contain a thin server that provides

file services to other hosts on the LAN using file services to other hosts on the LAN using network file access methodsnetwork file access methods

NAS: Strengths & NAS: Strengths & LimitationsLimitations

Relatively InexpensiveRelatively Inexpensive Ease of ManagementEase of Management More scalable and reliable More scalable and reliable

than DASthan DAS Accessible by any host OS Accessible by any host OS

anywhere on the networkanywhere on the network

Can cause high traffic loads on the Can cause high traffic loads on the LANLAN

Most NAS solutions are optimized Most NAS solutions are optimized for file-level access to storagefor file-level access to storage NFS (Network File System)NFS (Network File System) CIFS (Common Internet File CIFS (Common Internet File

System)System)


Storage Area Network Storage Area Network (SAN)(SAN)

Servers and strand-alone storage devices, Servers and strand-alone storage devices, connected by a dedicated networkconnected by a dedicated network

Any server can be configured to access any Any server can be configured to access any storage array and / or storage to storage accessstorage array and / or storage to storage access

Servers and storage can scale independentlyServers and storage can scale independently Block Level AccessBlock Level Access

SAN StorageSAN Storage

SAN TopologySAN Topology There are three basic methods of There are three basic methods of

communication using Fibre Channel communication using Fibre Channel infrastructureinfrastructure Point to point Topology (P-Point to point Topology (P-

to-P)to-P) A direct connection between two A direct connection between two

devicesdevices Based on Storage front end Based on Storage front end

portsports FC Arbitrated Loop Topology FC Arbitrated Loop Topology

(FC-AL)(FC-AL) A daisy chain connecting two or A daisy chain connecting two or

more devicesmore devices Maximum 127 devices can Maximum 127 devices can

connectconnect Fabric Topology Fabric Topology

Multiple devices connected via Multiple devices connected via switching technologiesswitching technologies

Maximum 16 million devices Maximum 16 million devices supportsupport

SAN: Strengths & SAN: Strengths & LimitationsLimitations

Servers and storage Servers and storage can scale can scale independentlyindependently

Does not impact and is Does not impact and is not impacted by LAN not impacted by LAN traffictraffic

Provides higher Provides higher availabilityavailability

Initial Implementation costs Initial Implementation costs more than NASmore than NAS

Can be complex to manageCan be complex to manage Requires specialized trainingRequires specialized training


DAS / NAS / SANDAS / NAS / SAN

Disk TechnologyDisk Technology

Disk Storage SubsystemDisk Storage Subsystem

Identify Physical ComponentsIdentify Physical Components

Main Physical ComponentsMain Physical Components

PlattersPlatters SpindleSpindle ArmsArms HeadsHeads

Platters

Spindle

Read/ Write Head

Logical StructuresLogical Structures Sector:Sector: Basic unit of data Basic unit of data

storage on a hard disk (512 storage on a hard disk (512 bytes)bytes)

Track:Track: Group of sectors Group of sectors would form a trackwould form a track

Cylinder:Cylinder: Group of Group of concentric circles (tracks)concentric circles (tracks)

Disk InterfacesDisk Interfaces ATA / IDE - ATA / IDE - ParallelParallel SATA - SATA - SerialSerial SCSI - SCSI - ParallelParallel SAS - SAS - SerialSerial FC - FC - SerialSerial FATA – FATA – SerialSerial SSD - SSD - SerialSerialTransmission Modes: Transmission Modes: ParallelParallel SerialSerial

Parallel transmissionParallel transmission Uses several wires, each wire sending one bit at the same Uses several wires, each wire sending one bit at the same

time as the others which was time as the others which was assumedassumed faster than straight faster than straight serial connections. serial connections.

Clock speed:Clock speed: There has to be additional signaling to let There has to be additional signaling to let the receiving device know what is being transmitted – i.e. 8 the receiving device know what is being transmitted – i.e. 8 bits being transmitted “now.”bits being transmitted “now.”

Serial transmissionSerial transmission

Sends bit by bit over a single line Two separate “wires” for sending / receiving

Two separate “wires” for sending / receiving

Send

Receive

Serial vs ParallelSerial vs Parallel

Full duplexFull duplex Data moves much Data moves much

faster than parallel faster than parallel because of single because of single stream and not tied to stream and not tied to particular clock speedparticular clock speed

Wraps many bits of data Wraps many bits of data into packets then into packets then transfers the packets at transfers the packets at a much higher speeda much higher speed

Up to 30 times faster Up to 30 times faster than parallelthan parallel

Less interferenceLess interference

Half duplexHalf duplex Processing Overhead – bit Processing Overhead – bit

arrangement with clock arrangement with clock speedspeed

Wide cable and connector Wide cable and connector sizesize

Crosstalk across wide Crosstalk across wide ribbon cable can cause ribbon cable can cause signal errors thereby signal errors thereby limiting cable lengthlimiting cable length

Signal SkewingSignal Skewing - the - the eight bits that leave at the eight bits that leave at the same time do not arrive at same time do not arrive at the other end at the same the other end at the same timetime

Fibre ChannelFibre Channel

Fibre ChannelFibre Channel, or , or FCFC, is a high-, is a high-speed network technology speed network technology (commonly running at 2-, 4-, 8- and (commonly running at 2-, 4-, 8- and 16-16-gigabit per second rates) per second rates) primarily used to connect primarily used to connect computer data storage

Fibre Channel Switched Fabric Fibre Channel Switched Fabric topology allows the connection of up topology allows the connection of up to 16 million devices, the theoretical to 16 million devices, the theoretical maximum. maximum.

http://en.wikipedia.org/wiki/Gigabit

http://en.wikipedia.org/wiki/Computer_data_storage

CacheCache

Disk CacheDisk Cache: a cache that resides : a cache that resides within a diskwithin a disk

Controller CacheController Cache: A cache that : A cache that resides within a controller and whose resides within a controller and whose primary purpose is to improve disk or primary purpose is to improve disk or array I/O performance.array I/O performance.

Provides a intermediate storage and bridge from something that is comparatively very fast to something that is comparatively slow

Write-Back & Write through CacheWrite-Back & Write through Cache

Write-ThroughWrite-Through Write-through caching, sometimes referred to as Write-through caching, sometimes referred to as conservative conservative

cache modecache mode, , Writes are written to both the cache and the disk before the Writes are written to both the cache and the disk before the

write is acknowledged as complete.write is acknowledged as complete.

Write-BackWrite-Back A write is A write is written to cachewritten to cache, , The The I/O is acknowledgedI/O is acknowledged as "complete" to the server that as "complete" to the server that

issued the write, issued the write, Some time later, the cached write is written or Some time later, the cached write is written or flushed to diskflushed to disk. .

When the application receives the I/O in complete When the application receives the I/O in complete acknowledgement, acknowledgement, it assumesit assumes the data is permanently stored the data is permanently stored on disk.on disk.

JBODs and RAIDsJBODs and RAIDs While a JBOD is a group of disks packaged While a JBOD is a group of disks packaged

in an enclosure and connected via a FC in an enclosure and connected via a FC loop, a RAID is a more sophisticated loop, a RAID is a more sophisticated device, that may improve performance device, that may improve performance and/or reliability of the storage deviceand/or reliability of the storage device

RAID is improving performances RAID is improving performances reading/writing information from a set of reading/writing information from a set of disks at the same time, and reliability disks at the same time, and reliability adding parity and/or mirroring information adding parity and/or mirroring information on multiple disks of the arrayon multiple disks of the array

RAID RAID (Redundant Array of Independent (Redundant Array of Independent Disk)Disk)

Widely used RAID LevelsWidely used RAID Levels RAID 0RAID 0 - Striping- Striping RAID 1RAID 1 - Mirroring- Mirroring RAID 3RAID 3 - Striping with dedicated Parity- Striping with dedicated Parity RAID 5RAID 5 - Striping with distributed Parity- Striping with distributed Parity RAID 6RAID 6 - Striping with distributed dual - Striping with distributed dual

parityparity RAID 10RAID 10 - Mirrored Striping- Mirrored Striping RAID 01RAID 01 - Striped Mirroring - Striped Mirroring

RAID 0RAID 0

Not really RAID – No redundant check dataNot really RAID – No redundant check data Single disk failure can result in loss of all dataSingle disk failure can result in loss of all data Better performance than single disk access for large filesBetter performance than single disk access for large files Good where performance is more important than Good where performance is more important than

redundancy.redundancy. Requires a minimum of 2 drives to implement Requires a minimum of 2 drives to implement

RAID 1RAID 1

100% data redundancy means no data loss if one disk fails, 100% data redundancy means no data loss if one disk fails, data can be retrieved from mirrored disk.data can be retrieved from mirrored disk.

Required two disks write operation per block but only one Required two disks write operation per block but only one read.read.

Good for small file where security is paramount.Good for small file where security is paramount. Requires a minimum of 2 drives to implement Requires a minimum of 2 drives to implement

RAID 3RAID 3

The data block is subdivided ("striped") and written on the The data block is subdivided ("striped") and written on the data disks. data disks.

Dedicated Parity diskDedicated Parity disk Very high Read & Write data transfer rate Very high Read & Write data transfer rate Requires a minimum of 3 drives to implement Requires a minimum of 3 drives to implement Disk failure has an insignificant impact on throughput Disk failure has an insignificant impact on throughput

RAID 5RAID 5

Each entire data block is written on a data disk Each entire data block is written on a data disk Distributed ParityDistributed Parity Highest Read & Medium Write data transaction rate Highest Read & Medium Write data transaction rate Requires a minimum of 3 drives to implement Requires a minimum of 3 drives to implement

RAID 6RAID 6 comprises block-level striping with comprises block-level striping with

double distributed parity. Double double distributed parity. Double parity provides fault tolerance up to parity provides fault tolerance up to two failed drivestwo failed drives

RAID 10RAID 10

Requires a minimum of 4 drives to implement Requires a minimum of 4 drives to implement High availability by combining features of RAID 0 High availability by combining features of RAID 0

performance and RAID 1 Redundancyperformance and RAID 1 Redundancy RAID 10 has the same fault tolerance as RAID RAID 10 has the same fault tolerance as RAID

level 1 level 1 Parity is not calculated so good write performanceParity is not calculated so good write performance Very expensive / High overheadVery expensive / High overhead

RAID 01RAID 01

RAID 01 is also called as RAID 0+1RAID 01 is also called as RAID 0+1 This is why it is called “mirror of This is why it is called “mirror of

stripes”. i.e the disks within the stripes”. i.e the disks within the groups are striped. But, the groups groups are striped. But, the groups are mirroredare mirrored

Parity CalculationParity Calculation Parity blockParity block A parity block is used by certain A parity block is used by certain RAID levels.. If a single drive in levels.. If a single drive in

the array fails, data blocks and a parity block from the working the array fails, data blocks and a parity block from the working drives can be combined to reconstruct the missing data.drives can be combined to reconstruct the missing data.

Given the table below, where each column is a disk, assume Given the table below, where each column is a disk, assume A1 = 00000111, A1 = 00000111, A2 = 00000101,A2 = 00000101,A3 = 00000000. A3 = 00000000.

Ap,Ap, generated by XORing generated by XORing A1, A2, and A3A1, A2, and A3, will then equal , will then equal 00000010. 00000010. If the second drive fails,If the second drive fails, A2 A2 will no longer be accessible, but can will no longer be accessible, but can be reconstructed by XORing be reconstructed by XORing A1, A3, and ApA1, A3, and Ap::

A1 XOR A3 XOR Ap = 00000101A1 XOR A3 XOR Ap = 00000101RAID array

A1A1 A2 A2 A3 A3 Ap Ap B1 B1 B2 B2 Bp Bp B3 B3 C1 C1 Cp Cp C2 C2 C3 C3 Dp Dp D1 D1 D2 D2 D3 D3

http://en.wikipedia.org/wiki/RAID

http://en.wikipedia.org/wiki/RAID_array

The End