building a cost-effective and high-performing public cloud

23
Building a cost- effective and high- performing public cloud Sander Cruiming, founder Cloud Provider 1

Upload: cloudprovider

Post on 19-Jun-2015

382 views

Category:

Technology


0 download

DESCRIPTION

Sander Cruiming, founder of Cloud Provider, shows how to build a cost-effective and high-performing public cloud to meet the today's high demands and requirements for cloud infrastructure. Presented at the HPC Advisory Council in Lugano, Switzerland on the 14th of March, 2013. The corresponding Youtube video with audio: http://www.youtube.com/watch?v=fhRH_yIlM7g Please check for more information: http://www.cloudprovider.net

TRANSCRIPT

  • 1. Building a cost-effective andhigh-performing public cloudSander Cruiming, founder Cloud Provider 1

2. Agenda Introduction of Cloud Provider Overview of our previous cloud architecture Challenges of that architecture due to evolvingcloud market and clients needs How we build an improved cloud architecture Comparison between the old and new setup2 3. Introduction: Company Cloud hosting and Infrastructure-as-a-Service(IaaS) provider in the Netherlands Founded in 2008, spin-off from a sharedhosting provider 2 products, both pay-as-you-go: Cloud Servers Cloud Apps Cloud platform is build using KVM3 4. Introduction:Differentiation Localized: support in Dutch, datacenter inAmsterdam Ease of use & simplicity: in-house developedmanagement portal with value-added tools Good performance/cost ratio: cloud based onSSD caching & Infiniband and low-latencynetwork starting at 13,25 euro per month 4 5. Introduction: Clients More than 500 clients and resellers Focus on developers, webhosts & ISPs,e-commerce, high-traffic sites Federation with other cloud providers to buy/selleachothers cloud capacity References:5 6. Introduction: Cloud portal6 7. Introduction: Cloud portal7 8. Previous cloud architecture (1) 4 layers: storage, hypervisors, management, backup Hypervisors were connected with SAN using iSCSIover gigabit ethernet Storage VLAN: Bonding 4 network interfaces for SAN,2 interfaces for each hypervisor to increasebandwidth 3 additional VLANs: public network, internalmanagement network, and backup network8 9. Previous cloudarchitecture (2)Internal management network InternetRouter HypervisorsSANControl servers Backup SANPublic network Storage networkBackup network 9 10. Cloud market evolving Explosion of interest and adoption since 2011 Number of clients growing, current clients deploymore and larger VMs Expected cloud market growth from US$70.1billion in 2012 to US$158.8 billion in 2014 Different workloads are deployed nowadays Beside test/dev, also production environments The rise of big data / high transaction volumeapplications and databases10 11. Clients higher demands Better reliability & availability Higher performance & lower latency External: internet connectivity between client andthe cloud Internal: connection between hypervisor and SAN,local network Competitive pricing 11 12. Challenges of previouscloud architecture (1) Internal storage network became aperformance bottleneck Clients experienced higher latency (iowait) Storage performance was inconsistent Creating backups (snapshots) took too much time Only 30 VMs per hypervisor to keep performanceunder control (= lower ROI) Difficult to manage and scale due to largenumber of networks and cables 12 13. Challenges of previouscloud architecture (2) 13 14. Alternativeinterconnects (1) Faster alternatives to gigabit iSCSI:Fibrechannel, 10 Gig-E iSCSI, Infiniband Important selection criteria: Performance Congestion control & Low latency Scalability Easy to manage Density: number of VMs per hypervisor Cost 14 15. Alternativeinterconnects (2) Host Cost adapterNumber ofQuality of connectivity Cost switchcard inports Service performance hypervisor 1 Gb iSCSI 2000 euro2 Gb/s 5x NIC(included) ethernet (24 ports)10 Gb iSCSI 1x HBA +24500 euro 10 Gb/s 530 euro ethernet1x NIC(16 ports)1x HBA +3000 euro Fibrechannel2 Gb/s 175 euro 1x NIC (18 ports)3500 euro Infiniband40 Gb/s 1x HCA500 euro(18 ports)15 16. Infiniband selected (1) Low latency: < 1 usec end-to-end High performance: 40 Gb/s host connectivity Consolidation: multiple fabrics on single cable Up to 8 virtual lanes No interdependency between between differenttraffic flows Highly scalable: tens of nodes possible Best performance/cost ratio 16 17. Infiniband selected (2) Other cloud providers are using Infinibandsuccessfully: 17 18. Infiniband selected (3) Mellanox InfiniScaleo IV QDR InfiniBandSwitch, 18 QSFP ports Mellanox MHQH19B-XTRConnectX-2 VPI adapter cardsingle-port 40Gb/s QSFP,PCIe2.0 x8 5.0GT/s18 19. New cloud architectureInfiniband switch 1 InternetRouterHypervisors SANControl serversBackup SANInfiniband switch 219 20. Results of new cloud architecture (1) Hdparm - timing buffered disk reads: Old setup: 16.94 MB/sec New setup: 83.09 MB/sec Seeker - random access time: Old setup: 16.23 ms New setup: 4.679 ms 20 21. Results of new cloud architecture (2) Larger number of VMs per hypervisor: Old setup: 30 VMs in average New setup: up to 120 VMs in average Duration for backup creation largely reduced Provisioning a new VM is faster Easier to manage and scale, smaller amount ofcables21 22. Results of new cloudarchitecture (3) Quote from one of our clients:We develop high-traffic business websites with the DrupalCMS. On the old platform, we experienced disk performanceissues, as Drupal needs fast storage access for a large numberof MySQL database queries and for file-based caching.The new cloud platform based on Infiniband has given a greatperformance boost to our clients websites, making us and ourclients happier.- Rick Bosscher, General Manager, Dycon.nl 22 23. Thank you! [email protected] 23