a2 vsphere architecture design

vSphere Architecture Design

VMware vSphere Plan and Design ServicesArchitecture Design

VMware vSphere Plan and Design ServicesArchitecture DesignforCustomerPrepared byJane Q. Consultant, Sr. Consultant VMware Professional [email protected] and Customer ConfidentialRevision HistoryDateRevAuthorCommentsReviewers

08 December 2010v2.2Pat CarriPubs Edits/FormattingJeff Friedman

07 December 2010v2.1Andrew HaldIntroduced vCAF, reorganized document flowJohn Arrasjid, Michael Mannarino, Wade Holmes, Kaushik Banerjee

04 November 2010v2.0Jeff FriedmanvSphere 4.1Andrew Hald

16 June 2009v1.0Mark Ewert, Kingsley Turner, Ken PolakowskiPang Chen

DELETE THE FOLLOWING HIGHLIGHTED TEXT AFTER YOU READ ITThis is representative sample deliverable of a Plan and Design for VMware vSphere engagement for use when building, rebuilding, or expanding a specific VMware vSphere design. Your actual deliverable for your customer will vary depending on the engagement scope, situation, environment, and requirements. You will need to update this document based upon your specific customer.

2010 VMware, Inc. All rights reserved. This product is protected by U.S. and international copyright and intellectual property laws. This product is covered by one or more patents listed at http://www.vmware.com/download/patents.html.VMware is a registered trademark or trademark of VMware, Inc. in the United States and/or other jurisdictions. All other marks and names mentioned herein may be trademarks of their respective companies.

VMware, Inc3401 Hillview AvePalo Alto, CA 94304www.vmware.com

VMware vSphere Plan and Design ServicesArchitecture Design

2010 VMware, Inc. All rights reserved.Page 2 of 79Design Subject Matter ExpertsThe people listed in the following table provided key input into this design.NameEmail AddressRole/Comments

2010 VMware, Inc. All rights reserved. VMware is a registered trademark of VMware, Inc.Page 3 of 79Contents1.Purpose and Overview91.1Executive Summary91.2Business Background91.3The VMware Consulting and Architecture Framework102.Conceptual Design Overview132.1Customer Requirements132.2Design Assumptions132.3Design Constraints142.4Use Cases143.vSphere Datacenter Design153.1vSphere Datacenter Logical Design153.2vSphere Clusters163.3Microsoft Cluster Service in an HA/DRS Environment203.4VMware FT204.VMware ESX/ESXi Host Design214.1Compute Layer Logical Design214.2Host Platform224.3Host Physical Design Specifications235.vSphere Network Architecture245.1Network Layer Logical Design245.2Network vSwitch Design255.3Network Physical Design Specifications295.4Network I/O Control306.vSphere Shared Storage Architecture326.1Storage Layer Logical Design326.2Shared Storage Platform336.3Shared Storage Design346.4Shared Storage Physical Design Specifications366.5Storage I/O Control367.VMware vCenter Server System Design387.1Management Layer Logical Design387.2vCenter Server Platform397.3vCenter Server Physical Design Specifications407.4vCenter Server and Update Manager Databases407.5Licenses438.vSphere Infrastructure Security448.1Overview448.2Host Security448.3vCenter and Virtual Machine Security448.4vSphere Port Requirements458.5Lockdown Mode and Troubleshooting Services459.vSphere Infrastructure Monitoring469.1Overview469.2Server, Network, and SAN Infrastructure Monitoring469.3vSphere Monitoring469.4Virtual Machine Monitoring4710.vSphere Infrastructure Patch/Version Management4810.1Overview4810.2vCenter Update Manager4810.3vCenter Server and vSphere Client Updates5011.Backup/Restore Considerations5111.1Hosts5111.2Virtual Machines5112.Design Assumptions5212.1Hardware5212.2External Dependencies5313.Reference Documents5413.1Supplemental White Papers and Presentations5413.2Supplemental KB Articles5613.3Supplemental KB Articles57Appendix A ESX/ESXi Host Estimation58Appendix B ESX/ESXi Host PCI Configuration62Appendix C Hardware BIOS Settings63Appendix D Network Specifications64Appendix E Storage Volume Specifications65Appendix F LUN Sizing Recommendations66Appendix G Security Configuration68Appendix H Port Requirements70Appendix I Monitoring Configuration74Appendix J Naming Conventions77Appendix K Design Log79

List of FiguresFigure 1. Datacenter Logical Design17Figure 2. Network Switch Design27Figure 3. SAN Diagram34

List of TablesTable 1. Infrastructure Design Qualities11Table 2. Infrastructure Design Quality Ratings12Table 3. Customer Requirements13Table 4. Design Assumptions13Table 5. Design Constraints14Table 6. Continuous Availability or High Availability16Table 7. Total Number of Hosts and Clusters Required16Table 8. VMware HA Cluster Configuration18Table 9. Option 1 Name or Option 2 Name21Table 10. VMware ESX/ESXi Specifications22Table 11. VMware ESX/ESXi Host Hardware Physical Design Specifications23Table 12. Option 1 Name or Option 2 Name24Table 13. Proposed Virtual Switches Per Host25Table 14. vDS Configuration Settings28Table 15. vDS Security Settings29Table 16. vSwitches by Physical/Virtual NIC, Port and Function29Table 17. Virtual Switch Port Groups and VLANs30Table 18. Virtual Switch Port Groups and VLANs30Table 19. Option 1 Name or Option 2 Name32Table 20. Shared Storage Logical Design Specifications33Table 21. Shared Storage Physical Design Specifications36Table 22. Storage I/O Enabled36Table 23. Disk Shares and Limits37Table 24. Option 1 Name or Option 2 Name38Table 25. vCenter Server Logical Design Specifications39Table 26. vCenter Server System Hardware Physical Design Specifications40Table 27. vCenter Server and Update Manager Databases Design40Table 28. vCenter Server and Update Manager Database Names41Table 29. SQL Server Database Accounts42Table 30. ODBC System DSN42Table 31. Lockdown Mode Configurations45Table 32. Estimated Update Manager Storage Requirements49Table 33. vCenter Update Manager Specifications49Table 34. Sources of Technical Assumptions for this Design52Table 35. VMware Infrastructure External Dependencies53Table 36. CPU Resource Requirements58Table 37. RAM Resource Requirements58Table 38. Proposed ESX/ESXi Host CPU Logical Design Specifications59Table 39. Proposed ESX/ESXi Host RAM Logical Design Specifications59Table 40. VMware vSphere Consolidation Ratios61Table 41. ESX/ESXi Host PCIe Slot Assignments62Table 42. ESX/ESXi Hostnames and IP Addresses64Table 43. VMFS Volumes65Table 44. NFS Volumes65Table 45. vSphere Roles and Permissions68Table 46. vCenter Virtual Machine and Template Inventory Folders to be used to Secure VMs69Table 47. ESX/ESXi Port Requirements70Table 48. vCenter Server Port Requirements71Table 49. vCenter Converter Standalone Port Requirements72Table 50. vCenter Update Manager Port Requirements73Table 51. SNMP Receiver Configuration74Table 52. vCenter SMTP Settings74Table 53. Physical to Virtual Windows Performance Monitor (Perfmon) Counters74Table 54. Modifications to Default Alarm Trigger Types76Table 55. Design Log79

1. Purpose and Overview1.1 Executive SummaryThis VMware vSphere architecture design was developed to support a virtualization project to consolidate 1,000 existing physical servers. The required infrastructure being defined here will be used not only for the first attempt at virtualization, but also as a foundation for follow-on projects to completely virtualize the enterprise and to prepare it for the journey to Cloud Computing.Virtualization is being adopted to slash power and cooling costs, reduce the need for expensive datacenter expansion, increase operational efficiency, and capitalize on the higher availability and increased flexibility that comes with running virtual workloads. The goal is for IT to be well-positioned to respond rapidly to ever-changing business needs.This document details the recommended vSphere foundation architecture to implement based on a combination of VMware best practices and specific business requirements and goals. The document provides both logical and physical design considerations encompassing all VMware vSphere-related infrastructure components, including requirements and specifications for virtual machines and hosts, networking and storage, and management. After this initial, foundation architecture is successfully implemented, the architecture can be rolled out to other locations to support a virtualization-first initiative, meaning that all future x86 workloads will be provisioned on virtual machines by default.Business BackgroundCompany and project background:Multinational manufacturing corporation with large retail sales and finance divisionsVision for the future of IT is to use virtualization as a key enabling technologyThis first foundation infrastructure to be located at a primary U.S. datacenter in Burlington, MassachusettsInitial consolidation project targets 1,000 mission-critical x86 servers

The VMware Consulting and Architecture FrameworkThe VMware Consulting and Architecture Framework (vCAF) is a set of tools for delivering all VMware consulting engagements in a standardized way. The framework guides the design process and creates the architecture design. vCAF is executed in the following phases:DiscoveryUnderstand the customers business requirements and objectives for the project.1. Capture the business and technical requirements, assumptions, and constraints for the project.1. Perform or secure a current state analysis of customers existing VMware vSphere environment.Development1. Conduct design workshops and interviews with the following subject matter experts: application, business continuity (BC) and DR, environment, storage, networking, security, server administration, and operations. The goal of these workshops is to transform the business and technical requirements into a logical design that is scalable and flexible.1. Discuss design options with tradeoffs and benefits. Compare the design choices with their impact against key infrastructure qualities as defined by vCAF.ExecutionBuild the vSphere infrastructure according to the physical design specifications.Execute verification procedures to confirm operational and technical success.ReviewIdentify next steps.Re-calibrate enterprise goals and identify any new or modified objectives.Match enterprise goals to future engagement of professional services.

The infrastructure qualities shown in the following table are used to categorize requirements and design decisions as well as assess infrastructure maturity.Table 1. Infrastructure Design Qualities

Design QualityDescription

AvailabilityIndicates the effect of a design choice on the ability of a technology and the related infrastructure to achieve highly available operation.Key metrics: % uptime

ManageabilityIndicates the effect of a design choice on the flexibility of an environment and the ease of operations in its management. Sub-qualities may include scalability and flexibility. Higher ratios are considered better indicators.Key Metrics:Servers per administratorClients per IT personnelTime to deploy new technology

PerformanceIndicates the effect of a design choice on the performance of the environment. This does not necessarily reflect the impact on other technologies within the infrastructure.Key Metrics:Response timeThroughput

RecoverabilityIndicates the effect of a design choice on the ability to recover from an unexpected incident which affects the availability of an environment.Key Metrics:RTO Recovery Time ObjectiveRPO Recovery Point Objective

SecurityIndicates the effect of a design choice to have a positive or negative impact on overall infrastructure security. Can also indicate whether a quality has an impact on the ability of a business to demonstrate or achieve compliance with certain regulatory policies.Key Metrics:Unauthorized access preventionData integrity and confidentialityForensic capabilities in case of a compromise

All qualities are rated as shown in the following table.Table 2. Infrastructure Design Quality RatingsSymbolDefinition

Positive effect on the design quality

oNo effect on the design quality or there is no comparison basis

Negative effect on the design quality

This document captures the design decisions made for the solution to meet customer requirements. In some cases, customer-specific requirements and existing infrastructure limitations or constraints might result in a valid but sub-optimal design choice.The primary goal of this design is to provide a service that corresponds with the business objectives of the organization. With financial constraints factored into the decision process, the key qualities to take into consideration are:AvailabilityRecoverabilityPerformanceManageabilitySecurity

Conceptual Design OverviewDELETE THE FOLLOWING HIGHLIGHTED TEXT AFTER YOU READ ITAt the top of this section, insert text describing the key motivations for this design effort. What are customer pain points? What are the factors driving business decisions in IT?Fill in the requirements, assumptions and constraints below with information gathered during your engagement.The key customer drivers and requirements guide all design activities throughout an engagement. Requirements, assumptions, and constraints are carefully logged so that all logical and physical design elements can be easily traced back to their source and justification.Customer RequirementsRequirements are the key demands on the design. Sources include both business and technical representatives.Table 3. Customer RequirementsIDRequirementSourceDate Approved

r101Tier 1 services must meet a one hour RTO.Fred Jones

r102PCI-compliant services require isolation from other services.David Johnson

Design AssumptionsAssumptions are introduced to reduce design complexity and represent design decisions that are already factored into the environment.Table 4. Design AssumptionsIDAssumptionSourceDate Approved

a101All services belonging to the Billing Department can be considered as Tier 2 for the purposes of DR planning.James Hamilton

a102Performance is considered acceptable if the end user does not notice a difference between the original platform and the new design.David Johnson

Design ConstraintsConstraints limit the logical design decisions and physical specifications. They are decisions made independent of this engagement that may or may not align with stated objectives.Table 5. Design ConstraintsIDConstraintSourceDate Approved

c101IBM will provide the server hardware.Fred Jones

c102Qlogic HBAs will be used in the ESX hosts.Fred Jones

Use CasesThis design is targeted at the following use cases:Server consolidation (power and cooling savings, green computing)Server infrastructure resource optimization (load balancing, high availability)Rapid provisioningServer standardizationThe following use cases are deferred to a future project:Server containment (new workloads)

vSphere Datacenter DesignvSphere Datacenter Logical DesignIn VMware vSphere, a datacenter is the highest level logical boundary. The datacenter may be used to delineate separate physical sites/locations or vSphere infrastructures with completely independent purposes.Within vSphere datacenters, VMware ESX/ESXi hosts are typically organized into clusters. Clusters group similar hosts into a logical unit of virtual resources, enabling such technologies as VMware vMotion, High Availability (HA), Dynamic Resource Scheduling (DRS), and VMware Fault Tolerance (FT).To address customer requirements, the following design options were proposed during the design workshops. For each design decision, the impact on each infrastructure quality is noted. The selected design option is then explained with the appropriate justification.DELETE THE FOLLOWING HIGHLIGHTED GUIDANCE TEXT AFTER YOU READ IT AND REMOVE THE HIGHLIGHTING FROM THE DESIGN DECISION TEMPLATE.The following design decision is an example. Follow the model below to communicate the design decisions appropriate to your customer and their requirements.Tier 2 Service AvailabilityCustomer requirements have not explicitly defined the service level availability for Tier 2 services. The following two options are available.Option 1: Continuous AvailabilityContinuously available services:Are redundant at the application levelHave no single points of failureThe drawbacks of continuous availability are:More expensive infrastructureNot-all applications are compatible with continuous availability methodsOption 2: High AvailabilityHighly available services:Limit single points of failureAre less expensive to supportThe drawbacks of high availability are:Some service downtime is possibleApplication awareness requires additional effortTier 2 services are not defined as mission critical, but they are still important to daily business operations of the customer. Providing no service availability efforts is not an acceptable option. Budgetary constraints must be balanced with availability requirements so that if a problem occurs, the service is restored within one hour, as stated in the service level agreement for Tier 2 services (r007).

Table 6. Continuous Availability or High AvailabilityDesign QualityOption 1Option 2Comments

AvailabilityBoth options improve availability, though Option 1 would guarantee a higher level.

ManageabilityoOption 1 would be harder to maintain due to increased complexity.

PerformanceooBoth design options have no impact on performance

RecoverabilityBoth options improve recoverability

SecurityooBoth design options have no impact on security

Legend: = positive impact on quality; = negative impact on quality; o = no impact on qualityDue to the lower cost and better manageability, customer has selected Option 2, High Availability. This design decision is reflected in the physical design specifications outlined below.vSphere ClustersAs part of this logical design, vSphere clusters are created to aggregate hosts. The number of hosts per cluster is shown in the following table.Table 7. Total Number of Hosts and Clusters RequiredAttributeSpecification

Number of hosts required to support 1,000 VMs17

Approximate number of VMs per host58.82

Maximum vSphere cluster size if hosts support more than 40 VMs each16 hosts

Capacity for host failures per cluster1 host

Dedicated hosts for maintenance capacity per cluster1 host

Number of usable hosts per cluster6 hosts

Number of clusters created3

Total usable capacity in hosts18 hosts

Total usable capacity in VMs(total usable hosts * true consolidation ratio)1,084

Assumptions and CaveatsThe total usable capacity in VMs does not account for shadow instances of VMs needed to support VMware FT. The number of protected VMs must be counted double.Host failures for VMware HA are expressed in the number of allowable host failures, meaning that the expected load should be able to run on surviving hosts. HA policies can also be applied on a percentage spare capacity basis.Dedicated hosts for maintenance assumes that a host is reserved strictly to offload running VMs from other hosts that must undergo maintenance. When not being used for maintenance, such hosts can also provide additional spare capacity to support a second host failure, or for unusually high demands on resources. Having dedicated maintenance hosts can be considered somewhat conservative, as spare capacity is being allocated strictly for maintenance activities. Such spare capacity is earmarked here to ensure that there is sufficient capacity to run VMs with minimal disruption.Hosts are evenly divided across the clusters.Clusters can be created and organized to enforce resource allocation policies such as:Load balancingPower managementAffinity rulesProtected workloadsLimited number of licenses available for specific applicationsFigure 1. Datacenter Logical Design

VMware HAEach cluster are configured for VMware High Availability (HA) to automatically recover VMs if either ESX/ESXi host fails, or if there is an individual VM failure. A host is declared failed if the other hosts in the cluster cannot communicate with it. A VM is declared failed if a heartbeat inside the guest OS can no longer be received.VMs are tiered in relative order of priority for restarts:High (for example, Windows Active Directory domain controller VMs).Medium (default).Low.Disabled. For non-critical VMs. Do not restart. Sacrifices resources for higher priority VMs (for example, QA and test VMs).The configuration settings for VMware HA are shown in the following table.Table 8. VMware HA Cluster ConfigurationAttributeSpecification

Enable host monitoringEnable

Admission controlPrevent VMs from being powered on if they violate availability constraints

Admission control policyCluster tolerates one host failure

Default VM restart priorityHigh (critical VMs)Medium (majority of VMs)Disabled (non-critical VMs)

Host isolation responsePower off VM

Enable VM monitoringEnable

VM monitoring sensitivityMedium

Setting ExplanationsEnable host monitoring. When HA is enabled, hosts in the cluster are monitored. If there is a host failure, the virtual machines on a failed host are restarted on alternate running hosts in the cluster.Admission control. Enforces availability constraints and preserves host failover capacity. Any operation on a virtual machine that decreases the unreserved resources in the cluster and violates availability constraints is not permitted.Admission control policy. Each HA cluster can support as many host failures as specified.Default VM restart priority. The priority level specified here is relative. VMs must be assigned a relative restart priority level for HA. VMs are organized into four categories: high, medium, low, and disabled. It is presumed that the majority of systems will be satisfied by the medium setting and are therefore left at default. VMs identified as high priority, such as the Active Directory VMs, are started before the medium priority VMs, which in turn are restarted before the VMs configured with low priority. If insufficient cluster resources are available, it is possible that VMs configured with low priority will not be restarted. To help prevent this situation, non-critical systems, such as QA and test VMs, are set to disabled. If there is a host failure, these VMs are not restarted, saving critical cluster resources for higher priority VMs.Host isolation response. Host isolation response determines what happens when a host in a VMware HA cluster loses its service console/management network connection, but continues running. A host is deemed isolated when it stops receiving heartbeats from all other hosts in the cluster and it is unable to ping its isolation addresses. When this occurs, the host executes its isolation response. To prevent the potential for multiple instances of each virtual machine to be running if a host becomes isolated from the network (causing other hosts to believe it has failed and automatically restart the hosts VMs), the VMs are automatically powered off upon host isolation.Enable VM monitoring. In addition to determining if a host has failed, HA can also monitor for virtual machine failure. When set to enabled, the VM monitoring service (using VMware Tools) evaluates whether each virtual machine in the cluster is running by checking for regular heartbeats from the VMware Tools process running in each guest OS. If no heartbeats are received, HA assumes that the guest operating system has failed, and HA reboots the VM.VM monitoring sensitivity. This affects how relatively quickly HA concludes that a VM failed. Highly sensitive monitoring results in a more rapid conclusion that a failure occurred. While unlikely, highly sensitive monitoring may lead to false identification of failures when the virtual machine in question is actually still working, but heartbeats have not been received due to factors such as resource constraints or network issues. Low sensitivity monitoring allows for more time before HA deems a VM to have failed. At the medium setting, HA restarts the VM if the heartbeat between the host and the VM was not received within a 60 second interval. HA also only restarts the VM after each of the first three failures every 24 hours to prevent repeated failed restarting of VMs that need intervention to recover.

Microsoft Cluster Service in an HA/DRS EnvironmentMicrosoft Cluster Service (MSCS) applies to Microsoft Cluster Service with Windows Server 2003 and Failover Clustering with Windows Server 2008.All hosts that run MSCS virtual machines are managed by a VMware vCenter Server system with VMware HA and DRS enabled. For a cluster of virtual machines on one physical host, affinity rules are used. For a cluster of virtual machines across physical hosts, anti-affinity rules are used. The advanced option for VMware DRS, ForceAffinePoweron, is set to 1, which enables strict enforcement of the affinity and anti-affinity rules that are created. The automation level of all virtual machines in an MSCS cluster are set to Partially Automated.Migration of MSCS clustered virtual machines is not recommended.VMware FTEach cluster also supports VMware Fault Tolerance (FT) to protect select critical VMs. The systems to protect are:2 Blackberry Enterprise servers2 Microsoft Exchange front-end servers2 Reporting serversThese six systems are distributed evenly amongst the three clusters resulting in two FT-protected virtual machines per eight hosts initially.All VMs to be protected by VMware FT have only one vCPU and disks configured eager-zeroed, also called thick-provisioned (not thin-provisioned). An eager-zeroed thick disk has all space allocated and zeroed out at creation time; this takes a bit longer for the creation time, but facilitates optimal performance and better security.VMware Fault Tolerance with VMware Distributed Resource Scheduler (DRS) are enabled. This process allows fault tolerant virtual machines to benefit from better initial placement and to be included in the cluster's load balancing calculations. Enable the Enhanced vMotion Compatibility (EVC) feature.On-Demand Fault Tolerance is scheduled for the two reporting servers during the quarter-end report period and then returned to the HA cluster during non-critical operations.FT traffic is supported with a pair of Gigabit Ethernet ports (see Section 5, vSphere Network Architecture). Because a pair of Gigabit Ethernet ports can support on average 4 to 5 FT-protected VMs per host, there is capacity for additional VMs to be protected by VMware FT.

VMware ESX/ESXi Host DesignCompute Layer Logical DesignThe compute layer of the architecture design encompasses the CPU, memory, and hypervisor technology components. Logical design at this level centers on performance and security.To address customer requirements, the following design options were proposed during the design workshops. For each design decision, the impact on each infrastructure quality is noted. The selected design option is then explained with the appropriate justification.DELETE THE FOLLOWING HIGHLIGHTED GUIDANCE TEXT AFTER YOU READ IT AND REMOVE THE HIGHLIGHTING FROM THE DESIGN DECISION TEMPLATE.The following Design Decision is an example. Please follow the model below to communicate the design decisions appropriate to your customer and their requirements. See Section 3.1 for an example.Design Decision 1Description of the design decisionOption 1: NameAdvantages:Advantage 1Advantage 2Drawbacks:Drawback 1Drawback 2Option 2: NameAdvantages:Advantage 1Advantage 2Drawbacks:Drawback 1Drawback 2Further details should be included here. Also highlight any relevant requirements, assumptions and/or constraints that will impact this decision.Table 9. Option 1 Name or Option 2 NameDesign QualityOption 1Option 2Comments






Legend: = positive impact on quality; = negative impact on quality; o = no impact on qualityWhich option was selected and why?Host PlatformThis section details the VMware ESX/ESXi hosts proposed for the vSphere infrastructure design. The logical components specified are required by the vSphere architecture to meet calculated consolidation ratios, protect against failure through component redundancy, and support all necessary vSphere features.Table 10. VMware ESX/ESXi SpecificationsAttributeSpecification

Host type and versionESXi 4.1 Installable

Number of CPUsNumber of coresTotal number of coresProcessor speed44162.4 GHz (2400 MHz)

Memory32GB

Number of NIC ports10

Number of HBA ports4

VMware ESXi was selected over VMware ESX because of its smaller running footprint, reduced management complexity, and significantly fewer number of anticipated software patches.The exact ESXi installable build version to be deployed will be selected closer to implementation and will be chosen based on the available stable and supported released versions at that time.

Host Physical Design SpecificationsThis section details the physical design specifications of the host and attachments corresponding to the previous section that describes the logical design specifications.Table 11. VMware ESX/ESXi Host Hardware Physical Design SpecificationsAttributeSpecification

Vendor and modelx64 vendor and model

Processor typeTotal number of coresQuad core x64-vendor CPU16

Onboard NIC vendor and modelOnboard NIC ports x speedNumber of attached NICsNIC vendor and modelNumber of ports/NIC x speedTotal number of NIC portsNIC vendor and model2 x Gigabit Ethernet4 (excluding onboard)NIC vendor and model2 x Gigabit Ethernet10

Storage HBA vendor and modelStorage HBA typeNumber of HBAsNumber of ports/HBA x speedTotal number of HBA portsHBA vendor and modelFibre Channel2/4GB2 x 4GB4

Number and type of local drivesRAID levelTotal storage2 x Serial Attached SCSI (SAS)RAID 1 (Mirror)72GB

System monitoringIPMI-based BMC

The configuration and assembly process for each system is standardized, with all components installed the same on each host. Standardizing not only the model, but also the physical configuration of the ESX/ESXi hosts, is critical to providing a manageable and supportable infrastructureit eliminates variability. Consistent PCI card slot location, especially for network controllers, is essential for accurate alignment of physical to virtual I/O resources. Appendix B contains further information on the host PCI placement.All ESX/ESXi host hardware including CPUs was selected following the vSphere Hardware Compatibility Lists and the CPUs were determined to be compatible with Fault Tolerance.

vSphere Network ArchitectureNetwork Layer Logical DesignThe network layer encompasses all network communications between virtual machines, vSphere management layer, and the physical network. Key infrastructure qualities often associated with networking include availability, security, and performance.To address customer requirements, the following design options were proposed during the design workshops. For each design decision, the impact on each infrastructure quality is noted. The selected design option is then explained with the appropriate justification.DELETE THE FOLLOWING HIGHLIGHTED GUIDANCE TEXT AFTER YOU READ IT AND REMOVE THE HIGHLIGHTING FROM THE DESIGN DECISION TEMPLATE.The following Design Decision is an example. Please follow the model below to communicate the design decisions appropriate to your customer and their requirements. See Section 3.1 for an example.Design Decision 1Description of the design decisionOption 1: NameAdvantages:Advantage 1Advantage 2Drawbacks:Drawback 1Drawback 2Option 2: NameAdvantages:Advantage 1Advantage 2Drawbacks:Drawback 1Drawback 2Further details should be included here. Also highlight any relevant requirements, assumptions and/or constraints that will impact this decision.Table 12. Option 1 Name or Option 2 NameDesign QualityOption 1Option 2Comments






Legend: = positive impact on quality; = negative impact on quality; o = no impact on qualityWhich option was selected and why?Network vSwitch DesignFollowing best practices, the network architecture complies with these design decisions:Separate networks for vSphere management, VM connectivity, vMotion traffic, and Fault Tolerance logging (VM record/replay) trafficSeparate network for NFS or iSCSI (storage over IP) used to store VM templates and guest OS installation ISO filesNetwork I/O control implemented to allow flexible partitioning of network traffic for virtual machine, vMotion, FT, and IP storage traffic across the physical NIC bandwidthRedundant virtual Distributed Switches (vDS) with at least three active physical adapter portsRedundancy across different physical adapters to protect against NIC or PCI slot failureRedundancy at the physical switch levelTable 13. Proposed Virtual Switches Per HostVirtual Standard (vSS) or Virtual Distributed Switch (vDS)FunctionNumber of Physical NIC Ports

vSS0Management Console and vMotion2

vDS1VM, Storage over IP and FT6

vSS 0 is dedicated to the management network and vMotion. Service consoles and VMkernel ports (used for vMotion and IP storage) do not migrate from host to host; these can remain on a Virtual Standard Switch (vSS.)vDS1, is allocated to virtual machine, IP Storage, and Fault Tolerance network traffic. This should be configured to a Disturbed Switch to take advantage of network I/O, Load Based Teaming, and Network vMotion. This vDS is configured to use six active Ethernet adapters. All physical network switch ports connected to these adapters are configured as trunk ports with spanning tree disabled. The trunk ports are configured to pass traffic for all VLANs used by the virtual switch.The physical NIC ports are connected to redundant physical switches.No traffic shaping policies are in place, Load-based teaming is configured for improved network traffic distribution between the pNICs and Network I/O Control enabled.VM network connectivity uses virtual switch port groups and 802.1q VLAN tagging to segment traffic into four VLANsTo support the network demands of up to 60 VMs per host, this vDS is configured to use six active Gigabit Ethernet adapters. All physical network switch ports connected to these adapters are configured as trunk ports with spanning tree disabled. The trunk ports are configured to pass traffic for all VLANs used by the virtual switch.The physical NIC ports are connected to redundant physical switches.

Figure 2. Network Switch Design

Table 14. vDS Configuration SettingsParameterSetting

Load balancingRoute based on physical NIC load

Failover detectionBeacon probing

Notify switchesEnabled

Rolling failoverNo

Failover orderAll active

vSwitch Configuration Setting ExplanationsLoad Balancing. Route based on physical NIC ensures vDS dvUplink capacity is optimized. Load-Based Teaming (LBT) avoids the situation of other teaming policies in which some of the dvUplinks in a DV Port Groups team were idle while others were completely saturated just because the teaming policy used is statically determined. LBT reshuffles port binding dynamically based on load and dvUplinks usage to make an efficient use of the available bandwidth. LBT only moves ports to dvUplinks configured for the corresponding DV Port Groups team. Note that LBT does not use shares or limits to make its determination while rebinding ports from one dvUplink to another.Failover Detection. In addition to link status, the VMkernel sends out and listens for periodic beacon probes on all network adapters in the team. This enhances link status, which relies exclusively on link integrity of the physical network adapter to determine when a failure occurs. Link status enhanced by beacon probing detects failures that are due to cable disconnects or physical switch power failures, as well as configuration errors or network interruptions beyond the local NIC termination point.Notify Switches. When enabled, this option sends out a gratuitous ARP whenever a new NIC is added to the team or when a virtual NIC begins using a different physical uplink on the ESX/ESXi host. This option helps to lower latency issues when a failover occurs or when virtual machines are migrated to another host using vMotion.Rolling Failover. Determines how a physical adapter is returned to active duty after recovering from a failure. When set to No, the adapter is returned to active duty immediately upon recovery. Setting it to Yes keeps the adapter inactive, even if it is recovered and requires manual intervention to return it to service.Failover Order. All physical adapters assigned to each vSwitch and port group are configured as Active adapters. No adapters are configured as standby or unused.

Table 15. vDS Security SettingsParameterSetting

Promiscuous modeReject (default)

MAC address changesReject

Forged transmitsReject

vDS Security Setting ExplanationsPromiscuous Mode. Setting to Reject at the vSwitch level protects against virtual machine virtual network adapters. Placing a VM virtual network adapter in promiscuous mode has no effect on which frames are received by the adapter.MAC Address Changes. Setting to Reject at the vSwitch level protects against MAC address spoofing. If the guest OS changes the MAC address of the adapter to anything other than what is in the .vmx configuration file, all inbound frames are dropped if the guest OS changes the MAC address back to match the MAC address in the .vmx configuration file, inbound frames are sent again.Forged Transmits. Setting to Reject at the vSwitch level protects against MAC address spoofing. Outbound frames with a source MAC address that is different from the one set on the adapter are dropped.Network Physical Design SpecificationsThis section expands on the logical network design in the corresponding previous section by providing details on the physical NIC layout and physical network attributes.Table 16. vSwitches by Physical/Virtual NIC, Port and FunctionvSwitchvmnicNIC / SlotPortFunction

00Onboard

N/A0Management Console and vMotion

021Management Console and vMotion

11Onboard

Quad PCIe Slot 70VM, FT, and Storage over IP traffic

131VM, FT, and Storage over IP traffic

15Quad PCIe Slot 7

Onboard



14Quad PCIe Slot 7

Onboard0VM, FT, and Storage over IP traffic


Table 17. Virtual Switch Port Groups and VLANsvSwitchPort Group NameVLAN ID

0MGMT-100100

0VMOTION-500500

1PROD-200200

0DEV-300300

0FT-600600

0NFS-650650

0N/A1

See Appendix D for more information on the physical network design specifications.Network I/O ControlVirtual Distributed Swtich1 (vDS1) are configured with Network I/O Control enabled. After Network I/O control is enabled, traffic through that virtual distributed switch is divided into the following network resource pools: FT traffic, iSCSI traffic, vMotion traffic, management traffic, NFS traffic and virtual machine traffic. This design specifies that virtual machine, iSCSI, and FT network traffic are dedicated to virtual distributed swtich1.The priority of the traffic from each of these network resource pools is set by the physical adapter shares and host limits for each network resource pool. Virtual machine traffic is set to High, the FT resource pool set to Normal, and the iSCSI traffic set to Low. These reservations apply only when the physical adapter is saturated.The iSCSI traffic resource pool shares do not apply to iSCSI traffic on a dependent hardware iSCSI adapter.Table 18. Virtual Switch Port Groups and VLANsNetwork Resource Pool Physical Adapter SharesHost Limit

Fault ToleranceNormalUnlimited

iSCSILowUnlimited

ManagementN/AN/A

NFSN/AN/A

Virtual MachineHighUnlimited

vMotionN/AN/A

Network I/O Settings ExplanationHost Limits. Host limits are the upper limit of bandwidth that the network resource pool can use.Physical Adapter Shares. Shares assigned to a network resource pool determine the total available bandwidth guaranteed to the traffic associated with that network resource pool. High. Sets the shares for this resource pool to 100.Normal. Sets the shares for this resource pool to 50.Low. Sets the shares for this resource pool to 25.Custom. A specific number of shares, from 1 to 100, for this network resource pool.

vSphere Shared Storage ArchitectureStorage Layer Logical DesignTo address customer requirements, the following design options were proposed during the design workshops. For each design decision, the impact on each infrastructure quality is noted. The selected design option is then explained with the appropriate justification.DELETE THE FOLLOWING HIGHLIGHTED GUIDANCE TEXT AFTER YOU READ IT AND REMOVE THE HIGHLIGHTING FROM THE DESIGN DECISION TEMPLATE.The following Design Decision is an example. Please follow the model below to communicate the design decisions appropriate to your customer and their requirements. See Section 3.1 for an example.Design Decision 1Description of the design decisionOption 1: NameAdvantages:Advantage 1Advantage 2Drawbacks:Drawback 1Drawback 2Option 2: NameAdvantages:Advantage 1Advantage 2Drawbacks:Drawback 1Drawback 2Further details should be included here. Also highlight any relevant requirements, assumptions and/or constraints that will impact this decision.Table 19. Option 1 Name or Option 2 NameDesign QualityOption 1Option 2Comments






Legend: = positive impact on quality; = negative impact on quality; o = no impact on qualityWhich option was selected and why?Shared Storage PlatformThis section details the shared storage proposed for the vSphere infrastructure design.Table 20. Shared Storage Logical Design SpecificationsAttributeSpecification

Storage typeFibre Channel SAN

Number of storage processors2 (redundant)

Number of switchesNumber of ports per host per switch2 (redundant)2

LUN size1TB

Total LUNs50

VMFS datastores per LUN1

VMFS version3.33

Shared Storage DesignThe following figure illustrates the design.Figure 3. SAN Diagram

Based on the results of the physical system assessment, it was determined that on average, each system has a 36GB system volume with 9 gigabytes used, and a 143GB data volume with 22 gigabytes used. After projecting for volume growth and providing 33% minimum free space per volume, for the task of estimating overall storage requirements it was determined that each virtual machine would be configured with a 12GB system volume and a 40GB data volume. Unless constrained by specific application or workload requirements, or special circumstances (such as being protected by VMware Fault Tolerance), all data volumes are provisioned as thin disks with the system volumes deployed as thick. This strategic over-provisioning saves an estimated 8.25TB of storage, assuming that on average, 50% of the currently available storage on each data volume remains unused. The consumption of each storage volume is monitored in production with alarms configured to alert if any approach capacity to provide sufficient time to source and provision additional disk.

With the intent to maintain at least 15% free capacity on each VMFS volume for VM swap files, snapshots, logs, and thin volume growth, it was determined that 48.7TB of available storage is required to support 1,000 virtual machines after accounting for the long term benefit of thin provisioning. This will be provided in 50 1TB LUNs (the additional 1.23 TB is for growth and test capacity). These LUNs will be zoned to all hosts and formatted as 50 VMFS datastores.1TB was selected because it provides the best balance between performance and manageability with approximately 20 VMs and 40 virtual disks per volume. Although larger LUNs up to 2TB (without extents) and 64TB (with extents) are possible, this size was chosen for several reasons. For manageability, it allows an adequately large portion of disks to better use resources and limit storage sprawl. A smaller size maintains a reasonable RTO and reduces the risks associated with losing a single LUN. In addition, the size limits the number of VMs that remain on a single LUN.Additionally, three NFS volumes are presented to each ESX/ESXi host for the storage of virtual machine templates, guest operating system installation, CD images (ISOs), and to provide administrators second-tier storage for log and VM archival and infrastructure testing. The separation of such files from VM files was done recognizing that these non-VM files can often have higher I/O characteristics.Each ESX/ESXi host is to be provisioned with a vSS switch supported by two physical Gigabit Ethernet adapters dedicated for IP storage connectivity.

Shared Storage Physical Design SpecificationsThis section details the physical design specifications of the shared storage corresponding to the previous section that describes the logical design specifications.Table 21. Shared Storage Physical Design SpecificationsAttributeSpecification

Vendor and modelStorage vendor and model.

TypeActive/passive.

ESX/ESXi host multipathing policyMost Recently Used (MRU). Set because the SAN is an active/passive array to avoid path thrashing. With MRU, a single path to the SAN is used until it becomes inactive, at which point it switches to another path and continues to use this new path until it fails; the preferred path setting is disregarded.

Minimum/Maximum speed rating of switch ports2GB/4GB.

See Appendix E for an inventory of VMFS and NFS volumes.Storage I/O ControlStorage I/O Control is enabled. This allows cluster-wide storage I/O prioritization, providing the ability to control the amount of storage I/O that is allocated to virtual machines during periods of I/O congestion. The shares are set per virtual machine and can be adjusted for each VM based on need.Table 22. Storage I/O EnabledDatastorePathStorage I/O Enabled

Prod_san01_02vmhba1:0:0:3 /dev/sda3 48f85575-5ec4c587-b856-001a6465c102yes

Prod_san01_07vmhba2:0:4:1 /dev/sdc1 48fbd8e5-c04f6d90-1edb-001cc46b7a18yes

Prod_san01_37vmhba32:0:1:1 /dev/sde1 48fe2807-7172dad8-f88b-0013725ddc92yes

Prod_san01_44vmhba32:0:0:1 /dev/sdd1 48fe2a3d-52c8d458-e60e-001cc46b7a18yes

Table 23. Disk Shares and LimitsVMDiskSharesLimit IOPS

ds007Disk 1Low500

kf002Disk 1 and 2Normal1000

jf001Disk 1High2000

rs003Disk 1 and 2Custom350

Shared I/O Control Settings ExplanationStorage I/O Enabled. Storage I/O is enabled per datastore. Navigate to the Configuration Tab > Properties to verify that the feature was enabled.Storage I/O Shares. Storage I/O shares are similar to VMware CPU and memory shares. Shares define the hierarchy of the virtual machines for distribution of storage I/O resources during periods of I/O congestion. Virtual machines with higher shares have higher throughput and lower latency.Limit IOPs. By default IOPS allowed for a virtual machine are unlimited. By allocating storage I/O resources, you then limit the IOPS allowed to a virtual machine. If a machine has multiple disks, you must set the same IOPS value for all the disks that access that virtual machine.

VMware vCenter Server System DesignManagement Layer Logical DesignThe Management LayerTo address customer requirements, the following design options were proposed during the design workshops. For each design decision the impact on each infrastructure quality is noted. The selected design option is then explained with the appropriate justification.DELETE THE FOLLOWING HIGHLIGHTED GUIDANCE TEXT AFTER YOU READ IT AND REMOVE THE HIGHLIGHTING FROM THE DESIGN DECISION TEMPLATE.The following Design Decision is an example. Please follow the model below to communicate the design decisions appropriate to your customer and their requirements. See Section 3.1 for an example.Design Decision 1Description of the design decisionOption 1: NameAdvantages:Advantage 1Advantage 2Drawbacks:Drawback 1Drawback 2Option 2: NameAdvantages:Advantage 1Advantage 2Drawbacks:Drawback 1Drawback 2Further details should be included here. Also highlight any relevant requirements, assumptions and/or constraints that will impact this decision.Table 24. Option 1 Name or Option 2 NameDesign QualityOption 1Option 2Comments






Legend: = positive impact on quality; = negative impact on quality; o = no impact on qualityWhich option was selected and why?vCenter Server PlatformThis section details the VMware vCenter Server proposed for the vSphere infrastructure design.Table 25. vCenter Server Logical Design SpecificationsAttributeSpecification

vCenter Server version4.1

Physical or virtual systemVirtual

Number of CPUsProcessor typeProcessor speed2VMware vCPUN/A

Memory 4GB

Number of NIC and ports1/1

Number of disks and disk sizes2: 12GB (C) and 40GB (D)

Operating system and SP levelWindows Server 2003 Enterprise R2 32-bit SP1

VMware vCenter Server, the heart of the vSphere infrastructure, is implemented on a virtual machine, as opposed to a standalone physical server. Virtualizing vCenter Server enables it to benefit from advanced features of vSphere, including VMware HA and vMotion.The exact vCenter Server build to be deployed will be selected closer to implementation and will be chosen based on the available stable and supported released versions at that time.

vCenter Server Physical Design SpecificationsThis section details the physical design specifications of the vCenter Server.Table 26. vCenter Server System Hardware Physical Design SpecificationsAttributeSpecification

Vendor and modelVMware VM virtual hardware 7

Processor typeVMware vCPU

NIC vendor and modelNumber of ports/NIC x speedNetworkVMware Enhanced VMXNET1 x Gigabit EthernetManagement network

Local disk RAID level N/A

vCenter Server and Update Manager DatabasesThis section details the specifications for the vCenter Server and Update Manager databases.Table 27. vCenter Server and Update Manager Databases DesignAttributeSpecification

Vendor and versionMicrosoft SQL Server 2005

Authentication methodSQL Server Authentication

Recovery methodFull

Database autogrowthEnabled in 1MB increments

Transaction log autogrowthIn 10% increments; restricted to 2GB maximum size

vCenter statistics level3

Estimated database size41.74GB

Setting ExplanationsAuthentication method. Per the current database security policies, SQL Server Authentication, and not Windows Authentication, will be used to secure the vCenter Server databases. A SQL Server account with strong password will be created to support vCenter Server and vCenter Update Manager access to their respective databases.Recovery method. The full recovery method helps to ensure that no data is lost if there is database failure between backups. Because it maintains complete records of all changes to the database within the transaction logs, it is critical the database is backed up regularly which truncates (grooms) the logs. The DBA team schedules incremental nightly and full weekend backups of the vCenter and vCenter Update Manager databases.Database autogrowth. The vCenter Server database can expand on demand in 1MB increments with no restriction on its growth.Transaction log autogrowth. The transaction log is restricted to a maximum 2GBs to prevent filling up the log volume. Because the vCenter Server and vCenter Update Manager databases are backed up nightly (truncating the logs), the 2GB maximum should be more than required.vCenter statistics level. Level 3 gives more comprehensive vCenter statistics than the default setting.Estimated database size. 41.74GB was calculated using the vCenter Advanced Settings tool, assuming 24 hosts and 1,084 VMs at the above statistics level.The corporate database, Microsoft SQL Server 2005, will be used, as there is currently a trained team of DBAs supporting several physical database servers on this platform. For this initial vSphere infrastructure, this resource is leveraged and databases for both the vCenter Server and vCenter Update Manager are hosted on a separate, production physical database server system.Table 28. vCenter Server and Update Manager Database NamesAttributeSpecification

vCenter database nameVC01DB01

Update Manager database nameVUM01DB01

The following table details the account to be created and its corresponding rights to each database. A strong password is assigned to this account and documented following established password storage procedures.Table 29. SQL Server Database AccountsAccount NameDatabaseRights

vc01VC01DB01dbowner

vc01VUM01DB01dbowner

vc01msdbdbowner*

* dbowner rights to the msdb database can and will be revoked following vCenter installation and creation of the vCenter Server and vCenter Update Manager databases.

Table 30. ODBC System DSNDatabaseODBCSystem DSNODBCClientDatabase ServerSQLAccount

vCenter ServerVC01DB01SQL Native ClientSQLPROD08vc01

Update ManagerVUM01DB01SQL Native ClientSQLPROD08vc01

DELETE THE FOLLOWING HIGHLIGHTED TEXT AFTER YOU READ ITThis sample is based on Enterprise Plus licensing. Be sure to educate your customer to the capabilities of each feature. The actual licensing that your customer has purchased will be based on their business requirements. You will need to update this section accordingly.LicensesFor this initial vSphere infrastructure, the Enterprise Plus vSphere license edition is used. This edition provides the following licensed features:Up to 8 physical cores per CPU and no physical memory limit per ESX/ESXi hostUpdate ManagervStorage APIs for Data ProtectionHigh Availability (HA)Thin provisioningData RecoveryvMotionHot add virtual hardware supportFault Tolerance (FT)vShield ZonesStorage vMotionDRSDPMvNetwork Distributed SwitchesHost ProfilesvStorage APIs for MultipathingVirtual Serial Port ConcentratorvStorage APIs for Array IntegrationNetwork I/O ControlStorage I/O ControlThe vCenter Server will be configured with the issued 25-character license keys and automatically installs the appropriate license on the ESX/ESXi hosts as they are added to inventory.The License Reporting Manager is used to view and generate reports on a quarterly basis for all license keys for vSphere 4.1 products in the virtual IT infrastructure.

vSphere Infrastructure SecurityOverviewSecurity is absolutely critical in this environment, and any security vulnerability or risk exposed by the new vSphere infrastructure would have a negative impact on future adoption of virtualization technology. To protect the business, existing security policies and procedures were considered and leveraged. Microsoft Active Directory users and groups were used to govern access. To reduce confusion that could undermine security, wherever possible the existing administration teams are granted the same access to the vSphere environment as they currently have to the physical infrastructure. For example, the physical network team is granted access to facilitate their responsibility of the virtual network infrastructure. However, end users will continue to access the virtual machines through the guest OS or application mechanisms and will not have access through VMware vSphere components or the vSphere Client directly. No access is granted that is not required to perform a specific, authorized job function.Host SecurityChosen in part for its limited management console functionality, ESXi is configured with a strong root password stored following the corporate password guidelines. ESXi lockdown mode is also enabled to prevent root access to the hosts over the network, and appropriate security policies and procedures are created and enforced to govern the systems. Because ESXi cannot be accessed over the network, sophisticated host-based firewall configurations are not required.A new team; ESX Admins will be created with a supporting Active Directory group. This, and only this, team has overall responsibility and access to the ESX/ESXi hosts.vCenter and Virtual Machine SecurityAccess to perform systems administration of the virtual machines is divided among the same teams currently responsible for the physical instances of the systems, leveraging the existing Active Directory implementation. Folders within vCenter Virtual Machine and Templates inventory are created to simplify assignment of these permissions to the appropriate virtual machines. Folders are created for each different system classification (such as HR, Finance, IT) and used to contain their respective virtual machines. Permissions are applied to these folders and mapped to the appropriate Active Directory group responsible for the management of the virtual machines they contain.A new team will also be created: Enterprise vSphere Administration are created with a supporting Active Directory group. This, and only this, team has overall and overarching responsibility and access to the entire vSphere infrastructure.The existing physical network and storage administration teams are also granted access, with their job responsibilities extended into the virtual infrastructure. Leveraging security capabilities of vSphere, the network and storage teams are granted access only to virtual networking and storage, respectively; that is, only what is required to perform their specific job responsibilities. Neither team has access to virtual machines, the datacenter, clusters, or any other aspect of the vSphere infrastructure. Similarly, the virtual machine admin teams do not have access to the configuration of network, storage, or any other aspects of the vSphere infrastructure, only the virtual machines they are responsible for. Only the new Enterprise vSphere Administration team has access to the entire infrastructure.As stated previously, no end users are granted access to the vSphere Infrastructure directly. They continue to leverage guest OS and application interfaces for connectivity. Appendix G contains detailed information regarding the assignment of VMware vCenter Roles to Microsoft Active Directory Groups used to secure the infrastructure.vSphere Port RequirementsAppendix H contains detailed information regarding which ports to open for communication between vSphere-related services.Lockdown Mode and Troubleshooting Services To remain compliant with the organizations security regulations, Lockdown Mode is enabled. All configuration changes to the vSphere environment are made accessing the vCenter Server with lockdown mode configured. Lockdown mode restricts access to host services on the ESXi server, but does not affect the availability of these services. If the ESXi host loses access to vCenter Server while running in Total Lockdown Mode, ESXi must be reinstalled to regain access to the host.

Table 31. Lockdown Mode ConfigurationsServiceLockdown ModeTotal Lockdown Mode

LockdownOnOn

Local Tech Support ModeOffOff

Remote Tech Support ModeOffOff

Direct Console User InterfaceOnOff

ESXi features three types of troubleshooting services; Local Tech Support Mode (TSM), Remote Tech Support Mode Service (SSH) and Direct Console User Interface Service (DCUI). In accordance with the organizations security policy TSM and SSH are enabled only when required. DCUI remains enabled to conform to design specifications.Lockdown and Troubleshooting Settings ExplanationLocal Tech Support Mode (TSM). Usable with physical access to the server console and root privileges.TSM provides a command-line interface to diagnose and repair VMware ESXi hosts. Support Mode should only be used at the request of VMware technical support.Remote Tech Support Service (SSH). Similar to TSM, but uses the secure protocol SSH for remote access to the server. Requires root privileges and provides a command-line interface to diagnose and repair VMware ESXi hosts. Support Mode should only be used at the request of VMware technical support.Direct Console user Interface Service (DCUI). Usable with physical access to the server and root privileges. The DCUI is used for basic configuration of the ESXi host. When running in Lockdown mode, you can log in locally to the direct console user interface as the root user and disable Lockdown Mode. You can then troubleshoot the issue using a direct connection to the vSphere Client or by enabling Tech Support Mode.

vSphere Infrastructure MonitoringOverviewBecause the uptime and health of the entire technology infrastructure is paramount, the current environment already has an implementation of an enterprise monitoring system. Capable of both SNMP-based and managed system-specific monitoring, such as leveraging Windows Events and Performance Counters, the system monitors all network, server, and storage systems, processing any events through a sophisticated event correlation engine to determine the appropriate response and notifications to send. The new vSphere infrastructure integrates into the monitoring system through the appropriately configured events and alerts.Server, Network, and SAN Infrastructure MonitoringAll of the physical systems, including the network and SAN, will continue to be monitored directly by the enterprise monitoring system which is configured to incorporate additional infrastructure required to support vSphere. The new physical servers purchased to run VMware ESX/ESXi are outfitted with IPMI Baseboard Management Controllers (BMC) used by the enterprise monitoring system to monitor system hardware status (such as processor temperature, fan speed, and the like). In the future, vSphere Distributed Power Management (DPM) will be considered, and can use these IPMI BMCs to automatically power ESX/ESXi Hosts on and off based on demand to help further power and cooling savings.vSphere MonitoringLeveraging the event monitoring and alarm system in vSphere, vCenter Server is configured to monitor the health and performance of all critical virtual infrastructure components, including the ESX/ESXi hosts, the clusters, VMware HA, Fault Tolerance, virtual machine operations such as vMotion, and the health of the vCenter Server itself. The events and conditions to be monitored and configured to alert are detailed in Appendix I.Upon the triggering of an alert, vCenter Server is configured to send SNMP traps to the enterprise management systems SNMP receiver. Although the same system is primarily responsible for event correlation and email alerting across the enterprise, vCenter Server is also configured to send email alerts for all triggered events to the vSphere Enterprise Administration group.The vSphere Enterprise Administrators group is also responsible for routinely reviewing and managing the health and system logs generated by the ESX/ESXi hosts, vCenter Server, and the virtual machines. These logs will be groomed and archived following corporate log retention policies and procedures. The following are monitored on a consistent basis to verify the health of the virtual infrastructure:VMware ESX/ESXi host hardware statusvCenter Server services statusVMware HA Healthcheck and operational statusCluster operational statusStorage performance statisticsNetwork performance statisticsNetwork I/O control statusStorage I/O control statusMemory utilization reportVirtual Machine MonitoringThe current enterprise management system provides monitoring of the 1,000 systems to be virtualized and continues performing this important task after the systems are converted to vSphere VMs. After reviewing the current monitoring configuration, it was determined that only minor changes are necessary to facilitate monitoring the systems after they are virtualized. The monitoring system primarily requires network connectivity to the virtual machines which is not impacted by the conversion, as their IP addresses and host names are not changing. However, the mechanism that monitors the performance of Windows virtual machines utilizes Windows Performance Monitor (Perfmon) counters and must be reconfigured to use new, virtualization-specific Windows Perfmon counters provided by VMware Tools. These counters, unlike their counterparts for physical components, are tuned for accurate assessment of virtualized Windows performance.Appendix I provides detailed monitoring system configuration information, including SNMP and SMTP settings, the list of alarms and events to leverage, and the Windows Performance Monitor counters to use by the enterprise monitoring system to monitor virtual machines.

vSphere Infrastructure Patch/Version ManagementOverviewMaintaining an up-to-date IT infrastructure is critical. The health, performance, and security of the entire business depends on the health, performance, and security of its supporting technology. Maintaining an up-to-date infrastructure can be a daunting task for IT administrators, but if not performed dependably and routinely, the infrastructure is at risk.The VMware vCenter Update Manager enterprise patch automation tool is implemented as part of the new vSphere infrastructure to keep the vSphere ESX/ESXi hosts and virtual machines VMware Tools up-to-date. Update Manager can provision, patch, and upgrade third-party modules such as EMC's PowerPath multipathing software. Administrators can also evaluate patches/updates to VMware vCenter Server and the vSphere Client and update those manually as required.vCenter Update ManagerVMware vCenter Update Manager is an automated patch management solution that applies patches and updates to VMware ESX/ESXi hosts, Microsoft Windows virtual machines, and select Linux virtual machines. vCenter Update Manager can also update VMware Tools and VMware virtual hardware in virtual machines. In addition to securing the datacenter against vulnerabilities and reducing or eliminating downtime related to host patching, automated ESX/ESXi host updates provide a common, installed version for hosts. Although VMware Fault Tolerance includes a versioning-control mechanism that allows the Primary and Secondary VMs to run on FT-compatible hosts at different but compatible patch levels, maintaining common versions among the hosts is the best practice. This is vital for the health of VMware Fault Tolerance, which requires the same build to be installed on all hosts supporting FT-protected VMs.vCenter Update Manager will be installed on the same system as vCenter Server and configured to patch/upgrade the ESX/ESXi hosts and VMware Tools installed within the virtual machines. It will, however, not be used to automatically update virtual machine hardware. Virtual machine hardware updates are evaluated and performed manually as needed.For this initial deployment, Update Manager is not leveraged to update virtual machines guest operating systems or applications, as there is already a robust mechanism in place for such patching. Because Update Manager is not used to patch VMs, the Update Manager Server can reside on the same system as the vCenter Server for simplicity. In the future, if Update Manager is used to update VMs, the Update Manager Server should be run on a separate dedicated system for performance reasons, so as not to overburden the vCenter Server.

Like vCenter, a dedicated database for Update Manager will be created on one of the database servers also housing the database for vCenter Server using the vCenter Update Manager Sizing Estimator from vmware.com, and using the following assumptions:No remediation of VMsRemediate ESXi 4.1+ hosts (no ESX 4.1+)1 concurrent ESXi host upgrades24 hostsPatch scan frequency for hosts: 4 per monthUpgrade scan frequency for hosts: 1 per monthPatching the vCenter Update Manager database is estimated to require 870 MB of storage for the first year and up to 2GBs of patch and 2GBs of temporary disk storage space on the Update Manager Server system for patches.Table 32. Estimated Update Manager Storage RequirementsEstimated Update Manager Database Size (first year)Estimated Patch Storage RequiredEstimated Temp Storage Required

870 MB2GB2GB

Table 33. vCenter Update Manager SpecificationsAttributeSpecification

Patch download sourcesSelect Download ESX 4.1 patchesUnselect Download ESX 3 patches, Download Linux VM patches, Download Windows VM patches

Shared repositoryD:\vCenterUpdateManager\vSpherePatches

Proxy settingsNone

Patch download scheduleEvery Sunday at 12:00AM EST

Email [email protected]

Update Manager baselines to leverageCritical and non-critical ESX/ESXi host patchesVMware Tools upgrade to match host

Virtual machine settingsSelect Snapshot virtual machines before remediation to enable rollbackSelect Dont delete snapshots

ESX/ESXi host settingsHost maintenance mode failure: RetryRetry interval: 30 MinutesNumber of retries: 3

vApp settingsSelect Enable smart reboot after remediation

Setting ExplanationsPatch download sources. Which patches to download.Shared repository. The vCenter Update Manager Server was configured with a data disk to use for storing patches at this location.Proxy settings. Settings for a proxy server if one is used to access the internet from the datacenter.Patch download schedule. The time and frequency to download new patches.Email notification. Who Update Manager automatically notifies when new patches are downloaded.Update Manager baselines to leverage. Update Manager baselines define a level of patches and updates to monitor for and download.Virtual machine settings. Initially, Update Manager will not be used to update virtual machines. However, this setting will be configured per best practices to prepare for the event that when VM patching is activated, a snapshot will be taken of each virtual machine prior to performing any remediation operations. This will enable rollback of patches that are applied by vCenter Update Manager, if necessary. These snapshots will not be automatically deleted by Update Manager. Members of the vSphere Administration group will delete the snapshots after determining that the patches have been successfully applied and are functioning correctly.ESX/ESXi host settings. Update Manager places a host into maintenance mode before applying patches. Maintenance mode automatically triggers the migration of any VMs running on the host to other hosts in the cluster to avoid VM downtime. If Update Manager and vCenter encounter problems putting a host into maintenance mode, this setting specifies what to do and how many times within the specified interval between attempts before abandoning the patch application to a particular host.vApp settings. vApps are logical groups of VMs. This setting uses the start order of VMs as defined with a vApp when powering on VMs. vApps often require powering on virtual machines in a specific order due to dependencies, and this is configured within the vApps properties.vCenter Server and vSphere Client UpdatesAdministrators routinely check for and evaluate new vCenter Server and vSphere Client updates. These are installed in a timely fashion following release and proper testing. VMware vSphere Client updates should be manually installed whenever vCenter Server is updated. Using conflicting versions of the vSphere Client and vCenter Server can cause unexpected results. vSphere Client automatically checks for and downloads an update if it exists when connecting to an updated vSphere Server.

Backup/Restore ConsiderationsHostsHost Profiles are created and used to restore ESX/ESXi hosts.Backing up hosts is not a necessary practice because a standard installation is relatively trivial and takes only minutes from start to finish. Administrators maintain accurate documentation of the ESX/ESXi networking and storage configurations to use for reference after a restore.Virtual MachinesThere currently is a robust enterprise backup system that is used to back up each physical system. The plan is to continue using this method to back up virtual machines. Restoring virtual machine guest operating systems, applications, and associated data follows the same method as for physical machines.The current production backup strategy calls for incremental backups nightly, full backups on Sundays, and full backups at month-end and year-end. Test, development, and QA VMs are backed up on an as-needed basis.In the future, virtualization-specific methods for backing up and restoring VMs will be considered.

Design AssumptionsHardwareHardware deployment must meet technical requirements for each product. If the hardware used deviates from the recommended hardware (see Section 13, Reference Documents), there must be re-qualification from the development team to make sure that this hardware supports all VMware products used in the deployment. The technical assumptions for this design are listed below.Table 34. Sources of Technical Assumptions for this DesignElementReference

ESX/ESXi and vCenter ServerESX/ESXi and vCenter Installation GuideESX/ESXi Configuration GuideBasic System Administration GuidevSphere 4.1 Configuration Maximums Guide

ESX/ESXi host hardwarevSphere Hardware Compatibility Lists

ESX/ESXi I/O adaptersvSphere Hardware Compatibility Lists

ESX/ESXi SAN compatibility Fibre Channel SAN Configuration GuideiSCSI SAN Configuration Guide

vMotion, HA, fault tolerancevSphere 4.1 Availability Guide

External DependenciesExternal dependencies address other systems or technologies that depend on or could be affected by the vSphere infrastructure. External dependencies are different assumptions in that they clearly identify dependent factors and the consequent implications.Table 35. VMware Infrastructure External DependenciesItemRequirements

Active DirectoryActive Directory is required to implement and operate the vSphere Infrastructure.

DNSDNS must be configured for connectivity between vCenter Server, Active Directory, VMware ESX/ESXi, and virtual machines.

NetworkNetwork congestion or failure prevents vMotion from migrating virtual machine and affects the ability of vCenter Server to manage VMware ESX/ESXi hosts. It can also negatively impact HA and FT.

Storage Area NetworkStability and performance of the SAN affects the virtual machines.

Time synchronizationAccurate time keeping and time synchronization is critical for a healthy vSphere infrastructure. All components including ESX/ESXi hosts, vCenter Server, the SAN, physical network infrastructure, and virtual machine guest operating systems must have accurate time keeping. This is especially critical for virtual machines protected by FT.

StaffProperly trained IT staff is critical for the correct implementation, operation, support, and enhancement of the vSphere infrastructure.

Policies and proceduresThe policies and procedures governing the use of information technology must be revised to properly incorporate the unique properties and capabilities of virtualization as implemented through this design.

Reference DocumentsSupplemental White Papers and PresentationsESX and vCenter Server Installation Guide for ESX 4.1, vCenter Server 4.1ESXi Installable and vCenter Server Setup Guide for ESXi 4.1 Installable, vCenter Server 4.1ESXi Embedded and vCenter Server Setup Guide for ESXi 4.1 Embedded, vCenter Server 4.1vSphere Datacenter Administration Guide for ESX 4.1, ESXi 4.1, vCenter Server 4.1http://www.vmware.com/pdf/vsphere4/r41/vsp_41_dc_admin_guide.pdfvSphere Resource Management Guide for ESX 4.1, ESXi 4.1, vCenter Server 4.1http://www.vmware.com/pdf/vsphere4/r41/vsp_41_resource_mgmt.pdfVMware ESXi and ESX Info Centerhttp://www.vmware.com/products/vsphere/esxi-and-esx/upgrade.htmlVMware Enterprise Infrastructure Support Life Cycle Policyhttp://www.vmware.com/support/policies/lifecycle/enterprise-infrastructure/index.htmlvSphere Management Assistant Guide for vSphere 4.1http://www.vmware.com/support/developer/vima/vma41/doc/vma_41_guide.pdfVMware Data Recovery Administration Guide for Data Recovery 1.2http://www.vmware.com/pdf/vdr_12_admin.pdfvSphere Compatibility Matrixeshttp://www.vmware.com/pdf/vsphere4/r40/vsp_compatibility_matrix.pdfConfiguration Maximums for VMware vSphere 4.1http://www.vmware.com/pdf/vsphere4/r41/vsp_41_config_max.pdfiSCSI SAN Configuration Guide for ESX 4.1, ESXi 4.1, vCenter Server 4.1http://www.vmware.com/pdf/vsphere4/r41/vsp_41_iscsi_san_cfg.pdfFibre Channel SAN Configuration Guide for ESX 4.1, ESXi 4.1, vCenter Server 4.1http://www.vmware.com/pdf/vsphere4/r41/vsp_41_san_cfg.pdfSetup for Failover Clustering and Microsoft Cluster Service for ESX 4.1, ESXi 4.1, vCenter Server 4.1http://www.vmware.com/pdf/vsphere4/r41/vsp_41_mscs.pdfVMware Network I/O Control: Architecture, Performance and Best Practices for VMware vSphere 4.1http://www.vmware.com/files/pdf/techpaper/VMW_Netioc_BestPractices.pdfvSphere Command-Line Interface Installation and Scripting Guide for ESX 4.1, ESXi 4.1, vCenter Server 4.1http://www.vmware.com/pdf/vsphere4/r41/vsp4_41_vcli_inst_script.pdfVMware vCenter Converter Installation and Administration Guide for vCenter Converter 4.2http://www.vmware.com/pdf/vsp_vcc_42_admin_guide.pdfvSphere Management Assistant Guide for vSphere 4.1http://www.vmware.com/support/developer/vima/vma41/doc/vma_41_guide.pdfVMware Compatibility Guideshttp://www.vmware.com/resources/compatibility/search.phpIntroduction to VMware vSphere for ESX 4.0, ESXi 4.0, vCenter Server 4.0http://www.vmware.com/pdf/vsphere4/r40/vsp_40_intro_vs.pdfvShield Zones Administration Guide for vShield Zones 1.0http://www.vmware.com/pdf/vsz_10_admin.pdfWhats New in VMware vSphere 4: Performance Enhancements: http://www.vmware.com/files/pdf/VMW_09Q1_WP_vSpherePerformance_P13_R1.pdfWhats New in VMware vSphere 4:Virtual Networking: http://www.vmware.com/files/pdf/VMW_09Q1_WP_vSphereNetworking_P8_R1.pdfWhat Is New in VMware vSphere 4: Storage: http://www.vmware.com/files/pdf/VMW_09Q1_WP_vSphereStorage_P10_R1.pdfNetwork Segmentation in Virtualized Environments http://www.vmware.com/files/pdf/network_segmentation.pdfProtecting Mission-Critical Workloads with VMware Fault Tolerance: http://www.vmware.com/files/pdf/resources/ft_virtualization_wp.pdfVMware ESX 3 802.1Q VLAN Solutions: http://www.vmware.com/pdf/esx3_vlan_wp.pdfCLARiiON Integration with VMware ESX: http://www.vmware.com/pdf/clariion_wp_eng.pdfRecommendations for Aligning VMFS Partitions: http://www.vmware.com/pdf/esx3_partition_align.pdfSecurity Design of the VMware Infrastructure 3 Architecture: http://www.vmware.com/pdf/vi3_security_architecture_wp.pdfMaking Your Business Disaster Ready with VMware Infrastructure: http://www.vmware.com/pdf/disaster_recovery.pdfAutomating High Availability (HA) Services with VMware HA: http://www.vmware.com/pdf/vmware_ha_wp.pdfESX 4 Patch Management Guide: http://www.vmware.com/pdf/vsphere4/r40/vsp_40_esxupdate.pdfBest Practices for Patching ESX: http://www.vmware.com/resources/techresources/1075Microsoft Exchange Server 2007 Performance on VMware vSphere 4 http://www.vmware.com/files/pdf/perf_vsphere_exchange-per-scaling.pdfImproving Scalability for Citrix Presentation Server: http://www.vmware.com/pdf/esx_citrix_scalability.pdfManaging VMware VirtualCenter Roles and Permissions: http://www.vmware.com/resources/techresources/826VMware vCenter Update Manager Performance and Best Practices http://www.vmware.com/pdf/Perf_UpdateManager40_Best-Practices.pdf

Supplemental KB ArticlesvMotion CPU Compatibility Requirements for Intel Processors: http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&externalId=1991vMotion CPU Compatibility Requirements for AMD Processors: http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&externalId=1992vMotion CPU Compatibility - Migrations Prevented Due to CPU Mismatch - How to Override Masks http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=1993&sliceId=1&docTypeID=DT_KB_1_1&dialogID=23256056&stateId=0 0 2325069Installing ESX.1 and vCenter 4. 41 Best Practices: http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=1009080&sliceId=2&docTypeID=DT_KB_1_1&dialogID=23256161&stateId=0%200%2023250853VMware High Availability Slot Calculation: http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=1010594&sliceId=1&docTypeID=DT_KB_1_1&dialogID=23256209&stateId=0%200%2023250906VMware vSphere 4.1 Software Compatibility Matrix: http://partnerweb.vmware.com/comp_guide/docs/vSphere_Comp_Matrix.pdfProcessors and Guest Operating Systems that Support VMware Fault Tolerance: http://kb.vmware.com/kb/1008027VMware Infrastructure Architecture Overview: http://www.vmware.com/pdf/vi_architecture_wp.pdfVirtualization Overview: http://www.vmware.com/pdf/virtualization.pdfWhats New in VMware vSphere 4:Virtual Networking: http://www.vmware.com/files/pdf/VMW_09Q1_WP_vSphereNetworking_P8_R1.pdfWhat Is New in VMware vSphere 4: Storage: http://www.vmware.com/files/pdf/VMW_09Q1_WP_vSphereStorage_P10_R1.pdfNetwork Throughput in a VMware Infrastructure: http://www.vmware.com/pdf/esx_network_planning.pdfNetwork Segmentation in Virtualized Environments http://www.vmware.com/files/pdf/network_segmentation.pdfThe vSphere Availability Guide: http://www.vmware.com/pdf/vsphere4/r40/vsp_40_availability.pdfProtecting Mission-Critical Workloads with VMware Fault Tolerance: http://www.vmware.com/resources/techresources/1094VMware ESX 3 802.1Q VLAN Solutions: http://www.vmware.com/pdf/esx3_vlan_wp.pdfCLARiiON Integration with VMware ESX: http://www.vmware.com/pdf/clariion_wp_eng.pdfRecommendations for Aligning VMFS Partitions: http://www.vmware.com/pdf/esx3_partition_align.pdfSecurity Design of the VMware Infrastructure 3 Architecture: http://www.vmware.com/pdf/vi3_security_architecture_wp.pdfMaking Your Business Disaster Ready with VMware Infrastructure: http://www.vmware.com/pdf/disaster_recovery.pdfAutomating High Availability (HA) Services with VMware HA: http://www.vmware.com/pdf/vmware_ha_wp.pdfESX 4 Patch Management Guide: http://www.vmware.com/pdf/vsphere4/r40/vsp_40_esxupdate.pdfBest Practices for Patching ESX: http://www.vmware.com/resources/techresources/1075Microsoft Exchange Server 2007 Performance on VMware vSphere 4 http://www.vmware.com/files/pdf/perf_vsphere_exchange-per-scaling.pdfImproving Scalability for Citrix Presentation Server: http://www.vmware.com/pdf/esx_citrix_scalability.pdfManaging VMware VirtualCenter Roles and Permissions: http://www.vmware.com/resources/techresources/826VMware vCenter Update Manager Performance and Best Practices http://www.vmware.com/pdf/Perf_UpdateManager40_Best-Practices.pdfSupplemental KB ArticlesvMotion CPU Compatibility Requirements for Intel Processors: http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&externalId=1991vMotion CPU Compatibility Requirements for AMD Processors: http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&externalId=1992vMotion CPU Compatibility - Migrations Prevented Due to CPU Mismatch - How to Override Masks http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=1993&sliceId=1&docTypeID=DT_KB_1_1&dialogID=23256056&stateId=0 0 2325069Installing ESX 4.1 and vCenter 4.1 best practices: http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=1009080&sliceId=2&docTypeID=DT_KB_1_1&dialogID=23256161&stateId=0%200%2023250853VMware High Availability slot calculation: http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=1010594&sliceId=1&docTypeID=DT_KB_1_1&dialogID=23256209&stateId=0%200%2023250906VMware vSphere 4.1 Software Compatibility Matrix: http://partnerweb.vmware.com/comp_guide/docs/vSphere_Comp_Matrix.pdfProcessors and Guest Operating Systems that Support VMware Fault Tolerance: http://kb.vmware.com/kb/1008027Appendix A ESX/ESXi Host EstimationTo determine the number of hosts required to consolidate the existing datacenters 1,100 physical x86 servers, the performance and utilization of the existing servers was analyzed using VMware Capacity Planner for 30 days. The analysis captured the resource utilization for each system, including average and peak CPU and RAM utilization.Out of the servers analyzed, 100 were disqualified from the consolidation project for the following reasons:Servers are already planned for decommission and there is no need to virtualizeIncomplete performance and utilization data; such servers to be deferred for further analysisServers use specialized hardware that cannot be virtualizedESX 4.1 supports USB device passthrough from an ESX or ESXi to a virtual machine.A total of 1,000 candidates were selected for this first virtualization initiative. Over the sampling period, the metrics in the following tables were observed.Table 36. CPU Resource RequirementsMetricAmount

Average number of CPUs per physical system2

Average CPU MHz2800 MHz

Average normalized CPU per physical system5663 MHz

Average CPU utilization per physical system6.5% (368.01 MHz)

Average peak CPU utilization per physical system9% (509.67 MHz)

Total CPU resources required for 1,000 VMs at peak509,670 MHz

Table 37. RAM Resource RequirementsMetricAmount

Average amount of RAM per physical system1024 MB

Average memory utilization62% (634.88 MB)

Average peak memory utilization70% (716.80 MB)

Total RAM required for 1000 VMs at peak before memory sharing716,800 MB

Anticipated memory sharing benefit when virtualized50%

Total RAM required for 1,000 VMs at peak with memory sharing358,400 MB

For estimating capacity, the target host platform has the specifications proposed in the following tables.Table 38. Proposed ESX/ESXi Host CPU Logical Design SpecificationsAttributeSpecification

Number of CPUs (sockets) per host4

Number of cores per CPU4

MHz per CPU core2,400

Total CPU MHz per CPU9,600

Total CPU MHz per host38,400

Proposed maximum host CPU utilization80%

Available CPU MHz per host30,720 MHz

Table 39. Proposed ESX/ESXi Host RAM Logical Design SpecificationsAttributeSpecification

Total RAM per host32,768 MB (32 GB)

Proposed maximum host RAM utilization80%

Available RAM per host26,214 MB

Estimation AssumptionsHosts sized for peak utilization levels rather than average utilization. This is to support all systems running at their observed peak resource levels simultaneously.CPU and memory utilization for each host capped at 80% (allow 20% for overhead and breathing room).Memory sharing: 50% (achieved through running the same guest OS, Windows Server 2003 Standard R2, across 90% of all VMs).

The following formula was used in calculating estimated required host capacity to support the peak CPU utilization of the anticipated VM workloads:

Total CPU required for total VMs at peak= # of ESX/ESXi Hosts Required

Available CPU per ESX/ESXi Host

Using this formula, the following estimated required host capacity was calculated for the planned vSphere infrastructure:

509,670 MHz (Total CPU)= 16.59 ESX/ESXi Hosts

30,720 MHz (CPU per Host)

The following formula was used in calculating the number of hosts required to support anticipated at peak RAM utilization:

Total RAM required for total VMs at peak= # of ESX/ESXi Hosts Required

Available RAM per ESX/ESXi Host

Using this formula, the following estimated required host capacity was calculated for the planned vSphere infrastructure: 358,400 MB (Total RAM)= 13.67 ESXi Hosts

26,214 MB (RAM per Host)

From a CPU workload perspective, 17 VMware ESX/ESXi hosts are needed, but