design size examples exchange

Design and Sizing Examples: Microsoft Exchange 2007 on VMware Infrastructure

of 19

© 2008 VMware, Inc. All rights reserved.

Design and Sizing Examples:Microsoft Exchange Solutions on VMware


Contents 1. Introduction ..................................................................................................................................... 3

1.1. Overview ............................................................................................................................................ 3

1.2. Benefits of Running Exchange Server 2007 on VMware Infrastructure 3 ......................................... 4

2. Sample Design .................................................................................................................................. 5

2.1. Sample Exchange Server 2007 on VMware Infrastructure Landscape .............................................. 5

2.2. Sample Exchange Physical Layout ...................................................................................................... 6

2.3. Sample Exchange Virtual Network Configuration .............................................................................. 7

2.4. Sample ESX Host Architecture/Storage Interaction ........................................................................... 9

2.5. Sample Logical Architecture ............................................................................................................. 10

2.6. Sizing and Configuration Guidelines ................................................................................................ 10

2.7. Exchange Server Building Blocks ...................................................................................................... 11

2.8. Sample Building Block Sizing – 4,000 users / Average Profile .......................................................... 12

3. Sizing and Deployment Examples ................................................................................................... 14

3.1. Example 1 – 4,000 Average Profile Mailboxes ................................................................................. 14

3.2. Example 2 – 16,000 Average Profile Mailboxes ............................................................................... 15

3.3. Example 3 – 64,000 Average Profile Mailboxes ............................................................................... 16

4. Design and Deployment Considerations ......................................................................................... 18

5. Summary ....................................................................................................................................... 18

© 2008 VMware, Inc. All rights reserved. of 19



1. Introduction

1.1. Overview Exchange can be a very complex application to deploy and there are many design decisions to be made to ensure a solid solution. We know that running Microsoft Exchange Server 2007 on VMware Infrastructure 3 can positively impact design, deployment, availability, and operations, but what does such a solution look like? In this document, we’ll explore a sample architecture design, illustrating an Exchange environment running on VMware Infrastructure 3. The focus of this architecture is to provide a high‐level overview of the solution components with diagrams to help illustrate key concepts. For detailed best practices, please refer to the Best Practices Guide in this Solution Kit. The sample design will cover the following subjects:

• Sample Exchange Server 2007 on VMware Infrastructure Landscape • Sample Exchange Physical Layout • Sample Exchange Virtual Network Configuration • Sample ESX Host Architecture/Storage Interaction • Sample Logical Architecture • Sizing and Configuration Guidelines • Exchange Server Building Blocks

The examples here show how these components contribute to the overall design and are intended only to provide a guideline. Customers should work with their infrastructure vendors to develop a detailed sizing and architecture plan designed for their individual requirements. After describing the concepts of our sample design, we’ll take a look at sizing examples of Exchange Server 2007 on VMware ® Infrastructure 3 for three different sized organizations:

• 4,000 Average mailbox profiles • 16,000 Average mailbox profiles • 64,000 Average mailbox profiles

It is important to note that this document describes samples that should be viewed as a model to help understand components and concepts. Official sizing for Exchange environments will vary based on business and technical requirements as well as server and storage hardware platforms. VMware recommends that you engage your server and storage vendors in your design plans or utilize one of the detailed, hardware‐specific reference architectures found on our website and in the Partner Resources Catalog.

of 19



1.2. Benefits of Running Exchange Server 2007 on VMware Infrastructure 3

• Simple and effective evaluation, test, and development environments. • Improved flexibility with Exchange 2007 server roles. • Simple and reliable Exchange disaster recovery (DR). • A better way to provision Exchange server resources. • Higher availability with less complexity. • Unprecedented control of Exchange servers with VMware VMotion. • Rapid provisioning with virtual machine (VM) templates. • Easier testing and troubleshooting with snapshots and clones. • Improved Exchange design flexibility to meet specific requirements. • Simplified maintenance and operation of essential legacy systems. • Full utilization of 64‐bit hardware.

of 19



2. Sample Design

2.1. Sample Exchange Server 2007 on VMware Infrastructure Landscape When deploying Exchange on VMware Infrastructure, the same rules generally apply as those in a physical design. For example, there are advantages to distributing workloads by separating the mailbox server from other peripheral server roles (CAS, Hub Transport, etc.) when you are working with physical servers, and those also apply when deploying Exchange on VMware Infrastructure. The figure below demonstrates what such an installation might look like conceptually:

of 19



2.2. Sample Exchange Physical Layout The figure below demonstrates a sample 16,000‐mailbox, average user profile environment where each of the Exchange server roles runs in its own virtual machine. Each ESX host has been sized to 16 CPUs and 48 GB of RAM to handle the workload of 2 Mailbox Server, 1 Hub Transport, and 1 Client Access Server virtual machines. To achieve best results in a VMware environment, it is a good practice to divide out each server role into its own virtual machine to allow for more efficient workload separation as well as to increase the amount of redundancy in the system.

The logical separation of the virtual machines is also an important part of the above design. When working in VMware environments it is important to classify and structure virtual machines logically as you would physical servers, and using resource pools is the best way to accomplish this goal. For example, resource pools allow for easy grouping and logical separation of production, test, and development workloads. You can also employ some physical separation as a part of your infrastructure, separating the test and development environments from the production environment on different ESX Servers as was done in this layout, but it is not necessary to do so. Resource pools also have the added benefit of ensuring that the most important workloads maintain priority for the use of the physical resources. To do this, each resource pool is allowed a certain number of “shares.” The number of shares designated for each resource pool will depend on the workloads for each virtual machine. In the above graphic, the Production Resource Pool would be given the most number of shares and the Development and Test Resource Pools would be given the least. In the event of resource contention, where two or more virtual machines are trying to use the

of 19



same resource (e.g. vCPU), the virtual machine with the most shares assigned to it takes priority while the other virtual machines will have to wait for the vCPU to become available. Please refer to the latest “Resource Management Guide” on the VMware Infrastructure 3 documentation website for more details on configuring and managing resource pools. When running Microsoft Exchange Server 2007 on VMware Infrastructure, it is important to consider which resource pools will reside on which hardware. It may be desirable (although not necessary) to separate the production environment on physical hardware, just make sure there are enough physical resources to provide the availability needed for proper operation of VMware HA, VMware DRS, and VMware VMotion (this will depend on the overall size of the environment).

2.3. Sample Exchange Virtual Network Configuration

This figure is designed to help illustrate how networking is handled at the ESX level. Each ESX server needs to have the virtual switches architected to handle the type of network traffic that will be assigned to each of the different virtual machines. The figure represents a sample configuration where the production resource pool is split between two different physical servers (to reflect redundancy for availability considerations). From a networking perspective you will want to make sure that the production network traffic remains separate from VMotion and Admin traffic. One of the most effective ways to handle the separation is by introducing VLAN technology to logically separate the traffic.

Each virtual machine (VM) acts independently, and will remain completely isolated until networking is configured. What makes the environment a little different than those in a physical world is that

of 19



the configuration needs to have an internal network configured to establish communication between VMs that reside on the same physical ESX server. This network traffic is handled through the virtual switch.

Each physical NIC could be configured to connect directly to an assigned VLAN, but the VMotion and Admin networks are not typically used heavily, while production network traffic is fairly constant with Exchange. One practice is to team all the NICs on the ESX server, connect them to trunk ports on the switch, and use VLAN tagging to direct the traffic at the switch level. This allows better utilization of bandwidth and frees up the majority of the ESX Server’s capacity for production traffic when the VMotion and Admin VLANs are not being heavily used.

of 19



2.4. Sample ESX Host Architecture/Storage Interaction The following figure is an example of how the ESX host and the VMs might interact with the underlying storage. In this example, each ESX host would run two Mailbox Server VMs, one Hub Transport VM, and one Client Access Server VM.

ESX Host Architecture for Exchange Server 2007 on VMware Infrastructure 3

of 19



2.5. Sample Logical Architecture Logical architectures for Exchange can get very complex, especially when designing large, multi‐site hierarchies. For simplicity, the example below is a basic single‐site, single‐domain design. Each Exchange Mailbox Server VM is equipped to handle 4,000 mailboxes, and each ESX host is designed to support 8,000 mailboxes. The entire solution is designed to support 16,000 total mail users.

Logical architecture for Exchange 2007 on VMware Infrastructure

2.6. Sizing and Configuration Guidelines Sizing of an Exchange Server 2007 environment is a complex process with many variables including business requirements, anticipated mailbox workloads, and hardware platform, to name a few. The

of 19



good news is that sizing an Exchange Server 2007 environment on VMware Infrastructure is nearly the same as sizing for physical servers. One advantage of configuring Exchange on VMware Infrastructure is the option to use predefined “building blocks” for the mailbox server role. Testing by VMware and its partners has focused on four primary sizes for mailbox virtual machine building blocks: 500, 1000, 2000, and 4000 users. Testing has validated performance and sizing for these configurations and each has known performance profiles which can be leveraged for easy Exchange server sizing, as well as for easily scaling the environment as additional Exchange servers are brought online. The sizing process begins with understanding and applying Microsoft’s guidelines for each server role, as represented by the following high‐level processes:

• Design the mailbox server building block. o Define current workloads using the Microsoft Exchange Server Profile Analyzer. o Choose an appropriate building block (500, 1000, 2000, and 4000 user blocks have

been tested and validated, although larger building blocks may be possible). o Apply Microsoft guidelines to determine the CPU requirements. o Apply Microsoft guidelines to determine the amount of memory required. o Utilize the Exchange 2007 Mailbox Server Role Storage Requirements Calculator

from Microsoft to determine storage requirements. • Design the peripheral server roles.

o Determine how many mailbox server building blocks are needed. o Calculate the number of mailbox server processor cores. o Use Microsoft Guidelines for Server Role Ratios to calculate processor and memory

requirements for the Hub Transport roles. o Use Microsoft Guidelines for Server Role Ratios to calculate processor and memory

requirements for the Client Access Server roles. o Allocate one or more virtual machines for each server role to satisfy the previously

calculated number of processor cores and amount of memory. • Determine how the virtual machines will be distributed across ESX hosts. • Aggregate virtual machine requirements plus some overhead to size each ESX host.

Storage sizing and configuration can vary depending on the storage array used and many vendors have unique enhancements to the storage solution that can increase availability, speed recovery, enhance performance, etc. To optimize performance and take advantage of these features, it is highly recommended that the storage partner be included in the design effort. There are many facets to an Exchange 2007 deployment besides sizing. Exchange 2007 can be deployed into some very complex, multi‐site architectures that should be designed with the assistance of an Exchange expert, whether that person is an internal company resource or a partner with experience deploying both Exchange and VMware infrastructures. The high‐level sizing guidelines defined above are described in detail in the Best Practices Guide in this Solution Kit.

2.7. Exchange Server Building Blocks The building block approach is a recommended best practice for creating a virtualized Exchange environment running on VMware Infrastructure 3 using pre‐sized virtual machine configurations.

of 19

http://www.microsoft.com/downloads/details.aspx?FamilyId=8F575F60-BD80-44AA-858B-A1F721108FAD&displaylang=en

http://msexchangeteam.com/files/12/attachments/entry438481.aspx



Exchange servers that have divided into virtual machine building blocks (as opposed to larger, monolithic Exchange servers) can simplify server sizing during the initial deployment and create a highly scalable solution using virtual machines with predictable performance patterns. Testing by VMware and its partners has focused on four primary sizes for mailbox virtual machine building blocks consisting of 500, 1000, 2000, and 4000 users. These configurations have known performance profiles that can be leveraged for rapid Exchange server sizing as well as easily scaling environments as additional Exchange servers need to be brought online. The table below presents some guidelines for sizing the mailbox server role in a virtual machine using the building block approach for mailboxes with the “average” user IOPS (Input/Output operations Per Second) profiles. The same principles are used for sizing the other profiles. Building block CPU and RAM sizing for mailboxes with average profiles Building Block 500 1000 2000 4000 Profile Average Average Average Average vCPU 1 1 2 4 Base RAM (GB) 2 2 2 2 Variable (MB/Mailbox) 3.5MB/Mailbox 3.5MB/Mailbox 3.5MB/Mailbox 3.5MB/Mailbox Total (GB) 4 6 9 16

Source: http://www.microsoft.com/technet/prodtechnol/exchange/2007/plan/hardware.mspx

2.8. Sample Building Block Sizing – 4,000 users / Average Profile Using the Microsoft sizing guidelines and the building block approach, we will apply the formula to size a 4,000‐user building block with an average mailbox profile. The following calculations are meant to serve as an example of the sizing process; customers are encouraged to use these as guidelines but must also evaluate specific requirements to determine the most optimal deployment models for their needs. Every environment is different and some organizations use email more heavily than others. To accurately determine your mailbox profile requirements, utilize the Microsoft Exchange Server Profile Analyzer. It is also highly recommended that you work with an internal or partner resource that is experienced with Exchange architectures to ensure proper performance in your environment. In our example, we will be using the average mailbox profile defined below:

• 10 messages sent/40 received per day • Average message size of 50KB • 500MB mailbox quota

Note that this example does not take into account a particular storage solution. Many of VMware’s storage partners have done extensive testing on building blocks of varying capacities and workload characteristics. Please refer to the Partner Resource Catalog for storage‐specific implementation details.

2.8.1. Mailbox Server Resource Requirements The following table summarizes the resource requirements for our 4,000‐user building block.

4,000‐user/Average Profile Building Block Requirements Exchange Role Physical Resources (per server)

of 19

http://www.microsoft.com/technet/prodtechnol/exchange/2007/plan/hardware.mspx




of 19

Mailbox Server CPU: 4 coresMemory: 16 GB OS and Application File Storage: 16 GB (OS & Application files) DB Storage: 20 x 400 GB 10K RPM FC/SCSI/SAS 3.5" Log Storage: 4 x 300 GB 15K RPM FC/SCSI/SAS 3.5" (Logs) Network: 1 Gbps

2.8.2. Guest VM Configuration The resource requirements for Exchange are detailed below in the VM configuration of our 4,000‐user, mailbox server building block.

Exchange VM configuration Exchange Role Virtual Hardware (per VM) Mailbox Server (4 VMs) CPU: 4 vCPU

Memory: 16 GB Storage: SCSI Controller 0 HDD 1: 16 GB (OS & Application files) HDD 2: 1101 GB (SG1‐SG7 Databases) HDD 3: 1101 GB (SG8‐SG14 Databases) HDD 4: 1101 GB (SG15‐SG21 Databases) HDD 5: 96 GB (SG1‐SG7 Logs) HDD 6: 96 GB (SG8‐SG14 Logs) HDD 7: 96 GB (SG15‐SG21 Logs) Network: NIC 1

2.8.3. Guest VM Storage Interaction




3. Sizing and Deployment Examples Using the 4,000‐user mailbox server building block and the sizing process detailed in the Best Practices document, this section helps to demonstrate how to scale out to accommodate a larger number of users.

3.1. Example 1 – 4,000 Average Profile Mailboxes The example below uses our 4,000‐user building block numbers to estimate the amount of processing and memory needed for the CAS and Hub Transport Roles. We then translate the estimated resources into virtual machine and host configurations.

3.1.1. Resource Requirements by Server Role Using Microsoft and VMware best practices, we can estimate the resource requirements of each server role based on server role ratios and Microsoft sizing guidelines. For an in‐depth look at the sizing and configuration process, please refer the Best Practices Guide in this Solution Kit. Resources required to support 4,000 average profile mailboxes Server Role #VMs vCPU

(per VM) vMemory(per VM)

OS/App Storage (per VM)

Network

Mailbox Server 1 4 16 GB 16 GB NIC1

Client Access Server

1 2 2 GB 16 GB NIC1

Hub Transport Server

1 2 2 GB 16 GB(+ 32 GB for protocol, tracking

logs, temp files)

NIC1

3.1.2. VM Distribution Deciding how to distribute the Exchange workload on physical servers depends on many factors, including availability/recovery model, corporate standards, and cost. By using two ESX hosts, we can implement advanced VMware features such as HA and DRS. Also, by configuring both hosts to connect to the same shared storage, we can utilize VMotion to reduce unplanned downtime. VM Distribution for 4,000 average users across 2 ESX hosts ESX Host VM(s)ESX Host 1 Exchange Mailbox VM 1 (4 vCPU/16GB RAM)

ESX Host 2 Exchange Client Access VM 2 (1 vCPU/2 GB RAM)

Exchange HUB Transport VM 2 (1 vCPU/2 GB RAM)

3.1.3. Host Specifications Each ESX host server should provide enough physical hardware to accommodate the planned workload and provide some headroom in the event of a VMware HA failover or planned VMotion migration of live VMs for host hardware maintenance.

of 19



ESX Host hardware configuration to support 4,000 average profile mailboxes ESX Host ConfigurationAll ESX Hosts 8 cores (2x4 or 4x2)

24 GB RAM 2 Fiber Channel HBAs 4 Gigabit Network Adapters

3.2. Example 2 – 16,000 Average Profile Mailboxes The example below uses our 4,000 user building block and peripheral server role calculations to show how our simple 4,000 user environment can scale out to support 16,000 users.

3.2.1. Resource Requirements by Server Role In this example, the Mailbox Server building block is the same; we’ve just added 3 more of them. We also added an extra CAS and an extra Hub Transport VM and upped the memory on the CAS VM to handle the extra workload from the mailbox servers. Resources required to support 16,000 average profile mailboxes Server Role #VMs vCPU


OS/App Storage (per VM)

Network



2 2 4 GB 16 GB NIC1



logs, temp files)

NIC1

3.2.2. VM Distribution Again, in this example, we’ve chosen to use two ESX hosts connected to shared storage to use advanced VMware features such as HA and DRS.

VM Distribution for 16,000 average users across two ESX hosts ESX Host VM(s)ESX Host 1 Exchange Mailbox VM 1 (4 vCPU/16GB RAM)

Exchange Mailbox VM 2 (4 vCPU/16 GB RAM) Exchange Client Access VM 1 (2 vCPU/4 GB RAM) Exchange Hub Transport VM 1 (2 vCPU/2 GB RAM)

ESX Host 2 Exchange Mailbox VM 3 (4 vCPU/16GB RAM)Exchange Mailbox VM 4 (4 vCPU/16 GB RAM) Exchange Client Access VM 2 (2 vCPU/4 GB RAM) Exchange Hub Transport VM 2 (2 vCPU/2 GB RAM)

of 19



3.2.3. Host Specifications Note that we are still only using two physical machines to support four times the number of mailboxes as the previous example; we’ve just increased the processor and memory specifications on the ESX Servers to support the added load. ESX Host hardware configuration to support 16,000 average profile mailboxes ESX Host VM(s)All ESX Hosts 16 cores (4x4)


3.3. Example 3 – 64,000 Average Profile Mailboxes This example takes our scaling method one step further by showing how the 4,000‐user building block and peripheral server roles calculations can be expanded to support 64,000 average profile mailboxes. Although we’ve used the 4,000‐user building block in this example, higher mailbox concentrations are certainly possible, depending on the specific workload. Mailbox Server VMs have been configured to run 11,000 mailboxes in production customer environments. That noted, the 4,000‐user building block is officially tested and recommended by our server and storage partners. Please refer to the Partner Resources Catalog in this Solution Kit for more information about building blocks and performance testing.

3.3.1. Resource Requirements by Server Role In this example, the Mailbox Server building block is the same and we’ve scaled to 16 VMs. We also increased the Client Access Server count to eight and the Hub Transport count to four. Resources required to support 64,000 average profile mailboxes Server Role # VMs vCPU


OS / App Storage (per VM)

Network



8 2 4 GB 16 GB NIC1



logs, temp files)

NIC1

3.3.2. Exchange VM Distribution In this example, we’ve increased the physical server count to eight ESX hosts and evenly balanced the initial VM placement across them. VM Distribution for 64,000 average users across 8 ESX hosts ESX Host VM(s)ESX Host 1 Exchange Mailbox VM 1 (4 vCPU/16GB RAM)

Exchange Mailbox VM 2 (4 vCPU/16 GB RAM) Exchange Client Access VM 1 (2 vCPU/4 GB RAM)

of 19



Exchange Hub Transport VM 1 (2 vCPU/2 GB RAM)ESX Host 2 Exchange Mailbox VM 3 (4 vCPU/16GB RAM)

Exchange Mailbox VM 4 (4 vCPU/16 GB RAM) Exchange Client Access VM 2 (2 vCPU/4 GB RAM) Exchange Hub Transport VM 2 (2 vCPU/2 GB RAM)

ESX Host 3 (extra headroom)

Exchange Mailbox VM 5 (4 vCPU/16GB RAM)Exchange Mailbox VM 6 (4 vCPU/16 GB RAM) Exchange Client Access VM 3 (2 vCPU/4 GB RAM)



ESX Host 6 Exchange Mailbox VM 11 (4 vCPU/16GB RAM)Exchange Mailbox VM 12 (4 vCPU/16 GB RAM) Exchange Client Access VM 6 (2 vCPU/4 GB RAM) Exchange Hub Transport VM 4 (2 vCPU/2 GB RAM)





3.3.3. Host Specifications Although we’ve increased to eight ESX hosts and 64,000 users, we are still utilizing the same server configuration as the 16,000‐user example. ESX host hardware configuration to support 64,000 average profile mailboxes ESX Host SpecificationAll ESX Hosts 16 cores (4x4)


of 19



4. Design and Deployment Considerations • Exchange Server 2007 aggressively utilizes all the memory provided to it in a guest. ESX Server

can support higher levels of memory over‐commitment if VMs share the same OS and application code pages. Even with page sharing, over‐commitment should be attempted with caution if users want to avoid performance impacts due to resource contention. VMware recommends setting "Memory Reservation" to the amount of memory configured for the VM.

• Follow Microsoft Guidelines for storage sizing using the Exchange 2007 Mailbox Server Role Storage Requirements Calculator.

• Use the latest processor generations (Intel Dual Core, Xeon 51xx, 53xx; AMD Rev. E/F) for their enhanced support for virtualization.

• VMware recommends at least four NIC ports per ESX host machine to address network traffic, VM security and isolation, VMotion, and Management (service console).

• VMware recommends at least two HBA ports per ESX host for redundancy. • The NIC/HBA ports are minimum recommendations for each ESX host. More ports may be

needed depending on the number of virtual machines and customer‐specific network and storage requirements. The number should be determined by a detailed sizing analysis with the infrastructure vendor.

5. Summary This document shows sample configurations of Exchange Server 2007 on VMware Infrastructure. These examples provide a high‐level guideline only and are not intended to reflect customer‐specific workloads. Customers need to work with their infrastructure vendors to build a detailed sizing and architecture design that meets their individual requirements.

of 19




© 2008 VMware, Inc. All rights reserved. of 19

VMware, Inc. 3401 Hillview Ave Palo Alto, CA 94304 USA Tel 650-427-5000 Fax 650-427-5001 www.vmware.com © 2008 VMware, Inc. All rights reserved. Protected by one or more of U.S. Patent Nos. 6,397,242, 6,496,847, 6,704,925, 6,711,672, 6,725,289, 6,735,601, 6,785,886, 6,789,156, 6,795,966, 6,880,022, 6,961,941, 6,961,806, 6,944,699, 7,069,413; 7,082,598 and 7,089,377; patents pending.

design size examples exchange

Documents