自動化負載平衡子計畫 第一季報告 (2009.07~2009.09) 董信煌...
TRANSCRIPT
Outline
• Objectives
• Resource Management in Xen
• CPU Schedulers of Xen
• Monitoring Performance of Xen
• Methodology
• Summary
• Outlook
Objectives
• 系統規劃與需求分析 . 了解及熟悉 Xen 環境中與資源分配及效能分析有關之功能 , 例如 Xen 對於多個執行於同一系統之虛擬機器 (VM) 的工作排程機制 (bvt, sedf, credit 等 scheduler)
Xen
• An open source server virtualization software package
• This virtual machine monitor (hypervisor) offers near-native VM performance through– Paravirtualization– Hardware assisted virtualization
Resource Management in Xen
• Each domain including host and guests consume three types of resource:– CPU– Memory– I/O
CPU Resource Management
• The main target of this project
• Each guest domain can be assigned one or more VCPUs (Virtual CPUs) to be realized by physical CPU (or CPUs in SMP systems)
• This realization process is called CPU scheduling
• Xen has built-in schedulers
Memory Resource Management
• Each guest domain can be assigned a predetermined amount of memory
• With proper monitoring, memory utilization can be determined
• The goal of memory resource management is to warrant each VM has enough memory without paging mechanism, and no VM wastes memory
• Domain memory can be dynamically adjusted at run time with ‘xm mem-set’
I/O Resource Management
• Current I/O model in Xen is done through drivers in Dom0
• Dom0’s memory assignment is critical for I/O performance
(Source: Cherkasova et al., Comparison Of the Three CPU Schedulers in Xen, Sigmetrics Perf Eval Review)
Terminology for CPU Scheduling
• Terms have been extended from the domain of process scheduling to the domain of CPU scheduling– Proportional Share/Fair Share– Work Conserving/Non Work Conserving– Preemptive/Non Preemptive
Proportional Share (PS)
• PS scheduling allocates CPU to VMs according to their shares (weights)
• PS scheduler provides instantaneous form of sharing
• Fair Share (FS) schedulers tend to provide a time-averaged form of proportional share based on the actual use over long time periods
Work Conserving (WC)
• In the WC-mode, the shares are merely guarantees – In a case of two VMs with equal shares, if one
is blocked, the other one can consume the entire CPU
• In a Non-WC mode, the shares are caps– In a case of two VMs with equal shares, if one
has consumed 50% of a CPU, it can not consume any more of the CPU
– Bad-behaved domains can be isolated
Preemptive
• Preemptive schedulers rerun the scheduling decision when a new guest becomes ready– The new guest may preempt the running
guest if it has a higher priority
• Non-Preemptive schedulers make decisions when the running guest gives up CPU
CPU Schedulers of Xen
• Three built-in schedulers– BVT (Borrowed Virtual Time)– SEDF (Simple Earliest Deadline First)– Credit
• The scheduler is determined when Xen boots up and can not be changed after that moment
• Each scheduler takes input parameters to fine tune the performance at run time
BVT
• Based on the concept of virtual time (VT) consumed by a guest domain (virtual machine)
• The domain with a lower VT has a higher priority
• PS scheduling• Work conserving• Low latency support for real time apps
through warping VT
SEDF
• Support for both WC and Non-WC mode, bad-behaved VM can be isolated when the cap is reached (Non-WC mode)
• Preemptive
• Real time apps support
• Queue is per CPU
• No global load balancing among multiple CPUs in SMP systems
Credit
• PS schedulers
• Support for WC and Non-WC mode
• Global load balancing in SMP systems
• Non-preemptive
• The default scheduler in Xen 3.0
Monitoring Performance of Xen
• Schedulers manage CPU resources for guest domains
• Performance of Xen can be monitored by– Xen-based tools– Apps-based tools
• Performance becomes a feedback to tune schedulers
Xen-based Monitoring
• Xen offers several utilities to monitor the performance of a Xen system including Dom0 and guest domains
– xenmon.py, a Python-based utility, was designed to compare Xen performance under different CPU weightings of the domains
– xenstore stores configuration, event and status information about Xen domains
Apps-based Monitoring
• VM performance can also be monitored through specific apps designated to run on the VM, e.g.– Web server– Database server– Computation server
Adaptive Scheduling
• Idea1. Use Credit scheduler to take advantages of
global load balancing in SMP systems2. Credit accepts two input parameters (weight
and cap) for each domain to fine tune its behavior
3. Use proper monitoring tools to get domain performance
4. Performance is fed back to adjust weight and cap
Methodology
Credit Scheduler
Weight
Cap
Monitored
Performance
Workload:
web, db, etc.
Performance feedback is used to tune parameters
Summary (1)
• In this quarter, we have done– Study on the resource management of Xen do
mains, particularly from the perspective of CPU resource management
– Study on Xen schedulers for managing CPU resources
– Study on Xen performance monitoring tools– Study on fine-tuning Xen schedulers at run tim
e
Summary (2)
• 系統規劃– 使用 open source 軟體為開發基礎– 使用 Xen 平台為 server virtualization 基礎 – 使用 commodity level 多核 (multi-core) 硬體為平台– 假設雲端系統為提供高效率之計算平台，例如使用 Ha
doop 之 map-reduce ，設計一個以 CPU 資源應用為主的自動化負載平衡機制
– 暫不考慮 I/O 、 memory 負載平衡– 先以單機多核的負載平衡為標的再拓展到多機群集
Summary (3)
• 需求分析– 以 CPU 資源應用為主，如何在 Xen 的環境下管理 CPU 之資源應
用– Xen 的內建 scheduler 分配實體 CPU 的資源給予 VM 之虛擬 VCP
U ， Xen 3.0 預設使用的 Credit scheduler 能使用 WC 與 non-WC mode ( 改善了 BVT) 及多核的負載平衡 ( 改善了 SEDF)
– Credit scheduler 接受參數輸入來設定每一個 domain 的 weight 及cap 值
– 預設之 weight 及 cap 值不一定對所有的 domain 來說都是最佳的，domain 實際執行的應用會影響到效能
– 必須能偵測 domain 之效能以回饋到 Credit scheduler 來動態調整每一個 domain 的 weight 及 cap 值
– 使用 Xen-based 或 Apps-based 方式來偵測 domain 之效能
Outlook
• Looking forward, we plan to – Xen 系統實做及資料收集 . 實做 Xen 環境及
上述之 VM scheduler 於多處理器平台 , 收集效能分析資料並參考服務品質 (QoS) 專案之研究 , 設定效能指標及環境變數 .