1 network appliance confidential |september 6, 2015 system engineer clive chiang...

1April 21, 2023Network Appliance Confidential |

System EngineerClive [email protected]

CDN的規劃與建置

SimpleFastReliable


CDN 的基本架構

Content DeliveryContent Storage

Content Reporting

Content Distribution

GSLB


CDN 的基本元件

Content Storage 資料層 (Data Layer) Content Delivery 資料層 (Data Layer) Content Distribution 資料層 (Data Layer) Content Reporting 資料層 (Data Layer) Global Request Redirection 網路層 (Network Layer)


Content Storage Architecture

Internet

Web Server Web Server Media Server Media Server

Load Balancer

Server Accelerator Farm

Content Storage

VLAN3

VLAN2

VLAN1

Data Center

Web Volume Media Volume

• A web volume shares to all web servers

• A media volume shares to all media servers


Content Delivery – On demand Caching

Media Server

Internet

Content Delivery

Encoding Server


Local DNS Srvr

(1)(5)

Trace of DNS queries for CDN sub-domain

CDN Provider

DNS Srvr(.x.com)

(4)

(3)(2)(6)

GRM DNS Srvr(.cdn.x.com)Top Level

DNS Srvrs(., .com)

(7)

InterNIC, etc.

Customer "x"ISP


Local DNS Srvr

GRM DNS Server Redundancy

GRM DNS Srvr1

DNS ResponseNS: GRM DNS Srvr1NS: GRM DNS Srvr2

Local DNS Server will failover quickly to alternative GRM DNS Servers if there is no response

Local DNS Server will failover quickly to alternative GRM DNS Servers if there is no response

Response can contain multiple NS records

Response can contain multiple NS records

Top LevelDNS Srvrs

GRM DNS Srvr2


Local DNS Srvr

GRM DNS Server Response

GRM DNS SrvrAgent3Agent2Agent1

DNS Responses1st: IP1, IP2, IP32nd: IP2, IP3, IP13rd: IP3, IP1, IP2

IP3IP2IP1

Failover– TTL is set low: changes are reflected quickly

Load-Balancing (crude, but effective)– The order is rotated for each response– Clients normally use 1st IP address– Note: Win2K uses "subnet prioritization"

and picks "closest" IP address`

Local DNS Server may cache result– Typically rotates addresses in responses

Failover– TTL is set low: changes are reflected quickly

Load-Balancing (crude, but effective)– The order is rotated for each response– Clients normally use 1st IP address– Note: Win2K uses "subnet prioritization"

and picks "closest" IP address`

Local DNS Server may cache result– Typically rotates addresses in responses

Response contains multiple IP addresses

Response contains multiple IP addresses


Local DNS Srvr

Effect of Local DNS Server

GRM DNS Srvr

Agent1

Agent2

Assumption Client is close to its

local DNS server (LDNS)

Assumption Client is close to its

local DNS server (LDNS)

GRM DNS Server receives request from LDNS

– Proximity measurements are from agents to LDNS

GRM DNS Server receives request from LDNS

– Proximity measurements are from agents to LDNS

DNS Queries and Responses

Proximity Measurements


Main Data Center

(7)

DNS Srvr

(1)(5) IP = DNS Agent2

DNS Based Routing:When it works

Remote Location

DNS Srvr

(4)

(3)

(2)

(6)


Application Level Request

Proximity Measurements

DNS Agent1

DNS Agent2

DNS Srvr

Top LevelDNS Srvrs


(3)

L7 Srvr

DNS Srvr

(1)

(2) IP = L7 Srvr

L7 Redirection

Remote Location

L7 Agent

Main Data Center



Redirection


(7)

DNS Agent1

DNS Srvr

Top LevelDNS Srvrs

(1)

DNS + L7 Based Request Redirection

Remote Location

L7 Agent1DNS Agent 2

(6)

Main Data Center



Redirection

L7 Srvr

DNS Srvr

(5) IP = DNS Agent1

(4)

(3)

(2)


Process:if (file type matches { "ppt", "pdf", ... }) {

redirect }else { serve request directly }

Process:if (file type matches { "ppt", "pdf", ... }) {

redirect }else { serve request directly }

Configurable List

Conditional Redirection Serving average sized files (~14kB) is quite fast Only redirect if file size is "likely" to be large

Conditional Redirection Serving average sized files (~14kB) is quite fast Only redirect if file size is "likely" to be large

HTTP Redirection


RTSP Redirection

(1) Request

(2) Redirect

(3) Request

L7 Server

L7 Agent

(1) RequestGET rtsp://srvr/file.rm RTSP/1.0

(2) Response302 RedirectLocation: rtsp://cache/srvr/file.rm

(3) RequestGET rtsp://cache/srvr/file.rm RTSP/1.0


MMS Redirection

Rewritten NetCache Response<ASX>

<ref href="mms://agent-ip/strm-srvr/strm.asf"/>

</ASX>

Browser Requesthttp://www.cdn.x.com/file.asx

Original Web Server Response<ASX>

<ref href="mms://strm-srvr/strm.asf"/>

</ASX>

The MMS protocol has no redirect capability ASX rewriting can be used to achieve a similar affect

The MMS protocol has no redirect capability ASX rewriting can be used to achieve a similar affect


Load Balancing and Failover

Remote LocationMain Data Center

Primary location

Failover location

Clients A Clients B

Center CachesEdge Caches for Clients A

Edge Caches for Clients B


Example Criteria for System Selection

System Selection

Criteria Notes

System HealthFailure or overload is a cause for removal from list of available systems.

Domain Based PoliciesAllow restriction based on location, supported protocols, etc.

Geographical LocationChoice can be narrowed down (at least to continent) by matching client IP address with IANA database

Client-Node ProximityDistance of client from each system can be determined dynamically.

Proximity information can also be preloaded.


Performance Planning

HTTP Forward and Reverse Proxy Cache– Target web objects capacity– Total disk numbers– Total disk space– Network interface

Target web site– Total web site capacity

Testing Program– Ex, Polymix4

Measurement– URL/s– Response Time


Performance Planning

WM Streaming Reverse Proxy Cache (non-transparent)– Total disk numbers ( 硬碟顆數要足夠 )– Total disk space– GbE Network interface ( 網路介面卡作 Trunking)– License ( 選用最高等級 )

Target Media Streaming– Bit rate (ex, 300Kbps,750Kbps,1Mbps,2Mbps,3Mbps)– Film length (ex, 60 min,90min)– Film numbers ( 數量越多越好 )

Testing Program– Windows Media Load Simulator

Measurement– ON-demand caching concurrent users

• Hit from Disk ( 所有影片是由 Disk 提供 )• Hit from Memory ( 所有影片是由 Memory 提供 )

– Live Splitting concurrent users


Singapore MOE

7 NetCaches (2 x C6100, 5 x C3100) with MMS,Real, QuickTime license

2xC6100 for their Internet Zone with Smartfilter 3.0 (NOC1 & NOC2)

– Layer 4 (Foundry ServerIron 400) for Transparent Caching (GE Interface)

4xC3100 for the Schools WAN Routers with about 400 schools (North,South,East & West)

– WCCP Enabled (FE Interface)

1xC3100 for the MOE HQ LAN Staff– WCCP Enabled (GE Enabled)

Network Appliance Confidential21

TANET 之 CDN 應用範例參考


CDN System Architecture

Internet

Web / MediaServer

DB Server

NetAppPrimary Storage

NetCache

Web MediaReverse Proxy

NetCache

GRM Agent

NetCache

GRM Agent

NetCache

GRM Agent

Data CenterRemote Site

Encoding/Editing Servers

GRM System


HTTP

DNS Server

Top LevelDNS Servers

GRM System Architecture

Remote Location

GRM L7 Agent1HTTP,MMS,Real

(6)

(1)

Data Center



(7) Redirection

GRM DNS Agent

GRM L7 Server2

(3)

HTTP

GRM DNS Agent

GRM L7 Server1

DFM CR




GRM Edge Group1

GRM Edge Group2

Streaming(8)

WMS

AuthoritativeDNS Server

(4)

(5)GRM DNSServer1

GRM DNSServer2

(2)


GRM Domain

在 edu.tw 下建立一個新的 domain=cdn.edu.tw– 屬於 GRM 轉向管理的 domain

此 domain 下有兩個 name server– GRMDNS1 ip1– GRMDNS2 ip2

在 cdn.edu.tw 下建立兩個 host record– grm ip3– grm ip4

每個地理區域依 CIDR 的集合來建立 Edge group– 每個 Edge group 依需求放置一台或多台的 NetCache


特色

不需使用 L4 switch 或 WCCP router 作透通式代理不需使用 GSLB 設備來達到全球負載平衡只需設計單一連結即可 . 任何地區的使用者的要求 ( 包括 http

和 streaming) 都會從當地的 Agent 直接提供服務 . Ex,– http://grm.cdn.edu.tw/page.html– http://grm.cdn.edu.tw/run.asp– http://grm.cdn.edu.tw/play.asx

可自行指定要轉向所需的副檔名每個 NetCache 可指定成只擔任反向代理模式 (Reverse Prox

y) 的 Web 及 Streaming Accelerator ，避免被使用者任意指定為正向代理快取伺服器來使用


特色 Name server 本身具有 DNS 的備援性 GRM DNS Server 會自動選擇合適的 GRM L7 server

– GRM L7 server 會自動負載平衡– 可隨時依負載狀況增加 GRM L7 server

GRM L7 Server 會自動選擇合適的 GRM Agent– 每個 Edge group 內的 Agent 會自動負載平衡– 可隨時依負載狀況增加 GRM Agent

任何一套系統的單一顆硬碟故障時– 不會造成服務中斷– 不會造成檔案流失– 不會造成日誌流失– 備援硬碟會自動進行重建 (Auto Rebuild)– 自動發出 email 和 SNMP 通知系統管理人員


TANET 之 CDN 應用範例參考查榜系統架構說明

NetApp Primary Storage– 可加速資料庫的存取速度

NetApp NetCache– 可加速網路的存取速度– Data Center

• 保護 Web Server• 減輕防火牆負擔

– Remote Site• 加快網頁的回應速度

GRM– 不需使用 L4 Switch/WCCP router 作透通式代理– 不需針對不同區域撰寫不同的網頁連結– 系統會自動將 user request 轉向到最佳的 NetCache


查榜系統方案一

使用 asp?no=12345 方式來查詢設定 NetCache 對此可強迫快取，並設定 TTL 結果

– 第一次的查詢結果從 Server 讀出– 第二次的查詢結果從 NetCache 讀出– 可減輕 Server 的負載– 加快查詢的回應速度


查榜系統方案二

將所有結果根據準考證號碼以程式產生對應的 html– 使用 no_12345.html 方式來給予查詢結果– 在 Web Server 上對這些 html 設定 TTL

利用 DFM 預先將這些 html 全部派送到 NetCache– 可另外設定 min-age / TTL / Lock time

結果– 所有的查詢結果都從 NetCache 讀出– 大幅減輕 Server 的負載– 加快查詢的回應速度

Network Appliance Confidential30

Q & A

Thank You

1 network appliance confidential |september 6, 2015 system engineer clive chiang...

Documents