How to Design and Develop DNS System

in CDN

Agenda

Intelligent DNS Resolution DNS Resolution Performance Requirement Relationship between Intelligent DNS and CDN traffic scheduling

Intelligent DNS Resolution

Process CDN & DNS support standard protocols Communication of Authoritative DNS System Consuming Characteristics

Process

CDN & DNS Supports Standard Protocols

rfc 1035 : basic rfc 2671 : EDNS0 rfc 3596 : AAAA ECS supports: draft-vandergaast-edns-client-subnet-04

Authoritative DNS Communication

Typical Messages Mainly UDP Single package request response Small package No-repeat five-tuple

Business aspect Delay-sensitive Distribution deployment

System Consuming

Network I/O Intensive Small package, high frequency One-time network I/O

CPU Intensive Small package, high frequency One-time network I/O Domain mainly for string query and processing Extra CDN strategy computing

Domain Resolution

Terminal interaction with local DNS Local DNS interaction with CDN DNS Local DNS interaction with DNS cluster

Terminal interaction with local DNS

Most terminal uses local DNS Local terminal

users

Local DNS terminal users

Local DNS interaction with CDN DNS

The user scale of local DNS varies significantly.

CDN DNS perspective Local DNS cache dilutes hotspots

Traffic scheduling perspective Each DNS visits cover varying sizes of users

CDN DNS

LDNS用户群

Local DNS interaction with DNS cluster

Local DNS Picking： Resolving Delay Sensitive Choose near NS for high probability

(short RTT) Choose far NS for low probability

(long RTT) Traffic scheduling perspective

Each NS has unequal visits

Local DNSExample: How to choose the best NS

1 5 9 13 17 21 25 29 33 37 41 45 49 53 57 61 65 69 73 77 81 85 89 93 97 1011051091131171211250

5

10

15

20

25

30

35

40

45

50

DNS Resolution Sequences

Dela

y m

s

DNS

Devi

ces

Local DNS Example: Punishiment under high delay

502502

5005075145215285355425495565635705775845915986056126196266336406476546616686756826896960

20

40

60

80

100

120

140

160

180

200

DNS Resolution Sequences

Dela

y m

s

DNSD

evice

s

Performance Requirement

High quality DNS system’s performance Performance index evaluation Performance test Network I/O selection New DNS data storage

High quality DNS system’s performance

Resemble ICMP echo services (ping)

Performance index evaluation

PerformanceC40M - 40Gb

C10M - 10Gb

C1M - 1Gb

C500K-C100 - 100Mb

C10K - 10Mb

C1K - 1Mb

C100K

Domain Resolution Quanlity

Huge DDoS

Small DDoS

DDoS

Huge website peak analysis

Active website

Single Machine Performance

Knot、 NSD、 BIND、 PowerDNS

BIND+DLZ+NOSQL

Hot-spot dilution’s impact on performances

Reason of hot-spot dilution Upstream Local DNS is cache-like services, which won’t be visited in before

expiration Hot-spot dilution effects

Online performance much lower than test performance High cache missing of software and hardware In Cloud, Higher cache missing of software and hardware

Beware Extra resources need to be spared during evaluation On storage and algorithm selection, extra resources need to be spared for

random access performance stability Replay or clone online traffic for DNS test.

Performance Test

queryperf DNSPerf tcpreplay tcpcopy Performance test devices

Traps in performace evaluation What is the parallel performance of CDN DNS?

Authoritative DNS’s purpose is to finish one-time response ASAP For CPU intensive business, better lower the parallel operation to avoid

getting out of control QPS is the principal index on evaluating CDN DNS’s performance

How about using queryperf/DNSPerf to evaluate online CDN DNS performance? Hard to cover the Local DNS variation of CDN DNS Fixed five-tuple makes it hard to show the lower level consuming Queryperf/DNSperf matches the testing device’s response rhythm to

perform fixed intensity stress test. Visit frequency of Internet usually follows Poisson distribution with

fluctuation Don’t forget human-caused visit frequency fluctuation

Regular probing, regular web crawler

Network I/O selection

Performance MagnitudeQ40M - 40GbQ10M - 10GbQ1M - 1Gb

Q100K - 100Mb Q10K - 10Mb

Q1K - 1Mb

Network I/ODPDK

PF_RING,NetmapBPS,netfilter

Socket

New DNS data storage Open LDAP LMDB

Lightning Memory-Mapped Database PowerDNS + LMDB

400kqps Knot DNS

700kqps

Load Balancer Selection Switch/Router + Anycast in the Node

First choice if condition allows Hardware Load Balancer Software Load Balancer

Intelligent DNS and CDN Traffic Scheduling

Example analysis Factors that impact DNS scheduling How to assess DNS scheduling result Scheduling alogrithm can be used

DNS Scheduling optimizing examples

Factors that impack intelligent DNS scheduling

Intelligent DNS doesn’t interact directly with terminal Some terminals change its Local DNS Local DNS Cache has impact on scheduling effection and expiration Local DNS covers varied sized user groups Local DNS optimizing action impacts data equity

Accuracy and precision of DNS Scheduling

0%

10%

20%

30%

40%

50%

60%

70%

High accuracy, low precision

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

80.00%

90.00%

Low accuracy, high precision

Actual Result Scheduling Target

Impacts of accuracy and precision of intelligent DNS

Precision Impact dynamic adjustment of the scheduling system Impact device bandwidth usage

Accuracy Impact prediction and planning of the scheduling system

DNS Scheduling basis

DNS statistics data Better not use directly

Intrinsic properties Local DNS IP location property Local DNS IP netowrk property

Stateless properties Hash characteristic Random

Common Scheduling Categories

Default equally divide by traffic By propotion IP categories

Static location categories Dynamic network link categories …

Characteristics of Common Scheduling Categories

Category Accuracy PrecisionBy location Really bad GoodDefault equally divide Very good Very goodRandom Accurate Not stableBy Local DNS IPs Bad Good

Combine intelligent DNS with other scheduling ways

Intelligent DNS for coarse-grained, other ways for finer grainuality Http 302 Cluster Limit connections and traffics on devices

Mixed uses in node for better productivity according to their own cases High quality and small traffic

High precision intelligent DNS scheduling Low demand and huge traffic

High precision intelligent DNS scheduling and cross-node scheduling Limit connections and traffics on auxiliary devices

Security Software pitfalls

Heterogeneous software backup for each other Open source DNS based

Function tailoring CDN is the most fundamental and stable part of the DNS software Useless code branches should be cut out

DDos traffic attacking High performance unusual DNS software can be designed with the bandwidth advantages of

CDN Introduce 3rd part DNS security products

Beware of the impact of security product introducing on traffic scheduling Improve the software performance to help the 3rd party security product and lower risks

High Availability

Distribution deployment Heterogeneous DNS software Platform images

Heterogeneous DNS

Combine two DNS software into one group 13 global roots of DNS are mixed in ues for BIND and NSD

Why heterogeneous DNS? Using the inconsistency of different DNS’s defects to ensure high

availability Being one of the critical system, defects in DNS can be catastrophic Local DNS cluster retrying can be contagious and paralyze the whole

sets of DNS Cost of development and maintenance

DNS is simple and the development cycle is short DNS is the relatively stable network protocol, changes in CDN related

part even smaller CDN related function

Can be done when stable Open source DNS might be short on customized CDN features, but they

are good fail-safe system.

DNS Clone Backup

Facing platform level and outside malfunctions

Platform level domain failure Domain registration failure Upper and top level domain failure Security threat

Spread the risk Don’t put all the eggs in one

basket Evacuation

Transfer customer to backup platform when under failure or threats

user1.cnd1.cn

user2.cnd1.cn

user3.cnd1.cn

user4.cnd1.cn

user5.cnd1.cnuser1.cnd2.co

muser2.cnd2.com

user3.cnd2.com

user4.cnd2.com

user5.cnd2.com

user1user2user3user4user5

CDN Platform 1

CDN Platform 2

Custom

er

Thanks!Beijing SpeedyCloud Technology Co., Ltd.,

For More Information, please visit: www.speedycloud.cnYou can also send email to: social@speedycloud.cn