![Page 2: Steve mushero china netcloud sz arch event 20120805.1](https://reader033.vdocuments.pub/reader033/viewer/2022051611/54bb09ee4a7959a46d8b45d9/html5/thumbnails/2.jpg)
1
Copyright © 2012 ChinaNetCloudServer Management Cloud computing Consulting
Running the World’s Internet Servers
Dealing with Chinese Network Anatomy
About Architecture
By Steve Mushero
![Page 3: Steve mushero china netcloud sz arch event 20120805.1](https://reader033.vdocuments.pub/reader033/viewer/2022051611/54bb09ee4a7959a46d8b45d9/html5/thumbnails/3.jpg)
2Running the World’s Internet Servers www.ChinaNetCloud.com ChinaNetCloud
Overview
Talk about Chinese Internet Structure Chinese Internet Problems & Issues
Strategies & Solutions Advice
![Page 4: Steve mushero china netcloud sz arch event 20120805.1](https://reader033.vdocuments.pub/reader033/viewer/2022051611/54bb09ee4a7959a46d8b45d9/html5/thumbnails/4.jpg)
3Running the World’s Internet Servers www.ChinaNetCloud.com ChinaNetCloud
About Steve
Founder, CEO & CTO of ChinaNetCloud Before that, CTO of Tudou ( 土豆网 )
Seven years in China From Silicon Valley, Seattle, New York
20+ years experience as CTO Experience in dev & ops
![Page 5: Steve mushero china netcloud sz arch event 20120805.1](https://reader033.vdocuments.pub/reader033/viewer/2022051611/54bb09ee4a7959a46d8b45d9/html5/thumbnails/5.jpg)
4Running the World’s Internet Servers www.ChinaNetCloud.com ChinaNetCloud
About ChinaNetCloud
Founded in Shanghai in 2008
by Silicon Valley Technology Guys
Vision: Run All the World's Internet Servers
We manage servers & systems
for Chinese Internet & Game Companies
Thousands of Servers - Hundreds of Customers
We've seen every type & size system
![Page 6: Steve mushero china netcloud sz arch event 20120805.1](https://reader033.vdocuments.pub/reader033/viewer/2022051611/54bb09ee4a7959a46d8b45d9/html5/thumbnails/6.jpg)
5Running the World’s Internet Servers www.ChinaNetCloud.com ChinaNetCloud
China Overview
China is World's largest Internet But Infrastructure has unique characteristics Difficult for world-class performance / user
experiences
Especially modern apps: – Mobile apps, ads, & e-commerce
End-user speed linked to success & profits
![Page 7: Steve mushero china netcloud sz arch event 20120805.1](https://reader033.vdocuments.pub/reader033/viewer/2022051611/54bb09ee4a7959a46d8b45d9/html5/thumbnails/7.jpg)
6 6
Running the World’s Internet Servers www.ChinaNetCloud.com
China Internet Basics
World's largest – 500M users
Very advanced / fast– 1-2-3 Mbps at home– 5-10-20 Mbps at work– 1-10-50 Gbps at IDC
But slow– Congestion– Regionality
![Page 8: Steve mushero china netcloud sz arch event 20120805.1](https://reader033.vdocuments.pub/reader033/viewer/2022051611/54bb09ee4a7959a46d8b45d9/html5/thumbnails/8.jpg)
7 7
Running the World’s Internet Servers www.ChinaNetCloud.com
China Telecom – South 21• China NetCom – North 10• China Mobile - GPRS• China UniCom – iPhone• CERNET - Students• Several more like Railcom
• Unicom bought Netcom• Mobile bought Railcom• Telecom bought Unicom CDMA
Monopoly ISPs
![Page 9: Steve mushero china netcloud sz arch event 20120805.1](https://reader033.vdocuments.pub/reader033/viewer/2022051611/54bb09ee4a7959a46d8b45d9/html5/thumbnails/9.jpg)
8Running the World’s Internet Servers www.ChinaNetCloud.com ChinaNetCloud
Internet Challenges
Poor interconnections Overloaded links Heavy-bandwidth uses
Within Region Between Regions Internationally
![Page 10: Steve mushero china netcloud sz arch event 20120805.1](https://reader033.vdocuments.pub/reader033/viewer/2022051611/54bb09ee4a7959a46d8b45d9/html5/thumbnails/10.jpg)
9 9
Running the World’s Internet Servers www.ChinaNetCloud.com
Internet ISP Basics
Each ISP is REGIONAL Actually one per province And one per large city And Mobile is separate
So not one company Ex: Telecom is 31 companies
They don't cooperate well
![Page 11: Steve mushero china netcloud sz arch event 20120805.1](https://reader033.vdocuments.pub/reader033/viewer/2022051611/54bb09ee4a7959a46d8b45d9/html5/thumbnails/11.jpg)
10 10
Running the World’s Internet Servers www.ChinaNetCloud.com
Bottlenecks
Within & Between Regions Heavy traffic
– P2P sharing– Video (Tudou, etc.)– Games, Music, etc.
Mobile speed limited– They have to buy bandwidth
Can use Mobile IDC, but . . .– Terrible service
![Page 12: Steve mushero china netcloud sz arch event 20120805.1](https://reader033.vdocuments.pub/reader033/viewer/2022051611/54bb09ee4a7959a46d8b45d9/html5/thumbnails/12.jpg)
11 11
Running the World’s Internet Servers www.ChinaNetCloud.com
Bottlenecks
ISPs don't care as no effect on revenue– Challenges of limited competition
Even worse as they use proxies, back channels
Especially in smaller provinces And CERNET for students
• Proxies in closets
![Page 13: Steve mushero china netcloud sz arch event 20120805.1](https://reader033.vdocuments.pub/reader033/viewer/2022051611/54bb09ee4a7959a46d8b45d9/html5/thumbnails/13.jpg)
12Running the World’s Internet Servers www.ChinaNetCloud.com ChinaNetCloud
Internet Challenges - Routing
BGP not common– Requires cooperation
– Back to 1990s methods Multi-line common
– Multi-IP per server
– Routing nightmare BGP increasingly used
– Especially for mobile
– Expensive, only high-end
![Page 14: Steve mushero china netcloud sz arch event 20120805.1](https://reader033.vdocuments.pub/reader033/viewer/2022051611/54bb09ee4a7959a46d8b45d9/html5/thumbnails/14.jpg)
13 13
Running the World’s Internet Servers www.ChinaNetCloud.com
International Links
Only a few links, bottlenecks Great Firewall of Fun Often unstable / highly variable
– Good one day
– Then bad for an hour, day, week Cannot depend on them
HK best, Japan next, beyond . . .
![Page 15: Steve mushero china netcloud sz arch event 20120805.1](https://reader033.vdocuments.pub/reader033/viewer/2022051611/54bb09ee4a7959a46d8b45d9/html5/thumbnails/15.jpg)
14Running the World’s Internet Servers www.ChinaNetCloud.com ChinaNetCloud
Internet Challenges - Students
Universities have separate network CERNET Hard to get to (expensive) Often use async routing to save money But useful for brands, games, etc.
– Use BGP / CDN
![Page 16: Steve mushero china netcloud sz arch event 20120805.1](https://reader033.vdocuments.pub/reader033/viewer/2022051611/54bb09ee4a7959a46d8b45d9/html5/thumbnails/16.jpg)
15Running the World’s Internet Servers www.ChinaNetCloud.com ChinaNetCloud
Internet Challenges - Mobile
Three carriers– Separate from land lines
Traditionally used Mobile IDC– Poor service
But also 50%+ on Wifi– Makes good routing & IDC hard
Moving to 3-line BGP
![Page 17: Steve mushero china netcloud sz arch event 20120805.1](https://reader033.vdocuments.pub/reader033/viewer/2022051611/54bb09ee4a7959a46d8b45d9/html5/thumbnails/17.jpg)
16Running the World’s Internet Servers www.ChinaNetCloud.com ChinaNetCloud
Industry Specific Issues
E-Commerce– Response time
– Reliability Advertising
– Performance Gaming
– Response time
– Multi-user zoning
– Big Downloads
![Page 18: Steve mushero china netcloud sz arch event 20120805.1](https://reader033.vdocuments.pub/reader033/viewer/2022051611/54bb09ee4a7959a46d8b45d9/html5/thumbnails/18.jpg)
17Running the World’s Internet Servers www.ChinaNetCloud.com ChinaNetCloud
BGP
Increasingly important 2, 3, 8 line – Very, very few 8 line Usually best, BUT . . . Expensive Not always good
– Poor links to other ISP
– Strict bandwidth limits Not always real
– NAT, other methods
![Page 19: Steve mushero china netcloud sz arch event 20120805.1](https://reader033.vdocuments.pub/reader033/viewer/2022051611/54bb09ee4a7959a46d8b45d9/html5/thumbnails/19.jpg)
18Running the World’s Internet Servers www.ChinaNetCloud.com ChinaNetCloud
IDC Data Centers
Increasingly important– Mobile and $$ driving
Highly variable bandwidth– Understand bandwidth
– When 100M is NOT ! Expansion a big problem Service a big problem Get what you pay for
![Page 20: Steve mushero china netcloud sz arch event 20120805.1](https://reader033.vdocuments.pub/reader033/viewer/2022051611/54bb09ee4a7959a46d8b45d9/html5/thumbnails/20.jpg)
19Running the World’s Internet Servers www.ChinaNetCloud.com ChinaNetCloud
Strategies Overview
Good Location Good Bandwidth Good Design CDN Monitor Advanced options
![Page 21: Steve mushero china netcloud sz arch event 20120805.1](https://reader033.vdocuments.pub/reader033/viewer/2022051611/54bb09ee4a7959a46d8b45d9/html5/thumbnails/21.jpg)
20Running the World’s Internet Servers www.ChinaNetCloud.com ChinaNetCloud
Locations
Single vs. Multi-location Multi-location very difficult
– Very rare in China
– Games or HA failover Best to choose 1 good location
– Grow and use 2nd for HA CHOOSE WELL
![Page 22: Steve mushero china netcloud sz arch event 20120805.1](https://reader033.vdocuments.pub/reader033/viewer/2022051611/54bb09ee4a7959a46d8b45d9/html5/thumbnails/22.jpg)
21Running the World’s Internet Servers www.ChinaNetCloud.com ChinaNetCloud
Locations
Best you can afford Ask about service Ask about expansion Think about mobile Avoid 3rd tier cities Avoid 3rd tier IDCs Avoid 2nd tier if you have $$ Think about clouds . . .
![Page 23: Steve mushero china netcloud sz arch event 20120805.1](https://reader033.vdocuments.pub/reader033/viewer/2022051611/54bb09ee4a7959a46d8b45d9/html5/thumbnails/23.jpg)
22Running the World’s Internet Servers www.ChinaNetCloud.com ChinaNetCloud
Choosing IDCs
Get the best you can afford Be careful, hard to change later Connectivity First
– Including DDoS, ARP, Net Service Second, important
– 24x7 Access
– Remote Hands ChinaNetCenter, 21ViaNet, SRT
![Page 24: Steve mushero china netcloud sz arch event 20120805.1](https://reader033.vdocuments.pub/reader033/viewer/2022051611/54bb09ee4a7959a46d8b45d9/html5/thumbnails/24.jpg)
23Running the World’s Internet Servers www.ChinaNetCloud.com ChinaNetCloud
Connectivity & Bandwidth
Get the best you can afford Think about users' location Single vs. 2 vs. 3 vs. 8 line Direct vs. BGP For mobile, remember Wifi
Buy the Best & CDN the Rest
![Page 25: Steve mushero china netcloud sz arch event 20120805.1](https://reader033.vdocuments.pub/reader033/viewer/2022051611/54bb09ee4a7959a46d8b45d9/html5/thumbnails/25.jpg)
24Running the World’s Internet Servers www.ChinaNetCloud.com ChinaNetCloud
Performance
Think small & fast Images, HTML, CSS, JS Follow all best practices
– YSlow, etc.
– Test & Analyze Use AJAX if possible Other push, XMPP, node.js CDN as much as you can
![Page 26: Steve mushero china netcloud sz arch event 20120805.1](https://reader033.vdocuments.pub/reader033/viewer/2022051611/54bb09ee4a7959a46d8b45d9/html5/thumbnails/26.jpg)
25Running the World’s Internet Servers www.ChinaNetCloud.com ChinaNetCloud
AJAX & Caching
A secret to perceived speed Use statics as much as you can
– Long statics & rename Use semi-statics
– Cache 1, 5, 60 minutes Carefully consider other objects
![Page 27: Steve mushero china netcloud sz arch event 20120805.1](https://reader033.vdocuments.pub/reader033/viewer/2022051611/54bb09ee4a7959a46d8b45d9/html5/thumbnails/27.jpg)
26Running the World’s Internet Servers www.ChinaNetCloud.com ChinaNetCloud
Push & Async
Latest trend to perceived speed XMPP
– Via OpenFire
– Messages, stats, updates Node.js Careful how you scale
– 250,000 per server ? Careful of bad connections
– Re-connect, lazy connect
![Page 28: Steve mushero china netcloud sz arch event 20120805.1](https://reader033.vdocuments.pub/reader033/viewer/2022051611/54bb09ee4a7959a46d8b45d9/html5/thumbnails/28.jpg)
27Running the World’s Internet Servers www.ChinaNetCloud.com ChinaNetCloud
Best practices via YSlow
Study Yahoo, Google & others Run YSlow & other tools Also GT Metrix
![Page 29: Steve mushero china netcloud sz arch event 20120805.1](https://reader033.vdocuments.pub/reader033/viewer/2022051611/54bb09ee4a7959a46d8b45d9/html5/thumbnails/29.jpg)
28Running the World’s Internet Servers www.ChinaNetCloud.com ChinaNetCloud
Leveraging CDNs
Another secret to perceived speed Also reduces expensive IDC bandwidth Careful of version, expiration, names WSA, too
– Whole Site Acceleration ChinaNetCenter, FastWeb, ChinaCache Clouds have CDN, too – AliYun, Amazon Use a CDN, Always !
![Page 30: Steve mushero china netcloud sz arch event 20120805.1](https://reader033.vdocuments.pub/reader033/viewer/2022051611/54bb09ee4a7959a46d8b45d9/html5/thumbnails/30.jpg)
29Running the World’s Internet Servers www.ChinaNetCloud.com ChinaNetCloud
Monitoring
Several good monitoring services From backbone or end user
– All around China
– Check each ISP
– Check each CDN But Expensive ! Network Bench, Gomez, Jiankongbao
![Page 31: Steve mushero china netcloud sz arch event 20120805.1](https://reader033.vdocuments.pub/reader033/viewer/2022051611/54bb09ee4a7959a46d8b45d9/html5/thumbnails/31.jpg)
30Running the World’s Internet Servers www.ChinaNetCloud.com ChinaNetCloud
Using Clouds
Very flexible Understand limits
– 100 Questions / Issues Good International - AWS Still limited in PRC
– But Improving
– Good Bandwidth, CDN
– Aliyun working hard to be AWS
– A few smaller – 21ViaNet Shanghai
![Page 32: Steve mushero china netcloud sz arch event 20120805.1](https://reader033.vdocuments.pub/reader033/viewer/2022051611/54bb09ee4a7959a46d8b45d9/html5/thumbnails/32.jpg)
31Running the World’s Internet Servers www.ChinaNetCloud.com ChinaNetCloud
Cross-Border International
Same but Different Consider cross-region
– PRC only
– Foreign only (where?)
– Both• Separate – Any Sync ?
• Integrated – Rackspace HK
Use cloud – Amazon
![Page 33: Steve mushero china netcloud sz arch event 20120805.1](https://reader033.vdocuments.pub/reader033/viewer/2022051611/54bb09ee4a7959a46d8b45d9/html5/thumbnails/33.jpg)
32Running the World’s Internet Servers www.ChinaNetCloud.com ChinaNetCloud
Summary
Chinese Internet Big But difficult Choose location & IDC carefully Design your app well Use CDN Monitor Be fast . . . users be happy . . .
![Page 34: Steve mushero china netcloud sz arch event 20120805.1](https://reader033.vdocuments.pub/reader033/viewer/2022051611/54bb09ee4a7959a46d8b45d9/html5/thumbnails/34.jpg)
33Running the World’s Internet Servers www.ChinaNetCloud.com ChinaNetCloud
Questions