![Page 1: Проектировка IPv6-оnly датацентра в Яндексе. Никита Широков](https://reader034.vdocuments.pub/reader034/viewer/2022052412/55892fffd8b42a50608b4639/html5/thumbnails/1.jpg)
![Page 2: Проектировка IPv6-оnly датацентра в Яндексе. Никита Широков](https://reader034.vdocuments.pub/reader034/viewer/2022052412/55892fffd8b42a50608b4639/html5/thumbnails/2.jpg)
IPv6 Only Data Centers in Yandex
Nikita V. Shirokov, network engineer, AS13238
![Page 3: Проектировка IPv6-оnly датацентра в Яндексе. Никита Широков](https://reader034.vdocuments.pub/reader034/viewer/2022052412/55892fffd8b42a50608b4639/html5/thumbnails/3.jpg)
Before we start
«How we launched IPv6 in Yandex» by Vladimir Ivanov
Two networks:
〉 Backbone (user's traffic handling)
〉 Technological network (so called «Fastbone»; bulk traffic)
!
https://tech.yandex.ru/events/yac/2012/talks/380/
3
![Page 4: Проектировка IPv6-оnly датацентра в Яндексе. Никита Широков](https://reader034.vdocuments.pub/reader034/viewer/2022052412/55892fffd8b42a50608b4639/html5/thumbnails/4.jpg)
Update on external IPv6 connectivity
mx.yandex.ru
4
mc.yandex.ru
![Page 5: Проектировка IPv6-оnly датацентра в Яндексе. Никита Широков](https://reader034.vdocuments.pub/reader034/viewer/2022052412/55892fffd8b42a50608b4639/html5/thumbnails/5.jpg)
Why we decided to build IPv6 only DC
〉We ran out of public v4 addresses (and almost ran out of 1918)
〉We think that public v6 is better that CGN for InterDC connectivity
5
![Page 6: Проектировка IPv6-оnly датацентра в Яндексе. Никита Широков](https://reader034.vdocuments.pub/reader034/viewer/2022052412/55892fffd8b42a50608b4639/html5/thumbnails/6.jpg)
Points of interest
〉WAN
〉 Firewalls and SLBs
〉 Intra DC Network (rack to rack etc)
〉 Services
6
![Page 7: Проектировка IPv6-оnly датацентра в Яндексе. Никита Широков](https://reader034.vdocuments.pub/reader034/viewer/2022052412/55892fffd8b42a50608b4639/html5/thumbnails/7.jpg)
WAN
IPv6 Ready
〉MPLS 6VPE for Intra Project traffic
〉 Native IPv6 for the rest
7
![Page 8: Проектировка IPv6-оnly датацентра в Яндексе. Никита Широков](https://reader034.vdocuments.pub/reader034/viewer/2022052412/55892fffd8b42a50608b4639/html5/thumbnails/8.jpg)
Firewalls
8
Current: optimized IPv6 kernel code and fw!
Future: Netmap based solution!
![Page 9: Проектировка IPv6-оnly датацентра в Яндексе. Никита Широков](https://reader034.vdocuments.pub/reader034/viewer/2022052412/55892fffd8b42a50608b4639/html5/thumbnails/9.jpg)
SLBs
9
WAN
L2 DC Fabric
![Page 10: Проектировка IPv6-оnly датацентра в Яндексе. Никита Широков](https://reader034.vdocuments.pub/reader034/viewer/2022052412/55892fffd8b42a50608b4639/html5/thumbnails/10.jpg)
SLBs(cont)
10
Issues:No checks inside tunnels
No 6over4 in LVS
Someone needs do
decapsulateL3 DC Fabric
WAN
![Page 11: Проектировка IPv6-оnly датацентра в Яндексе. Никита Широков](https://reader034.vdocuments.pub/reader034/viewer/2022052412/55892fffd8b42a50608b4639/html5/thumbnails/11.jpg)
Issues:
– No checks inside tunnels
– No 6over4 in LVS
– Someone needs do
– decapsulate
11
WAN
L3 DC Fabric
![Page 12: Проектировка IPv6-оnly датацентра в Яндексе. Никита Широков](https://reader034.vdocuments.pub/reader034/viewer/2022052412/55892fffd8b42a50608b4639/html5/thumbnails/12.jpg)
Our initial design for IPv6 only DC
12
BackboneFastbone
![Page 13: Проектировка IPv6-оnly датацентра в Яндексе. Никита Широков](https://reader034.vdocuments.pub/reader034/viewer/2022052412/55892fffd8b42a50608b4639/html5/thumbnails/13.jpg)
Services
Most of the services IPv6 ready since 2011+ (or so)
But:
〉 It’s easy to forget about monitoring
〉 or cluster’s management
〉 or replication’s transport
〉 or bootstrapping
13
![Page 14: Проектировка IPv6-оnly датацентра в Яндексе. Никита Широков](https://reader034.vdocuments.pub/reader034/viewer/2022052412/55892fffd8b42a50608b4639/html5/thumbnails/14.jpg)
Deploy. Phase 1
14
BackboneFastbone# show ipv6 bgp summary vrf …
BGP summary information for VRF …, address family IPv6 Unicast
BGP router identifier …, local AS number 65400
BGP table version is 288725, IPv6 Unicast config peers 210, capable peers 210
370 network entries and 370 paths using 45880 bytes of memory
BGP attribute entries [9/1224], BGP AS path entries [2/16]
BGP community entries [2/64], BGP clusterlist entries [0/0]
![Page 15: Проектировка IPv6-оnly датацентра в Яндексе. Никита Широков](https://reader034.vdocuments.pub/reader034/viewer/2022052412/55892fffd8b42a50608b4639/html5/thumbnails/15.jpg)
Not only IPv6 DC
RND: (or «I need something from github!111»)
〉 6to4 Nat. No need for high performance
Non Realtime servers needs to dl something from internet(such as webrobots/spiders etc)
〉 4over6 tunneling or Dedicated racks with /26 IPv4 subnets
〉 Future: high performance 6to4 Netmap bassed NAT/FW
15
![Page 16: Проектировка IPv6-оnly датацентра в Яндексе. Никита Широков](https://reader034.vdocuments.pub/reader034/viewer/2022052412/55892fffd8b42a50608b4639/html5/thumbnails/16.jpg)
Not only IPv6 DC
!
Realtime servers needs to dl/query something from internet (such as public dns resolvers etc)
〉 4over6 tunneling
〉 Future: high performance 6to4 Netmap bassed NAT/FW
16
![Page 17: Проектировка IPv6-оnly датацентра в Яндексе. Никита Широков](https://reader034.vdocuments.pub/reader034/viewer/2022052412/55892fffd8b42a50608b4639/html5/thumbnails/17.jpg)
Questions?
![Page 19: Проектировка IPv6-оnly датацентра в Яндексе. Никита Широков](https://reader034.vdocuments.pub/reader034/viewer/2022052412/55892fffd8b42a50608b4639/html5/thumbnails/19.jpg)
Additional Slides:
19
WAN
L2 DC Fabric
SLB check:
http get from 10.x.x.x to 10.x.x.y
external IP to VIP
external IP to 10.x.x.y
PBR: from 10.x.x.y thru 10.x.x.x
![Page 20: Проектировка IPv6-оnly датацентра в Яндексе. Никита Широков](https://reader034.vdocuments.pub/reader034/viewer/2022052412/55892fffd8b42a50608b4639/html5/thumbnails/20.jpg)
Additional Slides:
20
WAN
L3 DC Fabric
external IP to VIP
external IP to VIP encapsulated into SLB to Server
we cant http get on VIP @ SLB (will go to local IP @ loopback). therefore cant check if this address exist @ Server