nvidia dgx-1 超級電腦與人工智慧及深度學習
TRANSCRIPT
TAIPEI | SEP. 21-22, 2016
Eric Kang 康勝閔, Sep. 21 2016
NVIDIA DGX-1 超級電腦與人工智慧及深度學習
2
GPU Computing
NVIDIAComputing for the Most Demanding Users
Computing Human Imagination
Computing Human Intelligence
3
DEEP LEARNING EVERYWHERE
INTERNET & CLOUD
Image ClassificationSpeech Recognition
Language TranslationLanguage ProcessingSentiment AnalysisRecommendation
MEDIA & ENTERTAINMENT
Video CaptioningVideo Search
Real Time Translation
AUTONOMOUS MACHINES
Pedestrian DetectionLane Tracking
Recognize Traffic Sign
SECURITY & DEFENSE
Face DetectionVideo SurveillanceSatellite Imagery
MEDICINE & BIOLOGY
Cancer Cell DetectionDiabetic GradingDrug Discovery
4
DEEP LEARNING APPROACH
Deploy:
Dog
Cat
Honey badger
Errors
DogCat
Raccoon
Dog
Train:
DNN
DNN
5
72%74%
84%
88%
93%
96%
2010 2011 2012 2013 2014 2015
“SUPERHUMAN” RESULTSSPARK HYPERSCALE
ADOPTION
Deep Learning
ImageNet — Accuracy %
Cloud Services with AI Powered by NVIDIA
Alibaba/Aliyun Amazon Baidu eBay Facebook
Flickr Google iFLYTEK iQIYI JD.com
Orange Periscope Pinterest Qihoo 360 Shazam
Skype Sogou Twitter Yahoo Supermarket Yandex YelpHand-coded CV
Human
74%76%
6Source: IDC Worldwide Big Data and Analytics 2016 Predictions, November 2015. IDC FutureScape: Worldwide Digital Strategy Consulting 2016 Predictions, Nov 2015;
“By 2020, 80% of Big Data and Analytics deployments will need distributed micro analytics and 40% of all business analytics software will incorporate prescriptive analytics built on cognitive computing functionality. Both of these trends require a dramatic increase in processing power that could be enabled by GPUs.”
— IDC
“By 2018, over 50% of developer teams will embed cognitive services in their apps (vs 1% today) providing U.S. enterprises with over $60 billion annual savings by 2020.”
— IDC
AI — THE NEXT TRILLION $ IT OPPORTUNITY
7
Deep Learning is a massive opportunity
Data Scientist productivity is vital
NVIDIA is the choice of the deep learning world
DGX-1 is fast, instantly productive
NVIDIA DGX-1The Essential Tool of
Deep Learning Scientists
170 TFLOPS | 8x Tesla P100 16GB | NVLink Hybrid Cube Mesh2x Xeon | 8 TB RAID 0 | Quad IB 100Gbps, Dual 10GbE | 3U
8
TESLA P100 WITH NVLINKNew GPU Architecture to Enable the World’s Fastest Compute Node
Pascal Architecture NVLink CoWoS HBM2 Page Migration EnginePCIe
SwitchPCIe
Switch
CPU CPU
Highest Compute Performance GPU Interconnect for Maximum Scalability
Unifying Compute & Memory in Single Package
Simple Parallel Programming with Virtually Unlimited Memory
Unified Memory
CPU
Tesla P100
9
Engineered for deep learning | 170TF FP16 | 8x Tesla P100
NVLink hybrid cube mesh | Accelerates major AI frameworks
NVIDIA DGX-1WORLD’S FIRST DEEP LEARNING SUPERCOMPUTER
10
NVIDIA DEEP LEARNING SDKHigh Performance GPU-Acceleration for Deep Learning
COMPUTER VISION SPEECH AND AUDIO BEHAVIORObject Detection Voice Recognition Translation
Recommendation Engines Sentiment Analysis
DEEP LEARNING
cuDNN
MATH LIBRARIES
cuBLAS cuSPARSE
MULTI-GPU
NCCL
cuFFT
Mocha.jl
Image Classification
DEEP LEARNING SDK
FRAMEWORKS
APPLICATIONS
11
NVIDIA CUDNN
Building blocks for accelerating deep neural networks on GPUs
High performance deep neural network training and inference
Accelerates Caffe, CNTK, Tensorflow, Theano, Torch
Performance continues to improve over time
“NVIDIA has improved the speed of cuDNN with each release while extending the interface to more operations and devices at the same time.”
— Evan Shelhamer, Lead Caffe Developer, UC Berkeley
developer.nvidia.com/cudnn
AlexNet training throughput based on 20 iterations, CPU: 1x E5-2680v3 12 Core 2.5GHz.
0x
2x
4x
6x
8x
10x
12x
2014 2015 2016
K40(cuDNN v1)
M40(cuDNN v3)
Pascal(cuDNN v5)
12
NVIDIA DIGITSInteractive Deep Learning GPU Training System
Test Image
Monitor ProgressConfigure DNNProcess Data Visualize Layers
developer.nvidia.com/digitsgithub.com/NVIDIA/DIGITS
13
Instant productivity — plug-and-play, supports every AI framework
Performance optimized across the entire stack
Always up-to-date via the cloud
Mixed framework environments —containerized
Direct access to NVIDIA experts
DGX STACKFully integrated Deep Learning platform
14
NVIDIA DOCKER ON GITHUB
15
NVIDIA IMAGESPrebuilt and ready to use
16
DGX-1 CONTAINER LAUNCH FLOWCustomer data stays on premise
Web Browser
Node Management
User Authentication
Docker Image push/pull
Scheduler UI
HW/SW Metrics
LOCAL LAN
All Application Data
NFS Storage
DIGITS UI
Interactive Sessions
compute.nvidia.com 1. User schedules containers to run
3. User interacts with application
17
DIGITS FOR DGX-1A complete GPU-accelerated deep learning workflow
MANAGE TRAIN DEPLOY
DIGITS
DATA CENTER AUTOMOTIVE
TRAINTEST
MANAGE / AUGMENTEMBEDDED
GPU INFERENCE ENGINE
MODEL ZOO
18
BUILT FOR THE DATA CENTER
Data Center Ready24/7 Uptime
Boost data center throughput
Scalable Performance
Maximize reliability Simplify system operations
! !○
19
END-TO-END DESIGN FOR SYSTEM UPTIME 24/7 Uptime
Scalable Performance
Data Center Ready
Guaranteed QualitySystem Qual. Tests: Thermal, Stress, Airflow rate, Shock & Vibe
System Monitoring and Management for Tesla only
Dedicated Technical Staff for Failure Analysis
Extensive Qualification & Testing
Long Burn-in Testing
Zero Error Tolerance at Aggressive Clocks
Even with Differentiated Engineering 5% of GPUs are screened out
Differentiated Engineering
Low Operating Voltage for Long Term Reliability
Large Guard-band for Guaranteed Quality
Error Correction Code (ECC) for Data Integrity
20
DYNAMIC PAGE RETIREMENT MAXIMIZES UPTIME24/7 Uptime
Scalable Performance
Data Center Ready
GPU MEMORY
Uncorrectable Data Error causes application to
crash
Weak memory page is retired
Tesla GPU with Dynamic Page Retirement
GPU without Dynamic Page Retirement (DPR)
Weak memory is still active
1. Users lose productivity as jobs continue to crash
2. IT Managers need to physically open up the server and remove the bad GPU
3. Customer satisfaction risk with RMA process
1. Removes bad memory with simple reboot
2. No physical work required for IT
3. Negligible impact: <0.01% of memory is retired
!
21
DATA CENTER QUALIFIED BY SERVER OEMS24/7 Uptime
Scalable Performance
Data Center Ready
Server with Tesla GPU
Server with Unqualified GPU
Designed for max airflow through GPU
Supports airflow front-to-back & back-to-front
Lower power consumption
GPU Temp Running Linpack: 54C
Works against server airflow
Higher power consumption
Lower reliability
GPU Temp Running Linpack: 71C
Airflow
Temp: 54C
Temp: 71C
22
SCALE-OUT PERFORMANCE IN THE DATA CENTER24/7 Uptime
Scalable Performance
Data Center Ready
0
500
1000
1500
2000
8 16 32 64 96
Up to 2x Faster
Application Performance at Scale with GPUDirect RDMA
GPUDirect RDMAA
Direct transfers between GPUs
67% Lower GPU-to-GPU Latency
5x Higher GPU-to-GPU MPI Bandwidth
Tim
e-st
eps
per
Sec
# of Nodes
Hoomd-Blue ApplicationLJ Liquid Benchmark, 256K Particles
without RDMAwith RDMA
23
NVLINK DELIVERS SCALABLE PERFORMANCE24/7 Uptime
Scalable Performance
Data Center Ready
More than 45x Faster with 8x P100 Interconnected with NVLink
0x
5x
10x
15x
20x
25x
30x
35x
40x
45x
50x
Caffe/Alexnet VASP HOOMD-Blue COSMO MILC Amber HACC
2x K80 (M40 for Alexnet) 2x P100 4x P100 8x P100
Spee
d-up
vs
Dual
Soc
ket
Has
wel
l
2x Haswell
CPU
24
DATA CENTER GPU MANAGEMENT
24/7 Uptime
Scalable Performance
Device Management
• Device Identification
• Board Monitoring
• Clock Management
Per GPU Configuration & Monitoring
Data Center Ready
Enterprise-Grade Management Tool for Operating the Data Center
Active Health Monitoring ! Diagnostics &
System Validation
Runtime Health ChecksPrologue ChecksEpilogue Checks
Deep HW DiagnosticsSystem Validation Tests
Policy & Group Config Management
Pre-configured policiesJob level accountingStateful configuration
Power & Clock Mgmt.
Dynamic Power CappingSynchronous Clock Boost
!
Data Center GPU Manager (Tesla GPUs Only)
All GPUs Supported
25
DATA CENTER GPU MANAGER
24/7 Uptime
Scalable Performance
Data Center Ready
Integrated into Leading Industry Tools for HPC
Moab Cluster SuiteTORQUE
PBS Professional
IBM Platform HPCIBM Platform LSF
Bright Cluster Manager
StackIQ Boss for HPC with CUDA Pallet
Grid Engine
3rd PartySoftware
TAIPEI | SEP. 21-22, 2016
THANK YOU