microsoft big data 解 決方案與案例分享 rich ho technical architect 微軟技術中心
TRANSCRIPT
Microsoft Big Data 解決方案與案例分享
DBI 312
Rich HoTechnical Architect微軟技術中心
Agenda
• What is Big Data?• Microsoft Big Data Strategy• Key Benefits of Microsoft Big Data• Demo• Case Study
What is Big Data ?
By 2015, organizations that build a modern information management system will outperform their peers financially by 20 percent.
– Gartner, Mark Beyer, “Information Management in the 21st Century”
10x increase every five years
85% from new
data types
Dataexplosion
VolumeVelocityVariety
Easy Accessibility of External Data
Hadoop
Cloud
Cheap, Distributed Storage &
Processing
The world of Data is changing
Three V in Big Data
Relational Data
10x increase every 5 years
VOLUME
4.3 connected devices per adult
VELOCITY
VARIETY
85% from new data types
Big Data
We are moving to Big Data
Any Data, Any Size, Anywhere
Traditional Data
Big Data
Data Volume Petabytes+Terabytes
Data Variety UnstructuredStructured
Data Velocity StreamingBatch
Garbage? +++++-----
With Big Data, says:
以前我們將尿布和啤酒放在一起而創造新業績
現在我們將比老爸更早一步知道他女兒懷孕了 !
Traditional data flow
New exploratory data flow
What is Big Data?
Increasing Data Volumes
Increasing Data & Analysis Complexing
Emerging Technologies
Microsoft Big Data Strategy
Microsoft Big Data Stacks
NON-RELATIONAL
100111
DATA MANAGEMENT
SHAREAND GOVERN
DISCOVERAND RECOMMEND
TRANSFORMAND CLEAN
INSIGHTS
DATA ENRICHMENT
OPERATIONAL
SELF-SERVICE MOBILE
PREDICTIVE
REAL-TIMECOLLABORATIVE
MA
RK
ETPLA
CE
Exte
rnal D
ata
and S
erv
ices
RELATIONAL MULTIDIMENSIONAL STREAMING
Fast
L
oad
Source Systems
Historical Data(Beyond Active Window)
Summarize & Load
Big Data Sources (Raw, Unstructured) Data & Compute
Intensive Application
ERPCRM LOB
APPS
Integrate/Enrich
Enterprise Data Warehouse
Enterprise ETL with SSIS, DQS, MDS
Distributed environments for Non-structured data
Data Marts
Reporting
Multi-dimensional Analysis
Business Insights
Interactive Reports
Performance Scorecards
Crawlers
Bots
Devices
Sensors
Online Transaction
Enterprise Big Data Blueprint
Hadoop on Windows
Hadoop on Azure
Fast
L
oad
Source Systems
Historical Data(Beyond Active Window)
Summarize & Load
Big Data Sources (Raw, Unstructured) Data & Compute
Intensive Application
ERPCRM LOB
APPS
Integrate/Enrich
Hadoop on Windows Azure
Hadoop on Windows Server
Business Insights
Interactive Reports
Performance Scorecards
Crawlers
Bots
Devices
Sensors
Enterprise Data Warehouse
Data Marts
Reporting
Multi-dimensional Analysis
Enterprise ETL with SSIS, DQS, MDS
Online Transaction
Microsoft Big Data Platform
Fast
L
oad
Source Systems
Historical Data(Beyond Active Window)
Summarize & Load
Big Data Sources (Raw, Unstructured) Data & Compute
Intensive Application
ERPCRM LOB
APPS
Integrate/Enrich
Hadoop on Windows Azure
Hadoop on Windows Server
Business Insights
Interactive Reports
Performance Scorecards
Crawlers
Bots
Devices
Sensors
Enterprise Data Warehouse
Data Marts
Reporting
Multi-dimensional Analysis
Enterprise ETL with SSIS, DQS, MDS
Online TransactionSQL Server
EE
SQL Server PDW
SQL Server FT
Microsoft Big Data Platform
Fast
L
oad
Source Systems
Historical Data(Beyond Active Window)
Summarize & Load
Big Data Sources (Raw, Unstructured) Data & Compute
Intensive Application
ERPCRM LOB
APPS
Integrate/Enrich
Hadoop on Windows Azure
Hadoop on Windows Server
Business Insights
Interactive Reports
Performance Scorecards
Crawlers
Bots
Devices
Sensors
Enterprise Data Warehouse
Data Marts
Reporting
Multi-dimensional Analysis
Enterprise ETL with SSIS, DQS, MDS
Online Transaction
SQL Azure
Microsoft Big Data on Public Cloud
Hadoop on Azure (CTP)
Request a New Cluster (4 nodes, 2 TB)
Allocating NN & DN
Starting NN & DN
NN & DN ready! (after 9 mins)
Hadoop on Azure ready! (after 13 mins)
Access Hadoop on Azure through RDP
Key Benefits of Microsoft Big Data
Microsoft Big Data Benefits
• New Hadoop-based distribution on Windows / Azure
• Hive ODBC driver
• Hive add-in for Excel
Broader Access of Hadoop
• Analyze Big Data with familiar tools (Excel, PowerPivot, Power View)
• JavaScript based simple programming
Breakthrough Insights
• System Center Integration
• Active Directory Integration
Enterprise Ready
Breakthrough Insights
Hive ODBC Driver integrates Hadoop to SQL Server Analysis Services, PowerPivot, and Power View
Breakthrough Insights
Benefits
Key
Featu
res
Broader Access of Hadoop
Hive add-in for Excel
Hive ODBC driver
Broader Access of Hadoop
Benefits
Key
Featu
res
Enterprise Ready (SCOM Integration)
System Center to monitor performances of Head nodes and Data nodes
Enterprise Ready
Benefits
Key
Featu
res
1. Hadoop on Azure - JavaScript Console - Job Execution2. Excel Hive Connectivity3. Windows Marketplace & Hadoop on Azure Integration
Case Study: Mining Big Mobility Data
Microsoft Big Data Case Study: T-Drive
Microsoft Big Data Case Study: T-Drive
Video
Microsoft Research AsiaBig Data Case Study
Thank You
Connect. Share. Discussshttp://www.microsoft.com/taiwan/techdays2012/
Microsoft Certification & Training Resources
http://www.microsoft.com/learning/zh/tw/
Resources for IT Professionals
http://social.technet.microsoft.com/Forums/zh-tw/categories
/
Resources for Developershttp://social.msdn.microsoft.com/Forums/zh-tw/categories
/
Resources
請協助完成「本課程問卷」,並在離開教室時交給工作人員!
填妥大會背包中的大會問卷,可於活動第三天兌換問卷禮哦!
感謝您的合作。