iot, 디바이스부터머신러닝까지 놓치지않을꺼에욧insights azure sdk vs online...

28
IoT, 디바이스부터 머신러닝까지 놓치지 않을꺼에욧

Upload: others

Post on 22-Sep-2020

3 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: IoT, 디바이스부터머신러닝까지 놓치지않을꺼에욧Insights Azure SDK VS Online Domain Services HDInsight Machine Learning Stream Analytics Data Factory Event Hubs Mobile

IoT, 디바이스부터머신러닝까지

놓치지않을꺼에욧

Page 2: IoT, 디바이스부터머신러닝까지 놓치지않을꺼에욧Insights Azure SDK VS Online Domain Services HDInsight Machine Learning Stream Analytics Data Factory Event Hubs Mobile

Internal only

디바이스 및 센서 데이터 연결 및 처리 데이터 저장 및 성능 프레젠테이션 및 활용분석

Microsoft Data platform End-to-end 서비스

Page 3: IoT, 디바이스부터머신러닝까지 놓치지않을꺼에욧Insights Azure SDK VS Online Domain Services HDInsight Machine Learning Stream Analytics Data Factory Event Hubs Mobile

Internal only

Microsoft IoT platform End-to-end 서비스

각종 소형 디바이스 및 센서

IoT 허브(대량 Telemetry수집)

SQL Database(PaaS 형태의 SQL Server 서비스)

머신러닝(고급 분석 및기계학습)

Power BI(데이터 분석& 시각화 플랫폼)

대형 기계 장치 및 그 센서

Event Hub(대량 이벤트 수집)

Table/Blob 저장소(클라우드 저장소)

스트림 분석(실시간 분석)

서비스 버스(메시지 연결)

DocumentDB(NoSQL 문서DB 서비스)

HDInsight(클라우드 상의Hadoop 시스템)

데이터카탈로그(데이터 원본 검색)

데이터 레이크(빅데이터를 위한저장소 및 분석)

RevolutonAnalytics(고성능 R 통계분석)

Data Factory(클라우드 상의 ETL)

SQL DW(DW 특화 서비스)

Redis Cache(클라우드 앱을위한 Redis cache)

디바이스 및 센서

{ }

데이터 연결 및 처리 데이터 저장 및 성능 프레젠테이션 및 활용분석

Windows 10 IoT core

Page 4: IoT, 디바이스부터머신러닝까지 놓치지않을꺼에욧Insights Azure SDK VS Online Domain Services HDInsight Machine Learning Stream Analytics Data Factory Event Hubs Mobile

Devices

Windows 10 IoT Core

Page 5: IoT, 디바이스부터머신러닝까지 놓치지않을꺼에욧Insights Azure SDK VS Online Domain Services HDInsight Machine Learning Stream Analytics Data Factory Event Hubs Mobile

http://windows.Microsoft.com

Phone Small Tablet2-in-1s

(Tablet or Laptop)Desktops

& All-in-OnesPhablet Large TabletClassic Laptop

Xbox IoTSurface Hub Holographic

Windows 10

Page 6: IoT, 디바이스부터머신러닝까지 놓치지않을꺼에욧Insights Azure SDK VS Online Domain Services HDInsight Machine Learning Stream Analytics Data Factory Event Hubs Mobile

http://windows.Microsoft.com

One Store +One Dev Center

Reuse Existing Code

One SDK + Tooling

Adaptive User Interface

NaturalUser Inputs

One Universal Windows Platform

Page 7: IoT, 디바이스부터머신러닝까지 놓치지않을꺼에욧Insights Azure SDK VS Online Domain Services HDInsight Machine Learning Stream Analytics Data Factory Event Hubs Mobile

https://msdn.microsoft.com/en-

us/library/windows/hardware/dn914597(v=vs.85).aspx

Page 8: IoT, 디바이스부터머신러닝까지 놓치지않을꺼에욧Insights Azure SDK VS Online Domain Services HDInsight Machine Learning Stream Analytics Data Factory Event Hubs Mobile
Page 9: IoT, 디바이스부터머신러닝까지 놓치지않을꺼에욧Insights Azure SDK VS Online Domain Services HDInsight Machine Learning Stream Analytics Data Factory Event Hubs Mobile

DEMO

Client Programming(http://github.com

/KoreaEva/IoT)

Page 10: IoT, 디바이스부터머신러닝까지 놓치지않을꺼에욧Insights Azure SDK VS Online Domain Services HDInsight Machine Learning Stream Analytics Data Factory Event Hubs Mobile

Platform Services

Infrastructure Services

Web Apps

MobileApps

APIManagement

API Apps

Logic Apps

Notification Hubs

Content DeliveryNetwork (CDN)

Media Services

BizTalkServices

HybridConnections

Service Bus

StorageQueues

HybridOperations

Backup

StorSimple

Azure SiteRecovery

Import/Export

SQL Database

DocumentDB

RedisCache

AzureSearch

StorageTables

DataWarehouse

Azure AD Health Monitoring

AD PrivilegedIdentity Management

OperationalAnalytics

Cloud Services

BatchRemoteApp

ServiceFabric

Visual Studio

AppInsights

Azure SDK

VS Online

Domain Services

HDInsight MachineLearning

StreamAnalytics

Data Factory

EventHubs

MobileEngagement

Data Lake

IoT Hub

Data Catalog

Security & Management

Azure ActiveDirectory

Multi-FactorAuthentication

Automation

Portal

Key Vault

Store/Marketplace

VM Image Gallery& VM Depot

Azure ADB2C

Scheduler

Page 11: IoT, 디바이스부터머신러닝까지 놓치지않을꺼에욧Insights Azure SDK VS Online Domain Services HDInsight Machine Learning Stream Analytics Data Factory Event Hubs Mobile

Services

IoT HubStream Analytics

Page 12: IoT, 디바이스부터머신러닝까지 놓치지않을꺼에욧Insights Azure SDK VS Online Domain Services HDInsight Machine Learning Stream Analytics Data Factory Event Hubs Mobile

Internal only

Batch processing :

Azure Data Factory, Azure HDInsight

Hot Path Analytics : Azure Stream Analytics, Azure HDInsight Storm

데이터 저장소 및 처리

Real-time Analysis

Data-At-Rest Analysis

Machine Learning (R)

고급 분석

프레젠테이션 또는

비즈니스에 연결

앱 서비스 배포, 웹 게시

CRM, ERP 등 사내 시스템과의 연동 및 알림 제공

IoT Hub

일반적인 IoT 아키텍처

CoAP, AllJoyn, OPC

MQTT, 웹 소켓 또는

사용자 프로토콜

AMQPHTTPS, MQTT, OPC

CoAP, AllJoyn, OPC

Device

Device

Device

Device

Device

Device

AMQPHTTPS, MQTT,

OPC

VPN

LWM2M, HTTP, CoAP

저상소 및 분석

데이터 처리 및 저장 프레젠테이션 및 활용 영역디바이스 및 센서 디바이스와 저장소 간의 연결

통신 프로토콜

데이터의 흐름

Page 13: IoT, 디바이스부터머신러닝까지 놓치지않을꺼에욧Insights Azure SDK VS Online Domain Services HDInsight Machine Learning Stream Analytics Data Factory Event Hubs Mobile

Sensors on

Windows 10 IoT Core

SQL Database Machine Learning

Power BIStream AnalyticsIoT Hubs

Blob / Table

Serverless Architecture

Devices Management

Cortana

Cognitive Services

Page 14: IoT, 디바이스부터머신러닝까지 놓치지않을꺼에욧Insights Azure SDK VS Online Domain Services HDInsight Machine Learning Stream Analytics Data Factory Event Hubs Mobile

Internal only

Device Explorer

https://github.com/Azure/azure-iot-sdks

/tools/DeviceExplorer

Page 15: IoT, 디바이스부터머신러닝까지 놓치지않을꺼에욧Insights Azure SDK VS Online Domain Services HDInsight Machine Learning Stream Analytics Data Factory Event Hubs Mobile

DEMO

IoT Hub

Page 16: IoT, 디바이스부터머신러닝까지 놓치지않을꺼에욧Insights Azure SDK VS Online Domain Services HDInsight Machine Learning Stream Analytics Data Factory Event Hubs Mobile

Sensors on

Windows 10 IoT Core

SQL Database Machine Learning

Power BIStream AnalyticsIoT Hubs

Blob / Table

DEMO

SELECT *

INTO OUTPUT

FROM INPUT

Real time analysis

INPUT OUTPUTQUERY

Page 17: IoT, 디바이스부터머신러닝까지 놓치지않을꺼에욧Insights Azure SDK VS Online Domain Services HDInsight Machine Learning Stream Analytics Data Factory Event Hubs Mobile

DEMO

Stream Analytics Job

Page 18: IoT, 디바이스부터머신러닝까지 놓치지않을꺼에욧Insights Azure SDK VS Online Domain Services HDInsight Machine Learning Stream Analytics Data Factory Event Hubs Mobile

Sensors on

Windows 10 IoT Core

SQL Database Machine Learning

Power BIStream AnalyticsIoT Hubs

Blob / Table

DEMO

SELECT DeviceID, AVG(Temperature)as Temperature,

AVG(Humidity) as Humidity,

AVG(Dust) as Dust

INTO OUTPUTS

FROM INPUT

GROUP BY

DeviceID,

TumblingWindow(Minute, 1)

Real time analysis

INPUT OUTPUTQUERY

OUTPUTS

Page 19: IoT, 디바이스부터머신러닝까지 놓치지않을꺼에욧Insights Azure SDK VS Online Domain Services HDInsight Machine Learning Stream Analytics Data Factory Event Hubs Mobile

DEMO

SQL Server

Page 20: IoT, 디바이스부터머신러닝까지 놓치지않을꺼에욧Insights Azure SDK VS Online Domain Services HDInsight Machine Learning Stream Analytics Data Factory Event Hubs Mobile

Internal only

• Power BI는 클라우드 상에서 SaaS 형태로 서비스 되는 데이터 시각화 및 분석플랫폼

개요

• Office 제품, 특히 Excel과의 리포트 연동 기능• 화려한 대시보드 구성을 지원하는 툴 지원(Power BI Desktop)• 거의 모든 형식의 데이터 원본 지원• 데이터 원본에의 라이브 연결 지원• 실시간 대시보드 지원• Oracle, IBM과 같은 기타 DB와도 스케쥴링된 데이터 새로고침 지원• 대시보드 및 리포트, 데이터 모델 등을 미리 구성하여 조직에 배포할 수

있는 콘텐츠 팩 기능 지원• 조직의 콘텐츠 팩 뿐만 아니라 많이 사용하는 SaaS 솔루션들이 제공하는

콘텐츠 팩 이용 가능• 데이터셋을 통해 리포트 또는 대시보드를 만들 수 있으며 기본 제공되는

차트 이외에 사용자가 D3.js를 통해 개발한 시각화 요소(그래프, 차트)를생성 가능

• Andorid, iOS 등에 모바일 Native App 지원• KPI 관련 알림 기능, 코멘트 기능 등을 포함

특징

• 현재 실시간 연결 대시보드 연결은 Power BI REST API 또는 Azure Stream Analytics를 통해 가능

• 라이브 연결 지원은 SQL Server Analysis Services를 통해 지원

한계

클라우드 데이터 시각화 및 분석 플랫폼 (Power BI)

• 기업의 KPI 모음 대시보드• IT 관리 대시보드• 부서별 매출 분석 대시보드• 마케팅 트위터/Facebook 분석 대시보드

사용예시

프레젠테이션및 활용

Page 21: IoT, 디바이스부터머신러닝까지 놓치지않을꺼에욧Insights Azure SDK VS Online Domain Services HDInsight Machine Learning Stream Analytics Data Factory Event Hubs Mobile

DEMO

Power BI

Page 22: IoT, 디바이스부터머신러닝까지 놓치지않을꺼에욧Insights Azure SDK VS Online Domain Services HDInsight Machine Learning Stream Analytics Data Factory Event Hubs Mobile

Common Classes of Algorithms(Supervised|Unsupervised)

Classification Regression Anomaly Detection

Clustering

Page 23: IoT, 디바이스부터머신러닝까지 놓치지않을꺼에욧Insights Azure SDK VS Online Domain Services HDInsight Machine Learning Stream Analytics Data Factory Event Hubs Mobile

50°F 30°F 68°F 95°F1990

48°F 29°F 70°F 98°F2000

49°F 27°F 67°F 96°F2010

? ? ? ?2020

… … … ……

Known dataModelUnknown data

Weather forecast sample

Using known data, develop a model to predict unknown data.

Page 24: IoT, 디바이스부터머신러닝까지 놓치지않을꺼에욧Insights Azure SDK VS Online Domain Services HDInsight Machine Learning Stream Analytics Data Factory Event Hubs Mobile

Model (Regression)

90°F

-26°F

50°F 30°F 68°F 95°F1990

48°F 29°F 70°F 98°F2000

49°F 27°F 67°F 96°F2010

Using known data, develop a model to predict unknown data.

Predict 2020 Summer

Page 25: IoT, 디바이스부터머신러닝까지 놓치지않을꺼에욧Insights Azure SDK VS Online Domain Services HDInsight Machine Learning Stream Analytics Data Factory Event Hubs Mobile

Azure Machine Learning Ecosystem

Get/Prepare Data

Build/Edit Experiment

Create/Update Model

Evaluate Model Results

Publish Web Service

Build ML Model Deploy as Web ServiceProvision Workspace

Get Azure Subscription

Create Workspace

Publish an App

Azure Data

Marketplace

Page 26: IoT, 디바이스부터머신러닝까지 놓치지않을꺼에욧Insights Azure SDK VS Online Domain Services HDInsight Machine Learning Stream Analytics Data Factory Event Hubs Mobile

Data I/O

Taking Data & preparing for Analysis

Dimensionality reduction. E.g. Kinect measures 1000 points, 6 are relevant

Fitting Model selection; calibration; assessment

R – free scripts/graphics, many packages based on Vector Data.

Metrics to allow us to describe the data. E.g. Mean, Correlation…

Tools used for Text Input. E.g. ‘What is the theme of this essay?’

Page 27: IoT, 디바이스부터머신러닝까지 놓치지않을꺼에욧Insights Azure SDK VS Online Domain Services HDInsight Machine Learning Stream Analytics Data Factory Event Hubs Mobile

DEMO

Azure ML

Page 28: IoT, 디바이스부터머신러닝까지 놓치지않을꺼에욧Insights Azure SDK VS Online Domain Services HDInsight Machine Learning Stream Analytics Data Factory Event Hubs Mobile