20171013 데이터 분석가를 위한...

51
FAST CAMPUS Data Platform Seminar www.fastcampus.co.kr [email protected], 02- 501-9396 Copyright ⓒ FAST CAMPUS Corp. All Rights Reserved. 무단 전재 및 재배포 금지 1 데이터 분석가를 위한 데이터 플랫폼 Seminar 13. Oct. 2017 한종원, [email protected] Public Cloud 101

Upload: jongwon-han

Post on 23-Jan-2018

258 views

Category:

Technology


6 download

TRANSCRIPT

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

1

데이터�분석가를�위한�데이터�플랫폼�Seminar

13.�Oct.�2017�한종원,�[email protected]

Public�Cloud�101

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

•한종원�

•Python과�Cloud�Infra,�Lean/Agile�방법론�그리고�애플의�제품을�사랑.�

•2012년�석사�학위를�마치고,�startup을�시작(이때부터�AWS를�production�level에서�사용)�

•‘의미가�있는�일을,�올바르게�하고�싶다.’�

•경력�

•(현)�DevOps�전문�스타트업�‘HB�Smith’�대표�

•택시�O2O�서비스�스타트업�‘Kanizsa�Lab’의�backend�server�/�infra�devops�담당�

•Cloud�computing�전문�스타트업�'A2�company'�co-founder�(‘KINX’에�인수합병)�

•NEXON�'MapleStory�국내�Live�Team'에서�DBA,�SA로�근무�(산업�기능�요원)

발표자�소개

“Startup�경력�=�AWS�사용�기간”

https://www.linkedin.com/in/addnull/

2

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

•발표�대상:�초/중급�

•발표�순서�

•Hello�Cloud!�

•Public�Cloud�Providers�

•AWS�Use�Cases�for�Data�Engineering�

•Wrap�Up

Contents

(예상�발표�시간:�45분)

=�>�왜�필요하게�되었는가?�그리고�어떻게�발전해왔는가?

=>�AWS,�Azure,�GCP�그외�provider들의�비교�분석

=>�물리�서버�사용�경험이�있고,�public�cloud�경험은�거의�없는�분

3

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

Abstract

http://www.fastcampus.co.kr/data_seminar_platform/

4

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

Hello�Cloud!

5

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

•‘Cloud�Computing’�이란�?!?�

•다양한�정의�

•Data�저장과�접근을�인터넷으로�언제�어디서든�접근�가능한�기술�

•바로�사용�가능하고�사용한�만큼�과금되는�전산�기반�시설(infrastructure)들�

•전산�hardware�장비들의�가상화(virtualization)�기술

Hello�Cloud!

6

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

•‘Cloud�Computing’�이란�?!?�

•정의

Hello�Cloud!

전산�자원들의�공유하는�기술과�도구의�집합

왜�'Cloud�Computing'이�상업적으로�의미가�있는가?

7

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

•정의�

•여기서�'전산�자원’이란server,�storage,�network�같은 물리�hardware�장비�뿐만�아니라 platform과�application이라는software까지�포함

Hello�Cloud!

전산�자원들의�공유하는�기술과�도구의�집합

자료출처�https://en.wikipedia.org/wiki/Cloud_computing

8

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

•전산�자원을�공유한다?!?�

•피자를�먹는�네�가지�방법

Hello�Cloud!

자료출처�http://www.teenagemutantninjaturtles.com/fanzone/tmnt-pizza-toppings/

9

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

•Abstraction�level�

•음식점�피자:�돈만�있으면�OK!�

•배달�피자:�식탁과�돈�필요�

•냉동�피자:�전자렌지,�식탁,�돈�필요�

•홈피자:�집에서�AtoZ�모두�다�필요

Hello�Cloud!

자료출처�http://www.hostingadvice.com/how-to/iaas-vs-paas-vs-saas/

10

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

•Abstraction�level�

•음식점�피자:�

•배달�피자:�

•냉동�피자:�

•홈피자:

Hello�Cloud!

자료출처�http://www.hostingadvice.com/how-to/iaas-vs-paas-vs-saas/

Software�통채로�대여

Platform�까지�대여

Infrastructure만�대여

Cloud�안�씁니다..

11

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

•Abstraction�level�

•음식점�피자:�

•배달�피자:�

•냉동�피자:�

•홈피자:

Hello�Cloud!

자료출처�http://www.hostingadvice.com/how-to/iaas-vs-paas-vs-saas/

Software�통채로�대여

Platform�까지�대여

Infrastructure만�대여

Cloud�안�씁니다..

12

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

•‘Cloud�Computing’�은�

•‘Cloud�서비스’는��

•Cloud�서비스를�쓴다는�것 =�서비스�제공�업체가�미리�준비해둔�자원의�일부를�빌린다는�것

Hello�Cloud!

전산�자원들의�공유하는�기술과�도구의�집합

'Cloud�Computing'을�별도의�유무료�서비스�형태로�제3자에게�제공

13

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

•다만,�platform과�software를�빌리는�경우에는�digital�무형�자원이므로복사,�전달이�인터넷을�통해�쉽게�해결�

•그러나�infrastructure는�물리�장비라서�인터넷을�통한�대여가�어려움..�

•가상화(virtualization)�기술�필요성�대두

Hello�Cloud!

14

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

•가상화의�나쁜�예

Hello�Cloud!

'Matrix�(1999)'에서�실제�world를�묘사한�virtual�world(software)가�존재

15

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

•Cloud�서비스에서�가상화

Hello�Cloud!

실제�물리�machine을�묘사한�virtual�machine(VM)을�software로�구현

16

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

•Cloud�서비스에서�가상화�

•전�가상화�(vs)�반�가상화

Hello�Cloud!

자료출처�http://www.cloudvirtualizationspecialist.com/difference-application-virtualization-paravirtualization/

실제�물리�machine을�묘사한�virtual�machine(VM)을�software로�구현

17

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

•public�cloud�(vs)�private�cloud�

•Cloud�서비스의�물리�장비가�

•Cloud�서비스�업체에서�제공할�때�

•Cloud�서비스�사용자가�소유한�물리�장비일�때�

•그럼�왜,�누가�Cloud�서비스�시작하였는가?

Hello�Cloud!

->�private�cloud�(한정된�인원만�씀)

->�public�cloud�(즉,�불특정�다수가�함께�씀)

18

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

•public�cloud�역사�(2004년�Amazon�의�고민)�

•1년�중에�traffic�peak�일�때(예:�Christmas�season)는�수�많은�서버가�필요.�

•하지만�이런�기간은�1년�중에�매우�짧음�(2~3주)�

•대부분의�기간은�traffic�peak�대비�적은�수의�서버로도�충분함�

•이는�대부분의�시간�동안�많은�서버들의�사용률이�매우�낮은�문제�발생

Hello�Cloud!

19

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

Hello�Cloud!

자료출처�https://www.sugerendo.com/en/empresa/nuestras-tecnologias/amazon-web-services/

20

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

•public�cloud�역사�(2004년�Amazon�의�고민)�

•traffic�peak�일�때와�그렇지�않을�경우의�격차가�매년�더�커짐�

•언제�traffic�peak가�생길지�예측이�어려워짐�

•“서버를�필요할�때만�끌어다�쓰고�필요없을�때�반환할�수�없을까? 즉,�서버를�여러�기업에서�공유할�수�있을까?”�

•https://aws.amazon.com/about-aws/whats-new/2004/�:�SQS�출시�

•https://aws.amazon.com/about-aws/whats-new/2006/�:�EC2�출시

Hello�Cloud!

21

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

•물리�서버�(vs)�Cloud�서버

Hello�Cloud!

물리서버

AWS

AWS외

Cloud서버

22

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

•물리�서버(IDC)는�non-scalable�

•scale-out:�신규�서버�구매�

•scale-up:�부품�업그레이드->�소요�시간이�1주�이상�걸림�

•기업에게�“시간�=�돈”

Hello�Cloud!

23

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

•“Cloud�서버�쓰세요.�두�번�쓰세요.”�

•Cloud�서버를�쓴다는�것은 “보이지�않는�개발자가�24시간�365일알아서�서버를�관리해준다”는�것 그것도�아주�아주�아주�싼�연봉으로.

Hello�Cloud!

24

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

•“Cloud�서버�쓰세요.�두�번�쓰세요.”�

•Cloud�서버를�쓴다는�것은 “보이지�않는�개발자가�24시간�365일알아서�서버를�관리해준다”는�것 그것도�아주�아주�아주�싼�연봉으로.

Hello�Cloud!

Troubleshooting

API

HA

DR

Automation

Scale�Out

Scale�Up

BackupMigration

25

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

•Cloud�서버�사용의�이점�

•기업�경영자�입장:�비용절감(특히�인건비�절감)�

•‘홈피자’의�경우처럼�AtoZ를�모두�다�하려한다면,�

•‘냉동�피자’,�‘배달�피자’,�‘음식점�피자’�경우처럼�일부�위임을�해버리면,

Hello�Cloud!

적어도�3~10명�단위의�team이�3개�이상은�필요

전세계�대상으로�하는�서비스(약�20~50개국)를�운영에도핵심�인력�2~5명으로�충분

26

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

•Cloud�서버�사용의�이점�

•개발자�입장:�‘표준화'�

•‘홈피자’의�경우처럼�직접�물리�서버를�구비해서�관리한다면,�

•‘표준화’라는�단일한(homogeneous)�환경

Hello�Cloud!

통일되지�않은�다양한(heterogenous)�환경을�고려해야한다는�부담감

3R(Repeatable,�Reproducible,�Reliable)을�보장�

선배�개발자의�knowledge가�그대로�다음�세대에�전달되어�사용,�발전이�가능�

심지어�다른�국가,�다른�회사의�경험이�그대로�전파

27

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

Public�Cloud�Providers

28

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

•Public�cloud

Hello�Cloud!

AWS

AWS외

자료출처https://www.theregister.co.uk/2017/06/19/gartner_confirms_what_we_all_know_aws_and_microsoft_are_the_cloud_leaders_by_a_fair_way/

29

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

Public�Cloud�Providers

since�2006��������������������since�2008��������������������since�2010

30

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

•지원되는�기능과�심지어�가격�정책�면에서�대동소이�

•매우�치열하게�경쟁적으로�기능�upgrade와�가격�정책�benchmarking을�진행(release�announce가�1주�단위)�

•2017년�8월에�비교�자료일부�기능�상의�차이도�계속�없어지고�있는�상황

Public�Cloud�Providers

https://stackify.com/microsoft-azure-vs-amazon-web-services-vs-google-compute-comparison

31

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

•AWS�특장점�

•가장�오랫동안�운영된�서비스�(Since�2004)�

•가장�많은�use�case과�troubleshooting이�축적=>�사용자�입장에서(특히�개발자의�경우) 공식�문서와�인터넷�검색만으로도 큰�어려움�없이�사용�가능

Public�Cloud�Providers

32

거의�de�facto�급

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

•GCP�특장점�

•기존의�Google�서비스들과�Android�라는�막강한�echosystem들과�쉬운�연동

Public�Cloud�Providers

33

Android�App�개발자에게�선호됨

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

•Azure�특장점�

•'Azure�Stack'이라는�private�cloud를�지원

Public�Cloud�Providers

34

public�cloud에�막연한�불안감(data�외부�유출�등)이�있는�사용자를�위한�선택

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

•하지만,�이런�각�서비스�만의�고유의�장점과�차별점이�사라지는건�시간�문제�

•그�전까지�조금이라도�시장�점유율을�높이기�위해서�Cloud�서비스�업체�간의경쟁이�매우�치열하게�진행

Public�Cloud�Providers

35

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

AWS�Use�Cases�for�Data�Engineering

36

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

•‘자,�어디서부터�시작할까?’

AWS�Use�Cases�for�Data�Engineering

37

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

•Abstraction�Level�

•이�발표에서는�AWS�의�PaaS�/�SaaS�components�를�중점적으로�다룹니다.

AWS�Use�Cases�for�Data�Engineering

자료출처�http://www.hostingadvice.com/how-to/iaas-vs-paas-vs-saas/

38

IaaS는�사용자가�직접�관련�software를�처음부터�설치하는�것이므로�생략

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

•Big�data�4�단계�

•AWS에서�big�data�관련�주요�PaaS�/�SaaS�components

AWS�Use�Cases�for�Data�Engineering

39

collection�->�storing�->�analysis�->�visualization

collection storing analysis�&�visualization

RDS DynamoDB

S3

Kinesis QuickSight

ES AthenaEMR

Redshift

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

•fully�managed�Hadoop�

•HDFS�대신에�AWS�S3를�쓸�수도�있음�

•실시간�streaming�data�수집�(간단한�analysis�도�직접�처리�가능)�

•순간적으로�발생된�대규모�입력�요청도�문제없이�대응�

•원본�raw�data�저장도�가능하지만,간단한�aggregation,�emit�같은�data�가공도�지원

AWS�Use�Cases�for�Data�Engineering

40

EMR

복잡하고�오래걸리는�Hadoop�설치�및�성능�turning이�필요없음

Kinesis

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

•‘Object�Storage’�

•무제한�공간�제공�

•저장된�파일�개수�증감과�상관없이�일정한�file�access�time�보장(일반�block�storage와�차별되는�장점)�

•단,�한�file�내에�random�access는�불가능

AWS�Use�Cases�for�Data�Engineering

41

S3

기본�3�replica�backup

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

•fully�managed�MySQL�/�pgSQL�/�Oracle�/�MS�SQL�

•fully�managed�NoSQL�

•auto�scaling�지원�

•in-memory�cache�지원

AWS�Use�Cases�for�Data�Engineering

42

RDS

<=�RDB

DynamoDB DynamoDB�Accelerator�(DAX)

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

•Data�warehouse�

•기존�SQL�질의�처리�지원�

•S3,�RDS,�DynamoDB�에�저장된�data를�불러와서�병렬�처리�분석�

•즉,�structured,�semi-structured,�unstructured�data�모두�지원�

•in-memory�연산�기반의�visualization�생성�서비스�

•AutoGraph라는�data�type에�가장�적합한�형태의�visualization을자동�선택�해줌

AWS�Use�Cases�for�Data�Engineering

43

Redshift

QuickSight

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

•fully�managed�Elasticsearch�

•AWS�Kinesis�로부터�들어오는�data를�처리할�수�있음�

•AWS�S3�에�저장된�data를�대상으로�SQL�질의�처리

AWS�Use�Cases�for�Data�Engineering

44

ES

Athena

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

•AWS�사용법

AWS�Use�Cases�for�Data�Engineering

45

LEGO�블럭처럼�여러�개의�AWS�components를�조합해서�나만의�Infra�구축

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

•Data�Warehousing

AWS�Use�Cases�for�Data�Engineering

자료출처�https://aws.amazon.com/big-data/use-cases/

46

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

AWS�Use�Cases�for�Data�Engineering

자료출처�https://www.slideshare.net/awskorea/aws-big-data-and-analytics

47

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

Wrap�Up

48

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

•Abstract�

•Contents�

•Hello�Cloud!�

•Public�Cloud�Providers�

•AWS�Use�Cases�for�Data�Engineering

Wrap�Up

49

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

•저희는�API�개발팀을�위한�모니터링�서비스를�만들고�있습니다. �저희의�early�bird�고객이�되어주세요.�

•AWS�KRUG에서�다른�전문가와�함께하세요.�

•저희�기술�블로그에서�AWS팁을�가져가세요!

Call�To�Action

https://hbsmith.io

https://blog.hbsmith.io

https://www.fb.com/groups/awskrug/�http://www.awskr.org/slack/

50

FAST�CAMPUS�Data�Platform�Seminar

www.fastcampus.co.kr�[email protected],�02-�501-9396�

Copyright�ⓒ�FAST�CAMPUS�Corp.��All�Rights�Reserved.�무단�전재�및�재배포�금지

51

감사합니다.13.�Oct.�2017�

한종원,�[email protected]