introduction to treasuredata cloud

2014-03-08Hadoop Ecosystem Workshop Q1 2014

jazzwang@etusolution.com 1

Jazz's Lightening Talk

巨量資料的新商業模式？雲端資料分析服務

雲端運算Cloud

Computing

巨量資料Big

Data

2014-03-08Hadoop Ecosystem Workshop Q1 2014

jazzwang@etusolution.com 2

爵士閃電講

巨量資料的新商業模式？他山之石可以攻錯！

https://www.moedict.tw/他山之石可以攻錯

2014-03-08Hadoop Ecosystem Workshop Q1 2014

jazzwang@etusolution.com 3

爵士閃電講

巨量資料的新商業模式？他山之石可以攻錯！

案例：從日誌蒐集到線上分析

2014-03-08Hadoop Ecosystem Workshop Q1 2014

jazzwang@etusolution.com 4

生、流、存、算、用、看生、流、存、算、用、看

圖 1. Big Data Lifecycle 與產業發展

http://fredbigdata.blogspot.tw/2013/06/big-data-lifecycle.html

2014-03-08Hadoop Ecosystem Workshop Q1 2014

jazzwang@etusolution.com 5

Data Science Workflow

資料擷取蒐集

資料整理凈化

資料儲存管理

特徵萃取

視覺化分析

互動式查詢

資料模型/模式

說故事結果闡釋

Data Analysis: Just One Component of the Data Science Workflow, By Ben Lorica, 'Big Data Now 2013', OR'eilly

2014-03-08Hadoop Ecosystem Workshop Q1 2014

jazzwang@etusolution.com 6

Life of Big DataLife of Big Data ：蒐、存、取、析、用：蒐、存、取、析、用

蒐

存 取

析 用

2014-03-08Hadoop Ecosystem Workshop Q1 2014

jazzwang@etusolution.com 7

Treasure Data Cloud Treasure Data Cloud 的商業定位的商業定位

蒐

析 用

算

看

流生

存 取

2014-03-08Hadoop Ecosystem Workshop Q1 2014

jazzwang@etusolution.com 8

Data Flow of Log Analysis

用算 看流生 存

2014-03-08Hadoop Ecosystem Workshop Q1 2014

jazzwang@etusolution.com 9

Data Flow of Log Analysis (Enhanced ?)

2014-03-08Hadoop Ecosystem Workshop Q1 2014

jazzwang@etusolution.com 10

td-agent / flunetd

Unstructure → Unstructure

Unstructure → Structured

2014-03-08Hadoop Ecosystem Workshop Q1 2014

jazzwang@etusolution.com 11

爵士閃電講

巨量資料的新商業模式？他山之石可以攻錯！從日誌蒐集到線上分析

Live Demo:Treasure Data Cloud

2014-03-08Hadoop Ecosystem Workshop Q1 2014

jazzwang@etusolution.com 12

爵士閃電講

<聲明 >@ 2014-03-08

本人與 TreasureData公司無任何利益關係

(不代表未來不會合作)以下純屬個人的用戶經驗分享

PS.順便學一下 UI / UX

2014-03-08Hadoop Ecosystem Workshop Q1 2014

jazzwang@etusolution.com 13

1. Visit Web Console

https://console.treasuredata.com/login

2014-03-08Hadoop Ecosystem Workshop Q1 2014

jazzwang@etusolution.com 14

2. Sign up for free testing account

https://console.treasuredata.com/login

2014-03-08Hadoop Ecosystem Workshop Q1 2014

jazzwang@etusolution.com 15

3. Read Quick Start Tutorial

https://console.treasuredata.com/login

2014-03-08Hadoop Ecosystem Workshop Q1 2014

jazzwang@etusolution.com 16

4. Understand SampleDB Data Structure

https://console.treasuredata.com/login

2014-03-08Hadoop Ecosystem Workshop Q1 2014

jazzwang@etusolution.com 17

5. Submit Query …. online!

https://console.treasuredata.com/login

2014-03-08Hadoop Ecosystem Workshop Q1 2014

jazzwang@etusolution.com 18

Running via AJAX / RESTful method

https://console.treasuredata.com/login

2014-03-08Hadoop Ecosystem Workshop Q1 2014

jazzwang@etusolution.com 19

6. Get the result interactively

https://console.treasuredata.com/login

2014-03-08Hadoop Ecosystem Workshop Q1 2014

jazzwang@etusolution.com 20

Possible Use Cases

https://console.treasuredata.com/login

2014-03-08Hadoop Ecosystem Workshop Q1 2014

jazzwang@etusolution.com 21

Support High Availability

2014-03-08Hadoop Ecosystem Workshop Q1 2014

jazzwang@etusolution.com 22

fluentd plugins ( in_XXX, out_XXX )

http://fluentd.org/plugin/