SOFTWARE JAPAN 2016

CET(Capture EveryThing)

ICT

2

2

,

2006: 2006-2008:

2008-2015:

2013: 2015: /

3

=

5

CSWEB

AP

RD

IT

201210110

6

6

8

CET

CET

9

CET

10

CET

11

CET Capture EveryThing

ELB

Elasticsearch,Kibana

BigQuery

S3

APIELB Cloud BigtableGCP

Cloud Pub/Sub

Cloud Dataproc,Apache Spark

Air

12

Spark Streaming

UU

1

13

Spark Streaming URL

2

CET Capture EveryThing

ELB Cloud Pub/Sub

Cloud DataprocApache Spark

14

Amazon SNS

AWS Lambda

Slack

3

15

16

CET/

17

SQL / DB

CETGCP

18

AWSGCP1/10CET http://engineer.recruit-lifestyle.co.jp/techblog/2015-11-27-yoshida-interview-2/

CET(AET)

19

AWS S3

Elastic Search

TreasureData

RedShift

Spark Notebook

Docker

AWS S3

Cloud Bigtable

RUNDECK

Adhoc(EC2)

Spark(EMR)

Compute Engine

/

COLD: RedShift, TreasureData/

HOT: CETElastic Search /

1,000 (AWS)

r3.2xlarge

BigTableAPI

Rundeck Ansible

20

Rundeck

21

DBIDAUC

22

23

24

25

26

27

/

28

29

Play frameworkDeep)

30

31

32

SparkREC2 | Tech Blog | RECRUIT LIFESTYLEhttp://engineer.recruit-lifestyle.co.jp/techblog/2015-08-19-sparkr/

4gihyo.jp http://gihyo.jp/dev/serial/01/recruit-lifestyle/0004

1/3

33

@JAWS-UG Meguro #2 http://www.slideshare.net/RecruitLifestyle/ss-53400381

CET | Tech Blog | RECRUIT LIFESTYLE http://engineer.recruit-lifestyle.co.jp/techblog/2015-11-02-yoshida-interview-1/

2/3

34

AWSGCP1/10CET http://engineer.recruit-lifestyle.co.jp/techblog/2015-11-27-yoshida-interview-2/

3/3

35