waldo-gcp

Travel Service at GCPwaldo-gcp

Cage, F2E, Mitac

Road trip stopping at popular U.S. cities - Computing the optimal road trip across the U.S. | Randal S. Olson(http://goo.gl/pDNfp0)

http://goo.gl/pDNfp0

My own road trip (https://goo.gl/SzBwbM)

https://goo.gl/SzBwbM

Waldo-gcp overviewWhat is waldo

Demo

https://waldo-gcp.appspot.com/



Waypoints address sample text fileWhat is waldo

"台北市內湖區瑞光路227號1樓",

"高雄市鼓山區美術東二路106號",

"台南市長榮路一段175號",

"臺北市松山區南京東路五段123巷1弄15號",

"高雄市五福四路131號2樓"

Waldo-gcp Google Cloud Platform FeaturesWhat is waldo-gcp

Compute Storage

App Engine CloudDatastore

Cloud Storage

ComputeEngine

Waldo-gcp architecture

queue

MetadataWaldo Server

App Engine

Task Queues

Cloud Storage

Optimal-waypoints

managed VMs(python27)

...

endpoints API(OAuth 2)

Object Notific

ation

CloudeDatastore

snapshot

managed VMs(nodejs)

Android App

CSV

upload waypoints

Store/retrieve metadata

waypoint map

snapshotRESTful APIs for waypoints list, upload

waypoints

optimal road trip

Waldo-gcp dispatch.yaml

dispatch: - url: "*/favicon.ico" module: default

- url: "waldo-gcp.appspot.com/" module: default

- url: "*/notice/*" module: notice

- url: "*/waypoints/*" module: optimal-waypoints

- url: "*/screenshot/*" module: screenshot

- url: "*/google40fc41ccc276e5c5.html" module: ownership

- url: "*/*" module: "default"

Waldo-gcp architecture

Application hierarchy

[App Engine Modules in Python](https://goo.gl/p3nl48)

https://goo.gl/p3nl48

Compute

Cloud ComputingCompute

PaasIaaS

ApplicationDataRuntimeMiddlewareOSVirtualizationServersStorageNetworking


Saas


Infrastructure as a Service Platform as a Service Service as a Service

You manage Google manage

Managed VMsCompute | Managed VMs

● Managed VMs○ The Flexibility of Compute Engine○ The productivity of App Engine

● Provides best of both

○ IaaS + PaaS

Waldo-gcp storage - cloud datastore

queue


App Engine

Task Queues

Cloud Storage

Optimal-waypoints


...


Object Notific

ation

CloudeDatastore

snapshot

managed VMs(nodejs)

Android App

CSV

upload waypoints


waypoint map

snapshot

optimal road trip

RESTful APIs for waypoints list, upload

waypoints

Storage

Waldo-gcp StorageStorage

Cloud Datastore Cloud Storage Cloud SQL Compute Engine(Persistent Disk)

BlobNoSQL SQL Block

Waldo-gcp storage - cloud datastore

queue


App Engine

Task Queues

Cloud Storage

Optimal-waypoints


...


Object Notific

ation

CloudeDatastore

snapshot

managed VMs(nodejs)

Android App

CSV

upload waypoints


waypoint map

snapshot

optimal road trip


waypoints

Waldo-gcp storage - cloud storage

queue


App Engine

Task Queues

Cloud Storage

Optimal-waypoints


...


Object Notific

ation

CloudeDatastore

snapshot

managed VMs(nodejs)

Android App

CSV

upload waypoints


waypoint map

snapshot

optimal road trip


waypoints

Setup Cloud Storage Buckets

Waldo-gcp cloud storage

queue


App Engine

Task Queues

Cloud Storage

Optimal-waypoints


...


Object Notific

ation

CloudeDatastore

snapshot

managed VMs(nodejs)

Android App

CSV

upload waypoints


waypoint map

snapshot

optimal road trip


waypoints

Waypoints GCS BucketWaldo-gcp cloud storage

Waldo Server

Android App

endpoints API (OAuth 2)

Waldo-gcp-file

Waldo-gcp-file-in

Waldo-gcp-file-image

endpoint API upload waypoints address to

managed VMs upload optimal address to

managed VMs upload screenshot image to

GCS Bucket NotificationWaldo-gcp setup gcs object notification

Waldo Server

Android App


Waldo-gcp-file

Waldo-gcp-file-in


GCS Bucket NotificationWaldo-gcp storage

Waldo Server

Android App


Waldo-gcp-file

Waldo-gcp-file-in


● bucket push notification○ https only○ domain owned/notice

Google Cloud Storage Notification

class BucketNotificationHandler(webapp2.RequestHandler):

def post(self):

# Decode the name and bucket of the notification

def notification_decoder(object):

if 'kind' in object:

return dict(name=object['name'], bucket=object['bucket'], id=object['id'])

...

# Add a new task to the queue

taskqueue.add(url='/waypoints/', params={

'bucket': notification['bucket'],

'name': notification['name'],

'id': notification['id']

},queue_name='new-files')

...

Create the App Engine backend

Waypoints address text fileCreate the App Engine backend | Waldo-gcp-file

"台北市內湖區瑞光路227號1樓",

"高雄市鼓山區美術東二路106號",

"台南市長榮路一段175號",

"臺北市松山區南京東路五段123巷1弄15號",

"高雄市五福四路131號2樓"

Waypoints address distance_m & distance_sCreate the App Engine backend | Waldo-gcp-file-in

waypoint1 waypoint2 distance_m duration_s

台南市長榮路一段175號高雄市五福四路131號2樓 89300 4375

台南市長榮路一段175號高雄市鼓山區美術東二路 106號 84212 4243

台北市內湖區瑞光路227號1樓臺北市松山區南京東路五段 123巷1弄15號 5204 763

高雄市五福四路131號2樓高雄市鼓山區美術東二路 106號 4163 732

台北市內湖區瑞光路227號1樓高雄市五福四路131號2樓 356184 13972

臺北市松山區南京東路五段 123巷1弄15號高雄市鼓山區美術東二路 106號 354392 14071

臺北市松山區南京東路五段 123巷1弄15號高雄市五福四路131號2樓 359450 14230

台北市內湖區瑞光路227號1樓台南市長榮路一段175號 275002 10503

台北市內湖區瑞光路227號1樓高雄市鼓山區美術東二路 106號 352764 13541

臺北市松山區南京東路五段 123巷1弄15號台南市長榮路一段175號 276930 10674

Create Managed VMs Server● Optimal road trip ● screenshot

Waldo-gcp waypoint optimization

● Gather the distance between wall waypoints

● Google Map APIs

● genetic algorithm

optimal road trip

[nbviewer.ipython.org/github/rhiever/Data-Analysis-and-Machine-Learning-Projects/blob/master/optimal-road-trip/Computing the optimal road trip across the U.S..ipynb](http://goo.gl/SYoh53)

http://goo.gl/SYoh53

Pandas

● Python library● Powerful data structures for data

analysis, time series,and statistics

optimal road trip

http://pandas.pydata.org/



Using Managed VMsWaldo-gcp architecture

Waldo Server

Android App


/notice




- url: "*/*" module: default

bucket notification entry point

optimal road trip

screenshot

waldo-gcp

Managed VMsWaldo-gcp architecture

Waldo Server

Android App


/notice




- url: "*/*" module: default

Auto-scaled managed VMs(python27)

...

managed VMs(nodejs)

Containers on Managed VMs

FROM google/appengine-python27

# Install pandas

RUN apt-get update && apt-get install -y -q --no-install-

recommends python-pandas python-numpy

# Install Google Map APIs, Google Client Library

ADD requirements.txt /app/

RUN pip install --upgrade pip

RUN pip install -r requirements.txt

ADD . /app

Create the App Engine | Dockerfile

[Managed VMs - App Engine — Google Cloud Platform](https://goo.gl/4FFSmm)

https://goo.gl/4FFSmm

Optimal waypoint Module Configuration

application: waldo-gcp

module: optimal-waypoints

version: 1

runtime: python27

api_version: 1

threadsafe: yes

vm: true

automatic_scaling:

min_num_instances: 1

max_num_instances: 5

cool_down_period_sec: 60

cpu_utilization:

target_utilization: 0.5

Create the App Engine | app.yaml

Waldo-gcp screenshot

● Screenshot as a service

● Phantomjs + nodejs

snapshot server

[fzaninotto/screenshot-as-a-service](https://github.com/fzaninotto/screenshot-as-a-service)

https://github.com/fzaninotto/screenshot-as-a-service



Screenshot Module Configuration

application: waldo-gcp

module: screenshot

version: 1

runtime: custome

api_version: 1

threadsafe: yes

vm: true

resources:

cpu: .5

memory_gb: 1.3

manual_scaling:

instances: 1

snapshot server

Waldo-gcp

Road trip on Google Mapwaldo-gcp google map

optimal_route = [/*optimal waypoint calculate by genetic algorithm*/]

optimal_route = list(optimal_route)

optimal_route += [optimal_route[0]]

subset = 0

while subset < len(optimal_route):

waypoint_subset = optimal_route[subset:subset + 10]

output = "calcRoute(\"%s\", \"%s\", [" % (waypoint_subset[0], waypoint_subset[-1])

for waypoint in waypoint_subset[1:-1]:

output += "\"%s\", " % (waypoint)

if len(waypoint_subset[1:-1]) > 0:

output = output[:-2]

output += "]);"

print(output)

print("")

subset += 9

...

● Google Map APIs○ split max waypoints (11)

in one route

Road trip on Google Mapwaldo-gcp google map

calcRoute("605嘉義縣阿里山鄉二萬坪106號", "605嘉義縣阿里山鄉二萬坪106號", ["500彰化縣彰化市溫泉

路31號", "224九份風景區瑞芳區新北市 ", "971花蓮縣新城鄉海岸路", "555南投縣魚池鄉中山路599號"]);

function calcRoute(start, end, routes) {

switch (start) {

case "605嘉義縣阿里山鄉二萬坪106號" ,"605嘉義縣阿里山鄉二萬坪106號":

directionsDisplay1 = new google.maps.DirectionsRenderer(directionsDisplayOptions);

..

}

var request = {origin: start, destination: end, waypoints: waypts, optimizeWaypoints:

false, travelMode: google.maps.TravelMode.DRIVING

};

directionsService.route(request, function (response, status) {

case "605嘉義縣阿里山鄉二萬坪106號" ,"605嘉義縣阿里山鄉二萬坪106號":

directionsDisplay1.setDirections(response);

}

});

..

}

}

1. 605嘉義縣阿里山鄉二萬坪106號2. 500彰化縣彰化市溫泉路31號3. 224九份風景區瑞芳區新北市4. 971花蓮縣新城鄉海岸路5. 555南投縣魚池鄉中山路599號

Thanks you!

● [Here's Waldo: Computing the optimal search strategy for finding Waldo | Randal S. Olson](http://goo.gl/DSOZbv)

● [Computing the optimal road trip across the U.S. | Randal S. Olson](http://goo.gl/pDNfp0)

● [nbviewer.ipython.org/github/rhiever/Data-Analysis-and-Machine-Learning-Projects/blob/master/optimal-

road-trip/Computing the optimal road trip across the U.S..ipynb](http://goo.gl/SYoh53)

● [GoogleCloudPlatform/abelana-gcp](https://goo.gl/6L55oa)

Reference

http://goo.gl/DSOZbv



http://goo.gl/pDNfp0

http://goo.gl/SYoh53

https://goo.gl/6L55oa

《威利在哪裡？》（英文：Where's Wally?），也譯《威力在哪裡？》、《大家一起找沃裡？》、《聰明的沃

裡？》，是一套由英國插畫家Martin Handford創作的兒童書籍。這個書的目標就是在一張人山人

海的圖片中找出一個特定的人物——威利。威利穿著紅白條紋的襯衫並戴著一個絨球帽，手上拿

著木製的手杖，還戴著一副眼鏡。他總是會弄丟東西，如書本、野營設備甚至是他的鞋子，而讀者

也要幫他在圖中找出這些東西來。

wikipedia

http://goo.gl/D4mH6

Where's Waldo?

http://goo.gl/D4mH6

http://goo.gl/D4mH6

waldo-gcp

Internet