2006-05-18 10:00 – 14:00 @102a1 esaki laboratory pre-workshop discussion

2006-05-18 10:00 – 14:00

@102A1

Esaki Laboratory

Pre-WorkshopDiscussion

Chapter

• Introduction (22 pages)– Overview– Evaluation– Characteristics

• Accident (3 pages)– Accidents already occurred– Accidents we must assume

• Discussion (18 pages)

INTRODUCTION

• Historical System (weather.hongo)– Overview– Evaluation– Characteristics

• New Approach (live-e.hongo)– Concept (Sensor Abstraction)– System Overview– Characteristics

Live E! Project Overview

• A Sensor Network Infrastructure• E stands for three meanings

– Earth– Environment– Electronics

• Application– Public Service (such as for Natural Disaster)– Business– Education

INTRODUCTION

• Historical System (weather.hongo)– Overview– Evaluation– Characteristics

• New Approach (live-e.hongo)– Concept (Sensor Abstraction)– System Overview– Characteristics

Historical System

weather.hongo.wide.ad.jpWindows 2003 ServerMSSQL

APP APP APP

Upload with SOAP

Vaisala Sensors

Web Service(SOAP/XML)

WD

WDDBLog

Web Interface

Vaisala Sensor(WXT510)

• Measure Item– Temperature– Humidity– Pressure (Barometer)– RainFall– WindDir– WindSpeed– (CO2)

Serial Interface

iLon100

WXT510

RS232C

Lonworks

Internetweather.hongo

SOAP

WXT510s are connected to weather.hongo through Lonworks.

Behavior of Data Upload

neuron_id WNIID Time Temperature Humidity Pressure RainFall WindDir WindSpeed

TABLE WD

TABLE WDDBLog

CO2

neuron_id Time Temperature Humidity Pressure RainFall WindDir WindSpeed CO2

WeatherDataWrite

Update WD Insert WDDBLog

Vaisala SensorDataData

Current Data Table

Archive Data

Web Service

Each sensor has multiple sensor type

DATA MODEL

Inside weather.hongo

API (weather.hongo)

neuron_id WNIID Time Temperature Humidity Pressure RainFall WindDir WindSpeed

TABLE WD

TABLE WDDBLog

CO2

neuron_id Time Temperature Humidity Pressure RainFall WindDir WindSpeed CO2

Current Data Table

Archive Data

WeatherDataReadAll WDReadLogWeb Service

Select WD Select WDDBLog

XMLXMLXMLXML

The Latest Sensor Data Archived Sensor Data

Evaluation (weather.hongo)

• The Number of Sensor– About 50.

• Upload Frequency– Temperature, Humidity, Pressure, RainFall

• Every 5 minutes

– WindDir, WindSpeed• Every 20 seconds

• WDDBLog Table Size– Increases about 250,000 rows per day.– Current Row Size is 39,863,725 (2006-05-16 10:12:00)– A Query takes about 40 seconds.

Characteristics (weather.hongo)

• Designed by Echelon.• What the Sensor is:

– Vaisala sensor (at least, sensor on Lonworks)– Each sensor has multiple sensor type.– Sensors can be identified by NeuronID.

• No sensor meta-information– Sensor meta-information (sensor profile) must be prov

ided by other systems.

• Centralized– No scalability, No robustness

INTRODUCTION

• Historical System (weather.hongo)– Overview– Evaluation– Characteristics

• New Approach (live-e.hongo)– Concept (Sensor Abstraction)– System Overview– Characteristics

New Approach (live-e.hongo)

• What the Sensor is: (Sensor Abstraction)– Sensor ID of Live E!

• NeuronID is base on Lonworks system.• We must make a new ID system.• Globally-unique

– A single sensor type.• Each sensor must have only one value.• A Multiple sensor must be divided to single sensors.

– Profile• Meta-information that describes sensor.

Sensor ID

• sensor_id– Globally-unique ID– Format (Syntax Rule)

• Domain Name + /SensorModel/Location/SensorType

– Example• live-e.org/WXT510/0300000431b2/Temperature• live-e.unl.im.dendai.ac.jp/WM918/kanda/12/roof/Temperature• live-e.unl.im.dendai.ac.jp/WM918/kanda/12/roof/Humidity• live-e.unl.im.dendai.ac.jp/WM918/kanda/12/roof/Pressure• mew.co.jp/FS-Va-01/aoyama_elementary_school/Temperature

• Semantics– No meaning– Just a String

Sensor Data

• Each sensor has a single value.– The value can be determined by “sensor_id” an

d sampled “time”.

value = F( sensor_id , time)

Ex. The result of GetCurrentData(sensor_id);<d:Data xmlns:d=“http://live-e.org/Ver200603/Data”> <value time=“2006-05-15T02:34:53.000+09:00”>23.5</value><d:Data>

(Major) Sensor Profile

• sensor_id (Ex. live-e.org/WXT5 ---/Temperature)• sensor_model (Ex. WXT510)• sensor_type (Ex. Temperature)• latitude (Ex. 35.712194)• longitude (Ex. 139.76775)• location (Ex. 東大情報基盤センター )• address (Ex.東京都文京区弥生 2-11-16)

Sensor Profile Example

• GetProfile(“live-e.org/WXT510/ --- /Temperature”);

<p:Profile xmlns:p="http://live-e.org/Ver200603/Profile"> <sensor_id>live-e.org/WXT510/03000005c3a2/Temperature</sensor_id> <sensor_vendor>vaisala</sensor_vendor> <sensor_model>WXT510</sensor_model> <sensor_type>Temperature</sensor_type> <longitude>139.76775</longitude> <latitude>35.712194</latitude> <location> 東大情報基盤センター </location> <address> 東京都文京区弥生 2-11-16</address></p:Profile>

API (live-e.hongo)

• String GetCurrentDataAll();

• String GetCurrentData(String sensor_id);

• String GetCurrentDataByAreaRect(double,double,double,double);

• String GetCurrentDataByType(String sensor_type);

• String GetCurrentDataOf(String sensor_id_like);

• String GetDataByTimespan(String,Date,Date);

• String GetProfileAll();

• String GetProfile(String sensor_id);

• String GetProfileByAreaRect(double,double,double,double);

• String GetProfileByType(String sensor_type);

live-e.hongo.wide.ad.jp weather.hongo

APP APP APP

Web Service(SOAP/XML)

Debian EtchPostgreSQL8.1Java+Axis

WM918 FS-Va-01

WXT510

live-e.hongo

Weather Station(WM918)

• Measure Item– Temperature– Humidity– Pressure– RainFall– WindDir– WindSpeed WM918

PCRS232C

Internetlive-e.hongo

SOAP

Field Server (FS-Va-01)

• Measure Item– Temperature– Humidity– Pressure– RainFall– WindDir– WindSpeed– Solar_Radiation– Soil_Moisture– Soil_Temperature– Photo

FS-Va-01

Internetlive-e.hongo

SOAP

DataCenterMatsushita-Denko

Data Upload Protocol

• Protocol– SOAP

• Data Format– XML-like format

<Data> <LiveE-ID>live-e.unl.im.dendai.ac.jp/WM918/kanda/12/roof/Temperature</LiveE-ID> <Time>2006-05-12T04:34:42.0000+09:00</Time> <Value>23.1</Value></Data>・・・ <Data></Data> continues

Behavior of Data Upload

sensor_id time value

TABLE sensor_data_current TABLE sensor_data_log

sensor_id time value

DB_INSERT

Updatesensor_data_current

SensorsDataData

Web Service

data_index

Insertsensor_data_log

Characteristics (live-e.hongo)

• Improved– Sensor Abstraction– Profile Management

• The same as weather.hongo– Directly connected to the Database

• Upload Message Flow• Query Processing

– Centralized• All the sensors are managed in one database• No scalability

Accident

• Accidents Already Occurred– The Accident of New Measure Item– The Accident of Software Update

• Accidents we must assume.– Server Crush– Rigid System

• Difficulty of introducing new architecture or new model.– Database-oriented– Sensors are deployed without managing

The Accident of New Measure Item

• There is another data archive system.http://www.cnl.k.u-tokyo.ac.jp/~koba/live_e/index.php

– This system collects data by parsing HTML of web interface of weather.hongo.

http://weather.hongo.wide.ad.jp/WDShow/WDShow.aspx

• One day, a new measure item “DayRainFall” was added.– Of course, the HTML structure was changed.

• The archive system went into out of order.

The Accident of Software Update

• Sensor data upload software had a bug.– It had been already published.– Isi removed the bug.

• Published again• Announced

• One of us deployed the old version.– The fact was found out by human check.– live-e.hongo could not detect it.

WM918

PCRS232C

Internetlive-e.hongo

SOAP

Discussion

Discussion Agenda

1. Evaluate the system in many aspects– Scalability, Robustness, ... , etc.

2. Introduce related work.

3. Determine what we focus on ...• pub-sub? or range query? or any other?

4. Design prototype system

5. Determine what to do at the workshop

Evaluate Point

• Scalability• Robustness• Flexibility• Reliability • Efficiency• Functionality• Availability• Feasibility• Heterogeneity Management• Automation• Simplicity

Scalability

• What is O(N)? N: The Number of Node.

• Resource Capacity VS O(N).– R>>O(N) at N=1,000,000 → Scale– R<O(N) at N=1,000,000 → Does Not Scale

• Current System (live-e.hongo)– Does not scale!!

• What do we need to introduce to obtain scalability?

Robustness (Tolerance)

• What would happen if ... ?– A server crushed.– Disconnected.

• Current System– It must be a disaster.

• Lose the data archived• Out of Service• Cannot collect sensor data• Take time to comeback

• What do we have to do to obtain tolerance?

Flexibility

• Is it easy to extend ... ?– easy to add some functions ?– easy to add resources ? (such as disk space or CPU)

• Current System– Rigid. – Each parts are tightly dependent.– We need lots of energy to remodel.

• How do we design a flexible system?

Reliability

• Is the data correct?– Can we trust the value of sensors?– How deep can we detect irregular behavior?– Who has the responsibility of the data?– Certification

• Current System– Trust all the sensor even if one of them said -324℃

• What do we need to improve reliability?

Efficiency

• Low cost?– Time, Money, Power, Space ...– It is said... The cost is:

• big central > small distributed

• Current System– Unknown

• It is important to think of efficiency when we design system.

Functionality

• Complex Query OK?– Range query

• Current system– Good

• How do we obtain functionality within the system?

Availability

• Is Live E! system opened?• Example of open system

– Internet– DHT

• Current System– Not opend– Only the system manager can control the system.

• How do we provide availability?

Feasibility

• Can we execute the plan?– Human Resource– Money– Law

• We have to consider whether we can do it or not.

Heterogeneity Management

• How do we manage sensors uniformly?– Abstraction?– Namespace Management?

Automation

• What do we need to design system to work automatically?– When software updated.– When resource added.

Simplicity

• Simpler, More User

• Few would like to learn complex access methods.

RELATED WORK

• Content-Based Network– Publish-Subscribe Service

• DHT (Distributed Hash Table)– i3 (publish-subscribe on DHT)– PHT (range query on DHT)

• Caching

• Live Update

• Sensor Abstraction

What we focus on ...

• Publish - Subscribe Model

• Range Query

• Data Mining

• The Abstraction of Sensor

• Live Update

Prototype System

• Depends on what we focus on ...

• Talk about these:– Architecture– Interface and Protocol– Evaluation– Who

At the Workshop

• Evaluate the system.– What do you want to do?– I want to do something aggressive.

Ex. Increase virtual sensors until the system fails.