mmea platform
TRANSCRIPT
MMEA PlatformHarri Hytönen (Vaisala)September 23, 2015
Satellite
RadarIn-situ
Modelling Citizen science
Research
OverviewMMEA Platform = scalable real-time data processing with embedded notification, storage, routing, filtering and processing services.
Contributors
OSS Components• WSO2 Enterprise Service Bus (ESB)
– Integration solution• Service virtualization = proxy services• Message transformations• Transport (HTTP(S)/JMS/SMTP/FTP etc.)
switching– Authentication (WSO2 Identity Server) – Authorization– Monitoring and diagnostics
OSS Components• Apache ActiveMQ
– messaging backbone– quality of service
• PostgreSQL RDBMS• Apache Tomcat• Esper
– CEP engine• Wavellite
– framework for situation awareness• Octave
– High level language for computations• Puppet & Artifactory
– Deployment tools• CentOS
Commercial Components• Amazon AWS
– EC2 (for hosting)– RDS
• Profium Sense– RDF storage– Inference engine
• Confluence & JIRA– Documentation and issue tracking
MMEA Platform Deployment
Amazon EC2
binaries
config
buildssource code
Pilot - Indoor Air Quality (VTT, HiQ)deployment MMEA Testbed
Indoor Pilot MMEA platform
GetObservation
PutMeasurementComponents and
interfaces 2::Indoor observ ation data
source
GetObservation
PutMeasurement
«device»Measurement
dev ices
(from Components and interfaces 2)
GetObservationComponents and
interfaces 2::Measurement
Components and interfaces 2::
IndoorObserv ation
MMEA Framework::
SensorML
UI
GetObservationComponents and
interfaces 2::Indoor end-user system
UI
GetObservation
Components and interfaces 2::IndoorData
MMEAIndoorDataServ ice
PutMeasurement
MMEA Framework::MMEAObserv ationServ ice
GetObservation
MMEA Framework::MMEADataQualityServ ice
Other Use Cases• Helsinki area outdoor air quality• Weather stations data ingestion• Weather radar product generation• Hydrological data ingestion• Situational awareness applications
– Farmer’s situational awareness– Storm path detection– Semantic end user service
Complex Event Processing• Esper
– High speed event processing– Identifies meaningful events
• Targeted to real-time Event Driven Architectures• Designed for high-volume event correlation where millions of
events coming in would make it impossible to store them all to later query them using classical database architecture
• Some examples of applications in MMEA:– Detect patterns among events– Filter events and event aggregation– Generating notifications and alerts based on event patterns
Computations in the MMEA platform
ESB
Data source
EsperComplex event processing
Computation serviceOnline and offline computations
Mediator Mediator
Computation service
Computation service
Computation task1. Run preprocessor
2. Run Octave script
3. Run postprocessor
Workspace
Input data and parameters
Thank You – Questions?