● Complex Event ProcessingWhat ?

Why ?

● WSO2 Complex Event ProcessorScalability

Edge Computing

● Usecases ● What’s new is WSO2 CEP 4.0 ?

● Amount of data you analyse are big ● You don’t need data for long time● You don’t want to store all the data ● Time critical analysis

○ Finding opportunities before they’re gone○ Identify threats beforehand

• Filters & Transformations

• Windows and Aggregations • Joins• Sequences and Patterns

• Event Tables

Stateless• Filters & Transformations

Stateful Inmemory• Windows and Aggregations • Joins• Sequences and Patterns

Stateful Persisted• Event Tables

• Vertical Scaling• Horizontal Scaling

E.g. Calculate Mean

E.g. Calculate Mean

E.g. Calculate Mean

How about scaling median ?

How about scaling median ?

If & only if we can partition !

Spark Streaming

• Supports distributed processing• Runs micro batches• Not supports pattern & sequence detection

Spark Streaming

• Supports distributed processing• Runs micro batches• Not supports pattern & sequence detection

Apache Storm

• Supports distributed processing• Stream processing engine

Advantages

• Supports distributed processing• Supports Partitioning• Extendable• Open Source

Disadvantages

• Need to write Java code • Need to start from basic principles ( & data structures ) • Adoption for change is slow• No support to govern artifacts

Advantages

• Supports distributed processing• Supports Partitioning• Extendable• Open Source

Disadvantages

• No need to write Java code (Supports SQL like query language)

• No need to start from basic principles (Supports high level language)

• Adoption for change is fast• Govern artifacts using Toolboxes• etc ...

define stream StockStream (symbol string, volume int, price double);

@name(‘Filter Query’)from StockStream[price > 75]select *insert into HighPriceStockStream ;

@name(‘Window Query’)from HighPriceStockStream#window.time(10 min)select symbol, sum(volume) as sumVolume insert into ResultStockStream ;

define stream StockStream (symbol string, volume int, price double);

@name(‘Filter Query’)from StockStream[price > 75]select *insert into HighPriceStockStream ;

@name(‘Window Query’)partition with (symbol of HighPriceStockStream)begin

from HighPriceStockStream#window.time(10 min)select symbol, sum(volume) as sumVolume insert into ResultStockStream ;

end;

define stream StockStream (symbol string, volume int, price double);

@name(Filter Query’)@dist(parallel= ‘3')from StockStream[price > 75]select *insert into HightPriceStockStream ;

@name(‘Window Query’)@dist(parallel= ‘2')partition with (symbol of HighPriceStockStream)begin

from HighPriceStockStream#window.time(10 min)select symbol, sum(volume) as sumVolume insert into ResultStockStream ;

end;

Siddhi CEP

Siddhi CEP

• System Monitoring (Logs, Servers)• Finance (Financial trading)• Banking (Credit card fraud detection)• Marketing (Location & time based advertising)• Logistics (Taxi services, Vehicle parking)• Healthcare • E-commerce (Online retailers)• Telecommunication (Mobile offers)• System automation (Auto scaling)• IoT (Smart home, Smart buildings)• ...

• Distributed processing with Apache Storm• Improved High Availability• Improved Siddhi Query Language • Toolbox Support• Realtime Dashboard• Websocket + MQTT Support

And lots of extensions• Geo• Timeseries• NLP• PMML & WSO2 ML

Try out CEP 4.0 Alpha : https://github.com/wso2/product-cep/releases