infinispan 8 tour
TRANSCRIPT
![Page 2: Infinispan 8 Tour](https://reader031.vdocuments.pub/reader031/viewer/2022021918/58aa91531a28ab85678b49d3/html5/thumbnails/2.jpg)
About Infinispan• Apache license in-memory Key/Value data store
• API: Map based (ConcurrentMap) and JCache (JSR-107) compliant
• Embedded or Remote: Hot Rod, Rest, Memcached, Websocket
• Main cache modes: Distributed, Replicated, Local
• XA Transactions
• Events
• Distributed processing/Map Reduce
• Client language (C#, C++, Java, Javascript)
• Query: indexed with Lucene or non indexed
• 8.0.0.Final Released Aug-2015 / 8.1.0.Final Dec-2015
![Page 3: Infinispan 8 Tour](https://reader031.vdocuments.pub/reader031/viewer/2022021918/58aa91531a28ab85678b49d3/html5/thumbnails/3.jpg)
What’s new tl;dr• First based on Java 8
• New APIs leveraging the best java 8 has to offer
• New Admin Console
• New Integrations: Apache Hadoop, Apache Spark
• Querying: continuous, grouping, aggregation, Lucene 5, hybrid
• Expiration and eviction enhancements
![Page 4: Infinispan 8 Tour](https://reader031.vdocuments.pub/reader031/viewer/2022021918/58aa91531a28ab85678b49d3/html5/thumbnails/4.jpg)
New APIs
• Experimental Functional Map API
• Distributed Streams
![Page 5: Infinispan 8 Tour](https://reader031.vdocuments.pub/reader031/viewer/2022021918/58aa91531a28ab85678b49d3/html5/thumbnails/5.jpg)
Functional Map API motivations
• Not a replacement for the current Map/JCache API
• Asynchronous
• Lazy
• Lambda based
![Page 6: Infinispan 8 Tour](https://reader031.vdocuments.pub/reader031/viewer/2022021918/58aa91531a28ab85678b49d3/html5/thumbnails/6.jpg)
Functional Map API entry points AdvancedCache<String, String> cache = cacheManager.getCache().getAdvancedCache();
FunctionalMapImpl<String, String> functionalMap = FunctionalMapImpl.create(cache);
ReadOnlyMap<String, String> readOnlyMap = ReadOnlyMapImpl.create(functionalMap);
WriteOnlyMap<String, String> writeOnlyMap = WriteOnlyMapImpl.create(functionalMap);
ReadWriteMap<String, String> readWriteMap = ReadWriteMapImpl.create(functionalMap);
![Page 7: Infinispan 8 Tour](https://reader031.vdocuments.pub/reader031/viewer/2022021918/58aa91531a28ab85678b49d3/html5/thumbnails/7.jpg)
Functional Map API - ReadOnlyMap
<R> CompletableFuture<R> eval(K key, Function<ReadEntryView<K, V>, R> f);
<R> Traversable<R> evalMany(Set<? extends K> keys, Function<ReadEntryView<K, V>, R> f);
![Page 8: Infinispan 8 Tour](https://reader031.vdocuments.pub/reader031/viewer/2022021918/58aa91531a28ab85678b49d3/html5/thumbnails/8.jpg)
Functional Map API - ReadOnlyMap example
public class Transaction implements Serializable { int size; String script; boolean processed; public String getScriptHash() { byte[] digest = MessageDigest.getInstance(“SHA-256”).digest(script.getBytes("UTF-8")); return Base64.getEncoder().encodeToString(digest); } . . . }
CompletableFuture<String> value = readOnlyMap.eval("key223", entryView -> entryView.get().getScriptHash());
![Page 9: Infinispan 8 Tour](https://reader031.vdocuments.pub/reader031/viewer/2022021918/58aa91531a28ab85678b49d3/html5/thumbnails/9.jpg)
Functional Map API - ReadOnlyMap example
// key1, key2, ..., key10Set<String> keys = IntStream.rangeClosed(1, 10).boxed() .map(i -> "key" + i) .collect(Collectors.toSet()); Integer sum = readOnlyMap .evalMany(keys, entryView -> entryView.get().getSize()) .reduce(0, (a, b) -> a + b);
![Page 10: Infinispan 8 Tour](https://reader031.vdocuments.pub/reader031/viewer/2022021918/58aa91531a28ab85678b49d3/html5/thumbnails/10.jpg)
Functional Map API - WriteOnlyMap
CompletableFuture<Void> eval(K key, V value, BiConsumer<V, WriteEntryView<V>> f);
CompletableFuture<Void> eval(K key, Consumer<WriteEntryView<V>> f);
CompletableFuture<Void> evalMany(Map<? extends K, ? extends V> entries, BiConsumer<V, WriteEntryView<V>> f);
![Page 11: Infinispan 8 Tour](https://reader031.vdocuments.pub/reader031/viewer/2022021918/58aa91531a28ab85678b49d3/html5/thumbnails/11.jpg)
Functional Map API - WriteOnlyMap examples
// key1, key2, ..., key10Set<String> keys = IntStream.rangeClosed(1, 10).boxed() .map(i -> "key" + i).collect(Collectors.toSet()); writeOnlyMap.evalMany(keys, EntryView.WriteEntryView::remove).get();
CompletableFuture<Void> future = writeOnlyMap .eval("newKey", v -> v.set(new Transaction(5000, "13E3401")));future.thenAccept(e -> System.out.println(cache.get("newKey")));
Remove values from a set of keys:
Insert new value and print when done:
![Page 12: Infinispan 8 Tour](https://reader031.vdocuments.pub/reader031/viewer/2022021918/58aa91531a28ab85678b49d3/html5/thumbnails/12.jpg)
Functional Map API - Read/Write
Traversable<CompletableFuture<Void>> futures = readWriteMap.evalAll(view -> { Transaction t = view.get(); if(t.isProcessed()) { view.remove(); } if (t.getSize() > 1000) { return writeOnlyMap.eval(view.key() + "_", v -> { int newSize = t.getSize * 0.90; Transaction newTx = new Transaction(newSize, t.getScript()); v.set(NewTx); }); } return null; });
![Page 13: Infinispan 8 Tour](https://reader031.vdocuments.pub/reader031/viewer/2022021918/58aa91531a28ab85678b49d3/html5/thumbnails/13.jpg)
Distributed Streams
• Distributed implementation of java.util.stream.Stream over cache data
• Topology Aware
• Parallel
• Lambdas and Collectors must be serializable
![Page 14: Infinispan 8 Tour](https://reader031.vdocuments.pub/reader031/viewer/2022021918/58aa91531a28ab85678b49d3/html5/thumbnails/14.jpg)
Distributed Streams - Usage
CacheStream<Entry<String, Transaction>> entryStream = cache.entrySet().stream();
CacheStream<Transaction> valueStream = cache.values().stream();
CacheStream<String> keyStream = cache.keySet().stream();
![Page 15: Infinispan 8 Tour](https://reader031.vdocuments.pub/reader031/viewer/2022021918/58aa91531a28ab85678b49d3/html5/thumbnails/15.jpg)
Distributed Streams - Usage
Stream<Transaction> valueStream = cache.values().stream();Double averageSize = valueStream .filter(Transaction::isProcessed) .collect(Collectors.averagingInt(Transaction::getSize));
![Page 16: Infinispan 8 Tour](https://reader031.vdocuments.pub/reader031/viewer/2022021918/58aa91531a28ab85678b49d3/html5/thumbnails/16.jpg)
Distributed Streams - example
Double averageSize = valueStream .filter((Serializable & Predicate<Transaction>) Transaction::isProcessed) .collect(CacheCollectors.serializableCollector(() -> Collectors.averagingInt( (Serializable & ToIntFunction<Transaction>) Transaction::getSize) )
);
![Page 17: Infinispan 8 Tour](https://reader031.vdocuments.pub/reader031/viewer/2022021918/58aa91531a28ab85678b49d3/html5/thumbnails/17.jpg)
Admin Console
• Angular.js, bootstrap.js and PatternFly based
• Server mode
• Lifecycle/Monitoring
![Page 18: Infinispan 8 Tour](https://reader031.vdocuments.pub/reader031/viewer/2022021918/58aa91531a28ab85678b49d3/html5/thumbnails/18.jpg)
Admin Console - Demo• Start Domain controller
•Start slaves
docker run -it --name master -h master -e "SLAVES=1" gustavonalle/infinispan-server-domain
docker run -it --link master:master -h slave1 gustavonalle/infinispan-server-domain
![Page 19: Infinispan 8 Tour](https://reader031.vdocuments.pub/reader031/viewer/2022021918/58aa91531a28ab85678b49d3/html5/thumbnails/19.jpg)
Integrations• Hadoop and Spark Connectors
• Read/Write
• Infinispan Server as a datasource
• Separated Clusters
• Server mode only
• Task distribution with best locality
![Page 20: Infinispan 8 Tour](https://reader031.vdocuments.pub/reader031/viewer/2022021918/58aa91531a28ab85678b49d3/html5/thumbnails/20.jpg)
ArchitectureMaster
Worker Worker Worker Worker
![Page 21: Infinispan 8 Tour](https://reader031.vdocuments.pub/reader031/viewer/2022021918/58aa91531a28ab85678b49d3/html5/thumbnails/21.jpg)
Infinispan Spark Connector - Demo
• Start Infinispan docker run -it --name master -h master gustavonalle/infinispan-server-domain
• Start Spark docker run --name spark-master -ti gustavonalle/spark
• Start Spark shell docker exec -it spark-master /usr/local/spark/bin/spark-shell --master spark://172.17.0.3:7077 —-packages org.infinispan:infinispan-spark_2.10:0.2
![Page 22: Infinispan 8 Tour](https://reader031.vdocuments.pub/reader031/viewer/2022021918/58aa91531a28ab85678b49d3/html5/thumbnails/22.jpg)
Integrations - Demo import org.infinispan.spark._ import org.infinispan.spark.rdd._ import scala.util.Random
val wordList = scala.io.Source.fromFile("/usr/share/dict/words").getLines.foldLeft(List[String]())( (s, w) => w :: s)
val phrases = (0 to 400).toStream.map(i=> Random.nextInt(wordList.size)).sliding(4,4).map(_.map(wordList).mkString(" ")).toSeq
val phraseRDD = sc.parallelize(phrases).zipWithIndex.map(_.swap)
val config = new java.util.Properties config.put(“infinispan.client.hotrod.server_list”,“172.17.0.2:11222“) phraseRDD.writeToInfinispan(config)
val infinispanRDD = new InfinispanRDD[Long,String](sc, config) infinispanRDD.values.flatMap(_.split(" ")) .map( word => (word.length, 1) ) .reduceByKey(_+_) .sortBy(_._1).toArray .foreach{ case (size, count) => println(s"$size chars words: $count occurrences") }
![Page 23: Infinispan 8 Tour](https://reader031.vdocuments.pub/reader031/viewer/2022021918/58aa91531a28ab85678b49d3/html5/thumbnails/23.jpg)
Query - Aggregations
Query size = qf.from(Transaction.class) .select(Expression.avg("size"), Expression.max("size"), Expression.min(“size")) .having(“processed").eq(true).toBuilder() .build();
![Page 24: Infinispan 8 Tour](https://reader031.vdocuments.pub/reader031/viewer/2022021918/58aa91531a28ab85678b49d3/html5/thumbnails/24.jpg)
Query - Grouping QueryFactory qf = Search.getQueryFactory(cache); Query q = qf.from(Transaction.class) .select( Expression.property("accountId"), Expression.avg(“amount")) .groupBy(“accountId") .having(Expression.avg(“amount”)).lt(130.0).toBuilder() .orderBy(“accountId") .build();
![Page 25: Infinispan 8 Tour](https://reader031.vdocuments.pub/reader031/viewer/2022021918/58aa91531a28ab85678b49d3/html5/thumbnails/25.jpg)
Query - ContinuousQueryFactory<?> qf = Search.getQueryFactory(cache);Query query = qf.from(Person.class) .having("salary").lte(3000) .toBuilder().build();ContinuousQueryListener<Object, Object> listener = new MyListener<>(); ContinuousQuery<Object, Object> cq = new ContinuousQuery<>(cache); cq.addContinuousQueryListener(query, listener);
![Page 26: Infinispan 8 Tour](https://reader031.vdocuments.pub/reader031/viewer/2022021918/58aa91531a28ab85678b49d3/html5/thumbnails/26.jpg)
Query - Continuous/** * Listener for continuous query events. * * @author [email protected] * @since 8.1 */public interface ContinuousQueryListener<K, V> { /** * Receives notification that a cache entry has joined the matching set. * * @param key the key of the joining entry * @param value the joining entry or the Object[] projection if specified */ void resultJoining(K key, V value); /** * Receives notification that a cache entry has left the matching set. * * @param key the key of the leaving entry */ void resultLeaving(K key);}
![Page 27: Infinispan 8 Tour](https://reader031.vdocuments.pub/reader031/viewer/2022021918/58aa91531a28ab85678b49d3/html5/thumbnails/27.jpg)
Core - Eviction• Eviction can be triggered by memory size (estimation):
ConfigurationBuilder cfg = new ConfigurationBuilder();cfg .eviction().strategy(EvictionStrategy.LRU).type(EvictionType.MEMORY).size(MAX_MEMORY) .build();
• Works for primitives key/value, byte arrays
• Pluggable estimator for custom classes planned
![Page 28: Infinispan 8 Tour](https://reader031.vdocuments.pub/reader031/viewer/2022021918/58aa91531a28ab85678b49d3/html5/thumbnails/28.jpg)
Core - Expiration• New clustered expiration events
@Listener(clustered = true) protected class ClusterListener { @CacheEntryExpired public void onCacheEvent(CacheEntryEvent event) { log.debugf(“Received expiration event %s", event); }}
![Page 29: Infinispan 8 Tour](https://reader031.vdocuments.pub/reader031/viewer/2022021918/58aa91531a28ab85678b49d3/html5/thumbnails/29.jpg)
Thankshttps://github.com/infinispan/infinispan
https://issues.jboss.org/browse/ISPN
#infinispan Freenode
https://twitter.com/infinispan
http://infinispan.org/download
https://github.com/infinispan/infinispan-spark