JBoss 인피니스팬데이터그리드 플랫폼 따라잡기전재홍/JBoss User Group

인메모리 데이터그리드

Distributed Cache

• Fast access to data• Performance boost• Elasticity• High Availability• JSR 107 (Temporary Caching for the Java Platform)– Read, write, expiration, write-through, distribution

Distributed Cache (cont.)

Persistence(Database)

DistributedCacheCluster

Node

Node

Node

App

App

App

In-Memory Data Grid

• Evolution of distributed caches• Clustered by nature (shared across multiple servers)• Low response time• High throughput• Predictable scalability• High Availability• Querying• Task Execution (Map/Reduce)• JSR 347 (Data Grid for the Java Platform)

In-Memory Data Grid (cont.)

Persistence(DataStore)

In-MemoryData Grid

Node

Node

Node

App

App

App

IMDG for What?

• Sharing data (session, app states)• Toolkit for clustering• Performance (caching, in-memory processing)• Scalability• Database on cloud

Players

• Oracle Coherence• GridGain• HazelCast• IBM eXtreme Scale• GigaSpaces• VmWare GemFire• Terracotta• ScaleOut StateServer• JBoss Infinispan

Cache vs. Data Grid

• JSR 107 - Temporary Caching for the Java Platform– Basic interaction(read, write, expiry)– Transactions with JTA compatibility– Listener– Persistence: read-through, write-through, write-behind– Annotations– javax.cache.*

• JSR 347 - Data Grids for the Java Platform– Asynchronous, non-blocking API– Distributed code execution and map/reduce API– Group API for co-location– Annotations (CDI)– Eventually Consistent API– Querying– Configuration– javax.datagrid.*

인피니스팬

Infinispan

• Distributed in-memory key/value data grid and cache

• Distributed as library and server• Manageable• Open Source

DefaultCacheManager manager = new DefaultCacheManager();

// Cache<Integer, Ticket> cache = manager.getCache();Cache<Integer, Ticket> cache = manager.getCache(“myCache”);

Architecture: as library

Embedded library on the same JVM with Application

Infinispan

JVM

App

Architecture: as library, clustered

Cluster

Infinispan

JVM

App

Infinispan

JVM

App

Infinispan

JVM

App

Application doesn’t know it’s on cluster

l Use as library– More features

– Richer APIs

– Programmatic/ Declarative configuration

– Extendable/ embeddable

– Faster (API call)

Architecture: as server, clustered

• Use as server– Remote– Data tier shared by

multi-apps– App doesn’t affect

cluster– Non-java clients

• C++, .NET, Ruby, Python, Java

Cluster

Infinispan

JVM

Infinispan

JVM

Infinispan

JVM

App

App

App

Architecture: multi clusters

• Multi-clusters– By replication– By persistence– By replication

to other cluster (topology aware)

Cluster

Infinispan

JVM

Infinispan

JVM

Cluster

Infinispan

JVM

Infinispan

JVM

persistence

Clustering

• Peer-to-Peer– No central master, no single point of failure, no single

bottle neck

• JGroups– Reliable multicast communication library, nodes

discovery, sharing data, performing cluster scaling

• Consistent Hash– Hash based data distribution– How it finds where data locates

• Linear in nature: throughput, capacity

Cluster Mode: Replication(복제)

Replication Mode

Cache onServer 1

K,V

Cache onServer 2

K,V

Cache onServer 4

K,V

Cache onServer 3

K,V

cache.put(K,V)

Cluster Mode: Distribution(분산)

Distribution Mode(numOwners=2)

Cache onServer 1

K,V

Cache onServer 2

K,V

Cache onServer 4

Cache onServer 3

cache.put(K,V)

cache.get(K,V)

Cluster Mode: Invalidation(무효화)

Invalidation Mode

Cache onServer 1 K,V2

Cache onServer 2

K,V

Cache onServer 4

Cache onServer 3

cache.put(K,V2)

DB

Persistence

• Used for durability• Cache Store - Persistence Storage– File System, Cloud, Remote, JDBC, JPA, LevelDB,

Cassandra, HBase, MongoDB, BerkeleyDB, JDBM, REST

• CacheLoader, CacheWriter• Read-through, write-through, write-behind• Passivation, activation• Store chain• Shared store

Persistence (cont.)

• Passivation – write to persistence when evicted from memory (default)

• Activation – read to memory and remove from persistence

Transaction

• JTA Transaction Support• Support MVCC (Multi-Versioned Concurrency Control)

• Isolation Level– READ_COMMITTED (default)– REPEATABLE_READ

• Locking Mode– Optimistic Lock (default)– Pessimistic Lock

• Locking– Lock timeout– Lock striping

Query

• Query on values• JBoss Hibernate Search + Apache Lucene• Index Directory– Lucene Directory: in-memory, file system, JDBC– Infinispan Directory

• Distributed queries

Distributed Execution

• Executes codes on distributed nodes• Through a standard JDK ExecutorService interface• Use DistributedCallable extends

java.util.concurrent.Callable

Map/Reduce

• Based on Distributed Execution Framework• Mapper, Reducer, Collator, MapReduceTask

public interface Callator<KOut, Vout, R> {

R collate(Map<KOut, VOut>);}

public interface Mapper<KIn, VIn, KOut, VOut> extends Serializable {

void map(KIn key, VIn value, Collector<KOut, VOut> collector);}

public interface Reducer<KOut, VOut> extends Serializable {

VOut reduce(KOut reducedKey, Iterator<VOut> iter); }

Configuration: Declarative

• Eviction(제거)– on the

memory• Expiration(만료)

– on the cluster

<global><transport clusterName=“MyCacheCluster">

<properties><property name="configurationFile" value="jgroups-tcp.xml" />

</properties></transport><globalJmxStatistics enabled="true" />

</global>

<default><clustering mode="replication"><sync /></clustering>

</default>

<namedCache name="secureContextCache "><eviction strategy="LIRS" maxEntries="2000" /><expiration lifespan="600000" /><persistence passivation="false">

<store class="org.infinispan.persistence.file.SingleFileStore"fetchPersistentState="false" preload="true" shared="false"purgeOnStartup="true" ignoreModifications="false"><async enabled="true" flushLockTimeout=“1000"

shutdownTimeout="3000" modificationQueueSize="1500" threadPoolSize="3" /><properties>

<property name="location" value="${java.io.tmpdir}" /></properties>

</store></persistence>

</namedCache>

Configuration: Programmatic

• Configuration Based on XML

• Programmatic configuration

DefaultCacheManager manager = new DefaultCacheManager("infinispan-config.xml");Configuration baseConf = manager.getDefaultCacheConfiguration();Configuration config =new ConfigurationBuilder().read(baseConf).expiration().lifespan(50000).build();

manager.defineConfiguration(programmaticCache, config);Cache<String, String> cache = manager.getCache("secureContextCache");

DefaultCacheManager manager = new DefaultCacheManager();Configuration config = new ConfigurationBuilder()

.loaders()

.shared(false).passivation(false).preload(false)

.addCacheLoader()

.cacheLoader(new JdbcStringBasedCacheStore())

.addProperty("connectionFactoryClass","org.infinispan.loaders.jdbc.connectionfactory.ManagedConnectionFactory")

.addProperty("datasourceJndiLocation", "java:jboss/datasources/MySQLDS")

.addProperty("userName", "root")

.addProperty("password", "admin")

.async().threadPoolSize(10).build();manager.defineConfiguration(programmaticCache, config);Cache<String, String> cache = manager.getCache("secureContextCache");

Listener

• Listener on CacheManager– Node join/ leave, Cache start/ stop

• Cache– CRUD, Eviction/ Passivation– Rehashing/ Transaction completion@Listenerpublic class SimpleListener {

@CacheEntryCreatedpublic void dataAdded(CacheEntryCreatedEvent event) {

if (event.isPre()) {System.out.println("Before creating the entry:" + event.getKey());

} else {System.out.println("After creating the entry:" + event.getKey());

}…

}

DefaultCacheManager manager = new DefaultCacheManager();manager.addListener(listener);Cache<Integer, Ticket> cache = manager.getCache();cache.addListener(listener);

Asynchronous APIs

• put() and get() and remove() are synchronous– They wait for RPC and Locks (and maybe cache stores)

• The asynchronous API returns NotifyingFuture– Events are fired on completion of the operationNotifyingFuture<String> future = c.removeAsync(key);future.attachListener(new FutureListener<String>() {

@Overridepublic void futureDone(Future<String> future) {

try {future.get();System.out.printf ("The entry stored under key %s has been removed.", key);

} catch (ExecutionException e) {System.out.printf("Failed to remove %s!", key);

}}

});

Spring Integration

• Infinispan provider for Spring cache abstraction• infinispan-spring.jar

<cache:annotation-driven cache-manager=“myCacheManager"/><bean id="operationCacheManager"

class="org.infinispan.spring.provider.SpringEmbeddedCacheManagerFactoryBean"p:configurationFileLocation="classpath:infinispan -config.xml" />

@Cacheable(value = "secureContextCache", key="#contextId")public SecureLayerContext getSecureLayerContext(String contextId) {

return null;}

@CachePut(value = "secureContextCache", key="#contextId")public SecureLayerContext setSecureLayerContext(String contextId,

SecureLayerContext secureLayerContext) {return secureLayerContext;

}

@CacheEvict(value = "secureContextCache", key="#contextId")public void removeSecureLayerContext(String contextId) {

// Intentionally blank}

Customizing with Interceptors

• Infinispan follows Interceptor, Command/Visitor design patterns

• Custom interceptor extends BaseCustomInterceptor

Infinispan on JBoss AS 7 (WildFly 8)

• Used for session clustering, Hibernate L2 cache• Application gets cache with JNDI name using

@Resource• XML Configuration in server configuration file

<cache-container name="web" aliases="standard-session-cache" default-cache="repl"><transport lock-timeout="60000" /><replicated-cache name="repl" mode="ASYNC" batching="true">

<file-store /></replicated-cache>

</cache-container>

Marshalling

• JBoss Marshalling framework used for POJOs• User can provide custom Externalizer impls for

non-Serializable object• Custome marshaller can be provided as well

Client Implementations

Monitoring/Management

• Mbeans on CacheManager, Cache• RHQ (JON, JBoss Operations Network)

Radar Gun

• Data grid and distributed cache benchmarking framework

• Built to test Infinispan and other distributed data grid platforms

• https://github.com/radargun/radargun

적용 예

Use Cases: In Streaming Processing

Infinispan Data Grid

Use Cases: Data Grid Platform

In-Memory 대용량 데이터 처리를 위한 아키텍쳐 구조 제시 및 적용 사례,제 6회 한국 소프트웨어 아키텍트 대회 최우수상, 2013

Use Case: Session Clustering

• Store session information into cachein Spring MVC Interceptor

Case Study: Session Clustering

#1 Spring Cache Abstraction 사용하여 쉽게 다른 캐시 구현체 사용 가능하게

• ConcurrentHashMap, EHCache• Infinispan

#2 SecurityContext를 캐시에 저장• 기본적으로 HTTP Session에 저장

(HttpSessionSecurityContextRepository)• SecurityContextRepository 구현한

CacheSecurityContextRepository 작성하여 HTTP Session 대신 캐시를 사용하도록 함

Case Study: Infinispan with Spring

Loadbalancer

Client

Client

Client

InfinispanApplicationServices

SpringSecurity

WAS

InfinispanApplicationServices

SpringSecurity

WAS

InfinispanApplicationServices

SpringSecurity

WAS

. . .

Infinispan with Spring (cont.)

• infinispan-config.xml<global>

<transport clusterName=”MyCacheCluster">

<properties>

<property name="configurationFile" value="jgroups-tcp.xml" />

</properties>

</transport>

</global>

<default>

<clustering mode="replication">

<sync />

</clustering>

</default>

<namedCache name="securityContextCache">

<!– maxEntries means the maximum concurrent user connections-->

<eviction strategy="LIRS" maxEntries="10000" />

<!-- the max idle time is 30 minutes -->

<expiration maxIdle="1800000" />

</namedCache>

Infinispan with Spring (cont.)

• spring-cache-config.xml

• SpringEmbeddedCacheManagerFactoryBean

<cache:annotation-driven cache-manager="myCacheManager"/><bean id="myCacheManager"

class="my.domain.MyCacheManagerFactoryBean"p:configurationFileLocation="classpath:infinispan-config.xml" />

public class MyCacheManagerFactoryBean extends SpringEmbeddedCacheManagerFactoryBean {@Override public void afterPropertiesSet() throws Exception {

super.afterPropertiesSet();addListeners();

}private void addListeners() throws Exception {

SpringEmbeddedCacheManager cacheManager = getObject();if (cacheManager.getNativeCacheManager() instanceof EmbeddedCacheManager) {

cacheManager.getNativeCacheManager().addListener(new MyCacheManagerListener());}Collection<String> cacheNames = cacheManager.getCacheNames();for (String cacheName : cacheNames) {

SpringCache cache = cacheManager.getCache(cacheName);if (cache.getNativeCache() instanceof Cache) {

((Cache<?,?>)cache.getNativeCache()).addListener(new MyCacheListener());}}}}

Infinispan with Spring (cont.)

• Dao Implementation@Named("SecurityContextDao")

public class SecurityContextDaoImpl implements SecurityContextDao {

@Cacheable(value = "securityContextCache", key="#key")

public SecurityContext getSecurityContext(String key) {

return null;

}

@CachePut(value = "securityContextCache", key="#key")

public SecurityContext setSecurityContext(String key, SecurityContext securityContext) {

return securityContext;

}

@CacheEvict(value = "securityContextCache", key="#key")

public void removeSecurityContext(String key) {

// Intentionally blank

}

}

Infinispan with Spring (cont.)

• Spring Security’s SecurityContextRepositorypublic class MyCacheSecurityContextRepository implements SecurityContextRepository {

@Inject SecurityContextDao securityContextDao;

public SecurityContext loadContext(HttpRequestResponseHolder requestResponseHolder) {

...

return securityContextDao.getSecurityContext(authToken);

}

public void saveContext(SecurityContext context, HttpServletRequest request,

HttpServletResponse response) {

...

securityContextDao.setSecurityContext(authToken, context);

...

}

public boolean containsContext(HttpServletRequest request) {

...

return securityContextDao.getSecurityContext(key) != null;

}

}

Infinispan with Spring (cont.)

• spring-security-config.xml<bean id="cacheSecurityContextRepository"

class="my.domain.MyCacheSecurityContextRepository">

</bean>

<security:http pattern="/services/**" auto-config="false"

use-expressions="true" entry-point-ref="myAuthenticationEntryPoint"

authentication-manager-ref="operationsAuthenticationManager"

security-context-repository-ref="cacheSecurityContextRepository">

<security:custom-filter position="LOGOUT_FILTER" ref="myLogoutFilter" />

<security:custom-filter position="FORM_LOGIN_FILTER" ref="myAuthenticationFilter" />

<security:intercept-url pattern="/services/**" access="isFullyAuthenticated()" />

</security:http>

References

• infinispan.org• blog.infinispan.org• infinispan-ko.blogspot.com• facebook.com/groups/infinispan• red.ht/data-grid• coherence.oracle.com

감사합니다.