Transcript
Page 1: Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion

Rob Brooks-Bilson | @styggiti | rob.brooks-bilson.com

cf.Objective() 2014

Page 2: Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion

About Me

Sr. Director at Amkor Technology

Involved with ColdFusion for 18 years

Author of the O’Reilly’s books: Programming ColdFusion Programming ColdFusion MX

Adobe Community Professional for ColdFusion

Food Geek

Bed Jumper

Photographer

Page 3: Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion

More ColdFusion Goodness

Page 4: Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion

Agenda

What We’ll Cover And What We Wont…

What is caching?

What’s Ehcache?

Caching tags and functions

Any questions from Dan Wilson

Disclaimer

Caching Evolution in ColdFusion

Scale Up vs. Scale Out

Caching Architecture Review

Replicated Cache

BigMemory Go

Distributed Cache

Terracotta Management Console

Page 5: Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion

Legal Mumbo Jumbo: The Disclaimer

Much of what you’re about to see makes use of third-party libraries and applications

Some of it requires you to upgrade versions of libraries that ship with ColdFusion

There are bugs

There are workarounds

There is no official support from Adobe for much of what you’re about to see

Page 6: Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion

Ehcache Implementation : Caching Evolution

Core cache

Ehcache 1.6

9.0 Page caching

Fragment caching

Caching objects and data

ORM caching

Improved Hibernate plug-in, management, XA transactions, write-behind, bulk load, …

Snap-in support for distributed caching, via easy upgrade to Enterprise Ehcache

Ehcache 2.0

9.1 New and enhanced functions:

cacheGetSession, cacheGetMetadata

Support for new configuration properties

User-defined caches

copyOnRead &copyOnWrite, Explicit locking API, Nonstop Cache, Cache Search

New consistency modes: Strong and Eventual

Automatic Resource Control

Ehcache 2.5.1

10.0 Application level caching

Enhanced query cache

Six New functions

Cache region attribute in all functions

Presenter
Presentation Notes
Ehcache 2.8.2 is the current version as of this presentation (5/2014)
Page 7: Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion

Ehcache Implementation : Caching Evolution

Fast restartability

Terracotta Management Console

Ehcache 2.6.6

11.0 Bug fixes

cacheRemove now takes an array for ID

Ehcache 2.8.2

Current Version

Presenter
Presentation Notes
Ehcache 2.8.2 is the current version as of this presentation (5/2014)
Page 8: Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion

Scale Up vs. Scale Out

Scal

e U

p

Scale Out

What about cost? • Hardware • Data center space • Power • Redundancy • Software Licensing* • Operations & Sysadmin

How does this change in the cloud?

More CPU More RAM

More Machines

Page 9: Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion

Caching Architectures

In-Process (L1 Cache)

Operates in the same process (JVM) as the application server

Limited scalability for 32-bit systems

Must consider possible number of cache variants

Max JVM heap size

Can fail-over to disk storage

Fast

Data/object serialization is not required for memory based in-process cache

JVM

ColdFusion

Cache

In-Process Cache

Presenter
Presentation Notes
http://gregluck.com/blog/archives/2007/05/comparing_memca.html
Page 10: Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion

Caching Architectures

Out-of-Process (L2 Cache)

Operates in its own process, outside of the application server’s JVM

Highly scalable on both 32-bit and 64-bit platforms

Scale out

Utilize spare memory throughout the network

Slower than in-process caching

Data/objects must be serialized/deserialized

JVM

ColdFusion

Cache

JVM

Cache

Out-of-Process Cache

10

Page 11: Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion

Technology

Ehcache

Java caching API

In-process

Open Source / Commercial

Built-in to ColdFusion

BigMemory Go

Off-heap memory storage for the JVM

Stand Alone

Terracotta Management Console

Commercial

Terracotta

Distributed in-memory caching array

Out-of-process

Open Source no longer supported as of version 3.7.7 (Pairs with Ehcache 2.6.8)

Commercial: Part of BigMemory Max

BigMemory Max

Off-heap memory storage for the JVM

Distributed

Terracotta Management Console

Commercial

Page 12: Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion

Caching Architecture

Ehcache can be configured to run: Local: In-process (Ehcache [+

BigMemory Go])

Replicated: In-process (Ehcache [+ BigMemory Go])

Distributed: In-process and Out-of-process (Ehcache + BigMemory Max)

ColdFusion Server

Ehcache

ColdFusion Server

Ehcache

JVM

JVM

ColdFusion Server

Ehcache

ColdFusion Server

Ehcache

JVM

JVM

RMI, JMS,JGroups,Terracotta

In-Process Cache Replication

JVM

ColdFusion

Cache

In-Process Cache

JVM

ColdFusion

Cache

JVM

Cache

Out-of-Process Cache

Page 13: Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion

To Help Avoid Confusion…

Terracotta recently retired the Terracotta Server and discontinued the free versions of BigMemory Go and BigMemory Max

To avoid confusion, keep this in mind: Ehcache = In-Process Java Caching API – Comes

built-in with ColdFusion

BigMemory Go = Enterprise Ehcache + Off-Heap Memory Storage for the JVM on a single server - Commercial (90 day trial)

BigMemory Max = Enterprise Ehcache + Off-Heap Memory Storage for the JVM + Distributed In-Memory Caching via the Terracotta Server Array – Commercial (90 day trial)

Page 14: Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion

Still Confused?

BigMemory Go BigMemory Max

Distributed in-memory data store No (standalone JVM) Yes

In-memory capacity Unlimited (limit based on license) Unlimited (limit based on license)

Number of BigMemory mirrored stripes supported N/A Unlimited (limit based on license)

Number of application clients supported N/A Unlimited (limit based on license)

Fast search Yes Yes

Configurable data consistency guarantees N/A Yes

Terracotta Management Console™ Yes Yes

Fast Restartable Store™ Yes Yes

BigMemory-Hadoop Connector support Yes

In-process off-heap storage Yes Yes

Advanced security (authentication/authorization) N/A Yes

Enterprise support Yes Yes

Page 15: Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion

Cache Replication

Page 16: Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion

Scalability: Cache Replication (Clustering)

All supported cache types can be replicated

Each CF server in the cluster maintains its own in-process cache

Data is replicated among L1 caches

Replication via RMI, JMS, JGroups or Terracotta

Synchronous or asynchronous

Still hit JVM GC issues/limits

Simple configuration via ehcache.xml file

ColdFusion Server

Ehcache

ColdFusion Server

Ehcache

JVM

JVM

ColdFusion Server

Ehcache

ColdFusion Server

Ehcache

JVM

JVM

RMI, JMS,JGroups,Terracotta

In-Process Cache Replication

20

Page 17: Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion

Potential Replication Gotchas

Synchronous vs. Asynchronous delivery Asynchronous replication is the fastest method

Because it’s asynchronous the caller returns immediately

Messages are placed in a queue and batched via RMI as they are processed

Potential for data inconsistency exists

Synchronous

Removes potential for data inconsistency

Slower operation as caller waits for replication to complete before returning

Time To Idle Inconsistent with replicated caching

Data on some nodes will live longer than on others due to cache usage patterns

Do not use unless you don’t care about inconsistent data across cache nodes

Page 18: Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion

Replicated Cache Config Per Node – Step 1

Stop your ColdFusion Server

/lib/{servername}/ehcache.xml

Uncomment: <cacheManagerPeerProviderFactory

class="net.sf.ehcache.distribution.RMICacheManagerPeerProviderFactory" properties="peerDiscovery=automatic, multicastGroupAddress=230.0.0.1, multicastGroupPort=4446, timeToLive=1" propertySeparator="," />

Page 19: Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion

Replicated Cache Config Per Node – Step 2

<cacheManagerPeerListenerFactory class="net.sf.ehcache.distribution.RMICacheManagerPeerListenerFactory" properties="hostName=www1.foo.com, socketTimeoutMillis=120000" propertySeparator="," /> <cache name="myReplicatedCache" maxElementsInMemory="5000" eternal="false"> <cacheEventListenerFactory class="net.sf.ehcache.distribution.RMICacheReplicatorFactory“ /> </cache>

Page 20: Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion

Replicated Cache

It’s Really That Easy! *Don’t forget to restart ColdFusion

Page 21: Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion

BigMemory Go

Page 22: Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion

Problems with Application Scalability on the JVM

Slow Applications lead to caching

Large caches lead to latency from Garbage Collection pauses

GC pauses necessitate JVM tuning

Application usage and data grows

More caching is needed

More JVM tuning is needed

As cache size increases, more heap memory is used

More heap memory usage results in unpredictable and longer GC pauses

Eventually you hit a wall

Add more JVMs

Increased deployment and management complexity

Tune! Tune! Tune!

Presenter
Presentation Notes
Image courtesy of Steven Harris: http://dsoguy.blogspot.com/2010/09/little-bit-about-bigmemory-for-ehcache.html
Page 23: Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion

The Problem with Memory and the JVM

RAM is outpacing the JVM

32GB is now fairly standard on most servers

Amazon EC2 allows for up to 68.4GB

Most of that memory is used inefficiently if it’s used at all

Cached data ages differently than standard business objects which can confuse the Java garbage collector

The cache expiration determines when the data becomes garbage

And then there's tuning…

Page 24: Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion

GC Tuning – Need I Say More?

Page 25: Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion

BigMemory to the Rescue

Cache huge amounts of data with no GC pauses

Easy to implement Pure Java implementation

Works with all JVMs

Works with both standalone and distributed caches

No application code changes necessary to implement

Avoid GC because of pauses Off heap store on direct memory buffers

Use all available memory on your server (if you want to)

1 million puts per second

1 line of config to use it

JVM doesn’t have to search for garbage because we already know when a cache item needs to be thrown out (cache expiry)

Page 26: Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion

Tiered Memory Store and BigMemory Architecture

Page 27: Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion

Upgrade Ehcache and Install BigMemory Go

BigMemory Go requires a newer version of Ehcache

Adobe doesn’t officially support upgrading. Do so at your own risk!

There’s currently a bug you need to be aware of in CF 10 (as of updater 11): 3339491: After upgrading Ehcache,

ehcache.xml is ignored. You’ll need to use auth-ehcache.xml for now. (Marked as Fixed, but not yet deployed in a hotfix)

60

Page 28: Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion

Upgrade Ehcache and Install Big Memory: Step 1

Stop your ColdFusion server

Rename the following files in your servername/lib directory: ehcache-core-2.5.1.jar ehcache-core-2.5.1.jar.original

slf4j-api-1.5.6.jar slf4j-api-1.5.6.jar.original

slf4j-log4j12-1.5.6.jar slf4j-log4j12-1.5.6.jar.old

Page 29: Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion

Upgrade Ehcache and Install Big Memory: Step 2

Download BigMemory Trial (It will give you Max even if you select Go): http://terracotta.org/products/bigmemorygo

Extract the following files into your servername/lib directory: From /apis/ehcache/lib

ehcache-ee-<version>.jar slf4j-api-<version>.jar

From /common/lib bigmemory-<version>.jar

Copy the terracotta-license.key file you received by email after downloading BigMemory into your servername/lib directory

Page 30: Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion

Upgrade Ehcache and Install Big Memory: Step 3

Download the Slf4j version bundle that matches the version in your BigMemory download from http://www.slf4j.org/dist/

Extract the slf4j-log4j12-<version>.jar into your servername/lib directory

Page 31: Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion

Upgrade Ehcache and Install Big Memory: Step 4

ehcache.xml (auth-cache.xml): <cache name="sample-offheap-cache" maxElementsInMemory="100000" eternal="true" memoryStoreEvictionPolicy="LRU" overflowToOffHeap="true" maxMemoryOffHeap="2G"/>

Start your ColdFusion server

JVM Config in ColdFusion Admin: -XX:MaxDirectMemorySize=3G

Restart your ColdFusion server

Page 32: Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion

ColdFusion Memory Footprint w/ BigMemory Go

Page 33: Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion

Distributed Caching with BigMemory Max

Page 34: Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion

Scalability: Distributed Tiered Caching with BigMemory Max

Page 35: Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion

BigMemory Max

Page 36: Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion

Scalability: Distributed Caching (Commercial)

Commercial license

Runs out-of-process

Tiered caching

Hottest data in L1 (< 1 μs access)

Everything in L2 (< 2 ms access)

"Snap-in scale"

Works with single node Ehcache or replicated Ehcache

Simple config via ehcache.xml

High data capacity: 2TB+

Highly available

Enterprise Ehcache

Terracotta Server Array

Terracotta Server Array

ColdFusion

Enterprise Ehcache

ColdFusion

Terracotta Server Array

Terracotta Server Array

Page 37: Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion

• Stop your ColdFusion server • Rename the following files in your

servername/lib directory: ehcache-core-2.5.1.jar ehcache-core-2.5.1.jar.original slf4j-api-1.5.6.jar slf4j-api-1.5.6.jar.old slf4j-log4j12-1.5.6.jar slf4j-log4j12-1.5.6.jar.old

• Add the following jar files: From apis/ehcache/lib

ehcache-ee-<version>.jar slf4j-api-<version>.jar

From apis/toolkit/lib

terracotta-toolkit-runtime-ee-<version>.jar

Ehcache

Terracotta Server

ColdFusion

Ehcache

ColdFusion

Distributed Cache Configuration: Step 1/3

Page 38: Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion

Edit your ehcache.xml <ehcache>

<terracottaConfig url="someserver:9510"/>

<defaultCache maxElementsInMemory="10000" timeToLiveSeconds="120“

overflowToOffHeap="true" maxMemoryOffHeap="2G">

<terracotta clustered="true">

</defaultCache> </ehcache>

Ehcache

Terracotta Server

ColdFusion

Ehcache

ColdFusion

Distributed Cache Configuration: Step 2/3

Page 39: Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion

Start ColdFusion JVM Config in ColdFusion Admin:

-X:MaxDirectMemorySize=3G

Stop ColdFusion

Start the Terracotta server, then restart ColdFusion

bin/start-tc-server.sh bin/start-tc-server.bat

Ehcache

Terracotta Server

ColdFusion

Ehcache

ColdFusion

Distributed Cache Configuration: Step 3/3

Page 40: Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion

Terracotta Gotchas

The Terracotta server must be running before you bring up your ColdFusion server, otherwise your page request (that tries to do a cache operation) will spin until the connection times out and an error is returned

Terracotta can only be used with ColdFusion Enterprise. There’s a bug if you try to use it with the Developer Edition where ColdFusion will return an error telling you it’s an Enterprise Only feature.

Page 41: Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion

Cache Monitoring

Page 42: Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion

Installing the Terracotta Management Console

1. Ensure JAVA_HOME is set

2. Extract the BigMemory go files

3. Start the monitor server: /management-console/lib/start-tmc.bat

4. Configure ehcache (auth_cache.xml) and add monitoring line: <managementRESTService enabled="true" bind="0.0.0.0:9888" />

5. Open the web browser to http://localhost:9889/tmc

Page 43: Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion

Terracotta Management Console

Page 44: Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion

Q&A

Rob Brooks-Bilson

Senior Director

1900 South Price Road Chandler, AZ 85286

Tel (480) 786-7748 Fax (480) 821-9777 [email protected] Twitter: @styggiti


Top Related