Rob Brooks-Bilson | @styggiti | rob.brooks-bilson.com
cf.Objective() 2014
About Me
Sr. Director at Amkor Technology
Involved with ColdFusion for 18 years
Author of the O’Reilly’s books: Programming ColdFusion Programming ColdFusion MX
Adobe Community Professional for ColdFusion
Food Geek
Bed Jumper
Photographer
More ColdFusion Goodness
Agenda
What We’ll Cover And What We Wont…
What is caching?
What’s Ehcache?
Caching tags and functions
Any questions from Dan Wilson
Disclaimer
Caching Evolution in ColdFusion
Scale Up vs. Scale Out
Caching Architecture Review
Replicated Cache
BigMemory Go
Distributed Cache
Terracotta Management Console
Legal Mumbo Jumbo: The Disclaimer
Much of what you’re about to see makes use of third-party libraries and applications
Some of it requires you to upgrade versions of libraries that ship with ColdFusion
There are bugs
There are workarounds
There is no official support from Adobe for much of what you’re about to see
Ehcache Implementation : Caching Evolution
Core cache
Ehcache 1.6
9.0 Page caching
Fragment caching
Caching objects and data
ORM caching
Improved Hibernate plug-in, management, XA transactions, write-behind, bulk load, …
Snap-in support for distributed caching, via easy upgrade to Enterprise Ehcache
Ehcache 2.0
9.1 New and enhanced functions:
cacheGetSession, cacheGetMetadata
Support for new configuration properties
User-defined caches
copyOnRead ©OnWrite, Explicit locking API, Nonstop Cache, Cache Search
New consistency modes: Strong and Eventual
Automatic Resource Control
Ehcache 2.5.1
10.0 Application level caching
Enhanced query cache
Six New functions
Cache region attribute in all functions
Ehcache Implementation : Caching Evolution
Fast restartability
Terracotta Management Console
Ehcache 2.6.6
11.0 Bug fixes
cacheRemove now takes an array for ID
Ehcache 2.8.2
Current Version
Scale Up vs. Scale Out
Scal
e U
p
Scale Out
What about cost? • Hardware • Data center space • Power • Redundancy • Software Licensing* • Operations & Sysadmin
How does this change in the cloud?
More CPU More RAM
More Machines
Caching Architectures
In-Process (L1 Cache)
Operates in the same process (JVM) as the application server
Limited scalability for 32-bit systems
Must consider possible number of cache variants
Max JVM heap size
Can fail-over to disk storage
Fast
Data/object serialization is not required for memory based in-process cache
JVM
ColdFusion
Cache
In-Process Cache
Caching Architectures
Out-of-Process (L2 Cache)
Operates in its own process, outside of the application server’s JVM
Highly scalable on both 32-bit and 64-bit platforms
Scale out
Utilize spare memory throughout the network
Slower than in-process caching
Data/objects must be serialized/deserialized
JVM
ColdFusion
Cache
JVM
Cache
Out-of-Process Cache
10
Technology
Ehcache
Java caching API
In-process
Open Source / Commercial
Built-in to ColdFusion
BigMemory Go
Off-heap memory storage for the JVM
Stand Alone
Terracotta Management Console
Commercial
Terracotta
Distributed in-memory caching array
Out-of-process
Open Source no longer supported as of version 3.7.7 (Pairs with Ehcache 2.6.8)
Commercial: Part of BigMemory Max
BigMemory Max
Off-heap memory storage for the JVM
Distributed
Terracotta Management Console
Commercial
Caching Architecture
Ehcache can be configured to run: Local: In-process (Ehcache [+
BigMemory Go])
Replicated: In-process (Ehcache [+ BigMemory Go])
Distributed: In-process and Out-of-process (Ehcache + BigMemory Max)
ColdFusion Server
Ehcache
ColdFusion Server
Ehcache
JVM
JVM
ColdFusion Server
Ehcache
ColdFusion Server
Ehcache
JVM
JVM
RMI, JMS,JGroups,Terracotta
In-Process Cache Replication
JVM
ColdFusion
Cache
In-Process Cache
JVM
ColdFusion
Cache
JVM
Cache
Out-of-Process Cache
To Help Avoid Confusion…
Terracotta recently retired the Terracotta Server and discontinued the free versions of BigMemory Go and BigMemory Max
To avoid confusion, keep this in mind: Ehcache = In-Process Java Caching API – Comes
built-in with ColdFusion
BigMemory Go = Enterprise Ehcache + Off-Heap Memory Storage for the JVM on a single server - Commercial (90 day trial)
BigMemory Max = Enterprise Ehcache + Off-Heap Memory Storage for the JVM + Distributed In-Memory Caching via the Terracotta Server Array – Commercial (90 day trial)
Still Confused?
BigMemory Go BigMemory Max
Distributed in-memory data store No (standalone JVM) Yes
In-memory capacity Unlimited (limit based on license) Unlimited (limit based on license)
Number of BigMemory mirrored stripes supported N/A Unlimited (limit based on license)
Number of application clients supported N/A Unlimited (limit based on license)
Fast search Yes Yes
Configurable data consistency guarantees N/A Yes
Terracotta Management Console™ Yes Yes
Fast Restartable Store™ Yes Yes
BigMemory-Hadoop Connector support Yes
In-process off-heap storage Yes Yes
Advanced security (authentication/authorization) N/A Yes
Enterprise support Yes Yes
Cache Replication
Scalability: Cache Replication (Clustering)
All supported cache types can be replicated
Each CF server in the cluster maintains its own in-process cache
Data is replicated among L1 caches
Replication via RMI, JMS, JGroups or Terracotta
Synchronous or asynchronous
Still hit JVM GC issues/limits
Simple configuration via ehcache.xml file
ColdFusion Server
Ehcache
ColdFusion Server
Ehcache
JVM
JVM
ColdFusion Server
Ehcache
ColdFusion Server
Ehcache
JVM
JVM
RMI, JMS,JGroups,Terracotta
In-Process Cache Replication
20
Potential Replication Gotchas
Synchronous vs. Asynchronous delivery Asynchronous replication is the fastest method
Because it’s asynchronous the caller returns immediately
Messages are placed in a queue and batched via RMI as they are processed
Potential for data inconsistency exists
Synchronous
Removes potential for data inconsistency
Slower operation as caller waits for replication to complete before returning
Time To Idle Inconsistent with replicated caching
Data on some nodes will live longer than on others due to cache usage patterns
Do not use unless you don’t care about inconsistent data across cache nodes
Replicated Cache Config Per Node – Step 1
Stop your ColdFusion Server
/lib/{servername}/ehcache.xml
Uncomment: <cacheManagerPeerProviderFactory
class="net.sf.ehcache.distribution.RMICacheManagerPeerProviderFactory" properties="peerDiscovery=automatic, multicastGroupAddress=230.0.0.1, multicastGroupPort=4446, timeToLive=1" propertySeparator="," />
Replicated Cache Config Per Node – Step 2
<cacheManagerPeerListenerFactory class="net.sf.ehcache.distribution.RMICacheManagerPeerListenerFactory" properties="hostName=www1.foo.com, socketTimeoutMillis=120000" propertySeparator="," /> <cache name="myReplicatedCache" maxElementsInMemory="5000" eternal="false"> <cacheEventListenerFactory class="net.sf.ehcache.distribution.RMICacheReplicatorFactory“ /> </cache>
Replicated Cache
It’s Really That Easy! *Don’t forget to restart ColdFusion
BigMemory Go
Problems with Application Scalability on the JVM
Slow Applications lead to caching
Large caches lead to latency from Garbage Collection pauses
GC pauses necessitate JVM tuning
Application usage and data grows
More caching is needed
More JVM tuning is needed
As cache size increases, more heap memory is used
More heap memory usage results in unpredictable and longer GC pauses
Eventually you hit a wall
Add more JVMs
Increased deployment and management complexity
Tune! Tune! Tune!
The Problem with Memory and the JVM
RAM is outpacing the JVM
32GB is now fairly standard on most servers
Amazon EC2 allows for up to 68.4GB
Most of that memory is used inefficiently if it’s used at all
Cached data ages differently than standard business objects which can confuse the Java garbage collector
The cache expiration determines when the data becomes garbage
And then there's tuning…
GC Tuning – Need I Say More?
BigMemory to the Rescue
Cache huge amounts of data with no GC pauses
Easy to implement Pure Java implementation
Works with all JVMs
Works with both standalone and distributed caches
No application code changes necessary to implement
Avoid GC because of pauses Off heap store on direct memory buffers
Use all available memory on your server (if you want to)
1 million puts per second
1 line of config to use it
JVM doesn’t have to search for garbage because we already know when a cache item needs to be thrown out (cache expiry)
Tiered Memory Store and BigMemory Architecture
Upgrade Ehcache and Install BigMemory Go
BigMemory Go requires a newer version of Ehcache
Adobe doesn’t officially support upgrading. Do so at your own risk!
There’s currently a bug you need to be aware of in CF 10 (as of updater 11): 3339491: After upgrading Ehcache,
ehcache.xml is ignored. You’ll need to use auth-ehcache.xml for now. (Marked as Fixed, but not yet deployed in a hotfix)
60
Upgrade Ehcache and Install Big Memory: Step 1
Stop your ColdFusion server
Rename the following files in your servername/lib directory: ehcache-core-2.5.1.jar ehcache-core-2.5.1.jar.original
slf4j-api-1.5.6.jar slf4j-api-1.5.6.jar.original
slf4j-log4j12-1.5.6.jar slf4j-log4j12-1.5.6.jar.old
Upgrade Ehcache and Install Big Memory: Step 2
Download BigMemory Trial (It will give you Max even if you select Go): http://terracotta.org/products/bigmemorygo
Extract the following files into your servername/lib directory: From /apis/ehcache/lib
ehcache-ee-<version>.jar slf4j-api-<version>.jar
From /common/lib bigmemory-<version>.jar
Copy the terracotta-license.key file you received by email after downloading BigMemory into your servername/lib directory
Upgrade Ehcache and Install Big Memory: Step 3
Download the Slf4j version bundle that matches the version in your BigMemory download from http://www.slf4j.org/dist/
Extract the slf4j-log4j12-<version>.jar into your servername/lib directory
Upgrade Ehcache and Install Big Memory: Step 4
ehcache.xml (auth-cache.xml): <cache name="sample-offheap-cache" maxElementsInMemory="100000" eternal="true" memoryStoreEvictionPolicy="LRU" overflowToOffHeap="true" maxMemoryOffHeap="2G"/>
Start your ColdFusion server
JVM Config in ColdFusion Admin: -XX:MaxDirectMemorySize=3G
Restart your ColdFusion server
ColdFusion Memory Footprint w/ BigMemory Go
Distributed Caching with BigMemory Max
Scalability: Distributed Tiered Caching with BigMemory Max
BigMemory Max
Scalability: Distributed Caching (Commercial)
Commercial license
Runs out-of-process
Tiered caching
Hottest data in L1 (< 1 μs access)
Everything in L2 (< 2 ms access)
"Snap-in scale"
Works with single node Ehcache or replicated Ehcache
Simple config via ehcache.xml
High data capacity: 2TB+
Highly available
Enterprise Ehcache
Terracotta Server Array
Terracotta Server Array
ColdFusion
Enterprise Ehcache
ColdFusion
…
Terracotta Server Array
Terracotta Server Array
• Stop your ColdFusion server • Rename the following files in your
servername/lib directory: ehcache-core-2.5.1.jar ehcache-core-2.5.1.jar.original slf4j-api-1.5.6.jar slf4j-api-1.5.6.jar.old slf4j-log4j12-1.5.6.jar slf4j-log4j12-1.5.6.jar.old
• Add the following jar files: From apis/ehcache/lib
ehcache-ee-<version>.jar slf4j-api-<version>.jar
From apis/toolkit/lib
terracotta-toolkit-runtime-ee-<version>.jar
Ehcache
Terracotta Server
ColdFusion
Ehcache
ColdFusion
…
Distributed Cache Configuration: Step 1/3
Edit your ehcache.xml <ehcache>
<terracottaConfig url="someserver:9510"/>
<defaultCache maxElementsInMemory="10000" timeToLiveSeconds="120“
overflowToOffHeap="true" maxMemoryOffHeap="2G">
<terracotta clustered="true">
</defaultCache> </ehcache>
Ehcache
Terracotta Server
ColdFusion
Ehcache
ColdFusion
…
Distributed Cache Configuration: Step 2/3
Start ColdFusion JVM Config in ColdFusion Admin:
-X:MaxDirectMemorySize=3G
Stop ColdFusion
Start the Terracotta server, then restart ColdFusion
bin/start-tc-server.sh bin/start-tc-server.bat
Ehcache
Terracotta Server
ColdFusion
Ehcache
ColdFusion
…
Distributed Cache Configuration: Step 3/3
Terracotta Gotchas
The Terracotta server must be running before you bring up your ColdFusion server, otherwise your page request (that tries to do a cache operation) will spin until the connection times out and an error is returned
Terracotta can only be used with ColdFusion Enterprise. There’s a bug if you try to use it with the Developer Edition where ColdFusion will return an error telling you it’s an Enterprise Only feature.
Cache Monitoring
Installing the Terracotta Management Console
1. Ensure JAVA_HOME is set
2. Extract the BigMemory go files
3. Start the monitor server: /management-console/lib/start-tmc.bat
4. Configure ehcache (auth_cache.xml) and add monitoring line: <managementRESTService enabled="true" bind="0.0.0.0:9888" />
5. Open the web browser to http://localhost:9889/tmc
Terracotta Management Console
Q&A
Rob Brooks-Bilson
Senior Director
1900 South Price Road Chandler, AZ 85286
Tel (480) 786-7748 Fax (480) 821-9777 [email protected] Twitter: @styggiti