Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion

Rob Brooks-Bilson | @styggiti | rob.brooks-bilson.com
cf.Objective() 2014

About Me
 Sr. Director at Amkor Technology
 Involved with ColdFusion for 18
years
 Author of the O’Reilly’s books:
Programming ColdFusion
Programming ColdFusion MX
 Adobe Community Professional
for ColdFusion
 Food Geek
 Bed Jumper
 Photographer

Agenda
What We’ll Cover And What We Wont…
 What is caching?
 What’s Ehcache?
 Caching tags and functions
 Any questions from Dan Wilson
 Disclaimer
 Caching Evolution in ColdFusion
 Scale Up vs. Scale Out
 Caching Architecture Review
 Replicated Cache
 BigMemory Go
 Distributed Cache
 Terracotta Management Console

Legal Mumbo Jumbo: The Disclaimer
 Much of what you’re about to see makes use of third-party libraries and
applications
 Some of it requires you to upgrade versions of libraries that ship with
ColdFusion
 There are bugs
 There are workarounds
 There is no official support from Adobe for much of what you’re about to
see

Ehcache Implementation : Caching Evolution
Core cache
Ehcache 1.6
9.0
 Page caching
 Fragment caching
 Caching objects and data
 ORM caching
 Improved Hibernate plug-in,
management, XA transactions,
write-behind, bulk load, …
 Snap-in support for
distributed caching, via easy
upgrade to Enterprise
Ehcache
Ehcache 2.0
9.1
 New and enhanced functions:
cacheGetSession,
cacheGetMetadata
 Support for new configuration
properties
 User-defined caches
 copyOnRead &copyOnWrite,
Explicit locking API, Nonstop
Cache, Cache Search
 New consistency modes:
Strong and Eventual
 Automatic Resource Control
Ehcache 2.5.1
10.0
 Application level caching
 Enhanced query cache
 Six New functions
 Cache region attribute in all
functions

Ehcache Implementation : Caching Evolution
 Fast restartability
 Terracotta Management
Console
Ehcache 2.6.6
11.0
 Bug fixes
 cacheRemove now takes an
array for ID
Ehcache 2.8.2
Current Version

Scale Up vs. Scale Out
ScaleUp
Scale Out
What about cost?
• Hardware
• Data center space
• Power
• Redundancy
• Software Licensing*
• Operations & Sysadmin
How does this change in the cloud?
More CPU
More RAM
More Machines

Caching Architectures
In-Process (L1 Cache)
 Operates in the same process
(JVM) as the application server
 Limited scalability for 32-bit
systems
 Must consider possible number of
cache variants
 Max JVM heap size
 Can fail-over to disk storage
 Fast
 Data/object serialization is not
required for memory based in-process
cache
JVM
ColdFusion
Cache
In-Process Cache

Caching Architectures
Out-of-Process (L2 Cache)
 Operates in its own process,
outside of the application
server’s JVM
 Highly scalable on both 32-bit
and 64-bit platforms
 Scale out
 Utilize spare memory throughout
the network
 Slower than in-process caching
 Data/objects must be
serialized/deserialized
JVM
ColdFusion
Cache
JVM
Cache
Out-of-Process Cache
10

Technology
Ehcache
 Java caching API
 In-process
 Open Source /
Commercial
 Built-in to
ColdFusion
BigMemory Go
 Off-heap memory
storage for the JVM
 Stand Alone
 Terracotta
Management
Console
 Commercial
Terracotta
 Distributed in-
memory caching
array
 Out-of-process
 Open Source no
longer supported
as of version 3.7.7
(Pairs with
Ehcache 2.6.8)
 Commercial: Part
of BigMemory
Max
BigMemory Max
 Off-heap memory
storage for the JVM
 Distributed
 Terracotta
Management
Console
 Commercial

Caching Architecture
Ehcache can be configured to run:
 Local: In-process (Ehcache [+
BigMemory Go])
 Replicated: In-process (Ehcache [+
BigMemory Go])
 Distributed: In-process and Out-of-
process (Ehcache + BigMemory Max)
ColdFusion
Server
Ehcache
ColdFusion
Server
Ehcache
JVM
JVM
ColdFusion
Server
Ehcache
ColdFusion
Server
Ehcache
JVM
JVM
RMI, JMS,
JGroups,
Terracotta
In-Process Cache
Replication
JVM
ColdFusion
Cache
In-Process Cache
JVM
ColdFusion
Cache
JVM
Cache
Out-of-Process Cache

To Help Avoid Confusion…
 Terracotta recently retired the Terracotta
Server and discontinued the free versions of
BigMemory Go and BigMemory Max
 To avoid confusion, keep this in mind:
 Ehcache = In-Process Java Caching API – Comes
built-in with ColdFusion
 BigMemory Go = Enterprise Ehcache + Off-Heap
Memory Storage for the JVM on a single server -
Commercial (90 day trial)
 BigMemory Max = Enterprise Ehcache + Off-Heap
Memory Storage for the JVM + Distributed In-
Memory Caching via the Terracotta Server Array –
Commercial (90 day trial)

Still Confused?
BigMemory Go BigMemory Max
Distributed in-memory data store No (standalone JVM) Yes
In-memory capacity Unlimited (limit based on license) Unlimited (limit based on license)
Number of BigMemory mirrored stripes supported N/A Unlimited (limit based on license)
Number of application clients supported N/A Unlimited (limit based on license)
Fast search Yes Yes
Configurable data consistency guarantees N/A Yes
Terracotta Management Console™ Yes Yes
Fast Restartable Store™ Yes Yes
BigMemory-Hadoop Connector support Yes
In-process off-heap storage Yes Yes
Advanced security (authentication/authorization) N/A Yes
Enterprise support Yes Yes

Scalability: Cache Replication (Clustering)
 All supported cache types can be
replicated
 Each CF server in the cluster
maintains its own in-process cache
 Data is replicated among L1 caches
 Replication via RMI, JMS, JGroups
or Terracotta
 Synchronous or asynchronous
 Still hit JVM GC issues/limits
 Simple configuration via
ehcache.xml file
ColdFusion
Server
Ehcache
ColdFusion
Server
Ehcache
JVM
JVM
ColdFusion
Server
Ehcache
ColdFusion
Server
Ehcache
JVM
JVM
RMI, JMS,
JGroups,
Terracotta
In-Process Cache
Replication
20

Potential Replication Gotchas
 Synchronous vs. Asynchronous delivery
 Asynchronous replication is the fastest method
 Because it’s asynchronous the caller returns immediately
 Messages are placed in a queue and batched via RMI as they are processed
 Potential for data inconsistency exists
 Synchronous
 Removes potential for data inconsistency
 Slower operation as caller waits for replication to complete before returning
 Time To Idle
 Inconsistent with replicated caching
 Data on some nodes will live longer than on others due to cache usage patterns
 Do not use unless you don’t care about inconsistent data across cache nodes

Replicated Cache Config Per Node – Step 1
 Stop your ColdFusion Server
 /lib/{servername}/ehcache.xml
 Uncomment:
<cacheManagerPeerProviderFactory
class="net.sf.ehcache.distribution.RMICacheManagerPeerProviderFactory"
properties="peerDiscovery=automatic,
multicastGroupAddress=230.0.0.1,
multicastGroupPort=4446, timeToLive=1"
propertySeparator=","
/>

Replicated Cache Config Per Node – Step 2
<cacheManagerPeerListenerFactory
class="net.sf.ehcache.distribution.RMICacheManagerPeerListenerFactory"
properties="hostName=www1.foo.com, socketTimeoutMillis=120000"
propertySeparator=","
/>
<cache
name="myReplicatedCache"
maxElementsInMemory="5000"
eternal="false">
<cacheEventListenerFactory
class="net.sf.ehcache.distribution.RMICacheReplicatorFactory“
/>
</cache>

Replicated Cache
It’s Really That Easy!
*Don’t forget to restart ColdFusion

Problems with Application Scalability on the JVM
 Slow Applications lead to caching
 Large caches lead to latency from Garbage
Collection pauses
 GC pauses necessitate JVM tuning
 Application usage and data grows
 More caching is needed
 More JVM tuning is needed
 As cache size increases, more heap memory is used
 More heap memory usage results in unpredictable
and longer GC pauses
 Eventually you hit a wall
 Add more JVMs
 Increased deployment and management
complexity
 Tune! Tune! Tune!

The Problem with Memory and the JVM
 RAM is outpacing the JVM
 32GB is now fairly standard on most servers
 Amazon EC2 allows for up to 68.4GB
 Most of that memory is used inefficiently if it’s used at all
 Cached data ages differently than standard business objects
which can confuse the Java garbage collector
 The cache expiration determines when the data becomes garbage
 And then there's tuning…

GC Tuning – Need I Say More?

BigMemory to the Rescue
 Cache huge amounts of data with no GC pauses
 Easy to implement
 Pure Java implementation
 Works with all JVMs
 Works with both standalone and distributed caches
 No application code changes necessary to implement
 Avoid GC because of pauses
 Off heap store on direct memory buffers
 Use all available memory on your server (if you want to)
 1 million puts per second
 1 line of config to use it
 JVM doesn’t have to search for garbage because we already know when a cache item
needs to be thrown out (cache expiry)

Tiered Memory Store and BigMemory Architecture

Upgrade Ehcache and Install BigMemory Go
 BigMemory Go requires a newer
version of Ehcache
 Adobe doesn’t officially support
upgrading. Do so at your own risk!
 There’s currently a bug you need to
be aware of in CF 10 (as of updater
11):
 3339491: After upgrading Ehcache,
ehcache.xml is ignored. You’ll need to use
auth-ehcache.xml for now. (Marked as Fixed,
but not yet deployed in a hotfix)
60

Upgrade Ehcache and Install Big Memory: Step 1
 Stop your ColdFusion server
 Rename the following files in your servername/lib directory:
 ehcache-core-2.5.1.jar  ehcache-core-2.5.1.jar.original
 slf4j-api-1.5.6.jar slf4j-api-1.5.6.jar.original
 slf4j-log4j12-1.5.6.jar  slf4j-log4j12-1.5.6.jar.old

 Download BigMemory Trial (It will give you Max even if you select Go):
 http://terracotta.org/products/bigmemorygo
 Extract the following files into your servername/lib directory:
From /apis/ehcache/lib
ehcache-ee-<version>.jar
slf4j-api-<version>.jar
From /common/lib
bigmemory-<version>.jar
 Copy the terracotta-license.key file you received by email after downloading
BigMemory into your servername/lib directory

 Download the Slf4j version bundle that matches the version in your
BigMemory download from http://www.slf4j.org/dist/
 Extract the slf4j-log4j12-<version>.jar into your servername/lib directory

ehcache.xml (auth-cache.xml):
<cache name="sample-offheap-cache"
eternal="true"
memoryStoreEvictionPolicy="LRU"
overflowToOffHeap="true"
maxMemoryOffHeap="2G"/>
Start your ColdFusion server
JVM Config in ColdFusion Admin:
-XX:MaxDirectMemorySize=3G
Restart your ColdFusion server

ColdFusion Memory Footprint w/ BigMemory Go

Distributed Caching with BigMemory Max

Scalability: Distributed Tiered Caching with BigMemory Max

Scalability: Distributed Caching (Commercial)
 Commercial license
 Runs out-of-process
 Tiered caching
 Hottest data in L1 (< 1 μs access)
 Everything in L2 (< 2 ms access)
 "Snap-in scale"
 Works with single node Ehcache or
replicated Ehcache
 Simple config via ehcache.xml
 High data capacity: 2TB+
 Highly available
Enterprise
Ehcache
Terracotta
Server
Array
Terracotta
Server Array
ColdFusion
Enterprise
Ehcache
ColdFusion
…
Terracotta
Server
Array
Terracotta
Server Array

• Stop your ColdFusion server
• Rename the following files in your
servername/lib directory:
ehcache-core-2.5.1.jar  ehcache-core-
2.5.1.jar.original
slf4j-api-1.5.6.jar slf4j-api-1.5.6.jar.old
slf4j-log4j12-1.5.6.jar  slf4j-log4j12-1.5.6.jar.old
• Add the following jar files:
From apis/ehcache/lib
ehcache-ee-<version>.jar
slf4j-api-<version>.jar
From apis/toolkit/lib
terracotta-toolkit-runtime-ee-<version>.jar
Ehcache
Terracotta
Server
ColdFusion
Ehcache
ColdFusion
…
Distributed Cache Configuration: Step 1/3

Edit your ehcache.xml
<ehcache>
<terracottaConfig url="someserver:9510"/>
<defaultCache
timeToLiveSeconds="120“
overflowToOffHeap="true"
maxMemoryOffHeap="2G">
<terracotta clustered="true">
</defaultCache>
</ehcache>
Ehcache
Terracotta
Server
ColdFusion
Ehcache
ColdFusion
…

Start ColdFusion
JVM Config in ColdFusion Admin:
-X:MaxDirectMemorySize=3G
Stop ColdFusion
Start the Terracotta server, then restart
ColdFusion
bin/start-tc-server.sh
bin/start-tc-server.bat
Ehcache
Terracotta
Server
ColdFusion
Ehcache
ColdFusion
…

Terracotta Gotchas
 The Terracotta server must be running before you bring up your ColdFusion
server, otherwise your page request (that tries to do a cache operation) will
spin until the connection times out and an error is returned
 Terracotta can only be used with ColdFusion Enterprise. There’s a bug if you
try to use it with the Developer Edition where ColdFusion will return an error
telling you it’s an Enterprise Only feature.

Installing the Terracotta Management Console
1. Ensure JAVA_HOME is set
2. Extract the BigMemory go files
3. Start the monitor server: /management-console/lib/start-tmc.bat
4. Configure ehcache (auth_cache.xml) and add monitoring line:
<managementRESTService enabled="true" bind="0.0.0.0:9888" />
5. Open the web browser to http://localhost:9889/tmc

Q&A
Rob Brooks-Bilson
Senior Director
1900 South Price Road
Chandler, AZ 85286
Tel (480) 786-7748
Fax (480) 821-9777
rbils@amkor.com
Twitter: @styggiti

Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion (20)

More from ColdFusionConference (20)

Recently uploaded (20)

Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion