CQ5.x Maintenance Webinar 2013

© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
ADOBE
CUSTOMER
BROWN
BAG
SERIES
Andrew Khoury & Sham Hassan Chikkegowda
Customer Support Engineers, Adobe
For the best listening experience, we recommend using a
headset
TODAY’S TOPIC: AEM Monitoring and Maintenance

What’s Covered
1. Best practices for maintaining AEM instances.
2. Tips and tricks for proactive things to monitor.
3. Tips and tricks for reducing downtime when instances are corrupted.
4. Common pitfalls to avoid.

Topics covered
 Replication
DataStore Garbage Collection
Tar PM Optimization
Workflow




© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 4
Optimizing replication
Replication Tuning and
Monitoring

Monitoring Replication
• Replication can be monitored using JMX
• Each agent exposes an object
• The objects expose:
- If the agent is enabled
- If the queue is blocked, paused or invalid
- The number of entries in the queue
- Timing information
5

Configure a replication.log
6
Why?
• Keeps error.log from getting bloated
• Isolates replication errors and warnings
How?
• Add an Apache Sling Logging Logger:
- Log File: logs/replication.log
- Log Level: Info
- Logger: com.day.cq.replication
• Add an Apache Sling Logging Writer (for log rotation)
- Log File: logs/replication.log

Replication monitoring via
JMX And Log ConfigurationDEMONSTRATION

Replication: Initializing and Queuing
8
User activates
/content/geometrixx-outdoors/en/support
Durbo
package
/var/replication/data
Content is packaged
OSGi
event
is fired
Replication
AgentManager
handles the event
Sling Job Handler
queues/stores the job
slingevent:Job
/var/eventing/jobs

Replication: Processing Jobs
9
Sling
Job Handler
Process the job through the
correct TransportHandler
ERROR: Queue stuck
slingevent:Job
node
/var/eventing/jobs
Read job from queue

Replication: Troubleshooting
The queue can get stuck if:
• The Job Handler fails.
• Error while sending the data.
- Failure to create Durbo package from /var/replication/data
- Issue in class loading, network malfunction, etc.
• The receiving end fails to respond.
- Publish instance is experiencing high traffic or is unstable.
• The receiving end fails to import / receive the
replication during transport. For example:
- Failure to read the durbo package
- Failure to save/import the content
10

Optimizing Replication
Optimizations:
• Enable “File Storage” of replication durbo packages:
- Go to http://host:port/system/console/configMgr and login as admin
- Find “Adobe Granite Replication Content Factory” and click Edit
- Check the “replication.content.useFileStorage.name” checkbox and Save
• Set the connection timeout and socket timeout in the agent Extended
configuration to avoid socketConnect and socketRead hang situations.
- socketConnect can be 30 seconds = 30000
- socketRead timeout can be 10 minutes = 600000
11

12

Optimizations:
• In the “Triggers” tab of the replication agent:
- “No Status Update” - if you do not need a status to track that the replication happened.
- “No Versioning” - if you do not need versions of the pages or assets generated on
activations.
13

14

Reducing disk space usage
DataStore Garbage Collection

What is the DataStore?
The DataStore (default implementation):
• Stores binary JCR properties > 4KB (default).
• Only allows files to be added.
• Stores files under crx-quickstart/repository/repository/datastore.
• Files are referenced by the JCR internal node record (persistence manager
bundles).
16

How does the DataStore work?
1. A unique id is generated from contents of the file.
2. The id is used for the filename.
3. The datastore directory structure is built like this:
• datastore/{1st 2 chars id}/{2nd 2 chars id}/{3rd 2 chars id}/{id}
• E.g.
datastore/a2/b9/86/a2b98676377b947244bb954769def9b4fd625daa
4. Due to this, files are stored uniquely so the same file will only be stored
once.
17

DataStore Example
User uploads the same image to two different locations
18
AEM.png
Upload
Upload

a2b98676377b947244bb954769def9b4fd625daa
repository/repository/datastore
86
b9
a2
DataStore Example
19
JCR Node:
/content/dam/AEM.png/renditions/original/jcr:content/data property
JCR Node:
/content/dam/geometrixx-
media/AEM.png/renditions/original/jcr:content/data property

How does DataStore GC work?
DataStore GC (Garbage Collection) is the only way to delete DataStore files.
How does it work?
• “Scan” Phase
- Read all nodes in the JCR
- Update the timestamp of each of the referenced datastore files.
• “Delete” Phase
- Find all files with timestamp older than when it started the scan phase and deletes
them.
20

DataStore GC: Avoiding common Pitfalls
Before running DataStore GC, make sure:
• You have a backup of the datastore directory
• No issues with setting timestamps (this especially applies of your datastore is
on a network storage).
• Date and time is in sync between network storage and CQ server
• CQ5.3 or 5.4 with CRX2.2 - at least hotfix 2.2.0.70.
• CQ5.6.1 - do *NOT* use stopDataStoreGarbageCollection in JMX to stop the
GC process.
21

How to run DataStore GCDEMONSTRATION

DataStore GC
This error means you are missing a file from your datastore:
“DataStoreException: Record not found”
If you see that error then do not run datastore gc until you have restored the missing files.
To identify all missing files:
• Go to /crx/explorer/index.jsp and login as admin (/crx/index.jsp CQ5.4 and earlier).
• Browse to "Repository Configuration" => "Check Repository”.
• Check the "Data store consistency check" box.
• Click “Run”
• This will output a list of nodes referencing the missing files along with the path of the missing file.
• Since the files are unique, you can copy the missing files over from any other AEM (CQ) instances that
the same file.
See this article for more details:
• http://helpx.adobe.com/crx/kb/DataStoreGarbageCollection.html
23

How to run DataStore
Consistency Check
DEMONSTRATION

Tar Persistence Manager
Optimization

Part 2: Topics Covered
TarPM Optimization
- What is the TarPM & TarPM optimization?
- Best practices for maintaining AEM Instances
- Tips and tricks for proactive things to monitor
- Tips and tricks for reducing downtime & Speed up the process
Workflow Monitoring
- Supported Features
- Best practices for maintaining AEM Instances.
- Tips and tricks for proactive things to monitor
26

What is TarPM?
27
data_00000.tar
<cq_home>/crx-quickstart/repository/workspaces/crx.defau
data_00001.tar
Bundle 01
Bundle 02
Bundle 03
Bundle 04
• CRX Stores nodes and properties as one entity (bundle) in data tar file using Tar
Persistance Manager (Tar PM).
• Tar PM model is append only.

What is TarPM Optimization?
• Tar Optimization is a process which cleans up the old unreferenced data
to improve the overall performance & reduce disk usage.
28
bundle03
bundle04
bundle11
bundle14
data_00003.tar
TAR
OPTIMIZATIO
N
bundle01
bundle02
bundle03
bundle04
data_00001.tar
bundle11
bundle12
bundle13
bundle14
data_00002.tar
Unuse
d data
Deletes old
data tar file.
Deletes old
data tar file.

Data Tar file location.
• Default storage location of data tar files are
29
<cq_home>/crx-quickstart/repository
data_00000.tar
data_00001.tar
workspaces
crx.default
version
data_00004.tar
data_00005.tar
data_00010.tar
data_00011.tar
tarJournal
data_00012.tar
data_00013.tar

Best practices for maintaining AEM Instances
Regularly run tar optimization during the low usage hours of your
system.
• Automatic Schedule:-
- By default CRX automatically runs Tar PM optimization in midnight
between 2 AM to 5 AM.
- To reconfigure a scheduled optimization add a param autoOptimizeAt to
PersistenceManager element at repository.xml and workspace.xml.
• <PersistenceManager
class="com.day.crx.persistence.tar.TarPersistenceManager">
<param name="autoOptimizeAt" value="01:00-04:00" />
</PersistenceManager>
30

Manually execute TarPM optimization from JMX console
31

Manually by creating an optimization file “optimize.tar” in the desired workspace folder.
32

Tips and tricks for proactive things to monitor
• From JMX console monitor
- TarOptimizationRunningSince:- The
timestamp when the current Tar PM
optimization was started, or 0 if currently not
running.
- TarOptimizationWork:- The remaining
optimization work in KiloBytes.
- TarOptimizationRate:- TarPM optimization
rate in KB/sec.
33

How to monitor Tar
Optimization via JMX client
DEMONSTRATION

Tips and tricks for reducing downtime & Speed up the process
• Configure the TarOptimizationDelay property:- Number of milliseconds to sleep after
optimizing one transaction.
• Load the index files into the buffer cache every few minutes, from both the crx.default
and the version directory.
- cat <cq_home>/crx-quickstart/repository/workspaces/crx.default/index*.tar > /dev/null”
- cat <cq_home>/crx-quickstart/repository/version/index*.tar > /dev/null”
• To enable Index in memory option refer
http://dev.day.com/docs/en/crx/current/administering/persistence_managers.html#Configuring the indexInMemory
Property
35

Common Pitfalls to Avoid
• Scheduling tar optimization to run during the heavy usage hours of your
system.
• If you are optimizing tar files in a cluster, you need to ensure that the
Tar optimization time are set to the same value on all cluster nodes.
• Do not run tar optimization while you are backing up instance.
• Do not run tar optimization when datastore garbage collection is
running.
• Don’t ignore the message “File not found” exception when tar
optimization is running. Please contact daycare.
36

Workflow maintenance and Purge Obsolete
Workflow Data
Workflow Monitoring

Workflow Purge
• Workflows enable you to automate Experience Manager activities.
• Workflows events generate archived data.
• This data can quickly grow over time as workflows are processed.
• Purging this data on a regular schedule improves system performance.
• In AEM(CQ) 5.6 the Workflow JMX MBean support has been added in
order to maintain workflow system.
38

Regularly run workflow purge to
avoid the problems associated with
a large volume of data.
• Using user interface through Felix
console
• Using CURL
• Creating a JMX client that uses
Workflow Mbean
http://helpx.adobe.com/cq/kb/
workflow-monitor-via-jmx.html
39

Tips and tricks for proactive things to monitor
40

How to monitor workflow
via JMX client
DEMONSTRATION

CQ5.x Maintenance Webinar 2013

More Related Content

What's hot (20)

Viewers also liked (9)

Similar to CQ5.x Maintenance Webinar 2013 (11)

Recently uploaded (20)

CQ5.x Maintenance Webinar 2013