Date post: | 04-Jul-2015 |
Category: |
Technology |
Upload: | domenec-valles |
View: | 572 times |
Download: | 2 times |
Jconsole JMX data capture of a Camel server
This document contains screenshots taken on a productionserver. Its solely purpose is to share the collected data with
people on the Camel list and help diagnose some performance issues.
Memory
Threads
Camel threads• Some 1000 of them, most of them waiting for some lock, or parked.
• 424 <route> tags in a 4200 lines config file, there are some 626 routes available in theMbeans :-? (several routes are FTP from one URI to many URI)
Name: Camel (myCamel) thread #10 - ftp://retalixcon@SRV0043/exportState: WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@16c83b9
Total blocked: 147 Total waited: 2
Stack trace:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
java.util.concurrent.DelayQueue.take(DelayQueue.java:160)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
java.lang.Thread.run(Thread.java:662)
Name: Camel (myCamel) thread #192 - ftp://retalixcon@SRV0126/exportState: WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@e91163
Total blocked: 28 Total waited: 2
Stack trace:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
java.util.concurrent.DelayQueue.take(DelayQueue.java:160)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
java.lang.Thread.run(Thread.java:662)
Camel threads (continued)
Name: Camel (myCamel) thread #463 - ftp://retalixcon@SRV0153/export
State: WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1a1156f
Total blocked: 426 Total waited: 2
Stack trace:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
java.util.concurrent.DelayQueue.take(DelayQueue.java:160)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
java.lang.Thread.run(Thread.java:662)
Name: Camel (myCamel) thread #942 - ftp://retalixcon@SRV1004/export
State: WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@195f6a7
Total blocked: 130 Total waited: 2
Stack trace:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
java.util.concurrent.DelayQueue.take(DelayQueue.java:160)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
java.lang.Thread.run(Thread.java:662)
Quartz threads
• 10 of them, mostly seem to be waiting (which should be normal).
Name: DefaultQuartzScheduler_Worker-1
State: TIMED_WAITING on org.quartz.simpl.SimpleThreadPool$WorkerThread@1efb21d
Total blocked: 0 Total waited: 123.407
Stack trace:
java.lang.Object.wait(Native Method)
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:543)
Name: DefaultQuartzScheduler_Worker-5
State: TIMED_WAITING on org.quartz.simpl.SimpleThreadPool$WorkerThread@188145f
Total blocked: 0 Total waited: 123.428
Stack trace:
java.lang.Object.wait(Native Method)
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:543)
Classes
Summary
JMX Mbeans
• Many thread pools (some 100) of FileConsumer and FtpConsumer type alike.
• Many ProducerCache