git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Properly closing transport threads in ActiveMQ with listeners in Spring-servlet


Hi list,

I know this question has occasionally been asked before, but I can't
seem to figure out the correct method with the most recent ActiveMQ.

Our application runs with an in-memory database which is kept in sync by
listening to about 30 different topics on ActiveMQ which are monitored
via Spring-JMS' listener-setup via ActiveMQ's JMS-client (version 5.15.6).
The Spring version is the latest stable 5.x (altough it started with
2.x), is deployed on Tomcat 8.5 which runs on Java 10.

All libraries, including ActiveMQ are integrated in the war-archive
(i.e. no shared libraries).

Whenever the application is undeployed, Tomcat warns about Threads not
being closed:

WARNING [ContainerBackgroundProcessor[StandardEngine[Catalina]]]
org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads
The web application [appname] appears to have started a thread named
[ActiveMQ InactivityMonitor ReadCheckTimer] but has failed to stop it.
This is very likely to create a memory leak.

 17:05:20.267 WARNING
[ContainerBackgroundProcessor[StandardEngine[Catalina]]]
org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads
The web application [appname] appears to have started a thread named
[ActiveMQ Transport: tcp://servername/10.10.10.10:61616@49831] but has
failed to stop it. This is very likely to create a memory leak.

And some more.

I know in other discussions this has been dismissed as a non-issue/false
positive. But the ClassLoader for the specific app cannot be garbage
collected, which fixes all class-objects to remain, which locks all
static-properties (including some static caches of beans and other
objects) in memory. And the threads never seem to close (at least not in
several minutes) after undeploy; the connection does seem to have been
disconnected (at least, there are no consumers anymore on the broker).
Whether the memory leak is really due to the ActiveMQ-threads is
somewhat harder to proof, but I'd like to be able to eliminate that
option regardless.

Currently the configuration is in Spring-xml:
    <bean id="activeMqConnectionFactory"
class="org.apache.activemq.ActiveMQConnectionFactory" />
    <bean id="jmsConnectionFactory"
class="org.apache.activemq.pool.PooledConnectionFactoryBean"
destroy-method="destroy">
        <property name="connectionFactory"
ref="activeMqConnectionFactory" />
    </bean>
  <jms:listener-container connection-factory="jmsConnectionFactory"
destination-type="topic">
         <jms:listener destination="va_cache_advertisements"
ref="advertisementListener" />
    </jms:listener-container>

My initial configuration didn't use the PooledConnectionFactoryBean. But
while this reduces the amount of concurrent connections (and thus
threads), it didn't actually close any of the threads. The
advertisementListener is a MessageListener-implementation and has no
access to the JMS-connection.

I already have a ServletContextListener which is called upon undeploy
(to stop a few other leaking threads). And I did add ActiveMQ's
DefaultThreadPools's shutdown. I also tried a 'hack' to access
AbstractInactivityMonitor.stopMonitorThreads to stop the statically
cached threads/timers there. But both options don't seem to matter.

I've tried to find a proper resource on how to fix this, but most search
results seem to be very old and/or reported not to work.
So, how do I actually make sure those Threads (both inactivity
monitoring and tcp transport) are properly closed and/or any other
object that needs to be closed?

Thanks in advance,

Arjen