git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [artemis 2.1.0] taking 30+ minutes to boot & failover


https://github.com/apache/activemq-artemis/pull/2214


I will make it into 2.6.x as soon as the PR tests are finished.


If you could try a snapshot of master or 2.6.x (after I merge it of
course). Let me know if you need help buildling it.

On Wed, Jul 25, 2018 at 5:17 PM, Dan Langford <danlangford@xxxxxxxxx> wrote:
> Michael for the win!
> @Clebert im not opposed to coding up a test case it is just that i am
> swamped at the moment so im glad to hear that Michael already has a bead on
> this
>
> i did a *./artemis data exp* out of my 2.1.0 and then an *./artemis data
> imp* on a clean 2.6.2. the new system didnt have Address Settings and
> Security Settings that had previously been added via the jolokia api. i
> should look into the export options more to see if i just failed to provide
> a command line argument. fortunately in my scenario it was pretty trivial
> to recreate those settings quickly. so i did that. the import created all
> the addresses, routes, queues, and messages that i needed. it went pretty
> well. so my cluster is now on 2.6.2
>
> i plan to rework our usage of AddressSetting to not create a new entry for
> every Address we create but instead only create them for the few exceptions
> to our default configuration
>
> then i can start working on running the broker in CF with the cluster in
> more of a Master-Master mode.
> thanks again for all your help. feel free to respond if there is any more
> data you need from me.
>
> *side note: my broker is now spamming " AMQ224088: Timeout (10 seconds)
> while handshaking has occurred. " but thats another issue i will figure
> out. *
>
> my 30 minute startup times are now down to 30 seconds max. more like 10-15
> seconds regularly. THANK YOU ALL SO MUCH FOR YOUR HELP.
>
> On Wed, Jul 25, 2018 at 2:44 PM michael.andre.pearce
> <michael.andre.pearce@xxxxxx.invalid> wrote:
>
>> Clebert im able to recreate this and also worked out in code why this
>> happens. Ill try catch you on irc tomorrow or friday.
>>
>> Sent from my Samsung Galaxy smartphone.
>> -------- Original message --------From: Clebert Suconic <
>> clebert.suconic@xxxxxxxxx> Date: 25/07/2018  20:20  (GMT+00:00) To:
>> users@xxxxxxxxxxxxxxxxxxx Subject: Re: [artemis 2.1.0] taking 30+ minutes
>> to boot & failover
>> Yes.. it should include it.
>>
>>
>> Do you have an example of the API you used with 2.1?
>>
>>
>> I can't write a compatibility test this week.. but if you provide me
>> the example I will do it early next week.
>>
>>
>> In a perfect world, if you did it .. it would be great :)
>>
>>
>> On Wed, Jul 25, 2018 at 1:40 PM, Dan Langford <danlangford@xxxxxxxxx>
>> wrote:
>> > i tried 2.6.2 this morning to see if that was an improvement from 2.5.0 i
>> > tried a few months ago. unfortunately there was not. it may have failed
>> > much faster than 2.5.0 i dont recall the timing specifics but the error
>> is
>> > the same
>> >
>> > 11:15:50,853 ERROR [org.apache.activemq.artemis.core.server] AMQ224000:
>> > Failure in initialisation: java.lang.NegativeArraySizeException
>> > at
>> >
>> org.apache.activemq.artemis.api.core.SimpleString.readSimpleString(SimpleString.java:182)
>> > [artemis-commons-2.6.2.jar:2.6.2]
>> > at
>> >
>> org.apache.activemq.artemis.api.core.SimpleString.readSimpleString(SimpleString.java:171)
>> > [artemis-commons-2.6.2.jar:2.6.2]
>> > at
>> >
>> org.apache.activemq.artemis.api.core.SimpleString.readNullableSimpleString(SimpleString.java:158)
>> > [artemis-commons-2.6.2.jar:2.6.2]
>> > at
>> >
>> org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readNullableSimpleString(ChannelBufferWrapper.java:69)
>> > [artemis-commons-2.6.2.jar:2.6.2]
>> > at
>> >
>> org.apache.activemq.artemis.core.settings.impl.AddressSettings.decode(AddressSettings.java:736)
>> > [artemis-server-2.6.2.jar:2.6.2]
>> > at
>> >
>> org.apache.activemq.artemis.core.persistence.config.PersistedAddressSetting.decode(PersistedAddressSetting.java:95)
>> > [artemis-server-2.6.2.jar:2.6.2]
>> > at
>> >
>> org.apache.activemq.artemis.core.persistence.impl.journal.AbstractJournalStorageManager.newAddressEncoding(AbstractJournalStorageManager.java:1925)
>> > [artemis-server-2.6.2.jar:2.6.2]
>> > at
>> >
>> org.apache.activemq.artemis.core.persistence.impl.journal.AbstractJournalStorageManager.loadBindingJournal(AbstractJournalStorageManager.java:1466)
>> > [artemis-server-2.6.2.jar:2.6.2]
>> > at
>> >
>> org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.loadJournals(ActiveMQServerImpl.java:2601)
>> > [artemis-server-2.6.2.jar:2.6.2]
>> > at
>> >
>> org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.initialisePart2(ActiveMQServerImpl.java:2376)
>> > [artemis-server-2.6.2.jar:2.6.2]
>> > at
>> >
>> org.apache.activemq.artemis.core.server.impl.SharedNothingLiveActivation.run(SharedNothingLiveActivation.java:116)
>> > [artemis-server-2.6.2.jar:2.6.2]
>> > at
>> >
>> org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.internalStart(ActiveMQServerImpl.java:535)
>> > [artemis-server-2.6.2.jar:2.6.2]
>> > at
>> >
>> org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.start(ActiveMQServerImpl.java:474)
>> > [artemis-server-2.6.2.jar:2.6.2]
>> > at
>> >
>> org.apache.activemq.artemis.integration.FileBroker.start(FileBroker.java:111)
>> > [artemis-cli-2.6.2.jar:2.6.2]
>> > at org.apache.activemq.artemis.cli.commands.Run.execute(Run.java:82)
>> > [artemis-cli-2.6.2.jar:2.6.2]
>> > at
>> > org.apache.activemq.artemis.cli.Artemis.internalExecute(Artemis.java:149)
>> > [artemis-cli-2.6.2.jar:2.6.2]
>> > at org.apache.activemq.artemis.cli.Artemis.execute(Artemis.java:97)
>> > [artemis-cli-2.6.2.jar:2.6.2]
>> > at org.apache.activemq.artemis.cli.Artemis.execute(Artemis.java:124)
>> > [artemis-cli-2.6.2.jar:2.6.2]
>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> > [rt.jar:1.8.0_171]
>> > at
>> >
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>> > [rt.jar:1.8.0_171]
>> > at
>> >
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> > [rt.jar:1.8.0_171]
>> > at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_171]
>> > at org.apache.activemq.artemis.boot.Artemis.execute(Artemis.java:129)
>> > [artemis-boot.jar:2.6.2]
>> > at org.apache.activemq.artemis.boot.Artemis.main(Artemis.java:49)
>> > [artemis-boot.jar:2.6.2]
>> >
>> >
>> > right there in the AddressSettings.decode
>> >
>> >
>> > I am going to start looking into a journal export and then a journal
>> import
>> > and see what happens there.
>> > are programmatically created addresses / queues stored in the data export
>> > along with the address and security settings?
>> >
>> >
>> >
>> >
>> > On Wed, Jul 25, 2018 at 10:49 AM Clebert Suconic <
>> clebert.suconic@xxxxxxxxx>
>> > wrote:
>> >
>> >> @dan take a look at the compatibility tests in master.. We're help to
>> >> help if you don't understand anything.. it's using ClassLoaders and
>> >> Groovy so you can produce stuff with a combination of clients and
>> >> servers on 2.1.0 and consume on the current version.
>> >>
>> >> On Wed, Jul 25, 2018 at 12:04 PM, Clebert Suconic
>> >> <clebert.suconic@xxxxxxxxx> wrote:
>> >> > @Dan: what about this:
>> >> >
>> >> > you provide us a code path that could generate the incompatibility in
>> >> 2.1.0...
>> >> >
>> >> > an ultimate deal would be if you produced a compatibility test with
>> >> > 2.1.0.... you add some code that will generate the journal with 2.1.0,
>> >> > and then consume the messages on master (or 2.6.x)...
>> >> >
>> >> >
>> >> > if you generate the compatibility test showing the issue, we can try
>> >> fixing it.
>> >> >
>> >> >
>> >> > What about this idea? that would help you migrate into 2.6.x or
>> master.
>> >> >
>> >> > On Tue, Jul 24, 2018 at 7:25 PM, Clebert Suconic
>> >> > <clebert.suconic@xxxxxxxxx> wrote:
>> >> >> On Fri, Jul 20, 2018 at 4:47 PM, Dan Langford <danlangford@xxxxxxxxx
>> >
>> >> wrote:
>> >> >>> Thank you that was very helpful. we actually do have an address
>> >> settings
>> >> >>> entry for each queue. there could be a better pattern for us. but
>> >> currently
>> >> >>> our automated system for creating queues creates an address setting
>> at
>> >> the
>> >> >>> same time. i will look into improved patterns.
>> >> >>>
>> >> >>> as far as upgrading goes. i agree we really want to upgrade. until i
>> >> can
>> >> >>> find a work around for the config-delete-queues deserialization bug
>> >> >>> introduced in 2.2.0 i brought up back in April we will not be able
>> to
>> >> >>> easily move.
>> >> >>
>> >> >> You should be able to move to 2.6.2. if you're not able to I would
>> >> >> like to know where it failed.
>> >> >>
>> >> >>
>> >> >>>
>> >> >>> thanks again for all the help
>> >> >>>
>> >> >>> On Fri, Jul 20, 2018 at 8:56 AM Clebert Suconic <
>> >> clebert.suconic@xxxxxxxxx>
>> >> >>> wrote:
>> >> >>>
>> >> >>>> If you do not want to upgrade for any reason export the journal.
>> >> Cleani
>> >> >>>> uo.  Edit the text and remove the garbage (you will see) manually.
>> >>  Delete
>> >> >>>> all data and te import.
>> >> >>>>
>> >> >>>> (Make a backup to be safe of course)
>> >> >>>>
>> >> >>>>
>> >> >>>> But I still recommend the upgrade.
>> >> >>>>
>> >> >>>> On Fri, Jul 20, 2018 at 10:54 AM Clebert Suconic <
>> >> >>>> clebert.suconic@xxxxxxxxx>
>> >> >>>> wrote:
>> >> >>>>
>> >> >>>> > The address setting is the garbage I was talking about.  Upgrade
>> to
>> >> the
>> >> >>>> > latest broker and there will be a cleanup done at the load
>> before it
>> >> >>>> > starts.
>> >> >>>> >
>> >> >>>> >
>> >> >>>> > I highly recommend upgrade.
>> >> >>>> >
>> >> >>>> > On Fri, Jul 20, 2018 at 10:05 AM Justin Bertram <
>> >> jbertram@xxxxxxxxxx>
>> >> >>>> > wrote:
>> >> >>>> >
>> >> >>>> >> Analyzing thread dumps like this is pretty simple.  I generally
>> >> just
>> >> >>>> >> scroll
>> >> >>>> >> through and look for long stack-traces with lots of calls from
>> >> >>>> >> org.apache.activemq.artemis.  In your case every single thread
>> >> dump has
>> >> >>>> a
>> >> >>>> >> thread doing something like this:
>> >> >>>> >>
>> >> >>>> >> "main" #1 prio=5 os_prio=0 tid=0x00007f902800eb20 nid=0x74fe
>> >> runnable
>> >> >>>> >> [0x00007f9031675000]
>> >> >>>> >>    java.lang.Thread.State: RUNNABLE
>> >> >>>> >>     at
>> >> >>>> >>
>> >> >>>> >>
>> >> >>>>
>> >>
>> org.apache.activemq.artemis.core.settings.impl.HierarchicalObjectRepository.getPossibleMatches(HierarchicalObjectRepository.java:373)
>> >> >>>> >>     at
>> >> >>>> >>
>> >> >>>> >>
>> >> >>>>
>> >>
>> org.apache.activemq.artemis.core.settings.impl.HierarchicalObjectRepository.getMatch(HierarchicalObjectRepository.java:192)
>> >> >>>> >>     at
>> >> >>>> >>
>> >> >>>> >>
>> >> >>>>
>> >>
>> org.apache.activemq.artemis.core.paging.impl.PagingManagerImpl.reapplySettings(PagingManagerImpl.java:113)
>> >> >>>> >>     at
>> >> >>>> >>
>> >> >>>> >>
>> >> >>>>
>> >>
>> org.apache.activemq.artemis.core.paging.impl.PagingManagerImpl.onChange(PagingManagerImpl.java:108)
>> >> >>>> >>     at
>> >> >>>> >>
>> >> >>>> >>
>> >> >>>>
>> >>
>> org.apache.activemq.artemis.core.settings.impl.HierarchicalObjectRepository.onChange(HierarchicalObjectRepository.java:348)
>> >> >>>> >>     at
>> >> >>>> >>
>> >> >>>> >>
>> >> >>>>
>> >>
>> org.apache.activemq.artemis.core.settings.impl.HierarchicalObjectRepository.addMatch(HierarchicalObjectRepository.java:168)
>> >> >>>> >>     at
>> >> >>>> >>
>> >> >>>> >>
>> >> >>>>
>> >>
>> org.apache.activemq.artemis.core.settings.impl.HierarchicalObjectRepository.addMatch(HierarchicalObjectRepository.java:147)
>> >> >>>> >>     at
>> >> >>>> >>
>> >> >>>> >>
>> >> >>>>
>> >>
>> org.apache.activemq.artemis.core.settings.impl.HierarchicalObjectRepository.addMatch(HierarchicalObjectRepository.java:120)
>> >> >>>> >>     at
>> >> >>>> >>
>> >> >>>> >>
>> >> >>>>
>> >>
>> org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.recoverStoredConfigs(ActiveMQServerImpl.java:2424)
>> >> >>>> >>     at
>> >> >>>> >>
>> >> >>>> >>
>> >> >>>>
>> >>
>> org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.loadJournals(ActiveMQServerImpl.java:2374)
>> >> >>>> >>     at
>> >> >>>> >>
>> >> >>>> >>
>> >> >>>>
>> >>
>> org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.initialisePart2(ActiveMQServerImpl.java:2219)
>> >> >>>> >>     - locked <0x0000000080a8fce8> (a
>> >> >>>> >> org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl)
>> >> >>>> >>     at
>> >> >>>> >>
>> >> >>>> >>
>> >> >>>>
>> >>
>> org.apache.activemq.artemis.core.server.impl.SharedNothingLiveActivation.run(SharedNothingLiveActivation.java:109)
>> >> >>>> >>     at
>> >> >>>> >>
>> >> >>>> >>
>> >> >>>>
>> >>
>> org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.internalStart(ActiveMQServerImpl.java:518)
>> >> >>>> >>     at
>> >> >>>> >>
>> >> >>>> >>
>> >> >>>>
>> >>
>> org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.start(ActiveMQServerImpl.java:466)
>> >> >>>> >>     - locked <0x0000000080a8fce8> (a
>> >> >>>> >> org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl)
>> >> >>>> >>     at
>> >> >>>> >>
>> >> >>>> >>
>> >> >>>>
>> >>
>> org.apache.activemq.artemis.integration.FileBroker.start(FileBroker.java:111)
>> >> >>>> >>     - locked <0x000000008098be80> (a
>> >> >>>> >> org.apache.activemq.artemis.integration.FileBroker)
>> >> >>>> >>     at
>> >> org.apache.activemq.artemis.cli.commands.Run.execute(Run.java:73)
>> >> >>>> >>     at
>> >> >>>> >>
>> >> >>>>
>> >>
>> org.apache.activemq.artemis.cli.Artemis.internalExecute(Artemis.java:148)
>> >> >>>> >>     at
>> >> org.apache.activemq.artemis.cli.Artemis.execute(Artemis.java:95)
>> >> >>>> >>     at
>> >> org.apache.activemq.artemis.cli.Artemis.execute(Artemis.java:122)
>> >> >>>> >>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>> Method)
>> >> >>>> >>     at
>> >> >>>> >>
>> >> >>>> >>
>> >> >>>>
>> >>
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>> >> >>>> >>     at
>> >> >>>> >>
>> >> >>>> >>
>> >> >>>>
>> >>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> >> >>>> >>     at java.lang.reflect.Method.invoke(Method.java:498)
>> >> >>>> >>     at
>> >> >>>> org.apache.activemq.artemis.boot.Artemis.execute(Artemis.java:129)
>> >> >>>> >>     at
>> >> org.apache.activemq.artemis.boot.Artemis.main(Artemis.java:49)
>> >> >>>> >>
>> >> >>>> >> In every case it's the "main" thread which isn't surprising as
>> >> that is
>> >> >>>> the
>> >> >>>> >> thread responsible for starting the broker.  Also, you can
>> pretty
>> >> >>>> clearly
>> >> >>>> >> see in the trace that this is the thread starting the broker,
>> it's
>> >> >>>> loading
>> >> >>>> >> the journals, & restoring stored configuration (either address
>> >> settings
>> >> >>>> or
>> >> >>>> >> security settings).  I've seen high broker start times when
>> there
>> >> are
>> >> >>>> lots
>> >> >>>> >> and lots of addresses and lots of and lots of settings.  Do
>> either
>> >> (or
>> >> >>>> >> both) of these situations apply to you?
>> >> >>>> >>
>> >> >>>> >>
>> >> >>>> >> Justin
>> >> >>>> >>
>> >> >>>> >> On Fri, Jul 20, 2018 at 1:16 AM, Dan Langford <
>> >> danlangford@xxxxxxxxx>
>> >> >>>> >> wrote:
>> >> >>>> >>
>> >> >>>> >> > Dang I can’t easily upgrade past 2.1.0 because of the
>> >> >>>> >> config-delete-queues
>> >> >>>> >> > deserialization bug introduced in 2.2.0. Unless that bug was
>> >> squashed
>> >> >>>> in
>> >> >>>> >> > 2.6+. I don’t think I made a jira for it (vacation and work
>> >> load) but
>> >> >>>> we
>> >> >>>> >> > discussed it back in April. I should go confirm that bug on
>> 2.6
>> >> and
>> >> >>>> >> make a
>> >> >>>> >> > jira for that
>> >> >>>> >> >
>> >> >>>> >> > Thanks
>> >> >>>> >> > On Thu, Jul 19, 2018 at 5:46 PM Clebert Suconic <
>> >> >>>> >> clebert.suconic@xxxxxxxxx
>> >> >>>> >> > >
>> >> >>>> >> > wrote:
>> >> >>>> >> >
>> >> >>>> >> > > There is an issue I remember where the journal would have
>> some
>> >> dirt
>> >> >>>> >> that
>> >> >>>> >> > > was fixed on 2.3/0.
>> >> >>>> >> > >
>> >> >>>> >> > > I would ipgrade to 2.6.2.
>> >> >>>> >> > >
>> >> >>>> >> > > On Thu, Jul 19, 2018 at 6:34 PM Dan Langford <
>> >> danlangford@xxxxxxxxx
>> >> >>>> >
>> >> >>>> >> > > wrote:
>> >> >>>> >> > >
>> >> >>>> >> > > > would you be willing to help me translate these thread
>> dumps?
>> >> >>>> >> > > >
>> >> >>>> >> > > > i attached a Zip file with some thread dumps in them. i
>> will
>> >> also
>> >> >>>> >> share
>> >> >>>> >> > > > the fasthread.io links for each file. (i was struggling
>> >> getting
>> >> >>>> >> > > > fastthread to do a combo report with the threads in the
>> >> correct
>> >> >>>> >> order)
>> >> >>>> >> > > >
>> >> >>>> >> > > > artemis04-20180719-1525 https://goo.gl/d88azU
>> >> >>>> >> > > > artemis04-20180719-1530 https://goo.gl/G78qn3
>> >> >>>> >> > > > artemis04-20180719-1535 https://goo.gl/aMBSBw
>> >> >>>> >> > > > artemis04-20180719-1540 https://goo.gl/brKxxk
>> >> >>>> >> > > > artemis04-20180719-1545 https://goo.gl/RaXXCs
>> >> >>>> >> > > > artemis04-20180719-1550 https://goo.gl/r5dndK
>> >> >>>> >> > > > artemis04-20180719-1555 https://goo.gl/YJRLxe
>> >> >>>> >> > > >
>> >> >>>> >> > > > at :35, :45, and :55 the young+old gen space gets bigger
>> >> than at
>> >> >>>> the
>> >> >>>> >> > > other
>> >> >>>> >> > > > sample times. but i dont know what to look for in here to
>> >> >>>> determine
>> >> >>>> >> > what
>> >> >>>> >> > > > the broker is actually during during this time.
>> >> >>>> >> > > >
>> >> >>>> >> > > > thanks
>> >> >>>> >> > > >
>> >> >>>> >> > > > On Fri, Jul 6, 2018 at 10:42 AM Justin Bertram <
>> >> >>>> jbertram@xxxxxxxxxx
>> >> >>>> >> >
>> >> >>>> >> > > > wrote:
>> >> >>>> >> > > >
>> >> >>>> >> > > >> The first place I would start is grabbing thread dumps
>> >> every so
>> >> >>>> >> often
>> >> >>>> >> > to
>> >> >>>> >> > > >> see what the broker is actually doing during the 30+
>> >> minutes.
>> >> >>>> >> > > >>
>> >> >>>> >> > > >>
>> >> >>>> >> > > >> Justin
>> >> >>>> >> > > >>
>> >> >>>> >> > > >> On Fri, Jul 6, 2018 at 11:34 AM, Dan Langford <
>> >> >>>> >> danlangford@xxxxxxxxx>
>> >> >>>> >> > > >> wrote:
>> >> >>>> >> > > >>
>> >> >>>> >> > > >> > so my server startup times and failover times are
>> growing
>> >> >>>> pretty
>> >> >>>> >> > big.
>> >> >>>> >> > > >> but i
>> >> >>>> >> > > >> > dont really know where to start looking.
>> >> >>>> >> > > >> >
>> >> >>>> >> > > >> > here is a snippet of some logs to show you the time
>> >> stamps:
>> >> >>>> >> > > >> >
>> >> >>>> >> > > >> > 08:11:31,801 INFO  [org.apache.activemq.artemis.
>> >> >>>> >> > integration.bootstrap]
>> >> >>>> >> > > >> > AMQ101000: Starting ActiveMQ Artemis Server
>> >> >>>> >> > > >> >
>> >> >>>> >> > > >> > 08:11:31,818 INFO
>> >> [org.apache.activemq.artemis.core.server]
>> >> >>>> >> > > AMQ221000:
>> >> >>>> >> > > >> > live Message Broker is starting with configuration
>> Broker
>> >> >>>> >> > > Configuration
>> >> >>>> >> > > >> > (clustered=true,journalDirectory=./data/
>> >> >>>> >> > journal,bindingsDirectory=./
>> >> >>>> >> > > >> >
>> >> data/bindings,largeMessagesDirectory=./data/large-messages,
>> >> >>>> >> > > >> > pagingDirectory=./data/paging)
>> >> >>>> >> > > >> >
>> >> >>>> >> > > >> > 08:11:34,462 INFO
>> >> [org.apache.activemq.artemis.core.server]
>> >> >>>> >> > > AMQ221012:
>> >> >>>> >> > > >> > Using AIO Journal
>> >> >>>> >> > > >> >
>> >> >>>> >> > > >> > 08:11:34,493 INFO
>> >> [org.apache.activemq.artemis.core.server]
>> >> >>>> >> > > AMQ221057:
>> >> >>>> >> > > >> > Global Max Size is being adjusted to 1/2 of the JVM max
>> >> size
>> >> >>>> >> (-Xmx).
>> >> >>>> >> > > >> being
>> >> >>>> >> > > >> > defined as 1,073,741,824
>> >> >>>> >> > > >> >
>> >> >>>> >> > > >> > 08:11:34,555 INFO
>> >> [org.apache.activemq.artemis.core.server]
>> >> >>>> >> > > AMQ221043:
>> >> >>>> >> > > >> > Protocol module found: [artemis-server]. Adding
>> protocol
>> >> >>>> support
>> >> >>>> >> > for:
>> >> >>>> >> > > >> CORE
>> >> >>>> >> > > >> >
>> >> >>>> >> > > >> > 08:11:34,555 INFO
>> >> [org.apache.activemq.artemis.core.server]
>> >> >>>> >> > > AMQ221043:
>> >> >>>> >> > > >> > Protocol module found: [artemis-amqp-protocol]. Adding
>> >> protocol
>> >> >>>> >> > > support
>> >> >>>> >> > > >> > for: AMQP
>> >> >>>> >> > > >> >
>> >> >>>> >> > > >> > 08:11:34,556 INFO
>> >> [org.apache.activemq.artemis.core.server]
>> >> >>>> >> > > AMQ221043:
>> >> >>>> >> > > >> > Protocol module found: [artemis-hornetq-protocol].
>> Adding
>> >> >>>> >> protocol
>> >> >>>> >> > > >> support
>> >> >>>> >> > > >> > for: HORNETQ
>> >> >>>> >> > > >> >
>> >> >>>> >> > > >> > 08:11:34,556 INFO
>> >> [org.apache.activemq.artemis.core.server]
>> >> >>>> >> > > AMQ221043:
>> >> >>>> >> > > >> > Protocol module found: [artemis-mqtt-protocol]. Adding
>> >> protocol
>> >> >>>> >> > > support
>> >> >>>> >> > > >> > for: MQTT
>> >> >>>> >> > > >> >
>> >> >>>> >> > > >> > 08:11:34,556 INFO
>> >> [org.apache.activemq.artemis.core.server]
>> >> >>>> >> > > AMQ221043:
>> >> >>>> >> > > >> > Protocol module found: [artemis-openwire-protocol].
>> Adding
>> >> >>>> >> protocol
>> >> >>>> >> > > >> support
>> >> >>>> >> > > >> > for: OPENWIRE
>> >> >>>> >> > > >> >
>> >> >>>> >> > > >> > 08:11:34,556 INFO
>> >> [org.apache.activemq.artemis.core.server]
>> >> >>>> >> > > AMQ221043:
>> >> >>>> >> > > >> > Protocol module found: [artemis-stomp-protocol]. Adding
>> >> >>>> protocol
>> >> >>>> >> > > support
>> >> >>>> >> > > >> > for: STOMP
>> >> >>>> >> > > >> >
>> >> >>>> >> > > >> > 08:41:38,963 WARN
>> >> [org.apache.activemq.artemis.core.server]
>> >> >>>> >> > > AMQ222165:
>> >> >>>> >> > > >> No
>> >> >>>> >> > > >> > Dead Letter Address configured for queue DLQ in
>> >> AddressSettings
>> >> >>>> >> > > >> >
>> >> >>>> >> > > >> > 08:41:38,963 WARN
>> >> [org.apache.activemq.artemis.core.server]
>> >> >>>> >> > > AMQ222166:
>> >> >>>> >> > > >> No
>> >> >>>> >> > > >> > Expiry Address configured for queue DLQ in
>> AddressSettings
>> >> >>>> >> > > >> >
>> >> >>>> >> > > >> > 08:41:38,983 WARN
>> >> [org.apache.activemq.artemis.core.server]
>> >> >>>> >> > > AMQ222165:
>> >> >>>> >> > > >> No
>> >> >>>> >> > > >> > Dead Letter Address configured for queue ExpiryQueue in
>> >> >>>> >> > > AddressSettings
>> >> >>>> >> > > >> >
>> >> >>>> >> > > >> > 08:41:38,983 WARN
>> >> [org.apache.activemq.artemis.core.server]
>> >> >>>> >> > > AMQ222166:
>> >> >>>> >> > > >> No
>> >> >>>> >> > > >> > Expiry Address configured for queue ExpiryQueue in
>> >> >>>> >> AddressSettings
>> >> >>>> >> > > >> >
>> >> >>>> >> > > >> > 08:41:38,984 WARN
>> >> [org.apache.activemq.artemis.core.server]
>> >> >>>> >> > > AMQ222165:
>> >> >>>> >> > > >> No
>> >> >>>> >> > > >> > Dead Letter Address configured for queue example in
>> >> >>>> >> AddressSettings
>> >> >>>> >> > > >> >
>> >> >>>> >> > > >> > 08:41:38,985 WARN
>> >> [org.apache.activemq.artemis.core.server]
>> >> >>>> >> > > AMQ222166:
>> >> >>>> >> > > >> No
>> >> >>>> >> > > >> > Expiry Address configured for queue example in
>> >> AddressSettings
>> >> >>>> >> > > >> >
>> >> >>>> >> > > >> > 08:41:38,985 WARN
>> >> [org.apache.activemq.artemis.core.server]
>> >> >>>> >> > > AMQ222165:
>> >> >>>> >> > > >> No
>> >> >>>> >> > > >> > Dead Letter Address configured for queue exampleQueue
>> in
>> >> >>>> >> > > AddressSettings
>> >> >>>> >> > > >> >
>> >> >>>> >> > > >> > 08:41:38,986 WARN
>> >> [org.apache.activemq.artemis.core.server]
>> >> >>>> >> > > AMQ222166:
>> >> >>>> >> > > >> No
>> >> >>>> >> > > >> > Expiry Address configured for queue exampleQueue in
>> >> >>>> >> AddressSettings
>> >> >>>> >> > > >> >
>> >> >>>> >> > > >> >
>> >> >>>> >> > > >> > and it continues. i have 138 queues. i wonder if i need
>> >> to be
>> >> >>>> >> > looking
>> >> >>>> >> > > at
>> >> >>>> >> > > >> > PAGE configuration or some cache sizes. do i need to be
>> >> looking
>> >> >>>> >> at
>> >> >>>> >> > the
>> >> >>>> >> > > >> > number of messages persisted on these queues? where
>> would
>> >> you
>> >> >>>> >> look
>> >> >>>> >> > to
>> >> >>>> >> > > >> > determine why the startup times are so long?
>> >> >>>> >> > > >> >
>> >> >>>> >> > > >> > also, i know i need to upgrade but i cannot upgrade
>> off of
>> >> >>>> 2.1.0
>> >> >>>> >> due
>> >> >>>> >> > > to
>> >> >>>> >> > > >> > some deseralization changes introduced in 2.2.0. i
>> think
>> >> i have
>> >> >>>> >> > > another
>> >> >>>> >> > > >> > thread on here (that i need to update) regarding that
>> >> upgrade
>> >> >>>> >> issue.
>> >> >>>> >> > > >> >
>> >> >>>> >> > > >> > thanks for any ideas or insight you have for me
>> >> >>>> >> > > >> >
>> >> >>>> >> > > >>
>> >> >>>> >> > > > --
>> >> >>>> >> > > Clebert Suconic
>> >> >>>> >> > >
>> >> >>>> >> >
>> >> >>>> >>
>> >> >>>> > --
>> >> >>>> > Clebert Suconic
>> >> >>>> >
>> >> >>>> --
>> >> >>>> Clebert Suconic
>> >> >>>>
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Clebert Suconic
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Clebert Suconic
>> >>
>> >>
>> >>
>> >> --
>> >> Clebert Suconic
>> >>
>>
>>
>>
>> --
>> Clebert Suconic
>>



-- 
Clebert Suconic