git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Concurrent consumers from SEDA to CXF


Hi

Ah okay its likely because CXF can process asynchronously and you may
not have a limit on the seda/vm queue so it can buffer pending
inflight exchanges.
So try setting cxf to be synchronous, via synchronous=true on its endpoint.

On Tue, Jul 31, 2018 at 2:13 PM, Mirek Borsky <geraltofr@xxxxxxxx> wrote:
> Hi Claus,
>
> I would try to explain. Please ask for the clarification as needed.
>
> In the route in my example, if there will a delayer as the component that
> take 2 seconds to proceed, the route will look like this:
> <route>
>   <from uri="vm:sam.logger.to.server?concurrentConsumers=10" />
>   <delay><constant>2000</constant></delay>
> </route>
>
> there will be 10 inflight messages in this route when the seda queue is
> filled. The rate of processing of the messages from the seda queue will be
> around 5 messages per second. This is the expected behaviour when I set
> concurrentConsumers=10 on the seda queue..
>
> However, when instead of <delay/> there will be the cxf call to a service
> that take 2 seconds to answer, things will change. In my runtime
> nevironment, I can see 110 inflight messages in the route. Also the rate of
> processing of the messages from the seda queue is much higher, around 50 per
> second.
>
> The problem I perceive is not the number of concurrent requests for the cxf
> endpoint. Problem is the fact, that the combination of seda queue and cxf
> endpoint seems to exhaust all the available connections regardless of the
> concurrentConsumers setting.
>
> I hope I have provided the example that help you better understand the issue
> I am facing. If I have failed in doing so, please bear with me and try to
> ask differently.
>
> kind regards
> Miroslav Borský
>
>
> ---------- Původní e-mail ----------
> Od: Claus Ibsen <claus.ibsen@xxxxxxxxx>
> Komu: users@xxxxxxxxxxxxxxxx
> Datum: 30. 7. 2018 17:04:45
> Předmět: Re: Concurrent consumers from SEDA to CXF
> "Hi
>
> Can you try to explain the numbers a bit more, maybe with some example
> numbers?
>
> For http connections etc, then CXF has its own set of limitations,
> specially if you call the same remote server via the same client, eg
> same cxf-endpoint in Camel.
> That may have limitations on number of concurrent requests, that can
> be lower. So try to look at the CXF documentation about this.
>
> On Wed, Jul 25, 2018 at 9:16 PM, Mirek Borsky <geraltofr@xxxxxxxx> wrote:
>> Greetings,
>>
>> I have encountered a strange behavior with a simple camel route and I
> would
>> like to ask for help of more experienced users. First, let me quickly show
>
>> the route:
>>
>> The relevant part:
>> <from uri="vm:sam.logger.to.server?concurrentConsumers=10" />
>> <to uri="MonitoringService" />
>>
>> The whole blueprint:
>> <?xml version="1.0" encoding="UTF-8"?>
>> <osgi:blueprint xsi:schemaLocation="http://www.osgi.org/xmlns/blueprint/v
>> 1.0.0 https://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd"; xmlns:xsi
> =
>> "http://www.w3.org/2001/XMLSchema-instance"; xmlns:cxf="http://camel.
> apache.
>> org/schema/blueprint/cxf" xmlns:osgi="http://www.osgi.org/xmlns/blueprint/
> v
>> 1.0.0">
>> <camelContext id="TestSAMLoggerSeda-ctx" xmlns="http://camel.apache.org/
>> schema/blueprint" streamCache="true">
>> <route>
>> <from uri="vm:sam.logger.to.server?concurrentConsumers=10" />
>> <to uri="MonitoringService" />
>> </route>
>> </camelContext>
>> <cxf:cxfEndpoint xmlns:tns="http://www.talend.org/esb/sam/
>> MonitoringService/v1" id="MonitoringService" address="http://localhost:
> 8042/
>> MonitoringMock" serviceName="tns:MonitoringWebServiceService" endpointName
> =
>> "tns:MonitoringWebServicePort" wsdlURL="deploy/MonitoringService.wsdl">
>> <cxf:properties>
>> <osgi:entry key="dataFormat" value="PAYLOAD"/>
>> </cxf:properties>
>> </cxf:cxfEndpoint>
>> </osgi:blueprint>
>>
>> Now the expected behavior:
>> What I want to achieve is a queue from which 10 parallel threads are
>> processing messages, calling the cxf endpoint. The cxf endpoint takes
>> several seconds to answer, so I am expecting the speed of processing
>> somewhere about 1-2 messages per second.
>>
>> Observed behavior:
>> Using JMX I can see X+N inflight messages in the route, where X is the
>> maximum number of concurrent http connections to the cxf endpoint and N is
>
>> the number of concurrent users. For some reason, the workers that are
>> waiting for the cxf endpoint to answer are not counted against the
>> concurrentConsumers limit. Only the workers that are waiting for the
>> connection to be established are counted against the limit of
>> concurrentConsumers.
>>
>> Questions:
>> What is the correct behavior for the route?
>> If my configuration is incorrect, how can I achieve my target behavior?
>>
>> The camel version used is 2.20.
>>
>> If someone would be able to help me, and explain what I did wrong, I would
>
>> very much appreciate it.
>>
>> thank you for help
>> kind regards
>> Miroslav Borský
>
>
>
> --
> Claus Ibsen
> -----------------
> http://davsclaus.com @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2
> "



-- 
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2