git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Help with Splitter/Aggregator-like behavior


Hi

You can use the splitter and aggregator to do something like that.

The aggregator can group the splitted messages together based on that
frame, and it can do this in out of order.
And the output of the aggregator is routed separated from the input (async)

Maybe try to build a simple use-case / sample / unit test with just
the splitter and aggregator and see if you can build something.
And if not then maybe post a bit here again with what you have done,
so we can better help / understand your use-case.


On Thu, Sep 20, 2018 at 7:15 AM Ron Cecchini <roncecchini@xxxxxxxxxxx> wrote:
>
>
> So, I have a situation where I need something like a Splitter and an Aggregator.
> But as far as I can tell from reading and googling, maybe my situation is nonstandard?
> From what I can tell, a Splitter and Aggregator are used together within a single route.
> In my case, I need the Splitter and Aggregator separated into a sender and receiver, resp.
>
>
> I'm just looking for someone to tell me if the following fits squarely within the Splitter
> and Aggregator patterns - if so, I'll dig in and figure it out - or if there's another pattern
> or something else to try.
>
>
> Thank you in advance for your guidance, and sorry for being so verbose again (just trying to be clear).
>
>
> -----
>
>
> On the Splitter side, per usual, I need to split a big message into individual messages.
> However, I can't just split and let each individual message continue on the route.
> Instead, I need to "wrap" each individual message and stick some header information on it
>
>
> The situation is very much like the following, which is very UDP-like:
>
>
> Big messages come in, and they get split into "packages" of a preset size.
> All the individual "packages" can be said to belong to a "frame" of data.
> The header of the individual messages contain the Frame # and Package # and the Total #
> of packages in the frame so the receiver knows when it has received a full frame of data.
>
> Message: 1
>      Frame: 1 - Package: 1 - Total: 3
>      Frame: 1 - Package: 2 - Total: 3
>      Frame: 1 - Package: 3 - Total: 3
> Message: 2
>      Frame: 2 - Package: 1 - Total: 2
>      Frame: 2 - Package: 2 - Total: 2
>
>
> Etc.
>
>
> If I can't accomplish this with a split() of some kind, how could I do it with a regular Processor?
> Having a Processor manually split and bundle the data into "packages" is trivial.
> But how does the Processor then write the individual messages back to a "direct:processPackage" route point?
> Can a Processor invoke (write data to) a route, at some point in the middle of that route?
>
>
> -----
>
>
> The Aggregator, as you would expect, needs to do the opposite of the above:
>
>
> It needs to aggregate "packages" of data until it determines it has a full "frame".
> Then it bundles all the package payloads into a single, big message.
> When a frame is not full, data does not flow to the rest of the route.
> When the frame is full, the data is written to some route mid-point; e.g. "direct:translateMessage".
>
>
> So, can this sort of "asynchronous" aggregating be done?
> Can an aggregating Processor basically maintain state, and decide to write or not write to a route?
>
>
> Thank you again for any pointers.



-- 
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2