git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Proposal] Create new sub project for journaled messaging


Decanter is based in EventAdmin. I like the nice decoupling EventAdmin
gives us for decantr but it is limited by EventAdmin's deficiencies.
For example any Events sent before a subscriber arrives are lost.
The main advantage of the API compared to simple pub/sub is that you can go
back in the history (like in kafka).

So I think either the API directly or an EventAdmin based on the jounaled
messaging API could be very valuable for Decanter to make sure no startup
messages are lost.

Christian

Am Sa., 22. Dez. 2018 um 14:34 Uhr schrieb Jean-Baptiste Onofré <
jb@xxxxxxxxxxxx>:

> OK, it's clear now.
>
> It sounds a bit like Decanter (EventAdmin sent to a storage backend), no ?
>
> Regards
> JB
>
> On 22/12/2018 14:30, Christian Schneider wrote:
> > Hi JB,
> >
> > the idea is not to replace kafka. Instead we want to provide an API that
> > can abstract kafka as well as other journal based pub sub systems.
> > So actually one implementation of the api would bridge to kafka. We also
> > plan a mongo and a in memory impl.
> >
> > One use case for the api would be a new EventAdmin with history. Another
> is
> > logging that ensures no messages are lost at startup even if the real
> > backend appears later.
> >
> > Christian
> >
> > Am Sa., 22. Dez. 2018 um 13:13 Uhr schrieb Jean-Baptiste Onofré <
> > jb@xxxxxxxxxxxx>:
> >
> >> Hi Christian,
> >>
> >> I'm not sure I got the scope.
> >>
> >> Is it a new pub/sub system like Kafka or Google PubSub (or even
> >> ActiveMQ/AmazonMQ) ?
> >> Is it something like eventadmin on cloud (like we do in Cellar with
> >> Hazelcast) ?
> >>
> >> If it's just a pure pub/sub, I don't see a huge value compared to Kafka.
> >>
> >> About a spec, if we only have one impl of the spec, and the spec is
> >> mainly "focused" on the current impl, not sure it makes sense as well.
> >>
> >> Thoughts ?
> >>
> >> Regards
> >> JB
> >>
> >> On 22/12/2018 09:38, Christian Schneider wrote:
> >>> Some Background
> >>>
> >>> At Adobe I and two colleagues Timothee Maret and Alexei Krainiouk work
> on
> >>> making the AEM content publishing fit for the cloud. It is about
> >>> transporting content from author instances that are only visible to a
> >>> customer to outside facing instances that handle the load. For this
> >> effort
> >>> it is important to have a reliable pub/sub messaging with support of a
> >>> history where each consumer can decide where to start consuming from.
> >>>
> >>> We found that Apache Kafka is a good fit function wise but a little
> heavy
> >>> regarding deployment and management. So to be more versatile we thought
> >>> about encapsulating the messaging using an API and providing
> >>> implementations for different backends.
> >>> Unfortunately there is no existing API that solves this.
> >>>
> >>> Use case and API sketch
> >>>
> >>> So we created a description of the use cases as well as a sketch for a
> >>> minimal API.
> >>> See
> https://github.com/cschneider/journaled-events/blob/master/README.md
> >>>
> >>> Proposal
> >>>
> >>> We would like to develop this API as well as implementations as a sub
> >>> project at Apache Aries.
> >>> The main reason for choosing Aries is that David told us that there is
> >>> interest in a OSGi spec about a similar purpose. So we might also bring
> >>> this into an OSGi spec.
> >>> Alexei and Timothee are not yet committers at Aries. I think we can
> work
> >>> with them using the normal contribution model for the start and make
> them
> >>> committers after a few PRs.
> >>>
> >>> So what do you think?
> >>> As a first step I would like to discuss if the Aries PMC is interested
> in
> >>> giving this effort a home.  After that we can discuss the actual API as
> >>> well as possible usages and backends.
> >>>
> >>> Cheers,
> >>> Christian
> >>>
> >>
> >> --
> >> Jean-Baptiste Onofré
> >> jbonofre@xxxxxxxxxx
> >> http://blog.nanthrax.net
> >> Talend - http://www.talend.com
> >>
> >
> >
>
> --
> Jean-Baptiste Onofré
> jbonofre@xxxxxxxxxx
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>


-- 
-- 
Christian Schneider
http://www.liquid-reality.de

Computer Scientist
http://www.adobe.com