git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Proposal] Create new sub project for journaled messaging


OK, it's clear now.

It sounds a bit like Decanter (EventAdmin sent to a storage backend), no ?

Regards
JB

On 22/12/2018 14:30, Christian Schneider wrote:
> Hi JB,
> 
> the idea is not to replace kafka. Instead we want to provide an API that
> can abstract kafka as well as other journal based pub sub systems.
> So actually one implementation of the api would bridge to kafka. We also
> plan a mongo and a in memory impl.
> 
> One use case for the api would be a new EventAdmin with history. Another is
> logging that ensures no messages are lost at startup even if the real
> backend appears later.
> 
> Christian
> 
> Am Sa., 22. Dez. 2018 um 13:13 Uhr schrieb Jean-Baptiste Onofré <
> jb@xxxxxxxxxxxx>:
> 
>> Hi Christian,
>>
>> I'm not sure I got the scope.
>>
>> Is it a new pub/sub system like Kafka or Google PubSub (or even
>> ActiveMQ/AmazonMQ) ?
>> Is it something like eventadmin on cloud (like we do in Cellar with
>> Hazelcast) ?
>>
>> If it's just a pure pub/sub, I don't see a huge value compared to Kafka.
>>
>> About a spec, if we only have one impl of the spec, and the spec is
>> mainly "focused" on the current impl, not sure it makes sense as well.
>>
>> Thoughts ?
>>
>> Regards
>> JB
>>
>> On 22/12/2018 09:38, Christian Schneider wrote:
>>> Some Background
>>>
>>> At Adobe I and two colleagues Timothee Maret and Alexei Krainiouk work on
>>> making the AEM content publishing fit for the cloud. It is about
>>> transporting content from author instances that are only visible to a
>>> customer to outside facing instances that handle the load. For this
>> effort
>>> it is important to have a reliable pub/sub messaging with support of a
>>> history where each consumer can decide where to start consuming from.
>>>
>>> We found that Apache Kafka is a good fit function wise but a little heavy
>>> regarding deployment and management. So to be more versatile we thought
>>> about encapsulating the messaging using an API and providing
>>> implementations for different backends.
>>> Unfortunately there is no existing API that solves this.
>>>
>>> Use case and API sketch
>>>
>>> So we created a description of the use cases as well as a sketch for a
>>> minimal API.
>>> See https://github.com/cschneider/journaled-events/blob/master/README.md
>>>
>>> Proposal
>>>
>>> We would like to develop this API as well as implementations as a sub
>>> project at Apache Aries.
>>> The main reason for choosing Aries is that David told us that there is
>>> interest in a OSGi spec about a similar purpose. So we might also bring
>>> this into an OSGi spec.
>>> Alexei and Timothee are not yet committers at Aries. I think we can work
>>> with them using the normal contribution model for the start and make them
>>> committers after a few PRs.
>>>
>>> So what do you think?
>>> As a first step I would like to discuss if the Aries PMC is interested in
>>> giving this effort a home.  After that we can discuss the actual API as
>>> well as possible usages and backends.
>>>
>>> Cheers,
>>> Christian
>>>
>>
>> --
>> Jean-Baptiste Onofré
>> jbonofre@xxxxxxxxxx
>> http://blog.nanthrax.net
>> Talend - http://www.talend.com
>>
> 
> 

-- 
Jean-Baptiste Onofré
jbonofre@xxxxxxxxxx
http://blog.nanthrax.net
Talend - http://www.talend.com