git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Proposing an Apache Cassandra Management process


I agree with Jon and I think folks who are talking about tech debts in
Reaper should elaborate with examples about these tech debts. Can we be
more precise and list them down? I see it spread out over this long email
thread!!

On Sun, Sep 9, 2018 at 6:29 AM Elliott Sims <elliott@xxxxxxxxxxxxx> wrote:

> A big one to add to your list there, IMO as a user:
> * API for determining detailed repair state (and history?).  Essentially,
> something beyond just "Is some sort of repair running?" so that tools like
> Reaper can parallelize better.
>
> On Sun, Sep 9, 2018 at 8:30 AM, Stefan Podkowinski <spod@xxxxxxxxxx>
> wrote:
>
> > Does it have to be a single project with functionality provided by
> > multiple plugins? Designing a plugin API at this point seems to be a bit
> > early and comes with additional complexity around managing plugins in
> > general.
> >
> > I was more thinking into the direction of: "what can we do to enable
> > people to create any kind of side car or tooling solution?". Thinks like:
> >
> > Common cluster discovery and management API
> > * Detect local Cassandra processes
> > * Discover and receive events on cluster topology
> > * Get assigned tokens for nodes
> > * Read node configuration
> > * Health checks (as already proposed)
> >
> > Any side cars should be easy to install on nodes that already run
> Cassandra
> > * Scripts for packaging (tar, deb, rpm)
> > * Templates for systemd support, optionally with auto-startup dependency
> > on the Cassandra main process
> >
> > Integration testing
> > * Provide basic testing framework for mocking cluster state and messages
> >
> > Support for other languages / avoid having to use JMX
> > * JMX bridge (HTTP? gRPC?, already implemented in #14346?)
> >
> > Obviously the whole side car discussion is not moving into a direction
> > everyone's happy with. Would it be an option to take a step back and
> > start implementing such a tooling framework with scripts and libraries
> > for the features described above, as a small GitHub project, instead of
> > putting an existing side-car solution up for vote? If that would work
> > and we get people collaborating on code shared between existing
> > side-cars, then we could take the next step and think about either
> > revisit the "official Cassandra side-car" topic, or add the created
> > client tooling framework as official sub-project to the Cassandra
> > project (maybe via Apache incubator).
> >
> >
> > On 08.09.18 02:49, Joseph Lynch wrote:
> > > On Fri, Sep 7, 2018 at 5:03 PM Jonathan Haddad <jon@xxxxxxxxxxxxx>
> > wrote:
> > >> We haven’t even defined any requirements for an admin tool. It’s hard
> to
> > >> make a case for anything without agreement on what we’re trying to
> > build.
> > >>
> > > We were/are trying to sketch out scope/requirements in the #14395 and
> > > #14346 tickets as well as their associated design documents. I think
> > > the general proposed direction is a distributed 1:1 management sidecar
> > > process similar in architecture to Netflix's Priam except explicitly
> > > built to be general and pluggable by anyone rather than tightly
> > > coupled to AWS.
> > >
> > > Dinesh, Vinay and I were aiming for low amounts of scope at first and
> > > take things in an iterative approach with just enough upfront design
> > > but not so much we are unable to make any progress at all. For example
> > > maybe something like:
> > >
> > > 1. Get a super simple and non controversial sidecar process that ships
> > > with Cassandra and exposes a lightweight HTTP interface to e.g. some
> > > basic JMX endpoints
> > > 2a. Add a pluggable execution engine for cron/oneshot/scheduled jobs
> > > with the basic interfaces and state store and such
> > > 2b. Start scoping and implementing the full HTTP interface, e.g.
> > > backup status, cluster health status, etc ...
> > > 3a. Start integrating implementations of the jobs from 2a such as
> > > snapshot, backup, cluster restart, daemon + sstable upgrade, repair,
> > > etc
> > > 3b. Start integrating UI components that pair with the HTTP interface
> > from 2b
> > > 4. ?? Perhaps start unlocking next generation operations like moving
> > > "background" activities like compaction, streaming, repair etc into
> > > one or more sidecar contained processes to ensure the main daemon only
> > > handles read+write requests
> > >
> > > There are going to be a lot of questions to answer, and I think trying
> > > to answer them all up front will mean that we get nowhere or make
> > > unfortunate compromises that cripple the project from the start. If
> > > people think we need to do more design and discussion than we have
> > > been doing then we can spend more time on the design, but personally
> > > I'd rather start iterating on code and prove value incrementally. If
> > > it doesn't work out we won't release it GA to the community ...
> > >
> > > -Joey
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@xxxxxxxxxxxxxxxxxxxx
> > > For additional commands, e-mail: dev-help@xxxxxxxxxxxxxxxxxxxx
> > >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@xxxxxxxxxxxxxxxxxxxx
> > For additional commands, e-mail: dev-help@xxxxxxxxxxxxxxxxxxxx
> >
> >
>