git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Apache CloudStack Generic Key-Value and Communication Plug-in


That is interesting Ivan. You answer create another question.

You say that the plugin uses an Elasticsearch instance and that a clustered
ES is supported as
well. Is the plugin going to manage (install this ES) or you (the operator)
will have to deploy and manage the ES and only configure/point the ES
instance to ACS?


On Thu, Sep 20, 2018 at 11:34 PM, Ivan Kudryavtsev <kudryavtsev_ia@xxxxxxxxx
> wrote:

> Hello, Daan, Rafael.
>
> Ok, let's discover some points.
>
> Rafael:
> All the stuff is managed in Elasticsearch, clustered ES is supported as
> well. As for now, value datatype is 'keyword' and doesn't fit well for
> blobs. But, actually, there are no practical limitations except we don't
> want press the CS server with streaming for blobs as it's design is not
> well designed for long lasting jobs like that.
>
> To adress the case we also plan to develop artifact storage which is
> intended to keep blobs with versioning and managed security. Probably will
> use Ceph under the hood.
>
> Daan:
> Let's go with the cases. Actually, in the past I published the announcement
> for log management plugin which intention is to provide access to VM logs
> from the API and other interfaces. Now we develop CLI utility for it. That
> plugin helps to observe what logs are generate during VM deployment.
>
> This plugin (KV) helps to deploy complex topologies, like Docker Swarm,
> Kubernetes, MySQL Galera replicaset, DC/OS, monitor the state of the
> deployments and do trivial things:
>
> Observe current VM load, RAM availability, disk space. All these parameters
> can be observed in historical perspective or last values.
>
> A quorum example. Say, you deploy two VMs programmatically and they have to
> decide who is Docker master and who is worker, so you create shared storage
> and both VMs place they UUID in 'order' key. They check the history for
> 'order' and find who is the first. Next, the first generates cluster
> initialization and places secret key to 'secret'. After, worker reads it
> and joins the cluster. Want to add a new VM to the cluster, just pass that
> KV to it as well. Of cource, proper templates with bootstrap code are
> required.
>
> Actually, there are tons of very sophisticated deployments can be
> implemented in a very generic way with theese two plugins.
>
> Another piece of software to be developed is one-time accessor plugin which
> is important for secure sharing of various credentials when multi tenant
> VMs are used.
>
> Have a good day.
>
>
> пт, 21 сент. 2018 г., 2:21 Rafael Weingärtner <rafaelweingartner@xxxxxxxxx
> >:
>
> > That is very interesting Ivan!
> >
> > I have a few questions though:
> >
> >    - Do you have a limit on ”value” size? Can it be a binary of a file?
> >    - Where do you guys store these “keys” and ”values”?
> >
> >
> > On Wed, Sep 19, 2018 at 4:02 AM, Ivan Kudryavtsev <
> > kudryavtsev_ia@xxxxxxxxx>
> > wrote:
> >
> > > Hello, community.
> > >
> > > We glad to announce a new Open Source plug-in for Apache CloudStack
> which
> > > helps to maintain generic key-value storages. There are three types are
> > > supported right now:
> > >
> > > - temporary KV with TTL (many per account);
> > > - persistent KV bound to VM (one per VM);
> > > - persistent KV bound to account (many per account).
> > >
> > > KV storages support a commit-log based approach where old values are
> not
> > > erased with new ones and it provides the access to the last key/values
> > and
> > > to the history which helps to maintain things like quorums and other
> > > operations which require serialized access.
> > >
> > > The plugin uses Elasticsearch. I
> > >
> > > If you are interested to know more, contact our website for the full
> > > announcement and download details:
> > > https://bitworks.software/en/2018-09-19-acs-vm-kv-storage-plugin.html
> > >
> > > All questions, improvements and considerations are welcome!
> > >
> > > --
> > > With best regards, Ivan Kudryavtsev
> > > Bitworks LLC
> > > Cell: +7-923-414-1515
> > > WWW: http://bitworks.software/ <http://bw-sw.com/>
> > >
> >
> >
> >
> > --
> > Rafael Weingärtner
> >
>



-- 
Rafael Weingärtner