git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PROPOSE] Backup and Recovery Framework


Now outside of our ACS, I use Veeam for vCloud and vSphere and use Alike
for our XS pools. Inside ACS, the VMs use MaxBackup

On Thu, Jun 7, 2018 at 11:09 AM, Todd Pigram <todd@xxxxxxxxxxxxxx> wrote:

> I agree completely. That is why I use inside VM backup products currently.
>
> On Thu, Jun 7, 2018 at 11:07 AM, Will Stevens <wstevens@xxxxxxxxxxxx>
> wrote:
>
>> I have not been following this thread entirely, but I would not count on
>> our snapshot mechanism for backups.  If something like Veeam is used, then
>> an agent is in play and that agent will quiesce the running processes,
>> such
>> as MySQL.  This is important in order to do backup and DR as you will lose
>> transactions in memory and can potentially have a corrupted backup.
>>
>> Snapshots are not backups!!!
>>
>> *Will Stevens*
>> Chief Technology Officer
>> c 514.826.0190
>>
>> <https://goo.gl/NYZ8KK>
>>
>>
>> On Thu, Jun 7, 2018 at 10:13 AM Rafael Weingärtner <
>> rafaelweingartner@xxxxxxxxx> wrote:
>>
>> > Ok, now I start to understand.  So, we will be handling snapshots and
>> they
>> > will be sent to a third-party system.
>> >
>> > Now let’s proceed. If I configure this “feature” with external=false.
>> What
>> > does it do? Nothing? I mean, what is the purpose of this “external”
>> > parameter?
>> >
>> >
>> > On Thu, Jun 7, 2018 at 4:09 PM, Nicolas Vazquez <
>> > Nicolas.Vazquez@xxxxxxxxxxxxx> wrote:
>> >
>> > > Its format may depend on the provider, but under the hood it is a
>> > snapshot
>> > > which can be full or incremental and it is copied and stored in a
>> backup
>> > > repository in one of the backup servers. For example, if you use
>> Veeam,
>> > you
>> > > will find your backups stored as .vbk (for full backups) and .vib (for
>> > > incremental backups). It is the task of the backup provider to create
>> and
>> > > move these files to the backup repository.
>> > >
>> > > ________________________________
>> > > From: Rafael Weingärtner <rafaelweingartner@xxxxxxxxx>
>> > > Sent: Thursday, June 7, 2018 10:39:35 AM
>> > > To: dev
>> > > Subject: Re: [PROPOSE] Backup and Recovery Framework
>> > >
>> > > Ok, there are a lot of things, which are kind of blurry to me. So,
>> let’s
>> > > try to go step by step.
>> > >
>> > > Let me ask again. What is a backup of the VM? You mention
>> restore-point.
>> > > What is it? Is it a snapshot in the hypervisor (complete/partial
>> > snapshot)?
>> > > If it is not a snapshot,  what is it then? Are you installing some
>> agent
>> > in
>> > > the VM and then copying/mirroring the files in the VM’s file system?
>> > >
>> > > On Thu, Jun 7, 2018 at 3:26 PM, Nicolas Vazquez <
>> > > Nicolas.Vazquez@xxxxxxxxxxxxx> wrote:
>> > >
>> > > > Hi Daniel and Rafael,
>> > > >
>> > > >
>> > > > I have updated the diagram picture on the FS, please let me know if
>> the
>> > > > problem persists.
>> > > >
>> > > >
>> > > > @<mailto:daniel.herrmann@xxxxxxxxxxxxxxxx>Daniel you are right
>> about
>> > > > restoring VMs, please let me update the FS. Regarding your
>> question, we
>> > > > have considered the backups as a first class citizens, meaning that
>> a
>> > > > backup would continue existing unless its originating VM has been
>> > > > removed/expunged. This way, listBackups API method would list them
>> in
>> > > order
>> > > > to allow a user to restore a deleted VM from a backup.
>> > > >
>> > > >
>> > > > @Rafael the idea of the Dummy provider is simply returning the
>> expected
>> > > > entities from a backup provider. For example, the BackupManager
>> could
>> > ask
>> > > > the BackupProvider to list its backups and would expect a list of
>> > > backups.
>> > > > As the Dummy Provider implements the BackupProvider interface, it
>> has
>> > its
>> > > > own implementation of listBackups() method and simply returns some
>> mock
>> > > > entities to the Backup Manager. It does not store any backup, it
>> simply
>> > > > creates them in execution time and returns them.
>> > > >
>> > > >
>> > > > From the CloudStack point of view, Backups are stored externally, on
>> > the
>> > > > Backup and Recovery provider servers and are handled by them. We
>> keep
>> > > track
>> > > > of the existing backups in the Backup provider. Implementation may
>> > depend
>> > > > on the Backup and Recovery provider, would consist on snapshots, but
>> > are
>> > > > handled by the provider and not by CloudStack.
>> > > >
>> > > >
>> > > > Backup policies are created on the Backup and Recovery provider and
>> > > > provide retention options and periods as well as backup times. For
>> > > example,
>> > > > the provider may expose Golden, Silver and Bronze policies and have
>> > > > different options for each of them.
>> > > >
>> > > >
>> > > > The 'external' parameter on the listBackupPolicies API method would
>> > only
>> > > > ask the Backup and Recovery provider to list their policies if it is
>> > set
>> > > to
>> > > > true. If it is not set, or set to false, it just simply returns the
>> > > > existing mapped policies in CloudStack. Initially, CloudStack does
>> not
>> > > > store any backup policy, those should be imported/mapped by the
>> > > > 'importBackupPolicy' API method.
>> > > >
>> > > >
>> > > > Finally, we consider backups as restore points. As mentioned above,
>> > > > backups are stored on the Backup and Recovery provider servers, so
>> we
>> > > keep
>> > > > a mapping between the CloudStack entity and the backup, which can be
>> > used
>> > > > to restore VMs.
>> > > >
>> > > >
>> > > > Please let me know if anything.
>> > > >
>> > > >
>> > > > Regards,
>> > > >
>> > > > Nicolas Vazquez
>> > > >
>> > > > ________________________________
>> > > > From: Rafael Weingärtner <rafaelweingartner@xxxxxxxxx>
>> > > > Sent: Thursday, June 7, 2018 6:55:57 AM
>> > > > To: dev
>> > > > Subject: Re: [PROPOSE] Backup and Recovery Framework
>> > > >
>> > > > Hi Nicolas,
>> > > >
>> > > > The idea is great. I have some questions though.
>> > > >
>> > > > Before getting to the questions, your pictures are broken, or at
>> least
>> > > they
>> > > > seem broken to me.
>> > > >
>> > > > How would the “dummy” backup provider work? I mean, what does it do?
>> > > where
>> > > > does it storage the "backup"?
>> > > > What is the difference from this Backup to a Snapshot? Is it simply
>> > > storing
>> > > > a snapshot outside the cloud structure?
>> > > > How would these “backup policies” look like? I mean, what can I
>> > > configure?
>> > > > Regarding that parameter “external” of your backup policy, how
>> would it
>> > > > work if we set it to false?
>> > > > What are you defining as a backup (what do you mean by backup in the
>> > > > context of this feature?)? I mean, is it simply a snapshot file that
>> > you
>> > > > store somewhere else?
>> > > >
>> > > >
>> > > > Nicolas.Vazquez@xxxxxxxxxxxxx
>> > > > www.shapeblue.com<http://www.shapeblue.com>
>> > > > ,
>> > > > @shapeblue
>> > > >
>> > > >
>> > > >
>> > >
>> > > Nicolas.Vazquez@xxxxxxxxxxxxx
>> > > www.shapeblue.com
>> > > ,
>> > > @shapeblue
>> > >
>> > >
>> > >
>> > > > On Thu, Jun 7, 2018 at 6:05 AM, <daniel.herrmann@xxxxxxxxxxxxxxxx>
>> > > wrote:
>> > > >
>> > > > > Hi all,
>> > > > >
>> > > > > +1 for this feature request, thank you for bringing this to life!
>> > > Volume
>> > > > > snapshots are what our users currently use as backup solution,
>> which
>> > > has
>> > > > > some inherent problems such as handling large volumes and the
>> > handling
>> > > of
>> > > > > multiple volumes (consistency).
>> > > > >
>> > > > > One thought: when restoring a VM, I think there are two cases to
>> > > > consider:
>> > > > >
>> > > > > - The VM is still in CS and the user just wants to restore an old
>> > > version
>> > > > > of the VM
>> > > > > - The VM has already been deleted and the user wants to restore
>> the
>> > > > > deleted VM
>> > > > >
>> > > > > As listBackups is described as " listBackups API method. List
>> > existing
>> > > > > backups for a VM". Is the second use case supported at all?
>> > > > >
>> > > > > I like the approach not to define the backup policies in CS but in
>> > > > > whatever tool is used in the end, as otherwise a lot of features
>> > might
>> > > be
>> > > > > lost. We as services providers need to create them in the first
>> > place,
>> > > > but
>> > > > > I think this is the better solution.
>> > > > >
>> > > > > Regards
>> > > > > Daniel
>> > > > >
>> > > > > On 06.06.18, 14:43, "Nicolas Vazquez" <
>> > Nicolas.Vazquez@xxxxxxxxxxxxx>
>> > > > > wrote:
>> > > > >
>> > > > >     Hi all,
>> > > > >
>> > > > >
>> > > > >     We would like to introduce a new framework into CloudStack
>> that
>> > > will
>> > > > > allow adding Backup and Recovery providers as plugins. This
>> framework
>> > > > goal
>> > > > > is allowing users to backup their guest VMs for recovery purposes.
>> > > > >
>> > > > >
>> > > > >     Please find the FS on this link: https://cwiki.apache.org/
>> > > > > confluence/display/CLOUDSTACK/Backup+and+Recovery+Framework
>> > > > >
>> > > > >
>> > > > >     Regards,
>> > > > >
>> > > > >     Nicolas Vazquez
>> > > > >
>> > > > >     Nicolas.Vazquez@xxxxxxxxxxxxx
>> > > > >     www.shapeblue.com<http://www.shapeblue.com>
>> > > > >     ,
>> > > > >     @shapeblue
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > >
>> > > >
>> > > > --
>> > > > Rafael Weingärtner
>> > > >
>> > >
>> > >
>> > >
>> > > --
>> > > Rafael Weingärtner
>> > >
>> >
>> >
>> >
>> > --
>> > Rafael Weingärtner
>> >
>>
>
>
>
> --
>
>
> Todd Pigram
> http://about.me/ToddPigram
> www.linkedin.com/in/toddpigram/
> @pigram86 on twitter
> https://plus.google.com/+ToddPigram86
> Mobile - 216-224-5769
>
> PGP Public Key
> <http://pgp.mit.edu/pks/lookup?op=get&search=0x96B7B0F0C55933BB>
>
>


-- 


Todd Pigram
http://about.me/ToddPigram
www.linkedin.com/in/toddpigram/
@pigram86 on twitter
https://plus.google.com/+ToddPigram86
Mobile - 216-224-5769

PGP Public Key
<http://pgp.mit.edu/pks/lookup?op=get&search=0x96B7B0F0C55933BB>