git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Secondary storage issue with XenServer


Hello everyone,

 

                I recently found about Cloudstack and decided to give it a try and I really like it so far but I bumped into a very weird config issue.

                Here is my test environment.

               

                10.10.10.200 - XenServer 7.1 host (all free patches installed, ntp synced) eth0 is the management and also the only interface connected to the network. There are no bonds.

                               

                10.10.10.201 – Centos 7 NFS server:

                                - firewalld disabled

                                - /etc/exports is:

                                                                /cloud_storage/isos     *(rw,async,no_root_squash,no_subtree_check)

/cloud_storage/vms     *(rw,async,no_root_squash,no_subtree_check)

 

                10.10.10.210 - Cloudstack management server (OS is centos 7, ntp synced)

                                Management network range - 10.10.10.202-209

                                Guest network range – 10.10.10.211-219

                                Storage network range – no ips set.

                                Primary storage is set to use the XenServer host local storage.

 

                10.10.10.150my PC address in the test network (10.10.10.0/24, gate 10.10.10.1)

 

Zone, pod and cluster and host are created successfully.

But when I try to add secondary NFS server in Cloustack the operation fails.

Going through logs I see the following highlights (full log snipets are attached to this mail):

 

Cloudstack-server.log  -

 

                Line 193: 2018-08-21 11:04:08,865 WARN  [c.c.h.x.r.XenServerStorageProcessor] (DirectAgent-413:ctx-37d82b72) (logid:fac8aabf) createFileSR failed! due to the following: SR_BACKEND_FAILURE_61File SR creation error [opterr=fail to mount FileSR. Errno is 32]

                Line 221: 2018-08-21 11:04:08,865 WARN  [c.c.h.x.r.XenServerStorageProcessor] (DirectAgent-413:ctx-37d82b72) (logid:fac8aabf) Catch Exception com.cloud.utils.exception.CloudRuntimeException for template due to com.cloud.utils.exception.CloudRuntimeException: createFileSR failed! due to the following: SR_BACKEND_FAILURE_61File SR creation error [opterr=fail to mount FileSR. Errno is 32]

 

                XenServer SMlog

Aug 21 11:04:08 cloudstack-office SM: [20645] ['mount', '--bind', '/var/cloud_mount/12fb334c-64e6-3206-9373-7ae0627f83d2/template/tmpl/1/1', '/var/run/sr-mount/f517ce41-148e-4f88-9bdc-15d7e2d0eaa6']

Aug 21 11:04:08 cloudstack-office SM: [20645] FAILED in util.pread: (rc 32) stdout: '', stderr: 'mount: mount /var/cloud_mount/12fb334c-64e6-3206-9373-7ae0627f83d2/template/tmpl/1/1 on /run/sr-mount/f517ce41-148e-4f88-9bdc-15d7e2d0eaa6 failed: Stale file handle

Aug 21 11:04:08 cloudstack-office SM: [20645] '

Aug 21 11:04:08 cloudstack-office SM: [20645] Raising exception [61, File SR creation error [opterr=fail to mount FileSR. Errno is 32]]

Aug 21 11:04:08 cloudstack-office SM: [20645] lock: released /var/lock/sm/f517ce41-148e-4f88-9bdc-15d7e2d0eaa6/sr

Aug 21 11:04:08 cloudstack-office SM: [20645] ***** generic exception: sr_attach: EXCEPTION <class 'SR.SROSError'>, File SR creation error [opterr=fail to mount FileSR. Errno is 32]

               

                XenSever –XenSource.log –

Aug 21 11:05:38 cloudstack-office xapi: [debug|cloudstack-office|109042 UNIX /var/lib/xcp/xapi|dispatch:SR.add_to_other_config D:e87a119cfb36|api_effect] SR.add_to_other_config

Aug 21 11:05:38 cloudstack-office xapi: [ info|cloudstack-office|109033 |sm_exec D:143bd363be92|xapi] Session.destroy trackid=a5d69dd0a4351c2751078822b0a8844b

Aug 21 11:05:38 cloudstack-office xapi: [error|cloudstack-office|109033 |SR.attach D:070d838f0344|backtrace] sm_exec D:143bd363be92 failed with exception Storage_interface.Backend_error(_)

Aug 21 11:05:38 cloudstack-office xapi: [error|cloudstack-office|109033 |SR.attach D:070d838f0344|backtrace] Raised Storage_interface.Backend_error(_)

Aug 21 11:05:38 cloudstack-office xapi: [error|cloudstack-office|109033 |SR.attach D:070d838f0344|backtrace] 1/8 xapi @ cloudstack-office Raised at file sm_exec.ml, line 216

 

 

What is weird is that I am able to mount the NFS through XenServer -> new storage -> NFS wizard (using both NFSv3 and NFSv4 protocols). When added I am able to create disks (a.k.a. the host has write access).

 

Some NFS connection/accessibility tests were run from the XenServer hosts with the following results:

-          RPC test

                [root@cloudstack-office tmp]# rpcinfo -t 10.10.10.201 nfs

program 100003 version 3 ready and waiting

program 100003 version 4 ready and waiting

 

-          Mount scan

[root@cloudstack-office /]# showmount -e 10.10.10.201

Export list for 10.10.10.201:

/cloud_storage/vms  *

/cloud_storage/isos *

 

Note: both of those work manually but not through cloudstack

 

-          NFS sr-proble returns this for both shares:

[root@cloudstack-office tmp]# xe sr-probe type=nfs device-config:server=10.10.10.201 device-config:serverpath=/cloud_storage/vms

<?xml version="1.0" ?>

<SRlist/>

 

                Any idea why the NFS cannot be added as secondary storage L?

I am so out of ideas. Sorry for the long mail.

 

               

Best regards,

Jordan Kostov

 

Attachment: cloudstack_and_xen_logs.zip
Description: cloudstack_and_xen_logs.zip