Subject: Re: Multiple Metadata servers on pNFS/PVFS2
Yes, I recently learned that having multiple pNFS MDS servers are not
Has there been some work [either some benchmarks/new design] in this
Phil Carns wrote:
> Hi Nirmal,
> Are you trying to have multiple pNFS MDS servers, or multiple meta
> servers in the PVFS volume?
> I don't believe that the former is supported (but I was hoping someone
> else would chime in- I have not used the pNFS/PVFS implementation
> first hand).
> The latter should work fine without any particular changes to the
> installation steps except for the PVFS configuration files.
> In either case I am pretty sure that the bind option to mount is not
> going to do what you are looking for.
> Nirmal Thacker wrote:
>> I have a question regarding CITI's implementation of pNFS/PVFS2
>> Alternatively you could point me to a mailing lists for users of
>> pNFS/PVFS2 by the CITI group. I cannot seem to find a forum/group who
>> discuss issues of pNFS/PVFS2 [if it is not this one]
>> My problem is as follows:
>> I want to use more than a single Metadata server [MDS] in CITI's
>> pNFS/PVFS2. I use the instructions described here:
>> I understand these are not of the current build, but I am using an older
>> build of pNFS/PVFS2.
>> Now if there are two or more MDS nodes, then how will clients mount the
>> MDS nodes on a single mount point? I am familiar with the mount --bind
>> Is this what we must do?
>> This situation arises on two occasions
>> 1. When the MDS mounts a PVFS2 filesystem. Multiple MDS then mount the
>> MDS PVFS2 filesystems at one point.
>> To quote from the instructions :
>> // Load kernel interface
>> // Run on MDS
>> insmod /usr/local/bin/pvfs2-server/pvfs2.ko
>> /usr/local/bin/pvfs2-server/sbin/pvfs2-client -p
>> mount -t pvfs2 tcp://"mds_server_hostname":3334/pvfs2-fs /mnt/pvfs2
>> 2. When the clients mount an NFS4 filesystem [essentially a PVFS2
>> filesystem exported by the MDS]
>> To quote:
>> Step 7: Mount PVFS2 file system using pNFS
>> * Copy /etc/pvfs2tab file to every client
>> * Example:
>> // On client
>> insmod /usr/local/bin/pvfs2-layout/pvfs2-pnfs.ko
>> /usr/local/bin/pvfs2-layout/sbin/pvfs2-client -p
>> mount -t nfs4 "mds_server_name":/ /mnt/nfs4/
>> In both cases, multiple MDS cause the mounts to overlap. Although I have
>> not tried this out, I suspect this is would result in an incorrect
>> Have you come across this issue when you worked with pNFS/PVFS2 before?
>> Thanks for your time.
>> Nirmal Thacker
>> Pvfs2-users mailing list
>> [email protected]