Move a Spectrum Scale Filesystem to an new disk

Very few time but it's happened that you need to move a filesystem from one Disk system to another Disk system in IBM Spectrum Scale (GPFS).

And here is a little how to how to move a filesystem from one disk to another.

Basic overview

·        Create the new disks in Linux

·        Create a NSD on the new disks

·        Add the new NSD to the file system

·        Remove the old NSDs and let Spectrum Scale to migrate the data automatic.

·        Delete the NSDs

·        Delete the Disks in Linux

So let's start with Scanning the SCSI bus for new Disks on all NSD Servers

[root@server01~] echo "- - -" > /sys/class/scsi_host/hostX/scan

  

Replace the X with the correct SCSI Host adapter.

Create a NSD Stanza file on one of the Spectrum Scale Server, with your settings.

Down below can you see an example how it can look like.

%nsd: device=/dev/sdg nsd=mynewnsd01 servers=server01,server02,server03 usage=dataAndMetadata pool=system

Create the NSD by running mmcrnsd with flag -F and the text file you just created.

[root@server01~] mmcrnsd -F mynewnsd.txt mmcrnsd: Processing disk sdg mmcrnsd: Propagating the cluster configuration data to all   affected nodes.  This is an asynchronous process.

Let's now add the disk to our filesystem by using mmadddisk

[root@server01~] mmadddisk <FILESYSTEM> -F mynewnsd.txt  The following disks of ftp will be formatted on node server.domain.com:     mynewnsd01: size XXX00 MB Extending Allocation Map Checking Allocation Map for storage pool system Completed adding disks to file system XXXXX. mmadddisk: Propagating the cluster configuration data to all   affected nodes.  This is an asynchronous process.

Now let's remove the old NSDs from the filesystem.

It's probably recommended to run this command in Screen mode, just in case if something happening with you ssh session.

[root@server01~] mmdeldisk <filesystem> mynewnsd01 Deleting disks … Scanning file system metadata, phase 1 … Scan completed successfully. Scanning file system metadata, phase 2 … Scan completed successfully. Scanning file system metadata, phase 3 … Scan completed successfully. Scanning file system metadata, phase 4 … Scan completed successfully. Scanning user file metadata …  100.00 % complete on Mon Jun 20 12:01:25 2016  (     82176 inodes with total      38389 MB data processed) Scan completed successfully. Checking Allocation Map for storage pool system tsdeldisk completed. mmdeldisk: Propagating the cluster configuration data to all   affected nodes.  This is an asynchronous process.

When you have delete the disks you maybe want to rebalance the filesystem, either you can use "-b" together with mmdeldisk or you can run mmrestripefs

[root@server01~] mmrestripefs <filesystem> -b Scanning file system metadata, phase 1 … Scan completed successfully. Scanning file system metadata, phase 2 … Scan completed successfully. Scanning file system metadata, phase 3 … Scan completed successfully. Scanning file system metadata, phase 4 … Scan completed successfully. Scanning user file metadata …   10.22 % complete on Mon Jun 20 12:05:49 2016  (     81094 inodes with total       5673 MB data processed)   …. 100.00 % complete on Mon Jun 20 12:11:31 2016  (     82247 inodes with total      51263 MB data processed) Scan completed successfully.

Before you delete the devices in Linux do you need to delete the NSDs.

The command you need to use is mmdelnsd, but to be able to run that command do you need to pull out the NSD ID and to do that you need to run "mmlsnsd -m"

[root@server01~] mmlsnsd -m  Disk name         NSD volume ID           Device         Node name              Remarks -----------------------------------------------------------------------------------------------------------  oldnsd01        C0A8632E5640AE64   /dev/sdb       server01                server node  oldnsd01        C0A8632E5640AE64   /dev/sdb       server02                server node  oldnsd01        C0A8632E5640AE64   /dev/sdb       server03                server node ….

On each node do you need to

[root@server01~] mmdelnsd -p <NSD volume ID> mmdelnsd: Processing disk oldnsd01 mmdelnsd: Propagating the cluster configuration data to all   affected nodes.  This is an asynchronous process.

 Now can you run your SD device from Linux.

[root@server01~] echo 1 > /sys/block/sdX/device/delete

Replace X with the device you want to delete and in my case is it /dev/sdb.

If you have Multipath enabled your probably want to run "multipath -f <multipath device>