Design your own Global Name Space

I met a customer today where they look for a true global filesystem.
The customer has today a data center in Sweden and need to open another data center in north america.

They customer have today a calculation cluster based on open source. To be able to share data between their calucation nodes are they using NFS share today.
Based on regular NFS and the few functions they are using today did they see problems with either the NFS cache wasn't syncronized with the source or they client nodes access time to the source file was to slow.

Behind the source NFS Server was a regular midrange storage system from a large global disk vender.
And they even try with the vendors version of "flash system" that is SSD and not true flash, but still it is to slow.

The customer has today approx 1 PB of data and yearly are they growing with 2,5PB of data and each calculation batch is approx 40-80TB data per calculation.
The data are normally only used once or maybe twice during the data life time, and the life time per 2,5PB of data is 5-10 years.

What we at Cristie suggested was a IBM Spectrum Scale File System and a Spectrum Scale Server.
To be able to get a quick and cost effective solution did we choose does products to be able to have a full flexible and open filesystem that the customer can grow with any kind of hardware and still use easy building block solution such Spectrum Scale Server.
All archive data are we saving to a IBM TS4500 Tape library and using Spectrum Archive solution.

Instead of buying a true flash system or a Spectrum Scale Server with only SSD, did the customer ask for a less cost solution.
Because of the Spectrum Scale Server GL6 generates much better IOPS and GB/s then there existing solution will we only need a much smaller SSD cache then the customer first thought.

Because we are using Spectrum Scale Servers, do we need a server based gateway that we choose to run Linux, to be able to manage Spectrum Archive.
We did then fill up all server with maximum numbers of cheap SSDs from Lenovo, and to get better performace and fail-over cluster functionallity did we choose two nodes to a single GPFS SSD pool.

Now does the customer have a full Spectrum Scale solution using Disk, SSD and tape, and all working together as one single namespace filesystem. The customer don't need to use NFS Cache or regular slow NFS connetions anymore.
The customer get a true global software define filesystem, that is flexible and scalible with any hardware.
Now can do they have a solution they can expand easly in north america, but also expand the global namespace to other countries such Sweden, by using a software such Aspera to be able transfer data between the sites can they then save a lot of money buy not investing in a large network infrastructure.


Together with Cristie World Wide delivery functionallity did the customer make a success implementation, by sending the hardware to another country but still been invoicing in Sweden.


Comments

Post a Comment

Popular posts from this blog

Move a Spectrum Scale Filesystem to an new disk

Very few time but it's happened that you need to move a filesystem from one Disk system to another Disk system in IBM Spectrum Scale (GPFS). And here is a little how to how to move a filesystem from one disk to another. Basic overview ·         Create the new disks in Linux ·         Create a NSD on the new disks ·         Add the new NSD to the file system ·         Remove the old NSDs and let Spectrum Scale to migrate the data automatic. ·         Delete the NSDs ·         Delete the Disks in Linux So let's start with Scanning the SCSI bus for new Disks on all NSD Servers [root@server01~] echo "- - -" > /sys/class/scsi_host/hostX/scan    Replace the X with the correct SCSI Host adapter. Create a NSD Stanza file...
Read more

Manual Upgrade IBM Spectrum Protect 7.1.x to 8.1.x

Image
I'm ended up in a situations where a IBM Spectrum Protect (ISP) server was lock down with limitation who is allowed to login over SSH. The main reason of this is because the server was connected to a Linux based LDAP environment after the installation of ISP was done. So now did we end up in a situation where we not where able to let the local ISP user account couldn't ssh to itself. And because we couldn't ssh even though localhost to our ISP user (let's call it tsminst1). And I guess I'm not alone with this issue so I guess this could be a create blog post for my Spectrum Protect / TSM friends. First of all if you are using SuSE Linux Enterprise Server 11 you most run minimum SLES 11 Service Pack 4. Here is a quick and dirty how to upgrade your SLES 11 to SP4. For more information please go to following link: https://www.suse.com/support/kb/doc/?id=7016711 Before you start upgrade anything, make sure you backup the ISP database/volhistory and devic...
Read more

Upgrade GPFS 3.4 to Spectrum Scale 4.1.1

It's funny because it's true…. That´s how I want to start with today's story… And today will I be more technical then normally. Let's start from beginning before I complain too much and showing how easy it is. I have now heard twice where end-users of GPFS heard it isn't possible to upgrade from GPFS 3.4 filesystem to Spectrum Scale 4.x without to shut down the entire cluster. This is not true and I will now show it for you how easy it is and how it's works.  I want with today's topic explain how easy it is to upgrade a GPFS 3.4 filesystem to Spectrum Scale 4.1.1 without to shut down the cluster. This how my test system looks like and see the time stamp when I start with my upgrade. [root@gpfs1 gpfs]# date Tue Jul 28 14:52:21 CEST 2015 [root@gpfs1 src]# mmlscluster GPFS cluster information ========================   GPFS cluster name:         gpfstest.cristie.se   GPFS cluster ...
Read more