Optimising Data Storage Utilisation

An analysis of plausible scenarios to demonstrate the basic reasons behind low storage utilization, and the possible means for improving the same.

Let me begin my analysis through an anecdote.

Imagine if you were the CTO or the head of IT infrastructure of a company. There may have been instances when you would have approached the CEO or CFO for budgetary allocation for purchasing data storage.

It is but natural that if you were asked about the present utilisation of disk storage, having done your homework, you would have replied that your utilisation is around 40-50 per cent and substantiated it with data.

Herein lays the catch. The CFO promptly tells you to try to optimise data storage usage to around 75-80 per cent and then return for funds. You would have returned thinking what you should do or have done.

Now, let us try to analyse plausible scenarios to demonstrate the basic reasons behind low storage utilisation, and the possible means for improving the same.

Scenario 1:

You have multiple, heterogeneous storage systems (SAN/NAS) of different make, model and age. They do not talk to each other. Each one has a good amount of stranded space which cannot be made use of.

Solution:

Today there are tools and appliances available to integrate and consolidate the SAN and NAS boxes and then deploy virtualisation managed by a single console. This will help you reallocate the total available space and reclaim the stranded space for use.

Scenario 2:

You have a large number of servers running Windows, Solaris and different flavours of Unix. Each server has a large amount of DAS (Direct attached storage) which are not accessible by any other server. The disks are of heterogeneous interfaces e.g. SCSI, SAS, SATA etc. cannot be consolidated together in one array or box.

Solution:

We first need to know what spare capacity we have and where we have that. You may try mapping some shared drives or NFS mount across Unix platforms, but that’s not always productive. Today we have software available which will integrate new generation blade servers with FC SAN to old generation servers with DAS to give you a private cloud computing platform.

Scenario 3:

A project team owns the asset and is not agreeing to share the same. Initially they have allocated a certain amount of storage for the application/database, perhaps to take care of the business growth in five years. They have perhaps forgotten or never looked back, reviewed the usage, nor are they ready to share the disk space with any other application on the pretext of data security.

Solution:

You perhaps do not have a central support team having a storage resource monitoring tool to discover, manage and control the usage of storage .Perhaps your team has been a part of deliberate over provisioning of storage at the beginning of the project to avoid mid-term upgrades and Capex.

Here it is advisable to deploy an SRM management tool (open source tools available) and discourage over provisioning of disk space – use thin provisioning technology, which allows you to grow as you need. Charging back the project for disk space allotted will discourage over provisioning.

Scenario 4:

The storage may contain multiple copies of the same data (e.g. end user back-up for their mail files may have duplicate copies of attachments). This will result in in-effective use of storage.

Solution:

You may deploy de-duplication software (both at source and target level) to achieve efficiencies up-to 55 per cent in certain cases. This will also reduce back-up storage requirements, backup time, and operational costs for WAN replication. For back-up incremental backup instead of regular full backup will reduce storage requirements.

Scenario 5:

The projects do not have the discipline to cleanup storage space by:
(i) removing old unused files
(ii) archiving useful but old data appropriately

Solution:

Define a process for data clean-up on a regular basis. Identify useful data to be archived and use an archiving software to archive the data in lower tiered storage like NAS and Just a Box of Disks (JBODs).

Scenario 6:

In case you have the state–of-the-art storage boxes with FC and FCOE interfaces, and you want to further increase the utilisation, then a mix and match of thin provisioning, storage area virtualization, de-duplication can be deployed to enhance utilisation of the existing storage.

However, a word of caution! One should not go overboard with thin provisioning, as increase in thin provisioned storage may not be as smooth as it is claimed to be.

Website: http://www.ciol.com/