..

Storage Virtualization

Storage Virtualization is sort of a game changer. The more I think about Storage Virtualization and the advantages it brings in storage environments with adding flexibility with migrations, efficiency, automation, management and importantly adds features that your existing storage arrays that might natively not be supported.

Storage Virtualization will take any storage device that is physically connected to it and remap the physical disks to xLUNs. These xLUNs can now take advantage of all the native features of the Storage Virtualization Array (Engine). Features could include creating Storage Groups, Various Raid Types, Site-to-Site Replication, Pooling of disks, Thin Provisioning, Synchronous Copy, Asynchronous Copy, Local Copy, Stripping, Snapshots, VAAI and Automated Storage Tiering. Again doesn’t matter if your existing Storage natively does not support these features.

Two features that every customer wishes they had right now….VAAI and Policy based Automated Tiering (including Sub-LUN Tiering)

..

Storage Virtualization

..

Who Supports Storage Virtualization

There are several manufacturers that support Storage Virtualization today. Some of the leading storage virtualization arrays/engines include IBM SVC, EMC VPLEX, HP SVSP, HP P9500, Hitachi USPV/ VSP.

..

VAAI

Same can be said about VAAI (vStorage API for Array Integration), an amazing interface that VMware provides for its technology partners to offload rather intense storage related functions natively within storage devices, compared to the old approach where VMware Host did these tasks. This means Storage Processors need to be able to pick up these massive xcopy, lock operations and block zeroing.

Many storage vendors have already provided VAAI support while many have it on their roadmap and have planned release over the next few months. EMC Clariion was supported Day 1, 3Par similarly supports it with 2.3.1 MU2 InForm OS, while HDS supported VAAI on the AMS platform Day 1.

..

Automated Storage Tiering

Automated Storage Tiering is another great feature to have natively within storage arrays, but not every vendor supports it today.

Not all the data you have, need to be on the fastest tier, but as your application writes data, it can write to the fastest tier and then demote if the data is not being used. Similarly if there is any data that is frequently used, based on the policy can be moved up to a higher tier. So in short if you keep a good balance of SSD’s and SATA drives, you should be able to keep all your applications happy, all your users happy, all your DBA’s happy and importantly meet your SLA’s.

So initially the idea was to offer this at a LUN level. Create policy, if the LUN is busy based on the time of the day or the month of the year, move it to a faster tier. But then followed the concept of Sub-LUN Tiering. Well not the entire LUN needs to be moved, only a certain set of blocks, chunks, pages are hot and they need to be promoted to a faster tier. Helps tremendously reduce your operations on arrays and keeps cache free.

Compellent is considered a market leader in Automated Storage Tiering and were the first ones to take it to the market. Followed by HDS, EMC and 3Par. Not all storage vendors offer LUN tiering and Sub-LUN tiering with its storage platforms today.

..

Where am I going?

Well, for a second, lets think…..

The storage environment that you might have today, does not support all the needed features your applications and your business may require. Example VAAI and Automated Storage Tiering including Sub-LUN Tiering.

Why not push the physical Storage assets behind these Virtualization arrays / engines and start taking advantages of the native features offered within them including VAAI and Automated Storage Tiering including Sub-LUN Tiering.

If you are anxiously waiting for features from your existing storage vendors, which may be on their roadmap or may have been promised but never delivered, you do have a choice to closely look at Storage Virtualization and take advantages of these features without a major overhaul in your storage environment.

..

..

There are fundamental concepts of storage Economics, which typically include Thin Provisioning, Deduplication, Zero Page Reclaim, Compression, Reclaimation, Efficiency, Utilization, TCO, ROI, CAPEX, OPEX, etc.

Storage Economics is one of those subjects, everyone likes to hear about, but it’s hard to find it implemented in today’s storage environments.

With that said, a lot of vendors are natively trying to add the concepts of Storage Economics into their storage arrays. With some latest discussions we had with our current visit to Hitachi Japan on the topics of Storage Economics and the core concepts that help customer increase utilization & efficiency in storage environments, here is an attempt to shed some more light on the topic.

We use storage to run our business, to store structured and unstructured data. Data means everything these days. But have we thought about the economics associated with storage?

As consumers, we tend to consume more than necessary at times if we want to have enough buffer, or if we anticipate projected growth, business requirements, customer requirements, technology improvements, and the list goes on. Many vendors these days guarantee 20% more efficiency, 50% less storage, 50% less storage using Thin, etc, etc, etc.

There are several aspects one should consider related to Storage Economics, how your shrinking IT budgets can still meet up with your growing business requirements, and what you can do to keep a balance between both.

..

With various aspects of Storage Economics below, some may be applicable in the SMB space, some in the enterprise space, and some really at all levels. These may turn into the building blocks of your Storage Economics practice:

• It’s important to know what storage you have and where you have it.
• Try to move away from fat provisioning to thin provisioning.
• Use the concepts of Storage Virtualization to increase efficiency and utilization
• Run non-vendor specific SRM (Storage Resource Management) tools for storage optimization and storage management.
• Having a storage management tool is a must. You can still perform your daily task using various native element managers.
• Industry standard average storage utilization numbers range between 35 to 45%. If you can push your storage utilization numbers higher, it will help you drive the cost down phenomenally.
• Implement deduplication; verify your storage array supports deduplication natively. If not, it should be implemented in various parts of your storage like backup, unstructured data, etc.
• Run a heterogeneous environment with multiple vendors in it to keep balance relating to price structures.
• Though ILM is a forgotten word these days, make sure you run tiering within your storage environment that can help you move your data from higher SLA tiers to lower SLA tiers for cost containment purposes.
• Implement storage arrays that natively support Automated Storage Tiering and can automate the movement of data to the required Tiers based on time of the day, policies, spike in usage or business requirements.
• If there are native compression technologies available on the Storage Arrays for secondary or backup storage, implement those as a means to reduce your footprint.
• Look at extending the life of your storage arrays from a typical 3 years to 6 or 7 years.
• Leverage the use of outsourced computing models including Cloud technologies available in the market today. Could be private clouds or public clouds or a mesh of both technologies to reduce the storage footprint and management.
• Budget for your storage requirements and try to live by those even if you have to take drastic measures to keep it under control.
• Try to gain more operational efficiencies within the storage environments.
• Understand the TCO with any new storage purchase, as cost of new storage could include several aspects of implementation including migration, consulting, downtime, missed SLA’s, Training, management, etc.
• Try to reclaim storage as old host systems / server systems are retired or migrated.
• Check for inconsistencies in your Storage environment as those could result in missed SLA’s, downtime and penalties.
• Do not over provision and do not over budget. Its just storage, if you need more you can buy more, but having idle storage doing nothing for years in anticipation of future growth will heavily skew operational storage efficiencies.
• Do not create unnecessary storage management tasks and processes for your storage environment.
• Having backups and good working backups is very important, but do not tie down your storage with numerous copies of snaps, clones, mirrors, BCV’s, etc for a rainy day, rather have a DR plan and copy single instance of data remotely for DR purposes.
• Plot trends for your storage environment. See if trends can help you budget, forecast and provision your storage accurately.
• Remember the larger storage footprint you have, the larger your backup footprints will be, causing more storage space, more backup time windows, more network traffic, slower response times, more tapes, more offsite backups, more backup management cost and possibly more licensing cost.
• Get away from managing islands of storage; rather move to a more centralized storage management, long-term effects are amazing.
• Try to reduce licensing cost around storage software. The less storage you deploy, the less licensing per TB cost that you will pay.

..

There are many other factors you can implement; here are a few different posts from the past talking about this topic.

http://storagenerve.com/deepdive/storage-optimization/

..

A Google search on Storage Economics yields

http://www.google.com/search?q=storage+economics&btnG=Search&hl=en&sa=2

..

There are numerous areas of storage management that customers can try to bring in efficiencies that will help them better manage storage, reduce footprint, and reduce CAPEX and OPEX. It starts as a small practice within organizations and the value it creates grips the rest of the IT management teams.

In large organizations, there are Storage Architecture teams, Deployment teams, Provisioning teams and Operational Support teams, but seldom do we see a Storage Economics Team that helps drive utilization and efficiency through best economic practices within storage environments.

..

So take this opportunity and plant the seeds for your Storage Economics practice now.

..

Cheers
@storagenerve

.

Posts like these make me think how easy is it for people to make claims for something that they have no idea about. What I mean “something” is “storage in a customers environment”. Practically these are some very easy means to make money in the storage industry today. “Life is good” one walks into a customer without knowing their environment, applications, users, databases and blindly tell them that we can help you reclaim 70% of all your storage. Let us evaluate your environment, have our engineers come in perform a storage assessment, be resident here for a while, bill for the work to reclaim and redeploy the storage and yea help you buy the brand of storage we prefer for our customers.

We all know how optimized, well managed and efficient our storage environments are and why are they architected they way they are in your organization. If customers run a 60% utilized environment it means 40% of the storage is un-utilized but not necessarily reclaimable and re-deployable.

Picture Source: UPENN.EDU

The issues

Largely storage environments are heavily dependant upon the architecture, IOPS requirements, databases, vm’s, applications and many other variable factors that drive its performance.

Storage architectures in an organization typically encompass provision for growth of the existing file systems, databases and future requirements.

In between this growth, resource allocation, resource shifts and retirement of older host systems, there are usually holes that get created, which makes certain portions of this storage orphaned or reclaimable.

Storage Archiving to cheaper disk and tape is not always a practice in organizations, which can lead to off loading some of the structured and unstructured data from these systems.

Storage groups typically have a high turnover rate of employees, which creates a hole as someone new is being introduced to the environment and may need a ramp up time to understand the environment, applications and user needs.

Storage groups at time do not have written policies, procedures and guidelines on what and how the storage should be reclaimed for future use. Typically a lack of data management practices are also seen with related to moving the data to cheaper storage based on policies and lifecycle management.
Application, database and performance requirements are consistently growing, which makes purchasing new storage inevitable for newer applications. While the old apps and databases are still running, cost of migrating from the old systems to new ones cause additional budgeting issues.

There is a misconception that storage reclaimation is easy to achieve. 70% of your storage can be reclaimed and redeployed today.

.

Storage Management

Lack of defined processes, procedures, oversight, change control management, application needs, database demands, etc add more complexity to storage management environments making reclaimation a much harder task.

Lack of implementation of SRM (Storage Resource Management) tools in the environments adds another layer of complexity with storage management. Storage admins and managers typically true up monthly reports related to storage environments on excel spreadsheets.

Implementation of native features within storage should absolutely be considered before purchasing and deploying any new storage. Features like data deduplication, thin provisioning, automated tiering, zero page reclaim, vmware aware storage (api’s) and use of automated ILM policies.

Define, Define, Define……..all your process, procedures, exceptions….etc..

Yea and want to throw this out too…  Personally ran into one organization so far, where the storage manager was compensated (bonuses) based on the total reclaimed storage per year.

.

Political issues

The steepest battle with any storage optimization project is internal political issues within the organization.

Working at multiple levels either the C level or IT management level imposes additional challenges…

At times the management is possibly open to ideas around storage optimization exercises to reclaim the so-called 70% of all the reclaimable storage. But as this idea flows down to the local storage teams, its either killed or delayed because of political issues.

Going from the storage teams upwards causes similar issue with application teams, database teams, architects and then the money spenders or the C level executives.

“Did one think it was easy, when they walked in…”

.

The after effects

What are some of the effects of reclaiming 70% of storage in an organization…just a few I can highlight here.

• Large changes will be implemented in organizations at a Storage management level along with replacing key executives that made a decision to purchase all this storage..
• For many years to come that organization will not purchase storage, essentially use the existing “old” storage they have sitting on the floor.
• New Applications may still end up using older storage platforms creating storage management & performance issues.
• Customer may not be able to use latest technologies like Automated Tiering, Deduplication, Thin Provisioning, Zero Page reclaim, Power down disk, energy efficiency and many more.
• The larger problem it creates is the use of the storage on the floor for more than 3 / 5 years, where they start paying for hefty hardware and software maintenance charges beyond warranty.
• The company, the person that sold you storage assessment, storage reclaimation and storage redeployment will be in there to pitch you new storage products from a XYZ company…

And

• If every customer in the world reclaims about 70% of all the storage, I will leave the question upto you as to what will happen to the storage industry…. let the critics answer it…

.

The Journey

So anyone that comes and tells you that we will do a Storage Optimization for you today, have the results tomorrow and reclaim 70% of all your storage,………….….its nothing more than the “title of this post”.

As I like to call it, “It’s a Journey” to make your storage environment fully efficient, optimized and “beat the sh*t out of it”…..

It’s the process where the customer needs to be educated at every level within the organization by helping them create a “storage economics” practice that would enable them to achieve the right results..

Again its about establishing practices, policies, procedures, guidelines, tools, showing the importance at all levels and the biggest creating the awareness about it…

The shortest 5 rules to begin this journey….

• Storage rule 1#: Buy what you need, use what you buy
• Storage rule 2#: Define and follow your practices, policies and procedures.
• Storage rule 3#: Establish an on going storage economics practice in your organization
• Storage rule 4#: Use robust SRM tools to manage your storage environment.
• Storage rule 5#: Centralize storage management, resource, infrastructure

.

It’s a journey….or it turns into a pipe dream….