Archive for the ‘Storage’ Category

Storage optimization, a pipe dream

March 25th, 2010 3 comments


Posts like these make me think how easy is it for people to make claims for something that they have no idea about. What I mean “something” is “storage in a customers environment”. Practically these are some very easy means to make money in the storage industry today. “Life is good” one walks into a customer without knowing their environment, applications, users, databases and blindly tell them that we can help you reclaim 70% of all your storage. Let us evaluate your environment, have our engineers come in perform a storage assessment, be resident here for a while, bill for the work to reclaim and redeploy the storage and yea help you buy the brand of storage we prefer for our customers.

We all know how optimized, well managed and efficient our storage environments are and why are they architected they way they are in your organization. If customers run a 60% utilized environment it means 40% of the storage is un-utilized but not necessarily reclaimable and re-deployable.

Picture Source: UPENN.EDU

The issues

Largely storage environments are heavily dependant upon the architecture, IOPS requirements, databases, vm’s, applications and many other variable factors that drive its performance.

Storage architectures in an organization typically encompass provision for growth of the existing file systems, databases and future requirements.

In between this growth, resource allocation, resource shifts and retirement of older host systems, there are usually holes that get created, which makes certain portions of this storage orphaned or reclaimable.

Storage Archiving to cheaper disk and tape is not always a practice in organizations, which can lead to off loading some of the structured and unstructured data from these systems.

Storage groups typically have a high turnover rate of employees, which creates a hole as someone new is being introduced to the environment and may need a ramp up time to understand the environment, applications and user needs.

Storage groups at time do not have written policies, procedures and guidelines on what and how the storage should be reclaimed for future use. Typically a lack of data management practices are also seen with related to moving the data to cheaper storage based on policies and lifecycle management.
Application, database and performance requirements are consistently growing, which makes purchasing new storage inevitable for newer applications. While the old apps and databases are still running, cost of migrating from the old systems to new ones cause additional budgeting issues.

There is a misconception that storage reclaimation is easy to achieve. 70% of your storage can be reclaimed and redeployed today.


Storage Management

Lack of defined processes, procedures, oversight, change control management, application needs, database demands, etc add more complexity to storage management environments making reclaimation a much harder task.

Lack of implementation of SRM (Storage Resource Management) tools in the environments adds another layer of complexity with storage management. Storage admins and managers typically true up monthly reports related to storage environments on excel spreadsheets.

Implementation of native features within storage should absolutely be considered before purchasing and deploying any new storage. Features like data deduplication, thin provisioning, automated tiering, zero page reclaim, vmware aware storage (api’s) and use of automated ILM policies.

Define, Define, Define……..all your process, procedures, exceptions….etc..

Yea and want to throw this out too…  Personally ran into one organization so far, where the storage manager was compensated (bonuses) based on the total reclaimed storage per year.


Political issues

The steepest battle with any storage optimization project is internal political issues within the organization.

Working at multiple levels either the C level or IT management level imposes additional challenges…

At times the management is possibly open to ideas around storage optimization exercises to reclaim the so-called 70% of all the reclaimable storage. But as this idea flows down to the local storage teams, its either killed or delayed because of political issues.

Going from the storage teams upwards causes similar issue with application teams, database teams, architects and then the money spenders or the C level executives.

“Did one think it was easy, when they walked in…”


The after effects

What are some of the effects of reclaiming 70% of storage in an organization…just a few I can highlight here.

  • Large changes will be implemented in organizations at a Storage management level along with replacing key executives that made a decision to purchase all this storage..
  • For many years to come that organization will not purchase storage, essentially use the existing “old” storage they have sitting on the floor.
  • New Applications may still end up using older storage platforms creating storage management & performance issues.
  • Customer may not be able to use latest technologies like Automated Tiering, Deduplication, Thin Provisioning, Zero Page reclaim, Power down disk, energy efficiency and many more.
  • The larger problem it creates is the use of the storage on the floor for more than 3 / 5 years, where they start paying for hefty hardware and software maintenance charges beyond warranty.
  • The company, the person that sold you storage assessment, storage reclaimation and storage redeployment will be in there to pitch you new storage products from a XYZ company…


  • If every customer in the world reclaims about 70% of all the storage, I will leave the question upto you as to what will happen to the storage industry…. let the critics answer it…


The Journey

So anyone that comes and tells you that we will do a Storage Optimization for you today, have the results tomorrow and reclaim 70% of all your storage,………….….its nothing more than the “title of this post”.

As I like to call it, “It’s a Journey” to make your storage environment fully efficient, optimized and “beat the sh*t out of it”…..

It’s the process where the customer needs to be educated at every level within the organization by helping them create a “storage economics” practice that would enable them to achieve the right results..

Again its about establishing practices, policies, procedures, guidelines, tools, showing the importance at all levels and the biggest creating the awareness about it…

The shortest 5 rules to begin this journey….

  • Storage rule 1#: Buy what you need, use what you buy
  • Storage rule 2#: Define and follow your practices, policies and procedures.
  • Storage rule 3#: Establish an on going storage economics practice in your organization
  • Storage rule 4#: Use robust SRM tools to manage your storage environment.
  • Storage rule 5#: Centralize storage management, resource, infrastructure


It’s a journey….or it turns into a pipe dream….

EMC Symmetrix: VCMDB and ACLX

March 23rd, 2010 5 comments

VCMDB: Volume Control Manager Database

ACLX: Access Control Logix

VCM: Volume Control Manager device (where the database resides)

VCM Gatekeeper: Volume Control Manager Gatekeeper (database doesn’t reside on these devices)

SFS Volumes: Symmetrix File System Volumes


If you work with EMC Symmetrix systems, you know the importance of VCMDB. Introduced with Symmetrix 4.0 and used in every generation after that, VCMDB stands for Volume Control Manager Database). Also in the latest generation of systems the VCM device is at times also referenced as VCM Gatekeeper.

VCMDB is a relatively small device that is created on the Symmetrix system that allows for hosts access to various devices on the Symmetrix. VCMDB keeps an inventory of which devices have access to which host (HBA’s). Without a VCMDB in place, host systems will not be able to access the Symmetrix. The VCMDB should be backed up on regular intervals and would be helpful in a rainy day.

The VCMDB device size grew along with new generations of Symmetrix systems that got introduced, primarily a means to keep a track of more supported devices (hypers / splits) on these platforms. With the introduction of Symmetrix V-Max, the VCMDB concept is now a bit changed to ACLX (Access Control Logix). Access Logix is being used on the Clariion systems for years now.


Here are a few things to consider with VCMDB

  • On the older Symmetrix systems (4.0, 4.8, 5.0 and 5.5), the VCMDB (device) is mapped to all the channels, host
  • In these systems the VCMDB access is typically restricted by Volume Logix or ACL (access control lists)
  • With the Symmetrix DMX, DMX2 Systems – Enginuity Code 5670, 5671 the VCM device only requires to be mapped to the Management stations
  • Management stations include SYMCLI Server / Ionix Control Center Server / Symmetrix Management Console
  • At all given times on the DMX, DMX2 platforms, the VCMDB would need to be mapped to at least one station to perform online SDDR changes. Alternatively this problem of not having device mapped to at least one host can also be fixed by the PSE lab
  • Mapping VCMDB to multiple hosts, channels may make the device venerable to crashes, potential tampering, device attributes and data change
  • You can write disable VCMDB to avoid the potential of the above
  • With these systems, the host can communicate to the VCMDB via Syscalls
  • The VCM Edit Director Flag (fibrepath) needs to be enabled for management stations to see VCM device
  • The database (device masking database) of the VCMDB resides on the SFS volumes. This feature was introduced with DMX-3 / DMX-4 (5772 version of microcode). A 6 cylinder VCM Gatekeeper device is okay to use with these versions of microcode.
  • Starting Symmetrix V-Max systems, the concept of ACLX was introducted for Auto Provisioning etc.
  • VCM volumes are required to be mirrored devices like SFS volumes


Various different types of VCMDB

Type 0, Type 1, Type 2, Type 3, Type 4, Type 5, Type 6

  • Type 0: Symmetrix 4.0, 32 Director System, 16 cylinder device size, Volume Logix 2.x
  • Type 1: Symmetrix 4.8, 64 Director System, 16 cylinder device size, ESN Manager 1.x
  • Type 2: Symmetrix 5.0/5.5, 64 Director System, 16 cylinder device size, ESN Manager 2.x
  • Type 3: Symmetrix DMX, supports 32 fibre/ 32 iSCSI initiator records per port, 24 cylinder device in size. Enginuity 5569, Solutions Enabler 5.2, Support 8000 devices
  • Type 4: Symmetrix DMX/DMX-2, supports 64 fibre/ 128 iSCSI initiator records per port, 48 cylinder device in size. Enginuity 5670, Solutions Enabler 5.3, Supports 8000 devices
  • Type 5: Symmetrix DMX/DMX-2, supports 64 fibre / 128 iSCSI initiator records per port, 96 cylinder device in size, Enginuity 5671, Solutions Enabler 6.0, Supports 16000 devices
  • Type 6: Symmetrix DMX-3, DMX-4, supports 256 fibre / 512 iSCSI initiator records per port, 96 cylinder device in size, Enginuity 5771, 5772 Solutions Enabler 6.0, Supports 64000 devices


Notes about various Types of VCMDB

  • Type 3 of VCMDB can be converted to Type 4 VCMDB (code upgrade from 5669 to 5670 to 5671)
  • Solutions enabler 5.2 and Solutions Enabler 5.3 can read/write Type 3 VCMDB
  • Solutions enabler 5.3 can read/write Type 4 VCMDB
  • VCMDB device is recommended to be a certain size, but it is okay to use a larger size device if no choices are available.


Converting various types of VCMDB using SymCLI

  • If the device cylinder size is equal with a conversion you are attempting, the following will help you convert your VCMDB from type x to type y.
    • Backup the device
    • symmaskdb –sid <symmid> backup –file backup
    • Check the VCMDB type using
    • symmaskdb – sid <symmid> list database
    • Convert from type 4 to type 5
    • Symmaskdb – sid <symmid> convert –vcmdb_type 5 –file Covertfilename


To initialize VCMDB for the first time on a Symmetrix System

Within Ionix Control Center

  • Click on the Symmetrix array you are trying to initialize the VCMDB
  • Select Masking then VCMDB Management and then initialize
  • Select a new backup and create a file name
  • Create a file name with .sdm extenstion
  • Click on Activate the VCMDB
  • VCMDB backups are stored at \home\ecc_inf\data\hostname\data\backup\symmserial\
  • Also it will be viewable within Ionix Control Center at Systems/Symmetrix/VCMDB Backups/


With SymCLI

  • To query the VCMDB database
    • symmaskdb –sid <symmid> list database
    • To backup and init an existing VCMDB database
      • symmaskdb – sid <symmid> init –file backup

More technical deep dive coming soon on various other topics…including ACLX.



EMC Symmetrix: Calculations for Heads, Tracks, Cylinders, GB

March 22nd, 2010 No comments

Symmetrix Disk Drive

Here is the quick and dirty math on EMC Symmetrix Heads, Tracks, Cylinder sizes to actual usable GB’s of space.

Based on different generations of Symmetrix systems, here is how the conversions work.

Before we jump into each model type, lets look at what the basics are, with the following calculations.





There are s number of splits (hyper) per physical device.

There are n number of cylinders per split (hyper)

There are 15 tracks per cylinder (heads)

There are either 64 or 128 blocks of 512 bytes per track


All the calculations discussed here are for Open Systems (FBA) device types. Different device emulations like 3380K, 3390-1, 3390-2, 3390-3, 3390-4, 3390-27, 3390-54 have different bytes/track, different bytes/cylinder and cylinders/volume.


Symmetrix 8000/DMX/DMX-2 Series

Enginuity Code: 5567, 5568, 5669, 5670, 5671

Includes EMC Symmetrix 8130, 8230, 8430, 8530, 8730, 8830, DMX1000, DMX2000, DMX3000 and various different configurations within those models.

GB = Cylinders * 15 * 64 * 512 / 1024 / 1024 / 1024

eg: 6140 Cylinder devices equates to 2.81 GB of usable data

6140 * 15 * 64 * 512 / 1024 / 1024 / 1024 = 2.81 GB

Cylinders = GB / 15 / 64 / 512 * 1024 * 1024 * 1024


15 = tracks per cylinder

64 = blocks per track

512 = bytes per block

1024 = conversions of bytes to kb to mb to gb.


Symmetrix DMX-3/DMX-4 Series

Enginuity Code: 5771, 5772, 5773

Includes EMC Symmetrix DMX-3, DMX-4 and various different configurations within those models.

GB = Cylinders * 15 * 128 * 512 / 1024 / 1024 / 1024

Eg: 65520 Cylinder device equates to 59.97 GB of usable data

65540 * 15 * 128 * 512 / 1024 / 1024 / 1024 = 59.97 GB

Cylinders = GB / 15 / 128 / 512 * 1024 * 1024 * 1024

15 = tracks per cylinder

128 = blocks per track

512 = bytes per block

1024 = conversions of bytes to kb to mb to gb


Symmetrix V-Max

Enginuity Code: 5874

Includes EMC Symmetrix V-Max and various different configurations within this model.

GB = Cylinders * 15 * 128 * 512 / 1024 / 1024 / 1024

Eg: 262668 Cylinder device equates to 240.47 GB of usable data

262668 * 15 * 128 * 512 / 1024 / 1024 / 1024 = 240.47 GB

Cylinders = GB / 15 / 128 / 512 * 1024 * 1024 * 1024

15 = tracks per cylinder

128 = blocks per track

512 = bytes per block

8 bytes = 520-512 used for T10-DIF

1024 = conversions of bytes to kb to mb to gb

Drive format on a V-Max is 520 bytes, out of which 8 bytes are used for T10-DIF ( A post on DMX-4 and V-Max differences).

EMC Symmetrix: BIN file

March 12th, 2010 13 comments

EMC Symmetrix BIN file, largely an unknown topic in the storage industry and practically there is no available information related to it. This post is just an attempt to shed some light as to what a BIN file is, how it works, what’s in it and why is it essential with the Enginuity code.

Some EMC folks have capitalized on the BIN file as to the personality it brings to the Symmetrix, while the EMC competition always uses it against them as it introduces complexities in the storage environment with management and change control.


Personally I feel a Symmetrix wouldn’t be a Symmetrix if the BIN file weren’t there. The personality, characteristics, robustness, compatibility, flexibility, integration with OS’s, etc wouldn’t be there if the BIN file didn’t exist.


With the total number of OS’s, device types, channel interfaces and flags it supports today, sort of making it one of the most compatible storage arrays in the market. The configuration and compatibility on the Symmetrix can be verified using the E-Lab navigator available on Powerlink.


So here are some facts about the BIN file

  • Only used with Symmetrix systems (Enginuity Code)
  • BIN file stands for BINARY file.
  • BIN file holds all information about the Symmetrix configuration
  • One BIN file per system serial number is required.
  • BIN file was used with Symmetrix Gen 1 in 1990 and is still used in 2010 with Symmetrix V-Max systems.
  • BIN file holds information on SRDF configurations, total memory, memory in slots, serial number of the unit, number of directors, type of directors, director flags, engines, engine ports, front end ports, back end ports, drives on the loop, drives on the SCSI bus, number of drives per loop, drive types in the slots, drive speeds, volume addresses, volume types, meta’s, device flags and many more settings.
  • The setup for host connection if the OS is Open Systems or Mainframe environments using FICON, ESCON, GbE, FC, RF, etc is all defined in the BIN file. Also director emulations, drive formats if OSD or CKD, format types, drive speeds, etc is all defined in the BIN file.
  • BIN file is required to make a system active. It is created based on customer specifications and installed by EMC during the initial setup.
  • Any ongoing changes in the environment related to hardware upgrades, defining devices, changing flags, etc is all accomplished using BIN file changes.
  • BIN file changes can be accomplished 3 ways.
  • BIN file change for hardware upgrades is typically performed by EMC only.
  • BIN file change for other changes that are device, director, flags, meta’s, SRDF configurations etc is either performed through the SYMAPI infrastructure using SymCLI or ECC (Now Ionix) or SMC (Symmetrix Management Console) by the customer. (Edited based on the comments: Only some changes now require traditional BIN file change, typically others are performed using sys calls in enginuity environment)
  • Solutions enabler is required on the Symcli, ECC, SMC management stations to enable SYMAPI infrastructure to operate.
  • VCMDB needs to be setup on the Symmetrix for SymCLI, ECC, SMC related changes to work.
  • Gatekeeper devices need to be setup on the Symmetrix front end ports for SymCLI, ECC, SMC changes to work
  • For Symmetrix Optimizer to work in your environment, you need DRV devices setup on your Symmetrix.(EDITED based on comments: Only required until DMX platform. Going forward with DMX3/4 & V-Max platforms it uses sys calls to perform these Optimizer changes).


Back in the day

All and any BIN file changes on the Symmetrix 3.0, Symmetrix 4.0 used to be performed by EMC from the Service Processor. Over the years with introduction of SYMAPI and other layered software products, now seldom is EMC involved in the upgrade process.


Hardware upgrades

BIN File changes typically have to be initiated and performed by EMC, again these are the hardware upgrades. If the customer is looking at adding 32GB’s of Cache to the existing DMX-4 system or adding new Front End connectivity or upgrading 1200 drive system to 1920 drives, all these require BIN file changes initiated and performed by EMC. To my understanding the turn around time is just a few days with these changes, as it requires change control and other processes within EMC.


Customer initiated changes

Configuration changes around front end ports, creating volumes, creating meta’s, volume flags, host connectivity, configuration flags, SRDF volume configurations, SRDF replication configurations, etc can all be accomplished through the customer end using the SYMAPI infrastructure (with SymCLI or ECC or SMC).


Enginuity upgrade

Upgrading the microcode (Enginuity) on a DMX or a V-Max is not a BIN file change, but rather is a code upgrade. Back in the days, many upgrades were performed offline, but in this day and age, all changes are online and accomplished with minimum pains.



So EMC has moved quite ahead with the Symmetrix architecture over the past 20 years, but the underlying BIN file change requirements haven’t changed over these 8 generations of Symmetrix.

Any and all BIN file changes are recommended to be done during quite times (less IOPS), at schedule change control times. Again these would include the ones that EMC is performing from a hardware perspective or the customer is performing for device/flag changes.


The process

During the process of a BIN file change, the configuration file typically ending with the name *.BIN is loaded to all the frontend directors, backend directors, including the global cache. After the upload, the system is refreshed with this new file in the global cache and the process makes the new configuration changes active. This process of refresh is called IML (Initial Memory Load) and the BIN file is typically called IMPL (Initial Memory Program Load) file.

A customer initiated BIN file works in a similar way, where the SYMAPI infrastructure that resides on the service processor allows the customer to interface with the Symmetrix to perform these changes. During this process, the scripts verify that the customer configurations are valid and then perform the changes and make the new configuration active.

To query the Symmetrix system for configuration details, reference the SymCLI guide. Some standard commands to query your system would include symcfg, symcli, symdev, symdisk, symdrv, symevent, symhost, symgate, syminq, symstat commands and will help you navigate and find all the necessary details related to your Symmetrix. Also similar information in a GUI can be obtained using ECC and SMC. Both will allow the customer to initiate SYMAPI changes.

Unless something has changed with the V-Max, typically to get an excel based representation of your BIN file, ask your EMC CE.



You cannot run two BIN files in a single system, though at times the system can end up in a state where you can have multiple BIN files on various directors. This phenomenon typically doesn’t happen to often, but an automated script when not finished properly can put the system in this state. At this point the Symmetrix will initiate a call home immediately and the PSE labs should typically be able to resolve these issues.

Additional software like Symmetrix Optimizer also uses the underlying BIN file infrastructure to make changes to the storage array to move hot and cold devices based on the required defined criteria. There have been quite a few known cases of Symmetrix Optimizer causing the above phenomenon of multiple BIN files. , Though many critics will disagree with that statement. (EDITED based on comments: Only required until DMX platform. Going forward with DMX3/4 & V-Max platforms it uses sys calls to perform these Optimizer changes).


NOTE: One piece of advice, never run SYMCLI or ECC scripts for BIN file changes through a VPN connected desktop or laptop. Always run all necessary SymCLI / SMC / ECC scripts for changes from a server in your local environment. Very highly recommend, never attempt to administer your Symmetrix system with an iPhone or a Blackberry.

Hope in your quest to get more information on BIN files, this serves as the starting point..