Google+

Vaulting on EMC Symmetrix V-Max Systems

May 6th, 2009 1 comment

In this blog post we will cover the Vaulting technology related to the EMC Symmetrix V-Max and the DMX-4 technology. Vaulting is a key feature added on DMX-3 series of machines, which was further extended into the DMX-4’s and now the Symmetrix V-Max Systems. With power outages or major power fluctuations or other critical environmental conditions (temperature, water, fire and environmental hazards) in data centers, the vaulting technology saves all the data during the shutdown of the machine offering additional safety features with the customer’s data in transit (channels, cache – memory, etc).

 

Vaulting wasn’t available on the Symmetrix 3, 5 & 8 series of machines neither on the DMX nor the DMX-2 technology. With those machines, as a power hit, power fluctuation or environmental issue would be sensed; the BBU (battery backup unit) power would kick in keeping the machine in an online state for 3 minutes. During this time, all the I/O to and from the host is aborted; anything sitting in the cache is de-staged and written to disk drives. At that point, the entire machine goes into a DD (dead) state leaving itself offline (powered on but non-operational) or turning itself off if it’s a power outage.

 

With the DMX-3, DMX-4 and V-Max systems, since the number of cabinets has expanded; it becomes crucial that the data be saved before the Symmetrix turns itself offline or off. The concept of BBU has now changed to SPS (Standby Power Supply) modules that are locally attached to DAE’s (in Storage cabinets) and also in the System bay, that will keep the Symmetrix running for 5 minutes. During this time, the Symmetrix cuts off all I/O from the host, then lets the cache in the machine sync for consistency and then de-stage all the data from the cache – memory to the vault drives.

 

With the DMX-4 and V-Max as the memory is mirrored, it copies all the mirrored memory data to vault drives, creating literary two copies of the same data. In the later part of the blog post we will discuss what the rules of vaulting are and how is this technology deployed within the customers EMC Symmetrix storage environment.

 

The official definition of vault operation as referenced by EMC is: The vault operation is triggered when the Symmetrix system is powered down or transitioned offline or when environmental conditions initiate a vault situation.

 

During a power up procedure after the shutdown (vaulting), the hardware initializes and restores all the data in the cache – memory from the vault drives which verifies its integrity. Before another vault operation can be initialized, the SPS (Standby Power Supply’s) will have to be charged, which might take hours.

 

EMC Hopkinton Manufacturing (remember those HK serial numbers, they stand for HopKinton)

The picture as published by Boston Globe

 

 To read the blog post on EMC Serial Numbers, please see here

 

 The following are the Vault requirements within the V-Max Systems 

  1. Each director pair (2 – odd / even) on the V-Max system will require 200GB of vault space, that is 40 x 5GB chucks of dedicated vault data space
  2. The vault drives are M1 devices with not Raid or mirroring protection
  3. The vault drive cannot to be used by any host and is reserved for the Symmetrix
  4. Vault drives cannot be configured by Timefinder/Snap, virtual or dynamic sparing
  5. The data space created by the vault drives will be almost equivalent to the size of the cache – memory installed on the machine
  6. As mentioned in the previous blog post on enterprise flash drives, EFD’s cannot be used for vaulting operations
  7. For permanent sparing, 5 vault drives per loop are essential.


    One Correction as pointed byBasRaayman@StorageNerve Regarding vaulting/EFD’s on #V-Max. According to @storageanarchy this can be requested via RPQ: See http://bit.ly/5Lvrg 

    As referenced by TSA on his site, I quote “……You cannot order a DS8K with only SSDs (both Symmetrix DMX-4 and V-Max can be RPQ’d as flash-only configurations, although using HDDs for vault drives is more cost-effective).”

Enterprise Flash Drives (EFD) on EMC Symmetrix V-Max Systems


EMC has been a pioneer it its Flash Drive (EFD) Technology. With the V-Max Systems EMC has taken another leap in the Flash Drive technology to support larger and denser drives. Typically with EMC Symmetrix V-Max systems you will see support for 73GB, 146GB, 200GB and 400GB EFD’s.

 

 

The following are some of the characteristics and benefits of using Enterprise Flash Drive (EFD) with EMC Symmetrix V-Max Systems.

  1. 30X IOPS compared to traditional disk drives
  2. 98% less power consumption per I/O compared to traditional disk drives
  3. 60% lighter in weight than traditional disk drives
  4. 4GB interface available with Flash Drives
  5. No spindles, no mechanical failures
  6. Used for very high I/O, throughput and data hungry applications like Stock & Currency Trading, Algorithmic Trading, Data modeling, Real Time data needs, etc
  7. No special software required to manage the EFD technology on EMC’s
  8. Same looks feels of external casing of the EFD like the Traditional drives, fits in the same enclosure as a standard fiber channel drives
  9. Drives are seen by the storage and host as traditional fiber channel drives (73GB, 146GB, 200GB, 400GB)
  10. The above drives are also supported on the DMX-4 Series of machines
  11. Flash Drives are classified as Tier 0 in the customer’s ILM tiering
  12. Mirroring, Raid 5, Raid 6 is supported on Enterprise Flash Drives now, all members of the Raid group have to be on Enterprise Flash Drives
  13. Flash drives support MetaLUN technology and all the member volumes of a metalun have to be on flash drives for it to work
  14. Flash drives cannot be configured as Vault devices in V-Max or prior DMX series machines
  15. A hot spare of a flash drive have to be configured as another flash drive.
  16. Also the Symmetrix File Systems (SFS Volumes) cannot reside on Flash Drives
  17. EMC certified erasure services are not supported on Flash Drives. The Flash drives have to be scrubbed through EMC based appliances services. 

A nice blog post by Mark Peters at ESG about the SSD Technology

One Correction as pointed byBasRaayman@StorageNerve Regarding vaulting/EFD’s on #V-Max. According to @storageanarchy this can be requested via RPQ: See http://bit.ly/5Lvrg 

As referenced by TSA on his site, I quote “……You cannot order a DS8K with only SSDs (both Symmetrix DMX-4 and V-Max can be RPQ’d as flash-only configurations, although using HDDs for vault drives is more cost-effective).”

EMC Symmetrix Management Console (SMC – For Symmetrix V-Max Systems)


The Symmetrix Management Console is a very important step towards allowing customers take control of their Symmetrix V-Max Systems. With the new Symmetrix V-Max comes a new version of Symmetrix Management Console allowing customers to manage their EMC Symmetrix V-Max Systems through a GUI web browser interface with tons of new added features and wizards for usability. 

The Symmetrix Management Console was developed back in the day as a GUI to view customers Symmetrix DMX environment, over years it has evolved more to be a functional and operational tool to interface the machine for data gathering but also to perform changes. EMC Solutions Enabler Symcli is a CLI based interface to the DMX and V-Max Systems, but the SMC complements the CLI by allowing customers to perform more or less similar functions through a GUI. The looks & feels of SMC also resemble ECC (EMC Control Center) and customers sometime refer it as a ECC-lite (SMC). 

symmetrix-management-console-in-action

EMC Symmetrix Management Console in action monitoring EMC Symmetrix V-Max Systems

Some of the important features and benefits of the SMC for V-Max are listed below:

1)    Allows customers to manage multiple EMC Symmetrix V-Max Systems

2)    Increase customer management efficiency by using Symmetrix Management Console to automate or perform functions with a few set of clicks

3)    The Symmetrix Management Console 7.0 only works with Symmetrix V-Max systems

4)    The Symmetrix Management Console is installed on the Service Processor of the V-Max System and can also be installed on a host in the SAN environment.

5)    Customers can now do trending, performance reporting, planning and consolidation using SMC

6)    SMC will help customers reduce their TCO with V-Max Systems

7)    It takes minutes to install. Windows environment running a Windows Server 2003 along with IIS would be the best choice.

8 )    The interface the customers work on is a GUI. It has the looks and feels of ECC and the Console also integrates with ECC.

9)    New Symmetrix V-Max systems are configured and managed through the Symmetrix Management Console.

10) SMC also manages user, host permissions and access controls

11) Alert Management

12) From a free product, SMC now becomes a licensed product, which the customers will  have to pay for

13) It allows customers to perform functions related to configuration changes like creating and mapping masking devices, changing device attributes, flag settings, etc

14) Perform replication functions using SMC like Clone, Snap, Open Replicator, etc

15) SMC enables Virtual Provisioning with the Symmetrix V-Max arrays

16) Enables Virtual LUN technology for automated policies and tiering.

17) Auto Provisioning Group technology is offered through wizards in SMC

18) Dynamic Cache Partitioning: Allocates and deallocates cache based on policies and utilization.

19) Symmetrix Priority Controls

20) From the SMC, customers can now launch SPA (Symmetrix Performance Analyzer), this is more on the lines of Workload Analyzer which is a standard component of ECC Suite. This allows customers to view their storage & application performance & monitoring. SPA will can be obtained as a Add-on product from EMC based on licensing.

virtual-lun-technology-in-smc1

Virtual LUN Technology in works using a wizard

21) The SMC gives the customer capabilities for Discovery, Configuration, Monitoring, Administration and Replication Management.

22) SMC can be obtained from EMC Powerlink or through your account manager from EMC if you have an active contract in place with EMC for hardware/software maintenance or if your systems are under warranty.

Highly recommended management tool for SAN Admins and yea it’s not free anymore for V-Max Systems.   

To read the previous blog post on Symmetrix Management Console, as it relates to Symmetrix DMX-3 and DMX-4 machines.

A nice technical post by Steve Todd on V-Max and Symmetrix Management Console

A post by StorageZilla on Auto Provisioning Groups with SMC

Note: Both the pictures in this blog post have been derived from EMC’s Symmetrix Management Console Data Sheet.

Clariion SPCollects for CX, CX3, CX4


The following is the procedure for SPCollects on a Clariion, CX, CX3 and CX4 machines.

If you are running release 13 and above, you will be able to perform the SP Collects from the GUI of Navisphere Manager Software.

Using Navisphere perform the following steps to collect and transfer the SPCollects to your local drive.

  1. Login to Navisphere Manager
  2. Identify the Serial number of the array you want to perform the SP Collects on
  3. Go to SP A using expand (+)
  4. Right click on it and from the menu, select SPCollects
  5. Now go to SP B in the same array
  6. Right click on it and from the menu, select SPCollects
  7. Wait for 5 to 10 minutes depending on the how big your array is and how busy your array is
  8. Now right click on SP A and from the menu select File Manager
  9. From the window, select the zip file SerialNumberOfClariion_spa_Date_Time_*.zip
  10. From the window, hit the transfer button to transfer the files to your local computer.
  11. Follow a similar process ( 8, 9, 10) for SPB, from the File Manager
  12. The SP B file name will be SerialNumberOfClariion_spb_Date_Time_*.zip

For customers that do not have SPCollects in the menu (running release 13 below), there is a manual way to perform SPCollects using Navicli from your management console or an attached host system.

To gather SPCollects from SP A, run the following commands

navicli  –h  xxx.xxx.xxx.xxx  spcollect  –messner

Wait for 5 to 7 mins

navicli  –h  xxx.xxx.xxx.xxx  managefiles  –list

The name of the SPCollects file will be SerialNumberOfClariion_spa_Date_Time_*.zip

navicli  –h  xxx.xxx.xxx.xxx  managefiles  –retrieve

where xxx.xxx.xxx.xxx is the IP Address of SP A

For SP B, similar process like above, the name of the file you will be looking for is SerialNumberOfClariion_spb_Date_Time_*.zip

Where xxx.xxx.xxx.xxx will be the IP Address of SP B

SPCollects information is very important with troubleshooting the disk array and will give the support engineer all the necessary vital data about the storage array (environment) for troubleshooting.

The following data that is collected using the SP Collects from both the SP’s:

Ktdump Log files

iSCSI data

FBI data (used to troubleshoot backend issues)

Array data (sp log files, migration info, flare code, sniffer, memory, host side data, flare debug data, metalun data, prom data, drive meta data, etc)

PSM data

RTP data (mirrors, snaps, clones, sancopy, etc)

Event data (windows security, application and system event files)

LCC data

Nav data (Navisphere related data)

To read previous blog post on Clariion Technology, please see the following links

Clariion Cache: Page Size

Clariion Cache: Read and Write Caching

Clariion Cache: Navicli Commands

Clariion Cache: Idle, Watermark and Forced Flushing

Clariion Basics: DAE, DPE, SPE, Drives, Naming Convention and Backend Architecture

Clariion Flare Code Operating Environment

Or

Tag Clariion on StorageNerve