Google+

Archive

Archive for the ‘Storage’ Category

EMC Symmetrix: Dynamic Hot Spares

July 22nd, 2009 No comments

There are two types of sparing strategies available on EMC Symmetrix Series of machines.

Dynamic Hot Sparing:
Starting the Symmetrix 4.0, EMC had introduced dynamic hot spares in its Enginuity code to support customers against failing disk drives and reducing the probability of a data loss. Available there onwards on each version of Symmetrix, customers have been able to use this Hot Sparing technology. Today the Dynamic sparing is available on Symmetrix 4.0, Symmetrix 4.8, Symmetrix 5.0, Symmetrix 5.5, DMX, DMX2, DMX3, and DMX4 systems.

Permanent Spares: Was introduced starting the Symmetrix DMX3 products, now available on DMX4’s and V-Max systems. I believe, Enginuity code 5772 started supporting Permanent Spares to guard customers against failing disk drives to further help reduce any performance, redundancy and processing degradation on the Symmetrix systems with features that were not available with the Dynamic Hot Sparing.

Highlights of Permanent Sparing

Due to some design, performance, redundancy limitations and Symmetrix mirror positions, dynamic hot spares were becoming a bottleneck related to customer internal job processing, example: a failed 1TB SATA drive sync to dynamic spare might take more than 8 to 48 hours.  While a similar process to remove the dynamic spare and equalize the replaced drive might take the same. During this time the machine is more or less in a lock down (Operational but not configurable).

Due to these limitations, a concept of Permanent spares was introduced on EMC Symmetrix systems, which would help fulfill some gaps the Dynamic hot spares technology has. Following are the criteria for Dynamic Hot Spares.

To read about EMC Symmetrix : Permanent Hot Spares


Some important things to consider with Dynamic Hot Sparing

  1. Supported through microcode (Enginuity) version starting Symmetrix Family 4.0, support extended through all later releases of Enginuity until DMX-4 (5773).
  2. Dynamic Hot Spares configured and enabled in the backend by an EMC CE.
  3. No BIN file change is performed as the Dynamic Hot Spare gets invoked or removed upon a disk drive failure.
  4. No BIN file change is allowed until the Dynamic Hot Spare is removed from the active used devices pool and inserted back into the Spares pool.
  5. An EMC CE will need to attend site to replace the failed drive and put the dynamic hot spare back in the pool of devices available for sparing.
  6. Enginuity does not check for performance and redundancy when the dynamic hot spare is invoked.
  7. In the previous generation of Symmetrix systems, an exact match (speed, size, block size) was required with Dynamic hot spares. Starting I believe the 5772 (DMX3 onwards) version of microcode that requirement is not necessary. Now larger or smaller multiple dynamic spares can be spread across protecting multiple devices not ready, the one to one relationship (failed drive to dynamic spare) is not true any more.
  8. Related to performance on DMX3 systems and above, if correct dynamic spares are not configured, customers can see issues around redundancy and performance. Example, A 10K drive can be invoked automatically against a failed drive that is 15K causing performance issues. Also a drive on the same loop as other raid group devices can be invoked as a hot spare, potentially causing issues if the entire loop was to go down.
  9. Dynamic spares will not take all the characteristics of failed drives. Example, mirror positions.
  10. While the Permanent Spare or Dynamic Hot Spare is not invoked and is sitting in the machine waiting for a failure, these devices are not accessible from the front end (customer). The folks back at the PSE labs, will still be able to interact with these devices and invoke it for you incase of a failure or a proactive failure or for any reasons the automatic invoke fails.
  11. If a Permanent Spare fails to invoke, a Dynamic Hot Spare is invoked, if a Dynamic Hot Spare fails to invoke, the customer data stays unprotected.
  12. Dynamic Hot Spare is supported with RAID-1, RAID-10, RAID-XP, RAID-5 and various configurations within each Raid type.  Dynamic hot sparing does not work with RAID-6 devices.
  13. As far as I know for the V-Max systems, Dynamic hot sparing is not supported.


Some important benefits of Dynamic Hot Sparing

  1. Dynamic Hot Sparing kicks in when Permanent Sparing fails to invoke
  2. Provides additional protection against data loss

No BIN file change is performed with Dynamic Hot Sparing

As a requirement to all the new systems that are configured now, sparing is required. Hope this provides a vision into configuring your next EMC Symmetrix on the floor.

EMC Symmetrix: Permanent Sparing

July 21st, 2009 1 comment

There are two types of sparing strategies available on EMC Symmetrix Series of machines.

Dynamic Hot Sparing:
Starting the Symmetrix 4.0, EMC had introduced dynamic hot spares in its Enginuity code to support customers against failing disk drives and reducing the probability of a data loss. Available there onwards on each version of Symmetrix, customers have been able to use this Hot Sparing technology. Today the Dynamic sparing is available on Symmetrix 4.0, Symmetrix 4.8, Symmetrix 5.0, Symmetrix 5.5, DMX, DMX2, DMX3, and DMX4 systems.

Permanent Spares:
Was introduced starting the Symmetrix DMX3 products, now available on DMX4’s and V-Max systems. I believe, Enginuity code 5772 started supporting Permanent Spares to guard customers against failing disk drives to further help reduce any performance, redundancy and processing degradation on the Symmetrix systems with features that were not available with the Dynamic Hot Sparing.

Highlights of Permanent Sparing

Due to some design, performance, redundancy limitations and Symmetrix mirror positions, dynamic hot spares were becoming a bottleneck related to customer internal job processing, example: a failed 1TB SATA drive sync to dynamic spare might take more than 8 to 48 hours.  While a similar process to remove the dynamic spare and equalize the replaced drive might take the same. During this time the machine is more or less in a lock down (Operational but not configurable).

Due to these limitations, a concept of Permanent spares was introduced on EMC Symmetrix systems, which would help fulfill some gaps the Dynamic hot spares technology has. Following are the criteria for Permanent Spares.

Some important things to consider with Permanent Spares

  1. Permanent Spares are supported through the microcode (Enginuity) versions starting the DMX-3 (5772 onwards) into the latest generation Symmetrix V-Max Systems.
  2. The customer needs to identify and setup the devices for Permanent Spares using Solutions enabler or an EMC CE should perform a BIN file change on the machine to enable Permanent Spares and the associated devices.
  3. When the Permanent Spare kicks in upon a failing / failed drive, a BIN file change locally within the machine is performed using the unattended SIL. Any configuration locks or un-functional Service Processors will kill the process before it’s initiated, in this instance the Permanent Spare will not be invoked but rather will invoke the Dynamic Hot Spare.
  4. An EMC CE will not require attending the site right away to replace the drive since the Permanent Spare has been invoked and all the data is protected. All failed drives where Permanent spares have been invoked can be replaced in a batch. When the failed drive is replaced, it will become a Permanent spare and will go the Permanent spares pool.
  5. Configuration of Permanent Spares is initiated through BIN file change, during this process, the CE or the customer will required to consider Permanent Spares rules related to performance and redundancy.
  6. If a Permanent Spare cannot be invoked due to any reasons related to performance and redundancy, a Dynamic Hot Spare will be invoked against the failing / failed device.
  7. The Permanent Spare will take all the original characteristics of a failed disk (device flags, meta configs, hyper sizes, mirror positions, etc) as it gets invoked.
  8. The rule of thumb with permanent spares is to verify that the machine has required type / size / speed / capacity / block size of the related permanent spare drives configured.
  9. You can have a single Symmetrix frame with Permanent Spares and Dynamic Hot Spares both configured.
  10. While the Permanent Spare or Dynamic Hot Spare is not invoked and is sitting in the machine waiting for a failure, these devices are not accessible from the front end (customer). The folks back at the PSE labs, will still be able to interact with these devices and invoke it for you incase of a failure or a proactive measure or for any reasons the automatic invoke fails.
  11. Permanent spares can be invoked against Vault drives, if a permanent spare drive is available on the same DA where the failure occurred.
  12. Permanent spares can be configured with EFD’s. I believe for every 2 DAE’s (30+ drives) you have to configure one hot spare EFD (permanent spares).
  13. Permanent Spares supports RAID type RAID 1, RAID 10, RAID 5, RAID 6 and all configurations within.

Some important Benefits of Permanent Sparing

  1. Additional protection against data loss
  2. Permanent sparing reduces the number of times the data copy is required (one time) instead of dynamic spares that needs to data copy (two times).
  3. Permanent sparing resolves the problem of mirror positions.
  4. Permanent spares (failed) drives can be replaced in batches, do not require immediate replacement.
  5. Permanent spares do not put a configuration lock on the machine, while an invoked dynamic spare will put a configuration lock until replaced.
  6. Permanent spares obey the rules of performance and redundancy while Dynamic hot sparing does not.

As a requirement to all the new systems that are configured now, sparing is required. Hope this provides a vision into configuring your next EMC Symmetrix on the floor.

EMC Clariion FC4500 Systems: Determine the IP Address

July 16th, 2009 No comments

The EMC Clariion FC4500 Systems were the first generation of Clariion units after Data General was purchased by EMC.

Though these systems were pretty robust back in the day, compared to today’s advanced technology, it was just a JBOD.

For some related search terms and comments left by folks on the Storagenerve site, quite a few readers having been hunting for information related to the Clariion FC4500 Series of machines, never thought there were active users of this technology now, well I guess I was wrong.

At times to reset the Service Processor or to further investigate a certain problem, these units have to be assigned an IP Address or rather incase of a misplaced application, the customer needs to find the IP address of the unit.

This process is disruptive. I have tried it in the lab with success, but sometimes based on a certain flavor of Flarecode, I have also seen this process go unsuccessful.

The following is the procedure to recover or determine the IP address of the Clariion FC4500 units.

  • Select either SP a or SP b.
  • Using HyperTerminal and the following Serial port settings connect to the Serial Port of the Clariion unit on the back of the SP (Service Processor), 9600-8-N-1-N
  • To break the sequence, Please the shift key and then enter @$@$
  • At the Arom>menu
  • Now select Option 7
  • Now select Option 11
  • You should not be prompted to change to FCLI mode, enter 1
  • Now exit the menu using 0
  • Now select Option 1
  • Your Array should reboot now
  • As the Array comes up, please stay connected through the HyperTerminal
  • At the fcli> setlan
  • Hit Enter and list the IP information
  • Now you should change the mode back to Serial Application
  • Go back to the step 3 above and follow the process to perform a reboot again

The similar process needs to be followed for the other SP.

To read about similar processes for other Clariion Systems, please see below.

Clariion CX, CX3, CX4 – How to change Navisphere Manager Password

Clariion CX, CX3, CX4 – How to change IP Address of the SP

Clariion CX, CX3, CX4 – Settings for Dailup PPP into the Clariion machines

EMC Clariion FLARE Code Operating Environment

EMC AX4 Platform

July 16th, 2009 10 comments

On a few reader request, here is a blog post on EMC AX4 technology.

Previously in one of the post on StorageNerve, I had explained the evolution of the EMC Clariion Technology including the AX products and the flare code that is associated with the success of this platform, to read the blog post.

The following are the 4 available models within the AX4 Platform; the naming conventions will explain it further.

AX4-5F (Fiber Channel Host Connect)

AX4-5FSC (Fiber Channel Host Connect, Single RAID Controller)

AX4-5i (iSCSI Host Connect)

AX4-5iSC (iSCSI Host Connect, Single Raid Controller)
Drives per DAE (Drive Array Enclosure)

12


Maximum DAE’s (Drive Array Enclosure) supported per System

4


DPE (Disk Processor Enclosure)

12 drive per DPE

Maximum Drives Supported per System

60


Minimum Drives Supported per System

4 Drives including the Flare Drives (Flare code resides on the 1st 4 drives of the DPE, with do not remove stickers).

Naming Convention for Drives

Bus

Enclosure

Drive No

Possible Options

Bus 0 and 1

Enclosure 1, 2, 3, 4

DPE becomes enclosure 0

Drive 0 through 11 (12 Drives total)

B0_E3_D11 becomes Bus 0, Enclosure 3 and Drive 12

The following are the disk drives that are supported with the AX4 Platforms.

Capacity: 1TB

Speed: 7.2K

Type: SATA

Capacity: 750GB

Speed: 7.2K

Type: SATA

Capacity: 450GB

Speed: 15K

Type: SAS

Capacity: 400GB

Speed: 15K

Type: SAS

Capacity: 400GB

Speed: 10K

Type: SAS

Capacity: 300GB

Speed: 15K

Type: SAS

Capacity: 146GB

Speed: 15K

Type: SAS

As you notice above, only SAS and SATA drives are usable on an AX4 system with no support for FC or EFD’s.

Supported RAID Types with AX4 Systems

RAID 0, RAID 0+1, RAID 3 and RAID 5 Technologies.

CORRECTION: RAID 6 is now supported on AX-4 Platforms starting release 23 (Navisphere Express)

Supported Software on the AX4 Systems

Navisphere Express included

Navisphere Manager (Only with 2 Service Processors)

Navisphere QoS

Navisphere Analyzer

SnapView

MirrorView/S and /A

SANCopy

Ionix Control Center plug-in

PowerPath

RecoverPoint/SE and CDP/CRR

Replication Manager

Host Type Supported

Windows 2000/2003/2008

Linux

Solaris

HP-UX

AIX

VMware

Some very important characteristics of the AX4 Platforms

Very cost effective

Completes in the SMB space with low end IBM and HP Storage

Scales upto 60TB RAW Storage

Supports upto 64 Host systems connected

1GB Cache per Service Processor, 2GB max for dual processors.

4GB FC connectivity to host (fiber ports to switch / host)

You can run it with a single controller or dual controllers

iSCSI Support

SAS and SATA drives are supported

All Clariion Software supported

SPS (single included): Standby Power Supply – Battery allows the cache to destage during a forced shutdown resulting in no data loss

MetaLUN Technology supported

Virtual LUN Technology supported

Snaps and Clones supported

Supports Flare Release 23, 26 and 28.