Google+

Policy! Policy!! Policy!!!


It has been an exciting month, some new details are emerging related to automated storage tiering, workload distributions, workflow automation, SLA’s, QoS and how Policy based storage management can help solve these challenges. “Policy” as we all know in the “business world”, “advanced algorithms” as known in “scientific community” is used to solve complex storage challenges. This has been one of the favorite topics of discussion in the storage blogosphere these days.

Though there are two distinct groups of people, one favoring automation and the other half possibly thinking this technology brings no value-add in terms of how storage is utilized and managed today. This game was initially started by Compellent (Compellent Data Progression technology) about 4 years ago, then joined by Pillar Data Systems and now other OEM’s (including EMC, HDS, IBM) are starting to catchup on policy based automated storage tiering.

With private clouds in the near future and then hybrid clouds (a mesh of private and public clouds) in the horizon, automation, workload distribution, SLA’s, QoS will need to be monitored and managed to optimally run IT Infrastructures. Policy based management will create a new wave of storage management, automation and will act as a principle ingredient of hybrid clouds.

Generation 1 of policy based storage tiering works within a single storage subsystem.
Generation 2 in the near future should work across heterogeneous storage subsystems (by the same manufacturers).
Generation 3 over the next year or two will work across storage platforms irrelevant of the manufacturers.
Generation 3 of policy based management will include the entire stack of management. These products will be capable of not only managing the Storage, but also interact through policies at the Virtualization, Networking, Application, OS, Middleware and other layers in the stack of Infrastructure management..

We should see an up-rise of new emerging technologies that will create these external policy based engines for data movement automation. All infrastructure components including Storage, Virtualization, Networking, Application, OS, Middleware will provide the necessary API’s for these external engines to interact and enable data automation and workflow automation in the hybrid clouds (irrelevant of the manufacturers).

www links

Here are a few articles from the past month related to the topics of Policy, Automated Storage Tiering, Workloads, SLA’s and QoS.

Pillar (OEM)

http://blog.pillardata.com/pillar_data_blog/2009/10/autotiering-of-data.html

EMC (OEM)

http://flickerdown.com/2009/09/why-policy-is-the-future-of-storage/

http://flickerdown.com/2009/10/why-policy-is-the-future-of-storage-part-2/

http://stevetodd.typepad.com/my_weblog/2009/10/greenfield-monitoring-of-a-private-cloud.html

http://stevetodd.typepad.com/my_weblog/2009/09/federation-and-private-cloud.html

Compellent (Partner Blog)

http://blogs.cinetica.it/cinetica/2009/10/19/dear-mike/

http://blogs.cinetica.it/cinetica/2009/08/25/tiered-storage-and-new-features-for-the-rest-of-us/

HDS (OEM)

http://blogs.hds.com/hu/2009/09/ilm-revisited-intelligent-tiered-storage-for-file-and-content-data.html

Independents

http://www.storagemonkeys.com/index.php?option=com_myblog&show=the-end-of-history-or-just-the-beginning-.html&Itemid=136

http://thestoragearchitect.com/2009/10/18/enterprise-computing-do-we-need-fast-v1-emc/

http://www.theregister.co.uk/2009/09/22/emc_fast/

http://itknowledgeexchange.techtarget.com/storage-soup/hp-drops-roadmap-nuggets-at-storageworks-techday/

http://itknowledgeexchange.techtarget.com/storage-soup/spinnaker-founders-bring-avere-out-of-stealth/

http://breathingdata.com/2009/10/18/can-and-when-will-ssds-sata-replace-fcsas/

http://gestaltit.com/featured/top/gestalt/emc-unified-platform-storage-tiering/

http://storagenerve.com/2009/10/14/enhancements-to-emc-symmetrix-v-max-systems/

Your thoughts always welcome!!!

cheers
@storagenerve

  • LiemNCompellent

    Hi Devang, this is a great list and very comprehensive. Thanks for taking the time to put it all together as a resource for people doing research on tiered storage. Automated tiered storage has a big role to play in virtualized data centers and private/public clouds. If people want a fully fluid data center they'll need technology like policy based storage management to m automate the data movement.

    As you pointed out, Compellent has been shipping Data Progression tiered storage for a while so we haven't blogged as recently about it as other vendors. Still, I thought you and your readers might be interested in what we do have to say about it, so I'm sharing a few related links, a couple are posts from our blog at http://www.compellentblog.com. Since people are clearly interested in this topic, we’ll have more to share later on.

    Compellent Blog:
    Larry Aszmann, CTO, on Compellent's automated tiered storage patents and screenshots of GUI:
    http://www.compellentblog.com/Posts/2008/9/Pate

    Links to other blogs from Bob Fine, director of product management, on tiered storage:
    http://www.compellentblog.com/Tags/automated-ti
    Independent:

    Tony Asaro on Computerworld blog:
    http://blogs.computerworld.com/compellent_ilm

    It'll be interesting to see the response you'll get from this post. Please let me know if you have any questions. Thanks.
    Liem Nguyen, Compellent
    Twitter.com/liemnguyen
    lnguyen@compellent.com

  • Pingback: Tweets that mention Policy! Policy!! Policy!!! | StorageNerve -- Topsy.com()

  • sunshinemug

    Interesting post, Devang. As it happens, Ocarina is already in the category you're calling Generation 3. That is, the ability to migrate data among multiple tiers without the need to rehydrate data, with no need for vendor lock-in. You might want to read this recent post from our blog: http://onlinestorageoptimization.com/index.php/

  • Pingback: uberVU - social comments()

  • http://storagenerve.com storagenerve

    Hi Sunshine, i was so close at referencing this post in the list of reads, at one point i had even attached it to the post and then ended up removing. After reading it several times yesterday, i somehow felt it didn't include automation as its key ingredient though from a policy standpoint it was something i should have considered.

    I am looking forward to gaining some knowledge from the GestaltIT field day about the Ocarina Platform.

    How does Ocarina fit into the entire stack of infrastructure management at Generation 3. Lot of platforms today are at Gen 1 and are rapidly moving towards Gen 2. Gen 3 is what i call nirvana…

    Devang

  • http://www.ocarinanetworks.com/ Mike Davis

    Good summary Devang. It's funny, but your post wouldn't have been out of place 7 years ago when there was plenty of mindshare battle over the various policy-based HSM solutions, none of whom have successfully meaningfully changed the world of storage. But today it's just as relevant of a post! The obstacles to adoption are surely multi-vectored…not really being platform agnostic, nasty complexity, high license fees that offset any savings at less than 500TB, bugs, fear of competitive encroachment, and emphasis on HPC/scientific apps and not enterprise.

    There's a lot of discussion on policies for moving data. If we're really thinking in terms of cloud attributes, then we need to up-level our discussion based on the objective of tiering; The objective is primarily to adjust cost/GB (and maybe some other attributes) based on file lifecycle. Thus policies should be based on burdened-opex, performance, and reliability, which are attributes abstract from physical tiers. Again, this is in context of “cloud” solutions where multi-tenancy management and opex-based accounting are core runtime properties. In order to really deliver the value proposition for tiering, the idea of an ROI needs to be instantiated as an input to a policy decision, rather than just some salesman's tool.

  • http://storagenerve.com storagenerve

    Hi Sunshine, i was so close at referencing this post in the list of reads, at one point i had even attached it to the post and then ended up removing. After reading it several times yesterday, i somehow felt it didn't include automation as its key ingredient though from a policy standpoint it was something i should have considered.

    I am looking forward to gaining some knowledge from the GestaltIT field day about the Ocarina Platform.

    How does Ocarina fit into the entire stack of infrastructure management at Generation 3. Lot of platforms today are at Gen 1 and are rapidly moving towards Gen 2. Gen 3 is what i call nirvana…

    Devang

  • http://www.ocarinanetworks.com/ Mike Davis

    Good summary Devang. It's funny, but your post wouldn't have been out of place 7 years ago when there was plenty of mindshare battle over the various policy-based HSM solutions, none of whom have successfully meaningfully changed the world of storage. But today it's just as relevant of a post! The obstacles to adoption are surely multi-vectored…not really being platform agnostic, nasty complexity, high license fees that offset any savings at less than 500TB, bugs, fear of competitive encroachment, and emphasis on HPC/scientific apps and not enterprise.

    There's a lot of discussion on policies for moving data. If we're really thinking in terms of cloud attributes, then we need to up-level our discussion based on the objective of tiering; The objective is primarily to adjust cost/GB (and maybe some other attributes) based on file lifecycle. Thus policies should be based on burdened-opex, performance, and reliability, which are attributes abstract from physical tiers. Again, this is in context of “cloud” solutions where multi-tenancy management and opex-based accounting are core runtime properties. In order to really deliver the value proposition for tiering, the idea of an ROI needs to be instantiated as an input to a policy decision, rather than just some salesman's tool.