Next Steps for Dedupe: Dedupe 2.0 and the Cloud

Published by Jered Floyd | Filed under Jered Floyd, CTO

A few weeks ago, Tom wrote about the end of the road for Dedupe 1.0. There’s no question that Data Domain has won the Dedupe 1.0 game — that of deduplication for D2D backup — and while we’re still waiting to see which of EMC or NetApp gains the right to acquire them, we all know that both bidders have lost at the price they’re paying.

Disk-based backup is a huge market, don’t get me wrong, but it’s tiny compared with the amount of disk out there for archive and primary storage. Not only that, but as I wrote about in “Cutting Costs with Enterprise Archive”, deduplication for disk-based backup is based on “doing it wrong” — intentionally writing the same data again and again even though there’s no good reason for having configured your backup software to do so. Dedupe for backup was always a marketing game, one that Data Domain’s team excelled at winning.

You might note that we’ve never pursued this crowded D2D market head-on; instead we’ve focussed on where more data lives and nobody else has done a good job at addressing the much harder technical problems: Dedupe 2.0, archive and primary storage. If you’ve been to our website recently, you may have noticed that we’ve launched a new product in the Dedupe 2.0 space: Permabit Cloud Storage.

A few months ago, I made some snide comments about the “cloud” storage term. I’m not back-pedaling here, I still think that “cloud” is really “Storage as a Service 2.0″, but I do believe infrastructure is now mature enough that renting a home for your bits is now a viable thing to do.

The trap that I pointed out in my last message on cloud, and the reason I think EMC’s Atmos is misguided, is the problem of API proliferation. There are now dozens of simple, RESTful APIs out there competing for application adoption, and each locks you into a particular set of service providers. These standards will eventually coalesce and evolve, but in the meantime cloud providers are aggressively competing on the simplicity and power of their APIs… all while using pricey storage on the back end. Permabit Cloud Storage delivers massively scalable storage for these service providers that provides all the rich functionality they need to offer new web APIs, while staying agnostic on these interfaces by internally delivering conventional NFS and CIFS. Multiple service providers now use Permabit Cloud Storage to offer cloud services to their customers, freeing them from concerns over reliability, privacy and security.

Over at Storage Switzerland, Joseph Ortiz lays out the case for why cloud archive storage makes sense for small-to-medium business that have rapidly growing storage requirements. Permabit Cloud Storage provides infrastructure that allows service providers to deliver unbeatable pricing to these customers, as well as scalable systems that can be deployed as an internal corporate cloud. This is just one part of Permabit’s Dedupe 2.0 strategy — deduplication beyond the realm of just backup. There’s still more excitement to come.