Mini-mill phenomenon in Enterprise Storage sector

"Disruption is a theory : a conceptual model of cause and effect that makes it possible to better predict the outcomes of competitive battles in different circumstances. These forces almost always topple industry leaders when an attacker has harnessed them because disruptive strategies are predicated on competitors doing what is in their best and most urgent interest : satisfying their most important customers and investing where profits are most attractive. In a profit-seeking world, this is a safe bet." - The Innovator's Solution: Creating and Sustaining Successful Growth, by Michael E Raynor and Clayton M. Christensen

   Clayton M. Christensen in his all-time business classic named Innovator's Dilemma narrated how mini steel plants (called minimills) brought disruption in the US Steel market between 60s and 90s and pushed the established large Integrated steel mills out of business. It is a classic case of incumbents pursuing the path of higher profitability ceding the low-profitability space to new entrants and in turn helping the new entrant to get better in operation and finding the opportunity to replace the incumbent from more premium segments. 

Backblaze's comparison

The question is whether the similar pattern can happen in the storage industry. 
 But steel was commodity, you argue and storage system is not. Buyers know that EMC, NetApp, IBM, HP storage servers are expensive but IT managers buy them for the vendors guarantee for the system's reliability, features and the performance. While that argument does hold good, there were few who tried to build a business, the success of which were critically dependent on the cost of storage and they were astonished to find that they could bring down the cost by a magnitude if they built the entire system on their own.  Backblaze in 2009 published their finding of how much cost saving they could bring by building their own storage.  But Backblaze had a specific need and they had to develop/customize whole range of software to make their hardware useful for their business. Their chart shows that, even with all those R&D cost, they still managed to keep the cost of storage quite close to barebone disks. You may argue that  this could be a special case and it hardly proves the rule. 

Well, it wouldn't, had it been a single case. Gartner and other analysts have predicted that most enterprises will have cloud storage in their setup in next 3-4 years. When they talk about Cloud storage, they primarily mean the specific way of accessing and organizing storage, which is different from traditional methods of storage access, like NFS, CIFS or fibre-channel based block storage i.e. SAN. Beside renting storage from public cloud vendors e.g. Amazon, rackspace, enterprises may decide to build their own cloud infrastructure, especially if they have large user-base. Just like the PC hardware got freed from the clutch of IBM, HP and became commoditized at the hands of Taiwanese and Chinese ODMs few decades ago, storage server hardware is showing distinct signal of getting commoditized at the hands of Taiwanese ODMs like Quanta, Wistron.

Fact is, almost all enterprise storage vendors at present outsource their hardware manufacturing to ODMs in Taiwan [Quanta etc] or China [ where Jabil has a very large manufacturing base]. Traditionally storage OEMs have claimed to make their differentiation in their software. Now thanks to Open source movement, there are some proven solutions which can run on commoditized hardware. What prevents these enterprises to approach directly the ODM's for getting the customized hardware at cheap price?

Mike Yang, VP and GM for Quanta's
Cloud computing Business

 Nancy Gohring's recent article in IT World is an interesting read. She reports how Quanta has built itself as the vendor for data centre's customized hardware. Quanta, traditionally an ODM for server OEMs like Dell, HP is seen to be selling directly to the large-scale cloud providers after their first success story with Facebook around  five years ago. 

EE Times article tells how likes of Facebook, Google found viable alternative to likes of IBM, EMC, NetApp in Quanta and how Quanta recognized the opportunity to expand itself as the customized hardware vendor for big data centres. Facebook's need was to find a source for large number of low-cost hardware customised for their huge data centre. Clearly branded storage servers were found to be too costly and a little inflexible for them to sustain their business model. So they followed what other cloud providers like Rackspace, Google did. They approached the ODM, who have been supplying the unbranded hardware to the storage/server OEMs, and brought down their hardware cost drastically. The advantage that Facebook, Google or Rackspace have is that their unique model of services are driven entirely with software and they have a very strong software team who just needs powerful hardware [which ODM's manufactured based on customized design from Facebook] for their software to run. The necessary question is: does one need as big a data centres as the ones that Amazon, Rackspace have to justify the risk of taking this approach against the cost advantage that one gains. 
There are reports that second tier of cloud providers have already adopted this strategy of going directly to the ODMs for their hardware. Check this recent EE times article if you need reference.

Present Scenario

Future Scenario

Of course the cloud providers do not represent the general enterprise buyers who are the customer-base for enterprise storage/server vendors. Enterprises may be tempted but will never have the man-power or the risk-appetite to justify overhauling their existing IT setup..unless it becomes so cost-effective in the long run that they will be forced by their stakeholders [ It seems that Goldman Sach also bought hardware directly from quanta bypassing brands like Dell, HP ]. Remember, how virtualization swept every enterprise and forced every IT setup to adopt virtualization? Adopting public cloud could become a similar sweeping wave in next few years once the competition between Amazon, Rackspace, Google and second tier of Cloud providers flare up. Gartner says that Cloud and CRM will constitute large component in enterprise software spending in next two years. "It's very clear that mature regions like North America, Western Europe are focusing on public cloud computing, while emerging regions are focusing on private cloud computing," said Ms. Swineheart from Gartner during a recent press meet. Salesforce [leading CRM SAAS provider] already expanded their infrastructure anticipating larger customer acquisition. So one can reasonably expect that further expansion of Cloud infrastructure will reduce the revenue for storage OEMs and expand growth of Taiwanese commodity hardware ODMs like Quanta. All storage OEMs may not immediately see a revenue reduction since overall growth of enterprise IT expenditure will adequately compensate in absolute terms. Also following Clayton M. Christensen's conjecture, established players will find it unprofitable to compete with Quanta and instead focus on high-value customers who are too paranoid to try out commodity hardware. That will improve their profit margin and push their stock price, assuming there is enough growth in that segment. The pattern will continue till commodity hardware become baked enough to compete against branded hardware. Facebook's Open compute project can accelerate the process for a open standardized hardware design that people can use for enterprise servers.
However for the trend to be disruptive enough for the existing storage OEMs, enterprises need a viable ready-made software solution, an open source server software which can run on these commodity hardware which can directly be inducted, to data centres, like a HP or Dell server.  And that is one challenge that we do not have clear answer yet. While Rackspace sponsored openstack is becoming a strong contender for opensource software for cloud providers, it has some miles to go before standard-track enterprises can adopt it for their private cloud setup. But once they mature, OpenCompute certified commodity hardware with Openstack software could become a formidable entity to start the domino effect of disruption for the enterprise storage sector. Like enterprises running mainframes,  NAS and SAN will continue to exist in enterprise IT but major growth will be in cloud infrastructure.
  As Clayton M. Christensen explained, the disruptive solution always comes first as inferior alternative to existing solution and appeals to Niche buyers, in this case large scale cloud providers. But as the time goes opportunity pushes further innovations and soon the Industry gets a credible alternative to existing products at much lower cost-point at every customer segment. And when that happens Industry shifts to new structure quite fast creating new leaders of the Industry. That is how EMC became storage leader earlier pushing costly IBM servers. That is how Nucor became the leader riding on mini-mill disruption in US steel industry. Commodity enterprise hardware and open stack software have the potential to bring the mini-mill effect in enterprise storage sector but whether we will see a new leader in place of EMC depends on which path incumbents take. High tech software sectors are lot more connected compared to what steel Industry was in 70s-80s. EMC is in fact in the board of Openstack and so there are chances that incumbents will see the long-term pattern quite ahead and change themselves to survive the disruption force.

Commoditization of Enterprise Storage

Let's start with a question.
When was the last time that Enterprise Storage [NAS, DAS and SAN] Industry saw a new entrant challenging the incumbents and become successful?
Answer: 2001-2004 when NetApp successfully challenged dominance of EMC and picked up significant market shares in NAS space [NetApp today commands around 24% of NAS market while EMC has around 41%].
What happened at that time? Well, NetApp brought a new NAS appliance, in the industry, that was cheaper and lot more easier to manage for the IT folks. There is another side of the story. EMC, as we know today was not exactly the same in 2003. Between 2003 and 2005, EMC went through a transformation under the leadership of Mr. Tucci from being an engineering-driven organization to marketing-driven organization. They bought few companies which later on changed the way people look at EMC. Some of the dazzlers that they bought at that time are Documentam and VMware, much smaller but very interesting entities at that time. But these are all past data. Why are we bringing them now?  The next question will lead to the answer.
Question 2: Networking hardware segment [Routers, Switches] has already seen a mass-scale commoditization. HDD segment is also heavily commoditized. Surprisingly Enterprise Storage segment has resisted it till now. The question therefore is how are they doing it?
Well, the easy and quick answer is that the storage technology space is not as transparent as IETF-driven  networking technologies and more importantly Storage gears deal with storing and managing the organization data which is critical to organization's survival unlike Networking gears which only see the data in transit. High sensitivity to data  has made the organizations extremely risk-averse and a prisoner of sort to a particular set of vendors [who they bought the equipment from earlier].
Will my new system smoothly talk to old systems? How do I trust that my data will not mutate (get corrupted or changed irretrievably ) or storage infrastructure will not become unavailable and shaky when I bring a relatively unknown vendor in the same setup when the vendors themselves did not certify the configuration? I want my vendor to guarantee that the setup will be stable and I can demand that if the setup is relatively homogeneous.
Life of Data is another important aspect of the puzzle. Since regulatory compliance require firms to keep their internal data for a long period [some critical data must be stored for 25 years or more], IT managers tend to buy equipment from those vendors who are likely to survive for that long, and that leaves the buying decision heavily skewed towards the established players, making it a little difficult for new-entrants to find foot-hold.
All the leading Storage System vendors capitalize on this fear in customers' mind which kind of explains why they have been successful in resisting commoditization of storage systems and how they have been successful in pushing their servers and disk arrays to customers year after year. If you have invested in NetApp gears, you will continue to buy NetApp boxes and software and likewise for HP, EMC unless the cost dynamics pushes you to change.
  A curious thing to note that all the leading storage OEMs today use processors, HDDs and Flash drives from the same set of vendors and have their manufacturing outsourced to same part of the world but somehow they have maintained the product differentiation in customer's mind. Multiple Design patents and proprietary software are the secret sauce behind their dominance, they would argue. It is also curious to observe that relative positions in terms of market share between the top 5 did not see significant changes over the past 5-6 years with EMC leading the pack and IBM, NetApp, HP following EMC from a distance. Quite miraculously the incumbents have not really seen a challenger like NetApp in last half of a decade.
But is it really a miracle?
It is only when we look closely, the veil of miracle drops and we realize it is rather a vigilant and clever market maneuvering, by the incumbents, that did not allow the challenger to find success and sustain the way NetApp did. Each time, we saw a new entrant creating a niche for itself, we also knew that it is going to be bought over in a matter of months. When clustered Storage became hot domain for startup, we saw spinnaker got acquired by NetApp, and Isilon got bought over by EMC. If EMC bought XtremeIO, HP bought 3PAR. When Deduplication brought the spotlight on efficient storage, IBM bought Storwize. Datadomain was bought over by EMC just when people thought they would be the change-agent in disk-based back-up. As many observed, the incumbents somehow always managed to gobble up new entrants before they became challenger.
How long can they sustain? Well, they will sustain as long as the game is played in the same way by the new entrant and buyers continue to look at storage in the same way they have been looking.
But what happens if the game itself changes? Is it realistic to expect that the game will change when incumbents have low incentives to change the game? What incentives customers have to take up the risk of changing their way of looking at their IT puzzle?
We will explore that in next post.