NetApp's response to EMC's XtremeIO

After almost two years of maintaining the line that flash is good only as accelerator, NetApp  finally firmed up its all-flash storage array roadmap. First came EF540, a Flash version of EF5400, NetApp's present top-end disk-array a product-line that was brought through Engenio acquisition. Luckily this time NetApp decided not to take the path of ONTAP integration and adopted Engenio's SANtricity as its second OS. It helped NetApp to shorten the path between  technology acquisition and taking it to the market. A distinct sign that NetApp has permanently dropped its policy of one company one OS, NetApp also took up the challenge to develop a new OS for Flash drives. Tagged as project Mars, NetApp developed this product with a few of its old-timer technical stalwarts, isolated from it mainstream engineering development over last two years.  FlashRay, the new platform developed for the flash system, is expected to be released as beta, by around the end of this year. It probably will take another 6-12 months for NetApp to take the new product through its engineering verification process and mature it as a product. But the announcement has already created enough flutter in the market. It is clear that FlashRay is carved out of ONTAP. In all likelihood it has all the popular ONTAP gears like snapshot, deduplication, scale-out clustering. It is expected to support seamlessly all NetApp Data protection software like snap* suites but it is not clear how much of the FlashRay is not ONTAP.
After its failed attempt to acquire XtremeIO, NetApp did not have anything to offer to its customers looking for SSD-scale performance. EF540 has filled that lacuna in NetApp's portfolio especially in SAN space.  But FlashRay will be NetApp's flagship platform for SSDs and hybrid storage for the future. It is expected to counter EMC's yet to be launched high-performance Flash-array, being developed as Project-X. Project-X, if one goes by the buzz in the Industry, is built over XtremeIO's technology which EMC bought last year.
 From the engineering standpoint a design excavation was a long-due item for ONTAP, its storage OS for last 20 years. Fact is ONTAP already is quite heavy with years of additions of features and injections of codes to take care of diverse needs that it was not designed for to start with. Although WAFL[ONTAP's File system] designers took inspiration from Log-structured File system besides FFS and Episode from Berkeley in the beginning [see Dave's Blog], it was originally designed for a NAS server that maximized NFS write throughput. Obviously it cannot make efficient use of Flash read/write throughput unlike HDD. Flash wearing is another problem that WAFL needed to be made aware of [WAFL can recognize disk failure but memory loss due to wearing is a different problem] So WAFL needed an overhaul for it to be useful for all Flash-system or even hybrid storage arrays.
With XtremeIO, EMC got a head-start in all-flash array space. Though NetApp lost its bid to buy XtremeIO, it still could pick another Flash startup [Violin memory, whiptail among others] but  experience taught them that  a home-grown alternative, if one can build, is always a credible and safe bet. Spinnaker acquisition taught them that buying a startup technology and integrating it into its its existing portfolio is fraught with high complexity and delay. Datadomain acquisition taught them that buying a proven startup may not be that easy with EMC getting stronger every year. They had a proven Deduplication solution and it was that  home-grown Deduplication solution which saved them in front of the customers when Datadomain deal fell through. So they did not want to commit the same mistake for Flash. Some commented that NetApp took up Mars project after its acquisition bid for XtremeIO fell through but that hardly sounds logical. NetApp has a strong engineering capability, no doubt, but developing a winning product requires a lot more careful planning and long-term budget-commitment that hardly can be expected from a knee-jerk response to a failed acquisition attempt.
Many suspect that NetApp will keep its option of buying a Flash startup open [in fact some of its customers are buying flash-based storage controllers from a specific start-up with full knowledge that the startup is on radar of NetApp and they expect NetApp to buy that startup in the long run] but there are chances that FlashRay will find persistent and passionate support from the NetApp engineering community, starved for a long-time to work on ground-breaking innovations, making it another success story for NetApp engineering. Re-engineering Ontap and WAFL has also another advantage, they can reuse the experience to trim their mainstream version of Ontap.
                    How the flashArray will fare in the market is in the realm of speculation but this endeavour will  surely prove to its own people that NetApp still values its engineering team.

Innovations towards more energy- efficient storage

source: ComEd

Electrical Power Consumption is one large cost component in data centre expenses. As per some of the available reports, 30% of data centre energy bill is attributed to IT Equipment. Last year in September, New york Times published an article that highlighted how much energy bill these modern data centres run. As per that article [read here], Google’s data centers consume nearly 300 million watts and Facebook’s about 60 million watts. "Worldwide, the digital warehouses use about 30 billion watts of electricity, roughly equivalent to the output of 30 nuclear power plants, according to estimates industry experts compiled for The Times.", NYT reports.
Since power consumption has become such a big issue with data centres, there are many companies like ComEd whose business entirely is focused to solutions for reducing the energy use of data centers.
But given that 30% of that consumption is driven by servers and storage devices, the question arises as to why the storage vendors are not bringing more energy efficient storage. Fact that 'energy efficient storage' has not been highlighted to be one of the major trends in 2013, tells us that  addressing the problem is not very simple. Let us see why.

Electrical Power efficiency in disk based system

Storage systems at present are primarily disk-based. Even our backup system is predominantly disk-based except possibly the main-frame systems. With disks-based systems, historically the storage software are designed assuming that all attached disks are online i.e. disks have to be connected to the data bus of the storage controller and are powered on and are always available for the higher level . These systems traditionally cannot differentiate between a failed disk and a powered off disk. Fact is disk vendors, initially did not provide any different device state in the disk controller [electrical circuitry attached to the disk that controls read/write to the disk] that would identify that disks are powered down.  So the storage vendors too designed their File Systems /RAID Arrays with the assumption that disks are always powered on and in active state. Probably nobody imagined that hundreds of disks will be attached to a single storage controller one day. With the rising clamour for better power management, by 2008-2010, the disk vendors introduced power management scheme in SATA controllers. Since SATA is mostly used in near-line, backup and archive systems, and these systems have large number of disks which are not used all the time, one can power down 'idle' disks, if possible and bring considerable power saving. SATA provides two link power management states, in addition to the “Active” state. These states are “Partial” and “Slumber,” that, by specification, differ only by the command sent on the bus to enter the low power state, and the return latency. Partial has a maximum return latency of 10 microseconds, while Slumber has a maximum return latency of 10 milliseconds [further read].  Storage systems would need to tweak their File System and/or RAID controller to take advantage of SATA power management. Handling microseconds of latency is easier but handling miliseconds of latency requires major design change in the software.
EMC first brought this power management feature in their Clarion platform. The solution was to create a new RAID group and assign the powered down disks to that group after 30 min. idle state. The controller could recognize these disk states and can wait for maximum 10 seconds for the disks to come back to active state. EMC claims that this power down feature would save around 54% in average [ further read].  To my knowledge, other storage vendors are in the process of adopting this power saving feature in their controllers. If they haven't done already, it is probably because their disk based system would require pretty large design changes to accommodate these new states of disks. I personally was involved in analysis for one prominent storage vendors, and was made adequately aware of how deep the changes would go. However my take is that in next 3-4 years, most disk-based storage vendors will adopt the SATA power management.
That obviously leaves out a large number of systems that use FC/SAS disks. Fortunately SAS 2.1 brought in a new set of power management features which disk vendors are expected to adopt in next few years and SAS is expected to replace FC disks going forward, so we have a workable solution in the future.

Tape-based system as an alternative

Tape controllers on the other hand do not suffer such issues. Tapes, in fact are designed with specific attention to offline storage. One can backup the data to the tapes, take the cartridge out of the online system, store them in separate locker and insert them to the system when needed. Inside the vault, the tape cartridge do not consume any electrical power. They do however needs periodical data auditing since tape-read fails more frequently that disks.
But with the  new long-life  and a high-capacity LTO-5 and LTO-6 tapes, those problems are much reduced. Many are in fact bringing back tape storage in their system. EMC also is promoting tapes for backup data. Although it sounds like a regressive step, one must accept that tape does provide a considerable power saving option especially when it comes to storage for data backup and archival.

Little Longer-term Future

To envisage the future of power efficient storage, we need to look at the problem holistically.  One can power down idle disks. However more power is consumed by active disks. Data centres also spend considerable money in cooling the data centres. The pie chart at the top shows that almost 33% of total energy is spent in Cooling system and that cost is going to rise with rising global temperature.
     A better solution would therefore be to design storage media that consumes almost zero power when kept idle but also consumes much less power even in active state compared to existing hard disks. Much much better if these media can operate at room temperature which would translate to lower energy bill for cooling. Towards this, flash provides an excellent option. Flash storage [see previous post] consumes magnitude less power for regular read/write operation and consumes almost zero power when left idle. It also provides much higher random read/write throughput making it ideal for high-performance storage. At present its relative higher cost and limited write-span are the hindrance for  it to replace disks in mainstream storage. With time there is little doubt that further innovations will bring down the cost/GB drastically. Storage capacity also will be comparable to SAS. The biggest dampener for SSD/flash so far has been its number of writes limitation. A very recent article in IEEE Spectrum indicates that we already have a breakthrough. Macronix, a Taiwanese company has reported the invention of a self-healing NAND flash memory that survives more than 100 million cycles.
 Fact is they are yet to find the limit where it breaks. They strongly believe that it will survive a billion writes. Their method is simply a local heat treatment on the chip-set to lengthen the life of the media. If that invention works, we have an alternative storage solution that meets all our stated needs, namely, 1.consume low power, 2. can operate at room temperature, 3. provide both high capacity [~ around 2 TB] and high throughput and 5. consume a fraction of space compared to HDD [the IEEE Spectrum article can be accessed here].
In a couple of years the technology is very likely to matures with full-fledged induction of flash-only storage in mainstream storage systems. EMC's xtremeIO, whipTail, violin memory and other all-flash-storage systems are likely to define tomorrow's mainstream storage system.