From Scale out NAS to Scale out Database for Big Data

This post is the first review of NewSQL DBs, as promised in my last post and we will start with Clustrix. Clustrix is interesting for two reasons: 1. the architect and some of the principle designers came from NAS space (as opposed to database technology) and 2. they appear to be doing the Isilon equivalent in Database space.
Scale Out
 Scale out as a concept became popular with massive growth of Data centre. Essentially scale out means to scale horizontally i.e. adding more compute resources to the infrastructure pool in order to take care of growing demand. Essentially it means if you have one server for n users, to serve m*n users you simply need to add m similar servers. In contrast Scale-up means scaling vertically i.e. changing your server with a more powerful server when numbers of users increase.
Scale Out NAS
So scaling out is simpler, assuming one has enough rack-space in the data centre. During last decade, Scale-out NAS became a strong market driver and many innovation engines took off that promised faster, simpler and cheaper scale out. Huge growth of unstructured data in entreprise data centre pushed the scale-up model to the edge with each hardware replacement cost getting more and more expensive. Addditonally traditional NAS also comes with certain scaling limitations such as,
1. performance is limited by single head [multicore CPUs alleviated the issue just a little]
2. limited Namespace [ typically this is File system limitation brought in to ensure performance does not degrade unacceptably with large number of files]
3. migration of data between two NAS namespace is costly.
3PAR, Spinnaker and Islion were few of the innovation leaders in the space of Scale-out NAS. After NetApp bought Spinnaker, Isilon almost occupied this niche space entirely till EMC bought it.
What did Isilon bring technologically?
Isilon's OneFS clustered File System is the core of its technology. The OneFS software is designed with file-striping functionality across each node in a cluster, a fully distributed lock manager, caching, fully distributed meta-data, and a remote block manager to maintain global coherency and synchronization across the cluster. That enables Isilon to support a massive capacity of 15PB of file storage in a single file system, making EMC position Isilon NAS as the solution for Big Data for File-storage.
Scale-out Database Appliance

Experience of building the distributed lock manager has some relevance to Database and that gave confidence to the Isilon engineers when they started designing Clustrix. Concurrency management is one of the crucial challenge in designing a clustered database.  Clustrix claims to use MVCC [Multi-version concurrency Control] with a share-nothing architecture but the most important part of their innovation is building a distributed Query execution model which they call 'Sierra Distributed Engine'.  True to their philosophy of  "Bring Query to Data and not the Data to the Query",
Sierra’s most basic primitive is a compiled program called a query fragment. They can read, insert, update a container, execute functions, modify control flow, format rows, perform synchronization, and send rows to query fragments on other nodes. These query fragments are run on the nodes that contain the data. The communication between the nodes consists of just the intermediate and result rows needed for the queries. Many, many query fragments can operate simultaneously across the cluster. Those query fragments may be different components of the same query or parts of different queries.
Queries enter the system through the front-end network and are translated to an internal representation used by Sierra. Sierra then executes the queries in an efficient parallel fashion. Sierra uses SSDs to store the data, NVRAM to journal changes, and Infiniband to communicate with other nodes in the cluster. [Clustrix whitepaper]
Performance
Clustrix claims that transaction throughput /million of records in the database scales linearly with addition of nodes.
They support replication with MySQL, both Key-Value based query and SQL query as well as on-the-fly Schema change. The user does not need to worry about sharding / partitioning since basic performance bottleneck observed with RDBMS is removed here. Overall, Clustrix seem to have brought a strong NewSQL product.
For more information take a look at Clustrix resource page.