About a decade ago, new companies formed around building online applications in the areas of SaaS, Social Media, and other verticals that required the ability to scale effortlessly in multiple dimensions to support growth and peaks in demand. These companies and technologists built a new kind of infrastructure to service a rapidly-growing customer base that required real-time information. They relied upon low-latency storage resources directly installed in servers as direct-attached-storage (DAS) in order to put the data as close to the CPU as possible. The scale-out database technology that underpinned these applications could manage data across the cluster, and avoided the need to deploy traditional shared storage resources. Examples are shown below:
A key enabler for this was proprietary PCIe-connected flash SSDs, which could out-perform an entire storage array with a single device plugged into a server, and deliver much lower levels of latency. These products required custom software in the hosts since there were no standard protocols for PCIe-connected SSDs, until NVMe. We continue to see this architecture evolve and grow as more and more customers transition and adopt low-latency flash resources directly in these clusters using NVMe SSDs.
However, direct-attached storage still has all of the drawbacks that it did 20+ years ago before SANs were developed, in terms of manageability, capacity, and utilization. There have been no shared storage products that have enough performance and deliver low enough latency to service these highly-parallel, scale-out clustered applications running on large server clusters.
With the NVMe-Over-Fabrics protocol (NVMe-oF protocol driver is now standard in RHEL, CentOS, and Ubuntu), which delivers local NVMe SSD latency over a network, and a platform like Pavilion’s that can deliver SSD-class performance and latency to racks of servers from a single appliance without any host footprint, the benefits of shared storage can now be delivered in these environments for the first time.
Pavilion’s customers cover a broad spectrum of verticals, but all have one thing in common: clustered, parallel applications that rely on locally-attached low-latency flash storage in large server farms. Applications cover a lot of ground, from analytics to high-scale OLTP applications, and usually are leveraging clustered database technology.
Pavilion is delivering never-before-seen benefits in these environments in terms of both reducing infrastructure footprint and lowering operational cost, by delivering on the economics of shared, pooled storage for an entire rack (or racks), without sacrificing performance or latency in these high-scale environments.
DAS requires users to make storage provisioning decisions at procurement time, instead of application deployment time. Under-utilization of flash resources is the result, which can be as low as 20% at some of our customers. In contrast, rack-scale shared storage from a Pavilion array can be utilized at rates up to 80%. Pavilion has very low overhead with RAID6 (16+2), and provides thin provisioning to increase utilization even further. As a result, our customers deploy up to 75% less raw flash resources in their environment in many cases, leading to both capital and operational cost savings. In addition, for the first time they can take centralized snapshots of the entire clustered environment, allowing for easy handling of backups and test-dev copies of the data.
Another critical element required to deliver on these benefits is performance density. If you can’t deliver the required performance to racks of application servers, which all require direct-attached SSD speed and latency, you won’t be successful no matter how good your data management features are.
Finally, a product needs to do all this with minimal installation and operational complexity. Some vendors promote low-latency shared storage, but require custom software to be installed on all hosts. The days of installing proprietary storage stacks on hosts went away with the advent of NVMe and NVMe-oF, so don’t return to the dark ages of the early PCIe-connected flash days when proprietary drivers were required to get high performance and low latency.
Pavilion’s high-performance storage system can do it all – power racks of servers with flash resources without sacrificing performance or latency vs. direct-attached SSDs, all in a plug and play, easy to deploy fashion. As a result, Pavilion is delivering the economic benefits of shared storage to high-scale parallel applications.
If you are interested in learning more about Pavilion’s revolutionary storage platform and the benefits it can deliver in your environment, please contact us!