Set the Speed Dial for Pavilion Data

NVMe SSDs are the most expensive non-volatile storage in today’s data centers. But optimizing your ROI on NVMe can be tricky. Conventional wisdom says the highest performance with the lowest latency is achieved by installing the SSD directly into the server and scaling throughput or IOPS in a parallel fashion by adding more servers, each with its own NVMe SSD. However, increasing IOPS or throughput to match your workload is directly proportional to the amount of NAND Flash and power settings on the SSD. If your workload requires a small amount of storage, as most databases do, but your need…

Supercomputing Architecture for Any Data Center

Mapping the Human Genome was just the beginning for the life sciences discipline of bioinformatics.  Now, the goal is to apply every map of every person against a perfectly healthy baseline to identify sequences of DNA that carry diseases like diabetes, asthma, migraine, and schizophrenia.  However, even with the most powerful compute, network and storage resources on our planet, full genomic map comparisons are not possible. The International HapMap Project is a more efficient method to isolate genetic mutations using massively parallel processing with Hadoop® MapReduce and Spark® analytics to obtain statistically significant comparisons.  HapMap would not exist without the…

Disaggregation for Less Aggravation

NVMe SSDs ushered in an era of standards-based access to Solid State storage.  Previous protocols like SCSI and SAS were designed for spinning disk media and carried severe overhead that squelched the performance that can be delivered by NAND.  It’s no coincidence that IDC forecasts NVMe will make up more than 50% of all enterprise SSD shipments by 2020[1]. One step forward, two steps back While significant increases in IOPS and throughput can be delivered with today’s NVMe SSDs.  The deployment model is arcane.  By implementing NVMe inside of the server, we are back to the problems of direct-attached storage…

Pavilion’s perspective as vendors enter the NVMe game

NVMe Demands Storage System Re-Design, Not Retro-Fits We are experiencing exciting times in the storage industry.  During the week of May 1, 2018, Dell announced the NVMe-based PowerMax.   At Pavilion we welcome these kinds of announcements, since it accelerates the customer awareness around the fact that the future of storage is NVMe.   Let the loud and expensive bullhorn sound…its fascinating to watch.   Why?  Because the transition to all-NVMe storage is being validated before our eyes. You know you’re on to something important when the AFA storage incumbents make their retrofit product announcements.  Rather than doing the difficult, yet required work…

Ditch the DAS: The Benefits of Rack-Scale Flash

About a decade ago, new companies formed around building online applications in the areas of SaaS, Social Media, and other verticals that required the ability to scale effortlessly in multiple dimensions to support growth and peaks in demand. These companies and technologists built a new kind of infrastructure to service a rapidly-growing customer base that required real-time information. They relied upon low-latency storage resources directly installed in servers as direct-attached-storage (DAS) in order to put the data as close to the CPU as possible. The scale-out database technology that underpinned these applications could manage data across the cluster, and avoided…

Why NVMe? Why NVMe over Fabrics? Why Pavilion Data?

Applications are becoming increasingly parallel. What used to be done on a single application server is now spread across a cluster of servers operating in parallel. This allows for scaling in multiple dimensions. Need more bandwidth or compute power for your clustered application? Just add more servers to the cluster. Hyperscale customers have been doing this for nearly a decade now, and the number of customers that embrace this architecture are growing constantly. This scale-out approach also is used to scale storage performance and capacity. Traditional all-flash-arrays cannot offer enough density and/or performance to satisfy these applications. Therefore, the servers…

Don’t be fooled: Off-The-Shelf SSDs make a solid design choice in storage systems

Storage system vendors have chosen to integrate flash in two ways: incorporate standard off-the-shelf SSDs, or design their own flash modules and controllers.  Many of the early all-flash array pioneers, like Violin and TMS, designed their own custom flash modules for what were very sound reasons at the time.  The choice to go in one direction or another in this area revolves around several criteria, most notably Performance, Time-to-Market, and Cost.   I explore all 3 as it relates to this subject below: Performance Many vendors made the design choice to build their own flash controllers before NVMe products were available, due…