NVMe Servers: The Next Leap in Bare Metal Performance

A few years ago, SSD drives took the industry by storm - boosting performance for all kinds of use cases.  Now Packet is introducing the next revolution and its name is NVMe.

Before we get to the promise of NVMe: what is it?

NVM Express (or NVMe) is a driver standard for taking full advantage of flash media (aka SSDs) via PCIe interfaces.  While PCIe attached flash has been in play for several years via expensive, proprietary solutions (FusionIO anyone?), the industry has now successfully adopted the NVMe standard, allowing all kinds of innovation.  

In the next year or so, you’re going to see a LOT of NVMe flash hitting the market, especially for data-intensive applications, in which it can deliver six times the throughput of existing 6 Gb/s SATA SSDs with significantly lower latency.  Zoom zoom!

Wait, 6x LESS latency on reading and writing data?  How does this thing work?  Well, quite simply, it gets rid of the SATA bus and associated AHCI-based drivers and streamlines the I/O stack.  SATA was designed primarily for interfacing with mechanical hard disk drives (HDDs), and has become increasingly inadequate as SSDs have improved - frankly, the disks are faster than the connector.

By ripping out and not relying on plumbing that was built for a world of slow, queued up, spinning disks, NVMe flash drives offers reduced latency, increased Input/Output operations per second (IOPS), much more bandwidth to the processor, and lower power consumption, in comparison to SAS-based or SATA-based SSDs.

Packet and NVMe - Our Backstory

Packet’s product strategy prioritizes consistent availability over customization.  Our goal with each configuration is to really push the boundaries of performance, while providing a lot of value in terms of price.  

Looking at the market, we saw a dearth of good options for high IO workloads: big data, databases, etc.  The best option is the Rackspace High IO OnMetal box, which relies on a pair of super expensive 1.6TB Seagate Nytro WarpDrives.  This means you’re paying $1750 + management per box (or about $2.50 an hour before management).

AWS offers a high IO option (like the i2.4xlarge) but it still works out to $3.41 an hour.  We wanted to do better - our goal was less than $2 per hour.

We quickly became enamored with the emerging NVMe standard, and the ecosystem of flash drives coming to market that take advantage of these huge increases in performance with no real change to the underlying NAND.  Our friends at SuperMicro shipped us a FatTwin with 2 x 800GB Intel P3600 NVMe Flash Drives.  We were smitten!  But could we bring this tech to the masses, not just the chosen few with deep pockets?

Our Type 3 Config - A “Price to Performance” Powerhouse at Just $1.75 / hr

To ensure that this config would work for a broad enough range of power users, we beefed up our FatTwins with a healthy dose of extra speedy, low latency DDR4 RAM (128GB DDR-2100 ECC) and a generous amount of processing cores (16 physical / 32 hyperthreaded processing cores via 2 x E5-2640 v3 chips running at 2.6Ghz).  

We kept the boot disks fast with dual enterprise SSD’s.  This means that our Type 3 config can tackle intensive IO applications that also required fast processing threads.  It is primed for in-memory databases like Redis and Aerospike.  And with 32 cores, it even serves as a pretty awesome virtualization node for private cloud workloads (e.g. VMWare or OpenStack / Xen / KVM, etc).  

Massive Disk IO Requires a Better Network

With compute and storage demands increasing and scale-out architectures becoming more common, inter-server traffic has become a critical scalability bottleneck.  Most cloud providers limit “east west” traffic between servers -- either due to investment in the hardware or limitations in the software-based routers (e.g. OVS, tunnel overhead) that are the basis for most virtual clouds.   To help ensure maximum performance, we include bonded 2 x 10GbE SFP+ interfaces that offer high throughput, low latency, traffic between servers and have over-invested in our switching infrastructure to ensure maximum performance and reliability.

How It Compares

So, how’d we do?  Pretty good we think!  The specs of the Type 3 are off the charts and we managed to bring the price down to a point where just about anybody can harness the power of these incredible machines to tackle their intensive (or just well loved!) workloads.  As always, they’re available on-demand, by the hour via the Packet platform.