Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Background

NVMe has already established as storage choice for modern workloads requiring low latency and high performance. On cloud it is available as NVMe ephemeral resources that can be easily attached to computing instances.  Though this option  is within reach it comes with other challenges

  • Storage is ephemeral in nature and cannot be used when persistence is required as the contents do not survive stop
  • Compute instances cannot be stopped and it requires removal and recreating of instance if one does.  Now your applications need to consider this unnecessarily.
  • The minimum capacity is 475Gib even if the application will use only 1/10 of that capacity.
  • Hinder any solution that require  separation of storage and compute.
  • The Elastic Block Storage for a single compute instance will peak at 40K IOPS.

MayaScale provides NVMe Fabric over TCP on Google Cloud for high-performance shared storage and its performance is compared against running locally using the ephemeral local NVMe SSDs.\

Benchmark Setup

MayaScale works better when configured with multiple cores to provide same parallelism seen with NVMe hardware queues and also the compute instance network and storage performance is also tied to the number of provisioned CPU cores.  The software has very low memory requirement only.

  •  server node of machine type n1-highcpu-16 running MayaScale software
  • 16 CPU cores (Broadwell) and  ~14 GB memory
  • 4 Local NVMe SSDs of 375GB

  • client node of machine type n1-highcpu-16

FIO Performance Report


Column
width40%


4K IO

Local

IOPS( K)

NVMe-Of

IOPS (K)

random write100100
random read180180


4K IO

Local

IOPS (K)

NVMe-Of

IOPS (K)

random write200199
random read360360



Column
width40%


4K IO

Local

IOPS (K)

NVMe-Of

IOPS (K)

random write299299
random read522413


4K IO

Local

IOPS (K)

NVMe-Of

IOPS (K)

random write398368
random read659384




Conclusion

Mayascale is able to deliver same IOPS and Bandwidth  but with added latency even over NVME fabric using TCP.