MayaNAS: Using Wasabi Hot Object Cloud Storage

Wasabi Object storage on MayaNAS

Posted on April 25, 2019 at 11:09 AM


Introduction

With MayaNAS it is very easy to connect with all popular multi-cloud object storage for file storage and achieve extreme throughput for your file based workloads without using any additional resources on cloud, and cut down storage costs. It readily supports Azure Blob, Amazon S3, Google Cloud Storage and any S3-compatible object storage. In this blog we will be exploring how to connect with the Wasabi Hot Cloud Storage. which is described on their website as,
Wasabi Hot Cloud Storage is enterprise class, tier-free, instantly available and allows you to store an infinite amount of data affordably. Wasabi provides an S3-compliant interface to use with storage applications, gateways and other platforms.

Getting Started

To get started we will need a bucket name and the access key to connect to the Wasabi cloud storage. You can get those information from your Wasabi Console or create new one if needed. If you are new to Wasabi Cloud Storage they provide 30-day trial signup with no credit-card requirement.

Steps

Since It has fully S3-compliant interface we can make use of this hot storage with MayaNAS as follows:

  1. Define a cloud profile providing the access key and access URL information, which will be one of the following depending on the location of the bucket.
    • s3.wasabisys.com or s3.us-east-1.wasabisys.com
    • s3.us-west-1.wasabisys.com
    • s3.eu-central-1.wasabisys.com
    mayacli create cloud wasabi s3type=aws accesskey=AGZH7GAGXXXXXXXXXJO:e1eIotkPT5aIwasabiI6RBFWPro5FWQfwjLrEIp9 baseurl=https://s3.uswest1.wasabisys.com
  2. Check that your buckets are visible from MayaNAS CLI mayacli
     
    mayacli show cloud
    Name                           Cloud    Account              Location   BaseURL
    ------------------------------ -------- -------------------- ---------- --------
    wasabi                         aws      AGZH7GAG1Q8BI9TFN7JO            https://s3.uswest-1.wasabisys.com/
    
    mayacli show cloud wasabi type=list
    Name                             Other
    -------------------------------- --------
    testdisk1                        2019-04-25T05:10:32.000Z	
    testdisk3                        2019-04-25T05:10:06.000Z	
    
    mayacli show cloud wasabi bucket=testdisk3 type=size 
    Name                             Counted  Size    
    -------------------------------- -------- --------
    testdisk3                        16033    16810770458 (15.66G)
    
  3. Configure a disk of any size backed by this wasabi object storage bucket
    mayacli create disk wasabidisk1 cloud=wasabi size=10T blocksize=1M bucket=testdisk3
    mayacli set disk wasabidisk1 online=1
    
    
  4. ZFS zpool can be created with MayaNAS as follows.
    zpool create testwasabi /mnt/wasabidisk1
    zfs create -o recordsize=1m  testwasabi/testfs

FIO Performance

Run FIO benchmark program to evaluate basic sequential read and write performance. Here block size of 1MB with sync option is given to get the true performance without any caching skewing our results.
fio --ioengine=sync --direct=0 --fsync_on_close=1 \
--randrepeat=0 --nrfiles=1 --name=seqwrite --rw=write --bs=1m --size=10G \
--end_fsync=1 --fallocate=none --overwrite=0 --numjobs=10 \
--directory=/testwasabi/testfs/test --loops=1


fio --ioengine=sync --direct=0 \
--fsync_on_close=1 --randrepeat=0 --nrfiles=1  --name=seqread --rw=read \
--bs=1m --size=10G --end_fsync=1 --fallocate=none  --overwrite=0 --numjobs=10 \
--directory=/testwasabi/testfs --invalidate=1 --loops=1

From AWS c5n.2xlarge (8 vcpus, 21G) using single bucket
WasabiS3
Sequential Write367 MB/s502 MB/s
Sequential Read787 MB/s739 MB/s
Wasabi Object storage performance on MayaNAS
From AWS c5n.4xlarge (16 vcpus, 42G) with one bucket and three buckets
WasabiS3
1 bucket
Write 426 MB/s534 MB/s
Read881 MB/s896 MB/s
3 buckets
Sequential Write855 MB/s1318 MB/s
Sequential Read1577 MB/s1650 MB/s
Wasabi Object storage performance on MayaNAS
From AWS c5n.9xlarge (36 vcpus, 96G) with buckets 1, 3, and 5
WasabiS3
1 bucket
Write 354 MB/s537 MB/s
Read975 MB/s945 MB/s
3 buckets
Write849 MB/s1367 MB/s
Read2024 MB/s2674 MB/s
5 buckets
Write1321 MB/s2073 MB/s
Read2957 MB/s3242 MB/s
Wasabi Object storage performance on MayaNAS

Summary

Cloud native MayaNAS delivers exceptional throughput by connecting to multi-cloud object storage seamlessly even on a single instance using highly scalable S3 storage without the burden of cluster management or deploying other expensive disk resources. The results show that the performance scales linearly with S3 object storage as you add more bucketsn. Interestingly Wasabi storage provides very good read throughput compared to AWS S3. MayaNAS provides the best compatibility to your applications that was not designed for object storage by presenting traditional file storage to your applications with data integrity offered by ZFS.