Posted on April 25, 2019 at 11:09 AM
Introduction
With MayaNAS it is very easy to connect with all popular multi-cloud object storage for file storage and achieve extreme throughput for your file based workloads without using any additional resources on cloud, and cut down storage costs. It readily supports Azure Blob, Amazon S3, Google Cloud Storage and any S3-compatible object storage. In this blog we will be exploring how to connect with the Wasabi Hot Cloud Storage. which is described on their website as,
Wasabi Hot Cloud Storage is enterprise class, tier-free, instantly available and allows you to store an infinite amount of data affordably. Wasabi provides an S3-compliant interface to use with storage applications, gateways and other platforms.
Getting Started
To get started we will need a bucket name and the access key to connect to the Wasabi cloud storage. You can get those information from your Wasabi Console or create new one if needed. If you are new to Wasabi Cloud Storage they provide 30-day
trial signup with no credit-card requirement.
Steps
Since It has fully S3-compliant interface we can make use of this hot storage with MayaNAS as follows:
- Define a cloud profile providing the access key and access URL information, which will be one of the following depending on the location of the bucket.
- s3.wasabisys.com or s3.us-east-1.wasabisys.com
- s3.us-west-1.wasabisys.com
- s3.eu-central-1.wasabisys.com
mayacli create cloud wasabi s3type=aws accesskey=AGZH7GAGXXXXXXXXXJO:e1eIotkPT5aIwasabiI6RBFWPro5FWQfwjLrEIp9 baseurl=https://s3.uswest1.wasabisys.com
- Check that your buckets are visible from MayaNAS CLI mayacli
mayacli show cloud
Name Cloud Account Location BaseURL
------------------------------ -------- -------------------- ---------- --------
wasabi aws AGZH7GAG1Q8BI9TFN7JO https://s3.uswest-1.wasabisys.com/
mayacli show cloud wasabi type=list
Name Other
-------------------------------- --------
testdisk1 2019-04-25T05:10:32.000Z
testdisk3 2019-04-25T05:10:06.000Z
mayacli show cloud wasabi bucket=testdisk3 type=size
Name Counted Size
-------------------------------- -------- --------
testdisk3 16033 16810770458 (15.66G)
- Configure a disk of any size backed by this wasabi object storage bucket
mayacli create disk wasabidisk1 cloud=wasabi size=10T blocksize=1M bucket=testdisk3
mayacli set disk wasabidisk1 online=1
ZFS zpool can be created with MayaNAS as follows.
zpool create testwasabi /mnt/wasabidisk1
zfs create -o recordsize=1m testwasabi/testfs
FIO Performance
Run FIO benchmark program to evaluate basic sequential read and write performance. Here block size of 1MB with
sync option is given to get the true performance without any caching skewing our results.
fio --ioengine=sync --direct=0 --fsync_on_close=1 \
--randrepeat=0 --nrfiles=1 --name=seqwrite --rw=write --bs=1m --size=10G \
--end_fsync=1 --fallocate=none --overwrite=0 --numjobs=10 \
--directory=/testwasabi/testfs/test --loops=1
fio --ioengine=sync --direct=0 \
--fsync_on_close=1 --randrepeat=0 --nrfiles=1 --name=seqread --rw=read \
--bs=1m --size=10G --end_fsync=1 --fallocate=none --overwrite=0 --numjobs=10 \
--directory=/testwasabi/testfs --invalidate=1 --loops=1
From AWS c5n.2xlarge (8 vcpus, 21G) using single bucket
| Wasabi | S3
|
|---|
| Sequential Write | 367 MB/s | 502 MB/s |
| Sequential Read | 787 MB/s | 739 MB/s |
From AWS c5n.4xlarge (16 vcpus, 42G) with one bucket and three buckets
| Wasabi | S3
|
|---|
| 1 bucket
|
|---|
| Write | 426 MB/s | 534 MB/s |
| Read | 881 MB/s | 896 MB/s |
| 3 buckets
|
|---|
| Sequential Write | 855 MB/s | 1318 MB/s |
| Sequential Read | 1577 MB/s | 1650 MB/s |
From AWS c5n.9xlarge (36 vcpus, 96G) with buckets 1, 3, and 5
| Wasabi | S3
|
|---|
| 1 bucket
|
|---|
| Write | 354 MB/s | 537 MB/s |
| Read | 975 MB/s | 945 MB/s |
| 3 buckets
|
|---|
| Write | 849 MB/s | 1367 MB/s |
| Read | 2024 MB/s | 2674 MB/s |
| 5 buckets
|
|---|
| Write | 1321 MB/s | 2073 MB/s |
| Read | 2957 MB/s | 3242 MB/s |
Summary
Cloud native MayaNAS delivers exceptional throughput by connecting to multi-cloud object storage seamlessly even on a single instance using highly scalable S3 storage without the burden of cluster management or deploying other expensive disk resources. The results show that the performance scales linearly with S3 object storage as you add more bucketsn. Interestingly Wasabi storage provides very good read throughput compared to AWS S3. MayaNAS provides the best compatibility to your applications that was not designed for object storage by presenting traditional file storage to your applications with data integrity offered by ZFS.