Linux Clients
Recent Linux kernel distributions have support for NVMe host tcp driver but you may have to load the driver.
To Load the driver
# modprobe nvme-tcp
- To discover this NVMe target from another client with NVMe over fabrics support.
[root@ip-172-31-20-45 ~]# nvme discover -t tcp -a 50.50.50.50 -s 4420 Discovery Log Number of Records 1, Generation counter 1 =====Discovery Log Entry 0====== trtype: tcp adrfam: ipv4 subtype: nvme subsystem treq: not specified portid: 3 trsvcid: 4420 subnqn: nqn.2022-20.com.zettalane:hanvme.stor traddr: 50.50.50.50 eflags: not specified sectype: none
For the client to perform IO on the discovered NVMe target, it has to be connected and configured with the host as follows.
# nvme connect -t tcp -a 50.50.50.50 -s 4420 -n nqn.2022-20.com.zettalane:hanvme.stor # nvme list Node Generic SN Model Namespace Usage Format FW Rev --------------------- --------------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- -------- /dev/nvme0n1 /dev/ng0n1 3fdbecf743216d21 Linux 1 53.69 GB / 53.69 GB 512 B + 0 B 3.10.0-1
To disconnect the NVMe device
# nvme disconnect -n nqn.2022-20.com.zettalane:hanvme.stor NQN:nqn.2022-20.com.zettalane:hanvme.stor disconnected 1 controller(s)
Windows Clients
Windows NVMe host tcp drivers are available from 3rd party vendor StarWind. Once the Free NVMEoF initiator is installed.
- To discover MayaScale NVMe target 50.50.50.50 from Windows client 10.1.0.6
C:\Program Files\StarWind Software\StarNVMeoF> StarNVMeoF_Ctrl.exe discovery_tcp 50.50.50.50 10.1.0.6 StarNVMeoF Controller Application v 1.9.0.567 Our Miniport #2 (ver 1.9 build 560) Discovering of TCP addr 10.1.0.5:4420 from 10.1.0.6... NET_LUID: 0x6008001000000 Discovery returned: gen 3, numrec 1 1) subnqn nqn.2018-07.com.zettalane:mayascale1.10a0500 NVMe: port 0x0, cntlid 0xffff, trtype TCP, addr 0.0.0.0 : 4420, adrfam 1, asqsz 32 For the client to perform IO on the discovered NVMe target, it has to be connected and configured with the host as follows.
C:\Program Files\StarWind Software\StarNVMeoF> StarNVMeoF_Ctrl.exe insert_tcp 10.1.0.5 10.1.0.6 nqn.2018-07.com.zettalane:mayascale1.10a0500 nqn.2018-07.com.zettalane:mayascale1.10a0500 StarNVMeoF Controller Application v 1.9.0.567 Our Miniport #2 (ver 1.9 build 560) HostId 0x8CB3F1480812FD4B-655FA8ECC1E51D04 - HostName certvm NET_LUID: 0x6008001000000 Connecting to TCP target at addr 10.1.0.5:4420 from 10.1.0.6 SubNQN=nqn.2018-07.com.zettalane:mayascale1.10a0500 HostNQN=nqn.2018-07.com.zettalane:mayascale1.10a0500 nq=0 qd=0 core=0... Controller connected: TargetId = [0] numLuns = 2, numIoQs = 2, ioQDepth = 128, firstCore = 0 Adding persistent entry for target 0... Persistent entry for target 0 addedTo disconnect the NVMe device
C:\Program Files\StarWind Software\StarNVMeoF> StarNVMeoF_ctrl remove 0 StarNVMeoF Controller Application v 1.9.0.567 Our Miniport #2 (ver 1.9 build 560) Disconnecting ControllerId 0... Controller disconnected: Id= [0]