Merge pull request #73 from pkoutoupis/feature/nvme-export-support

Feature/nvme export support

CHANGELOG.md

@@ -1,3 +1,14 @@
+### Release 8.0.0 ###
+
+- module: Updated for 5.14 and 5.15 kernels
+- utility: Added NVMe Target support / framework
+- utility: Added support for dm-writecache status readout in API
+- utility: Module checker code now traverses via sysfs
+- utility: Added module checker in daemon
+- documentation: Clean up formatting and add content
+- packaging: Fixed dependencies in spec and debian control files.
+- misc: Cleaned up and optimized scripts to enable rapiddisk/cache on root during boot (thank you Matteo Tenca)
+
 ### Release 7.2.1 ###
 
 - module: Added support for RHEL 8.4 kernel

README.md

@@ -2,8 +2,91 @@
 
 Author: Petros Koutoupis (<[email protected]>)
 
-Building and Installing the rapiddisk kernel modules and utilities
-------------------------------------------------------------------
+## About the RapidDisk Project
+
+RapidDisk contains a set of advanced Linux RAM Drive and Caching kernel
+modules. The user space utilities allow you to dynamically allocate RAM
+as block devices to either use them as stand alone disk drives or even
+map them as caching nodes to slower local (or remote) disk drives. The
+same utilies provide users with the capability to export the same volumes
+across an NVMe Target network.
+
+### Caching Policies
+
+Leverage a high speed RAM drive to add speed to a slower volume by
+utilizing the (Linux native) Device-Mapper framework. Enable a Write /
+Read-through or Write Around Least Recently Used or LRU (FIFO) cache.
+
+#### Write-Through / Read-Through Caching
+
+This is where an application treats cache as the main data store and
+reads data from it and writes data to it. The cache is responsible for
+reading and writing this data to the permanent storage volume, thereby
+relieving the application of this responsibility.
+
+In this mode, all writes are cached to a RapidDisk RAM drive but are also
+written to disk immediately. All disk reads are cached. Cache is not persistent
+over device removal, reboots, or after you remove the Device-Mapper mapping.
+This module does not store any cache metadata on RapidDisk volumes but instead
+in memory outside of RapidDisk. You can map and unmap a cache drive to any
+volume at any time and it will not affect the integrity of the data on the
+persistent storage drive.
+
+#### Write-Around Caching
+
+Write Around caching shares some similarities with the Write-Through
+implementation. However, in this method, only read operations are cached
+and not write operations. This way, all read data considered hot can
+remain in cache a bit longer before being evicted.
+
+### RESTful API
+
+The RapidDisk Daemon (rapiddiskd) enabled remote management of RapidDisk
+volumes. The management commands are simplified into a set of GET and POST
+commands.
+
+An example of a GET command:
+
+```console
+# curl -s --output - 127.0.0.1:9118/v1/listRapidDiskVolumes|jq .
+{
+  "volumes": [
+    {
+      "rapiddisk": [
+        {
+          "device": "rd1",
+          "size": 67108864
+        },
+        {
+          "device": "rd0",
+          "size": 67108864
+        }
+      ]
+    },
+    {
+      "rapiddisk_cache": [
+        {
+          "device": "rc-wa_loop7",
+          "cache": "rd0",
+          "source": "loop7",
+          "mode": "write-around"
+        }
+      ]
+    }
+  ]
+}
+```
+
+An example of a POST command:
+
+```console
+# curl -X POST -s 127.0.0.1:9118/v1/createRapidDisk/128|jq .
+{
+  "status": "Success"
+}
+```
+
+## Building and Installing the rapiddisk kernel modules and utilities
 
 Change into the project's parent directory path.
 
@@ -41,8 +124,7 @@ For utility information please reference the rapiddisk manual page:
 # man 1 rapiddisk
 ```
 
-Inserting/Removing the rapiddisk / rapiddisk-cache kernel modules
------------------------------------------------------------------
+## Inserting/Removing the rapiddisk / rapiddisk-cache kernel modules
 
 To insert the rapiddisk module:
 
@@ -68,8 +150,7 @@ To remove the rapiddisk-cache module:
 # modprobe -r rapiddisk-cache
 ```
 
-Building and installing / uninstalling the tools ONLY
------------------------------------------------------
+## Building and installing / uninstalling the tools ONLY
 
 Installing:
 
@@ -83,22 +164,19 @@ Uninstalling:
 # make tools-uninstall
 ```
 
-Installing modules for DKMS support
------------------------------------
+## Installing modules for DKMS support
 
 ```console
 # make dkms-install
 ```
 
-Uninstalling modules for DKMS support
------------------------------------
+## Uninstalling modules for DKMS support
 
 ```console
 # make dkms-uninstall
 ```
 
-Managing the RapidDisk daemon service
---------------------------------------------------------
+## Managing the RapidDisk daemon service
 
 After installation, to start the service via systemd:
 
@@ -123,4 +201,3 @@ To start the service at boot via systemd:
 ```console
 # systemctl enable rapiddiskd.service
 ```
-

conf/Makefile

@@ -12,7 +12,7 @@ all:
 
 .PHONY: install
 install: all
-	@echo Installing rapiddisk cluster and rapiddiskd daemon files.
+	@echo Installing rapiddisk high availability and rapiddiskd daemon files.
 	$(MKDIR) $(DESTDIR)$(CFG_DIR)
 	$(CP) $(SERVICE).* $(DESTDIR)$(CFG_DIR)
 	$(CHMOD) -R 755 $(DESTDIR)$(CFG_DIR)
@@ -22,7 +22,7 @@ install: all
 
 .PHONY: uninstall
 uninstall: 
-	@echo Uninstalling rapiddisk cluster and rapiddiskd daemon files.
+	@echo Uninstalling rapiddisk high availability and rapiddiskd daemon files.
 	$(RM) $(DESTDIR)$(CFG_DIR)
 	$(RM) $(DESTDIR)$(SYSD_DIR)/$(SERVICE)d.*
 

conf/rapiddiskd.service

@@ -1,5 +1,5 @@
 [Unit]
-Description=an enclosure monitoring daemon
+Description=A rapiddisk network management daemon
 After=network.target
 
 [Service]

debian/changelog

@@ -1,3 +1,14 @@
+rapiddisk (8.0.0-1) released; urgency=medium
+
+* module: Updated for 5.14 and 5.15 kernels
+* utility: Added NVMe Target support / framework
+* utility: Added support for dm-writecache status readout in API (github issue #65)
+* utility: Module checker code now traverses via sysfs
+* utility: Added module checker in daemon
+* documentation: Clean up formatting and add content
+* packaging: Fixed dependencies
+* misc: Cleaned up and optimized scripts to enable rapiddisk/cache on root during boot (thank you Matteo Tenca)
+
 rapiddisk (7.2.1-1) released; urgency=medium
 
 * module: Added support for RHEL 8.4 kernel

debian/control

@@ -1,13 +1,14 @@
 Package: rapiddisk
-Version: 7.2.1-1
+Version: 8.0.0-1
 Section: base
 Priority: optional
 Architecture: amd64
-Depends: dkms, gcc, libjansson4
+Depends: dkms, gcc, libjansson4, libmicrohttpd12
 Maintainer: Petros Koutoupis <[email protected]>
 Description: RapidDisk
  The RapidDisk software defined advanced RAM drive and storage
  caching solution. This suite includes a collection of modules,
  configuration files, and command line utilities for managing
- RapidDisk enabled storage volumes.
+ RapidDisk enabled storage volumes and accessing them either
+ locally or across an NVMe Target network.
 

debian/postinst

@@ -16,9 +16,9 @@ fi
 case "$1" in
 
   configure)
-    dkms add -m rapiddisk -v 7.2.1
-    dkms build -m rapiddisk -v 7.2.1
-    dkms install -m rapiddisk -v 7.2.1
+    dkms add -m rapiddisk -v 8.0.0
+    dkms build -m rapiddisk -v 8.0.0
+    dkms install -m rapiddisk -v 8.0.0
     echo "rapiddisk max_sectors=2048 nr_requests=1024" >> /etc/modules
     echo "rapiddisk-cache" >> /etc/modules
     echo "dm_mod" >> /etc/modules

debian/prerm

@@ -14,7 +14,7 @@ fi
 
 case "$1" in
   remove|upgrade|deconfigure)
-    dkms remove -m rapiddisk -v 7.2.1 --all
+    dkms remove -m rapiddisk -v 8.0.0 --all
   ;;
 
   failed-upgrade)

doc/API.txt

@@ -142,6 +142,7 @@ ex) curl -X POST -s 127.0.0.1:9118/v1/createRapidDiskCache/rd0/sdb/write-through
 Supported caching policies:
 * write-through
 * write-around
+* writeback (dangerous)
 
 status code: 200 (on success)
 
@@ -197,3 +198,84 @@ Retrieve cache statistics from an existing rapiddisk-cache volume. Example outpu
     }
   ]
 }
+
+If it is writeback cache, it will look like this:
+
+{
+  "statistics": [
+    {
+      "cache_stats": [
+        {
+          "device": "rc-wb_sdd",
+          "errors": 0,
+          "num_blocks": 16320,
+          "num_free_blocks": 16320,
+          "num_wb_blocks": 0
+        }
+      ]
+    }
+  ]
+}
+
+-------------------------
+GET /v1/listAllNVMeTargets
+
+ex) curl -s --output - 10.0.0.185:9118/v1/listAllNVMeTargets|jq .
+
+status code: 200 (on success)
+
+Retrieve cache statistics from an existing rapiddisk-cache volume. Example output:
+
+{
+  "targets": [
+    {
+      "nvmet_targets": [
+        {
+          "nqn": "rd1-test",
+          "namespace": 2,
+          "device": "/dev/rd2",
+          "enabled": "true"
+        },
+        {
+          "nqn": "rd1-test",
+          "namespace": 1,
+          "device": "/dev/rd1",
+          "enabled": "true"
+        }
+      ]
+    },
+    {
+      "nvmet_ports": [
+        {
+          "port": 1,
+          "address": "10.0.0.185",
+          "protocol": "tcp",
+          "nqn": "rd1-test"
+        }
+      ]
+    }
+  ]
+}
+
+-------------------------
+GET /v1/listAllNVMePorts
+
+ex) curl -s --output - 10.0.0.185:9118/v1/listAllNVMePorts|jq .
+
+status code: 200 (on success)
+
+Retrieve cache statistics from an existing rapiddisk-cache volume. Example output:
+
+{
+  "targets": [
+    {
+      "nvmet_ports": [
+        {
+          "port": 1,
+          "address": "10.0.0.185",
+          "protocol": "tcp"
+        }
+      ]
+    }
+  ]
+}

doc/Makefile

@@ -9,13 +9,13 @@ all:
 
 .PHONY: install
 install: all
-	@echo Installing rapiddisk man page.
+	@echo Installing rapiddisk man pages.
 	$(MKDIR) $(DESTDIR)$(MAN_DIR)
 	$(CP) rapiddisk*.1 $(DESTDIR)$(MAN_DIR)/
 
 .PHONY: uninstall
 uninstall: 
-	@echo Uninstalling rapiddisk man page.
+	@echo Uninstalling rapiddisk man pages.
 	$(RM) $(DESTDIR)$(MAN_DIR)/rapiddisk*.1
 
 .PHONY: clean