Merge pull request #145 from pkoutoupis/defect/minor_cleanup_and_9.0_…

…release

Defect/minor cleanup and 9.0 release

.gitignore

@@ -1,6 +1,7 @@
 *.a
 *.o
 *.o.*
+*.d
 *.mod
 *.mod.*
 *.ko

CHANGELOG.md

@@ -1,3 +1,29 @@
+### Release 9.0.0 ###
+- module: Added code to capture page count metrics.
+- module: Updated ioctl support.
+- module: Fixed page alloc usage decrement on discard.
+- module: Added shrink volume support.
+- module: Added support for 5.19 Linux kernels (thank you Andrea Righi).
+- module: Added support for 6.0 Linux kernels.
+- utility: Added code to lock/unlock RAM device.
+- utility: Added flag to suppress header in stdout (thank you Matteo Tenca).
+- utility: Added support for RapidDisk "model" branding and support for revalidate size in NVMe Target logic.
+- utility: Traced, found and fixed all discovered memory leaks (a huge undertaking, thank you Matteo Tenca!!!).
+- utility: Rewrite daemon to execute commands from a shared library instead of popen to utility (another huge undertaking, thank you Matteo Tenca!!!).
+- utility: Added checks in URL parsing logic for REST API (thank you Matteo Tenca).
+- utility: Fix forking logic in daemon and remove need for realpath() usage (thank you Matteo Tenca).
+- utility: Define and standardize error messaging for consistency and improve verbose mode (thank you Matteo Tenca).
+- utility: Fix resize operation error messaging (github issue #142).
+- utility: Cleaned up and optimized NVMe Target management.
+- scripts: Fixed error checking in NVMe Target hostnqn script file.
+- scripts: Added fio execution script file examples.
+- scripts: Added valgrind test script to check for memory leaks (thank you Matteo Tenca).
+- build: Update Makefiles and add support for CPPFLAGS, CFLAGS, LDFLAGS, etc. (thank you Matteo Tenca).
+- build: Update Debian and RPM packaging with updated depends (thank you Matteo Tenca).
+- documentation: Added dm-writecache stats notes.
+- documentation: Added contrib file and cleaned up README and man pages.
+- documentation: Added doxygen documentation support (thank you Matteo Tenca).
+
 ### Release 8.2.0 ###
 - module: Fixed support for 5.14.
 - documentation: Updated README files.

Makefile

@@ -1,4 +1,4 @@
-# Copyright © 2016 - 2022 Petros Koutoupis
+# Copyright © 2016 - 2023 Petros Koutoupis
 # All rights reserved.
 #
 # This program is free software: you can redistribute it and/or modify

README.md

@@ -95,8 +95,8 @@ An example of a POST command:
 Change into the project's parent directory path.
 
 > To build the rapiddisk management utility, you will need to have the
-> `libjansson` and `libmicrohttpd` development library files installed
-> on your host system.
+> `libjansson`, `libpcre2`, `libdevmapper` and `libmicrohttpd`
+> development library files installed on your host system.
 >
 > You are required to having either the full kernel source or the kernel
 > headers installed for your current kernel revision.
@@ -206,3 +206,69 @@ To start the service at boot via systemd:
 ```console
 # systemctl enable rapiddiskd.service
 ```
+
+## Managing RapidDisk as an NVMe Target
+There are a few things that need to be known when using the NVMe Target
+features of the RapidDisk suite.
+
+### Loading the NVMe Target Kernel Modules
+In order to map any RapidDisk device and export it in the NVMe Target
+framework, the nvmet and the nvmet-tcp or nvmet-rdma kernel modules must be
+inserted.
+
+```console
+# modprobe nvmet nvmet-tcp
+```
+
+### Enabling NVMe Target Ports
+At least one Ethernet interface will need to be configured as a target
+port to export the RapidDisk volume from.
+
+```console
+# rapiddisk -i eth -P 1 -t tcp
+```
+
+### Exporting Targets
+When exporting a volume, a RapidDisk volume and a target port must be
+defined. If a host NQN is not defined, the administration utility will
+provide access to any host NQN. Note - a target can be exported across more
+than one target port.
+
+```console
+# rapiddisk -e -b rd3 -P 1
+```
+
+If a host NQN is defined, access is restricted to only those host NQNs.
+Note - the following command example can be repeated multiple times to add
+additional host NQNs for the specified target export.
+
+```console
+# rapiddisk -e -b rd3 -P 1 -H nqn.host1
+```
+
+### Unexporting Targets
+Unexporting RapidDisk volumes looks a bit different than exporting. If a
+host NQN is defined for a specified target, only that NQN will be removed from
+accessing the exported target.
+
+```console
+# rapiddisk -x -b rd3 -H nqn.host1
+```
+
+Removing all allowed host NQNs will revert access to any and all host NQNs
+requesting access to the target.
+
+If a target port is defined, the exported target will not be exported from the
+interface if one condition is met: the target has no defined allowed host NQNs.
+
+```console
+# rapiddisk -x -b rd3 -P 1 -H nqn.host1
+```
+OR
+
+```console
+# rapiddisk -x -b rd3 -P 1
+```
+
+And if there are no defined allowed host NQNs and the target is not being exported
+across any target ports, the entire target is removed from the subsystem.

conf/Makefile

@@ -1,4 +1,4 @@
-# Copyright © 2016 - 2022 Petros Koutoupis
+# Copyright © 2016 - 2023 Petros Koutoupis
 # All rights reserved.
 #
 # This program is free software: you can redistribute it and/or modify

conf/rapiddisk.sh.pacemaker

@@ -1,6 +1,6 @@
 #!/bin/bash
 #
-# Copyright (C) 2015 - 2022 Petros Koutoupis. All rights reserved.
+# Copyright (C) 2015 - 2023 Petros Koutoupis. All rights reserved.
 #
 
 PATH=/bin:/sbin:/usr/bin:/usr/sbin

conf/rapiddisk.sh.rgmanager

@@ -1,6 +1,6 @@
 #!/bin/bash
 #
-# Copyright (C) 2015 - 2022 Petros Koutoupis. All rights reserved.
+# Copyright (C) 2015 - 2023 Petros Koutoupis. All rights reserved.
 #
 
 PATH=/bin:/sbin:/usr/bin:/usr/sbin

doc/Makefile

@@ -1,4 +1,4 @@
-# Copyright © 2016 - 2022 Petros Koutoupis
+# Copyright © 2016 - 2023 Petros Koutoupis
 # All rights reserved.
 #
 # This program is free software: you can redistribute it and/or modify

doc/rapiddisk.1

@@ -32,6 +32,9 @@ Export a RapidDisk block device as an NVMe Target.
 -f
 Erase all data to a specified RapidDisk device (dangerous).
 .TP
+-g
+Do not print header.
+.TP
 -H
 The host to export / unexport the NVMe Target to / from.
 .TP
@@ -63,6 +66,9 @@ The port to export / unexport the NVMe Target to / from.
 Define cache policy: write-through (wt), write-around (wa) or writeback (wb) (dangerous) (default: write-through).
 Writeback caching is supplied by the dm-writecache kernel module and is not intended for production use as it may result in data loss on hardware/power failure.
 .TP
+-q
+List all system memory and block device resources.
+.TP
 -R
 Revalidate size of NVMe export using existing RapidDisk device.
 .TP
@@ -130,6 +136,46 @@ rapiddisk -e -b rd3 -P 1 -H nqn.host1
 rapiddisk -R -b rd0
 .TP
 rapiddisk -x -b rd3 -P 1 -H nqn.host1
+.SH MANAGING RAPIDDISK AS AN NVME TARGET 
+There are a few things that need to be known when using the NVMe Target features of the RapidDisk suite.
+.TP
+1.
+In order to map any RapidDisk device and export it in the NVMe Target framework, the nvmet and the nvmet-tcp or nvmet-rdma kernel modules must be inserted.
+.TP
+2.
+At least one Ethernet interface will need to be configured as a target port to export the RapidDisk volume from.
+.RS
+.P
+ex) rapiddisk -i eth -P 1 -t tcp
+.RE
+.TP
+3.
+When exporting a volume, a RapidDisk volume and a target port must be defined. If a host NQN is not defined, the administration utility will provide access to any host NQN. Note - a target can be exported across more than one target port.
+.RS
+.P
+ex) rapiddisk -e -b rd3 -P 1
+.P
+If a host NQN is defined, access is restricted to only those host NQNs. Note - the following command example can be repeated multiple times to add additional host NQNs for the specified target export.
+.P
+ex) rapiddisk -e -b rd3 -P 1 -H nqn.host1
+.RE
+.TP
+4.
+Unexporting RapidDisk volumes looks a bit different than exporting. If a host NQN is defined for a specified target, only that NQN will be removed from accessing the exported target.
+.RS
+.P
+ex) rapiddisk -x -b rd3 -H nqn.host1
+.P
+Removing all allowed host NQNs will revert access to any and all host NQNs requesting access to the target.
+.P
+If a target port is defined, the exported target will not be exported from the interface if one condition is met: the target has no defined allowed host NQNs.
+.P
+ex) rapiddisk -x -b rd3 -P 1 -H nqn.host1
+.P
+ex) rapiddisk -x -b rd3 -P 1
+.P
+And if there are no defined allowed host NQNs and the target is not being exported across any target ports, the entire target is removed from the subsystem.
+.RE
 .SH EXIT STATUS
 rapiddisk returns a zero exit status if no error occurs during operation. A non-zero value is returned on error.
 .SH AUTHORS

doc/rapiddiskd.1

@@ -7,14 +7,17 @@ rapiddiskd [ -h | -v ] [ options ]
 rapiddiskd is a daemon intended to listen for API requests to manage RapidDisk functions.
 .SS Options
 .TP
+-d
+Remain in the foreground (implies -V).
+.TP
 -h
 Invoke the help menu.
 .TP
 -p
 Change port to listen on (default: 9118).
 .TP
 -V
-Enable debug messages to stdout (this is ugly)
+Enable debug messages to stderr (this is ugly).
 .TP
 -v
 Display the version number.

module/Makefile

@@ -1,4 +1,4 @@
-# Copyright © 2011 - 2022 Petros Koutoupis
+# Copyright © 2011 - 2023 Petros Koutoupis
 # All rights reserved.
 #
 # This program is free software: you can redistribute it and/or modify
@@ -15,7 +15,7 @@
 #
 # SPDX-License-Identifier: GPL-2.0-only
 
-VERSION = 8.2.0
+VERSION = 9.0.0
 
 ifeq ($(KSRC),)
 	KSRC := /lib/modules/$(shell uname -r)/build

module/dkms.conf

@@ -1,5 +1,5 @@
 PACKAGE_NAME="rapiddisk"
-PACKAGE_VERSION="8.2.0"
+PACKAGE_VERSION="9.0.0"
 BUILT_MODULE_NAME[0]="rapiddisk"
 BUILT_MODULE_NAME[1]="rapiddisk-cache"
 DEST_MODULE_LOCATION[0]="/kernel/rapiddisk/"

module/rapiddisk-cache.c

@@ -1,5 +1,5 @@
 /*******************************************************************************
- ** Copyright © 2011 - 2022 Petros Koutoupis
+ ** Copyright © 2011 - 2023 Petros Koutoupis
  ** All rights reserved.
  **
  ** This program is free software: you can redistribute it and/or modify
@@ -54,7 +54,7 @@
 	} \
 } while (0)
 
-#define VERSION_STR	"8.2.0"
+#define VERSION_STR	"9.0.0"
 #define DM_MSG_PREFIX	"rapiddisk-cache"
 
 #define READCACHE	1
@@ -1248,7 +1248,7 @@ cache_status(struct dm_target *ti, status_type_t type, unsigned status_flags,
 
 static struct target_type cache_target = {
 	.name    = "rapiddisk-cache",
-	.version = {8, 2, 0},
+	.version = {9, 0, 0},
 	.module  = THIS_MODULE,
 	.ctr	 = cache_ctr,
 	.dtr	 = cache_dtr,
@@ -1290,4 +1290,4 @@ MODULE_LICENSE("GPL");
 MODULE_AUTHOR("Petros Koutoupis <[email protected]>");
 MODULE_DESCRIPTION("RapidDisk-Cache DM target is a write-through caching target with RapidDisk volumes.");
 MODULE_VERSION(VERSION_STR);
-MODULE_INFO(Copyright, "Copyright 2010 - 2022 Petros Koutoupis");
+MODULE_INFO(Copyright, "Copyright 2010 - 2023 Petros Koutoupis");

module/rapiddisk.c

@@ -1,5 +1,5 @@
 /*******************************************************************************
- ** Copyright © 2011 - 2022 Petros Koutoupis
+ ** Copyright © 2011 - 2023 Petros Koutoupis
  ** All rights reserved.
  **
  ** This program is free software: you can redistribute it and/or modify
@@ -41,7 +41,7 @@
 #include <linux/radix-tree.h>
 #include <linux/io.h>
 
-#define VERSION_STR		"8.2.0"
+#define VERSION_STR		"9.0.0"
 #define PREFIX			"rapiddisk"
 #define BYTES_PER_SECTOR	512
 #define MAX_RDSKS		1024
@@ -1021,4 +1021,4 @@ MODULE_LICENSE("GPL");
 MODULE_AUTHOR("Petros Koutoupis <[email protected]>");
 MODULE_DESCRIPTION("RapidDisk is an enhanced RAM disk block device driver.");
 MODULE_VERSION(VERSION_STR);
-MODULE_INFO(Copyright, "Copyright 2010 - 2022 Petros Koutoupis");
+MODULE_INFO(Copyright, "Copyright 2010 - 2023 Petros Koutoupis");

pkg/debian/changelog

@@ -1,3 +1,32 @@
+rapiddisk (9.0.0-1) UNRELEASED; urgency=medium
+
+  * module: Added code to capture page count metrics.
+  * module: Updated ioctl support.
+  * module: Fixed page alloc usage decrement on discard.
+  * module: Added shrink volume support.
+  * module: Added support for 5.19 Linux kernels (thank you Andrea Righi).
+  * module: Added support for 6.0 Linux kernels.
+  * utility: Added code to lock/unlock RAM device.
+  * utility: Added flag to suppress header in stdout (thank you Matteo Tenca).
+  * utility: Added support for RapidDisk "model" branding and support for revalidate size in NVMe Target logic.
+  * utility: Traced, found and fixed all discovered memory leaks (a huge undertaking, thank you Matteo Tenca!!!).
+  * utility: Rewrite daemon to execute commands from a shared library instead of popen to utility (another huge undertaking, thank you Matteo Tenca!!!).
+  * utility: Added checks in URL parsing logic for REST API (thank you Matteo Tenca).
+  * utility: Fix forking logic in daemon and remove need for realpath() usage (thank you Matteo Tenca).
+  * utility: Define and standardize error messaging for consistency and improve verbose mode (thank you Matteo Tenca).
+  * utility: Fix resize operation error messaging (github issue #142).
+  * utility: Cleaned up and optimized NVMe Target management.
+  * scripts: Fixed error checking in NVMe Target hostnqn script file.
+  * scripts: Added fio execution script file examples.
+  * scripts: Added valgrind test script to check for memory leaks (thank you Matteo Tenca).
+  * build: Update Makefiles and add support for CPPFLAGS, CFLAGS, LDFLAGS, etc. (thank you Matteo Tenca).
+  * build: Update Debian and RPM packaging with updated depends (thank you Matteo Tenca).
+  * documentation: Added dm-writecache stats notes.
+  * documentation: Added contrib file and cleaned up README and man pages.
+  * documentation: Added doxygen documentation support (thank you Matteo Tenca).
+
+ -- Petros Koutoupis <[email protected]>  Wed, 28 Dec 2022 08:56:52 +0000
+
 rapiddisk (8.2.0-1) UNRELEASED; urgency=medium
 
   * module: Fixed support for 5.14.

pkg/debian/copyright

@@ -4,15 +4,15 @@ Upstream-Contact: Petros Koutoupis <[email protected]>
 Source: https://github.com/pkoutoupis/rapiddisk/
 
 Files: *
-Copyright: 2015-2022 Petros Koutoupis <[email protected]>
+Copyright: 2015-2023 Petros Koutoupis <[email protected]>
 License: GPL-2.0+
 
 Files: pkg/debian/*
-Copyright: 2022 Matteo Tenca <[email protected]>
+Copyright: 2022-2023 Matteo Tenca <[email protected]>
 License: GPL-2.0+
 
 Files: scripts/rapiddisk-rootdev/*
-Copyright: 2022 Matteo Tenca <[email protected]>
+Copyright: 2022-2023 Matteo Tenca <[email protected]>
 License: GPL-2.0+
 
 License: GPL-2.0+