Longhorn v1.6.2-rc1

DON'T UPGRADE from/to any RC/Preview/Sprint releases because the operation is not supported.

Resolved Issues in this release

Improvement

• [BACKPORT][v1.6.2][UI][IMPROVEMENT] Allow users to request backup volume update 8539 - @a110605
• [BACKPORT][v1.6.2][IMPROVEMENT] Handle credential secret containing mixed invalid conditions 8541 - @mantissahz
• [BACKPORT][v1.6.2][IMPROVEMENT] Allow users to request backup volume update 8154 - @mantissahz
• [BACKPORT][v1.6.2][IMPROVEMENT] Investigate performance bottleneck in v1 data path 8511 - @PhanLe1010 @roger-ryao
• [BACKPORT][v1.6.2]Mirror the quay.io/openshift/origin-oauth-proxy image to Longhorn repo similar to what we are doing for CSI sidecar images 8334 - @PhanLe1010 @roger-ryao
• [BACKPORT][v1.6.2][IMPROVEMENT] Fall back to a running instance-manager if a default is not available 8491 - @derekbit
• [BACKPORT][v1.6.2][IMPROVEMENT] Avoid misleading log messages in longhorn manager while syncing danger zone settings 8383 - @yangchiu @mantissahz
• [BACKPORT][v1.6.2][IMPROVEMENT] Prevent unnecessary updates of instanceManager status 8421 - @yangchiu @derekbit
• [BACKPORT][v1.6.2][IMPROVEMENT] Only sync log settings to running instance manager pod 8489 - @derekbit
• [BACKPORT][v1.6.2][IMPROVEMENT] Do not terminate nfs-ganesha in share-manager pod after failing to access recovery backend 8346 - @derekbit @chriscchien
• [BACKPORT][v1.6.2][IMPROVEMENT] Expose virtual size of qcow2 backing images 8322 - @chriscchien
• [BACKPORT][v1.6.2][IMPROVEMENT] Improve environment_check script for NFS protocol bug and the host system self diagnosis 8277 - @james-munson @chriscchien
• [BACKPORT][v1.6.2][IMPROVEMENT] Add dmsetup and dmcrypt utilities check in environment check script 8218 - @derekbit

Bug

• [BACKPORT][v1.6.2][BUG] Backup marked as "completed" cannot be restored, gzip: invalid header 8377 - @derekbit @roger-ryao
• [BACKPORT][v1.6.2][BUG] Valid backup secret produces error message: "there is space or new line in AWS_CERT" 8477 - @mantissahz
• [BACKPORT][v1.6.2][BUG] Secret for backup not found 8512 - @mantissahz
• [BACKPORT][v1.6.2][BUG] Lost connection to unix:///csi/csi.sock 8493 - @ejweber @roger-ryao
• [BACKPORT][v1.6.2][BUG] Longhorn Helm uninstall times out. 8409 - @ChanYiLin @roger-ryao
• [BACKPORT][v1.6.2][BUG] Disable tls 1.0 and 1.1 on webhook service 8388 - @ChanYiLin @roger-ryao
• [BACKPORT][v1.6.2][BUG] share-manager-pvc appears to be leaking memory 8426 - @derekbit @roger-ryao
• [BACKPORT][v1.6.2][BUG] longhorn-manager build failed 8410 - @yangchiu @mantissahz
• [BACKPORT][v1.6.2][BUG] RWX volume is hang on Photon OS 8279 - @yangchiu @PhanLe1010
• [BACKPORT][v1.6.2][BUG] BackupTarget conditions don't reflect connection errors in v1.6.0 8223 - @ejweber @chriscchien

Misc

• [BACKPORT][v1.6.2]DOCS - Incorrect documentation on pre-upgrade checker configuration 8342 - @yangchiu

Contributors

• @ChanYiLin
• @PhanLe1010
• @a110605
• @chriscchien
• @derekbit
• @ejweber
• @innobead
• @james-munson
• @mantissahz
• @roger-ryao
• @yangchiu

Longhorn v1.5.5

Longhorn v1.5.5 Release Notes

This latest version of Longhorn 1.5 introduces several improvements and bug fixes that are intended to improve system quality, resilience, and stability.

The Longhorn team appreciates your contributions and anticipates receiving feedback regarding this release.

Note

For more information about release-related terminology, see Releases.

Installation

Important

Ensure that your cluster is running Kubernetes v1.21 or later before installing Longhorn v1.5.5.

You can install Longhorn using a variety of tools, including Rancher, Kubectl, and Helm. For more information about installation methods and requirements, see Quick Installation in the Longhorn documentation.

Upgrade

Important

Ensure that your cluster is running Kubernetes v1.21 or later before upgrading from Longhorn v1.4.x or v1.5.x (< v1.5.5) to v1.5.5.

Longhorn only allows upgrades from supported versions. For more information about upgrade paths and procedures, see Upgrade in the Longhorn documentation.

Deprecation & Incompatibilities

For information about important changes, including feature incompatibility, deprecation, and removal, see Important Notes in the Longhorn documentation.

Post-Release Known Issues

For information about issues identified after this release, see Release-Known-Issues.

Resolved Issues

Improvements

• [BACKPORT][v1.5.5][IMPROVEMENT] Cannot read/write to block volume when the container is run as non-root 8123 - @PhanLe1010 @chriscchien
• [BACKPORT][v1.5.5][IMPROVEMENT] Do not terminate nfs-ganesha in share-manager pod after failing to access recovery backend 8347 - @derekbit @chriscchien
• [BACKPORT][v1.5.5][IMPROVEMENT] Expose virtual size of qcow2 backing images 8321 - @shuo-wu @chriscchien
• [BACKPORT][v1.5.5][IMPROVEMENT] Improve logging in CSI plugin when mount fails. 8286 - @james-munson @chriscchien
• [BACKPORT][v1.5.5][IMPROVEMENT] Upgrade support bundle kit version to v0.0.36 8161 - @c3y1huang @roger-ryao
• [BACKPORT][v1.5.5][IMPROVEMENT] Improve environment_check script for NFS protocol bug and the host system self diagnosis 7972 - @james-munson @roger-ryao

Bug Fixes

• Security issues in latest longhorn docker images 8372 - @c3y1huang @chriscchien
• [BACKPORT][v1.5.5][BUG] Backup marked as "completed" cannot be restored, gzip: invalid header 8378 - @derekbit @chriscchien
• [BACKPORT][v1.5.5][BUG][v1.6.0-rc1] Failed to run instance-manager in storage network environment 8305 - @yangchiu @ejweber
• [BACKPORT][v1.5.5][BUG] Replica rebuild failed 8257 - @shuo-wu @chriscchien
• [BACKPORT][v1.5.5][BUG] longhorn manager pod fails to start in container-based K3s 7948 - @ChanYiLin @chriscchien
• [BACKPORT][v1.5.5][BUG] persistence.removeSnapshotsDuringFilesystemTrim Helm variable is unreferenced 7951 - @ejweber @roger-ryao
• [BACKPORT][v1.5.5][BUG] Failed to restore a backup to file by the scripts/restore-backup-to-file.sh with a CIFS backup target. 8127 - @mantissahz @roger-ryao
• [BACKPORT][v1.5.5][BUG] Longhorn api-server PUT request rate 8153 - @ejweber @roger-ryao
• [BACKPORT][v1.5.5][BUG] A replica may be incorrectly scheduled to a node with an existing failed replica 8116 - @ejweber @chriscchien
• [BACKPORT][v1.5.5][BUG] potential risk to unmap a negative number 8236 - @Vicente-Cheng @roger-ryao
• [BACKPORT][v1.5.5][BUG] Use config map to update default-replica-count won't apply to default-replica-count.definition.default if the value equal to current default-replica-count.value 8135 - @james-munson @chriscchien
• [BACKPORT][v1.5.5][BUG] LH manager reboots due to the webhook is not ready 8036 - @ChanYiLin @chriscchien
• [BACKPORT][v1.5.5][BUG] Can't use longhorn with Generic ephemeral volumes 8201 - @ejweber @roger-ryao
• [BACKPORT][v1.5.5][BUG] Volume cannot attach because of the leftover non-empty volume.status.PendingNodeID after upgrading Longhorn 7996 - @james-munson @roger-ryao
• [BACKPORT][v1.5.5][BUG] no Pending workload pods for volume xxx to be mounted 8082 - @c3y1huang @roger-ryao

Miscellaneous

• [TASK] update go-iscsi-helper and go-common-lib in v1.5.x 7960 - @ChanYiLin
• [BACKPORT][v1.5.5][REFACTOR] move mount point check function to common lib 8125 - @ChanYiLin @roger-ryao

Contributors

• @ChanYiLin
• @PhanLe1010
• @Vicente-Cheng
• @c3y1huang
• @chriscchien
• @derekbit
• @ejweber
• @innobead
• @james-munson
• @mantissahz
• @roger-ryao
• @shuo-wu
• @yangchiu

Longhorn v1.5.5-rc2

DON'T UPGRADE from/to any RC/Preview/Sprint releases because the operation is not supported.

Resolved Issues in this release

Improvement

• [BACKPORT][v1.5.5][IMPROVEMENT] Cannot read/write to block volume when the container is run as non-root 8123 - @PhanLe1010 @chriscchien
• [BACKPORT][v1.5.5][IMPROVEMENT] Do not terminate nfs-ganesha in share-manager pod after failing to access recovery backend 8347 - @derekbit @chriscchien
• [BACKPORT][v1.5.5][IMPROVEMENT] Expose virtual size of qcow2 backing images 8321 - @shuo-wu @chriscchien
• [BACKPORT][v1.5.5][IMPROVEMENT] Improve logging in CSI plugin when mount fails. 8286 - @james-munson @chriscchien
• [BACKPORT][v1.5.5][IMPROVEMENT] Upgrade support bundle kit version to v0.0.36 8161 - @c3y1huang @roger-ryao
• [BACKPORT][v1.5.5][IMPROVEMENT] Improve environment_check script for NFS protocol bug and the host system self diagnosis 7972 - @james-munson @roger-ryao

Bug

• [BACKPORT][v1.5.5][BUG][v1.6.0-rc1] Failed to run instance-manager in storage network environment 8305 - @yangchiu @ejweber
• [BACKPORT][v1.5.5][BUG] Replica rebuild failed 8257 - @shuo-wu @chriscchien
• [BACKPORT][v1.5.5][BUG] longhorn manager pod fails to start in container-based K3s 7948 - @ChanYiLin @chriscchien
• [BACKPORT][v1.5.5][BUG] persistence.removeSnapshotsDuringFilesystemTrim Helm variable is unreferenced 7951 - @ejweber @roger-ryao
• [BACKPORT][v1.5.5][BUG] Failed to restore a backup to file by the scripts/restore-backup-to-file.sh with a CIFS backup target. 8127 - @mantissahz @roger-ryao
• [BACKPORT][v1.5.5][BUG] Longhorn api-server PUT request rate 8153 - @ejweber @roger-ryao
• [BACKPORT][v1.5.5][BUG] A replica may be incorrectly scheduled to a node with an existing failed replica 8116 - @ejweber @chriscchien
• [BACKPORT][v1.5.5][BUG] potential risk to unmap a negative number 8236 - @Vicente-Cheng @roger-ryao
• [BACKPORT][v1.5.5][BUG] Use config map to update default-replica-count won't apply to default-replica-count.definition.default if the value equal to current default-replica-count.value 8135 - @james-munson @chriscchien
• [BACKPORT][v1.5.5][BUG] LH manager reboots due to the webhook is not ready 8036 - @ChanYiLin @chriscchien
• [BACKPORT][v1.5.5][BUG] Can't use longhorn with Generic ephemeral volumes 8201 - @ejweber @roger-ryao
• [BACKPORT][v1.5.5][BUG] Volume cannot attach because of the leftover non-empty volume.status.PendingNodeID after upgrading Longhorn 7996 - @james-munson @roger-ryao
• [BACKPORT][v1.5.5][BUG] no Pending workload pods for volume xxx to be mounted 8082 - @c3y1huang @roger-ryao

Misc

• [TASK] update go-iscsi-helper and go-common-lib in v1.5.x 7960 - @ChanYiLin
• [BACKPORT][v1.5.5][REFACTOR] move mount point check function to common lib 8125 - @ChanYiLin @roger-ryao

Contributors

• @ChanYiLin
• @PhanLe1010
• @Vicente-Cheng
• @c3y1huang
• @chriscchien
• @derekbit
• @ejweber
• @innobead
• @james-munson
• @mantissahz
• @roger-ryao
• @shuo-wu
• @yangchiu

Longhorn v1.5.5-rc1

DON'T UPGRADE from/to any RC/Preview/Sprint releases because the operation is not supported.

Resolved Issues in this release

Improvement

• [BACKPORT][v1.5.5][IMPROVEMENT] Expose virtual size of qcow2 backing images 8321 - @chriscchien
• [BACKPORT][v1.5.5][IMPROVEMENT] Cannot read/write to block volume when the container is run as non-root 8123 - @PhanLe1010 @chriscchien
• [BACKPORT][v1.5.5][IMPROVEMENT] Improve logging in CSI plugin when mount fails. 8286 - @james-munson @chriscchien
• [BACKPORT][v1.5.5][IMPROVEMENT] Allow users to request backup volume update 7983 -
• [BACKPORT][v1.5.5][IMPROVEMENT] Upgrade support bundle kit version to v0.0.36 8161 - @c3y1huang @roger-ryao
• [BACKPORT][v1.5.5][IMPROVEMENT] Improve environment_check script for NFS protocol bug and the host system self diagnosis 7972 - @james-munson @roger-ryao

Bug

• [BACKPORT][v1.5.5][BUG] longhorn manager pod fails to start in container-based K3s 7948 - @ChanYiLin @chriscchien
• [BACKPORT][v1.5.5][BUG][v1.6.0-rc1] Failed to run instance-manager in storage network environment 8305 - @yangchiu @ejweber
• [BACKPORT][v1.5.5][BUG] RWX volume is hang on Photon OS 8280 - @PhanLe1010
• [BACKPORT][v1.5.5][BUG] persistence.removeSnapshotsDuringFilesystemTrim Helm variable is unreferenced 7951 - @ejweber @roger-ryao
• [BACKPORT][v1.5.5][BUG] Failed to restore a backup to file by the scripts/restore-backup-to-file.sh with a CIFS backup target. 8127 - @mantissahz @roger-ryao
• [BACKPORT][v1.5.5][BUG] Longhorn api-server PUT request rate 8153 - @ejweber @roger-ryao
• [BACKPORT][v1.5.5][BUG] A replica may be incorrectly scheduled to a node with an existing failed replica 8116 - @ejweber @chriscchien
• [BACKPORT][v1.5.5][BUG] potential risk to unmap a negative number 8236 - @Vicente-Cheng @roger-ryao
• [BACKPORT][v1.5.5][BUG] Replica rebuild failed 8257 - @shuo-wu @chriscchien
• [BACKPORT][v1.5.5][BUG] Use config map to update default-replica-count won't apply to default-replica-count.definition.default if the value equal to current default-replica-count.value 8135 - @james-munson @chriscchien
• [BACKPORT][v1.5.5][BUG] LH manager reboots due to the webhook is not ready 8036 - @ChanYiLin @chriscchien
• [BACKPORT][v1.5.5][BUG] Can't use longhorn with Generic ephemeral volumes 8201 - @ejweber @roger-ryao
• [BACKPORT][v1.5.5][BUG] Volume cannot attach because of the leftover non-empty volume.status.PendingNodeID after upgrading Longhorn 7996 - @james-munson @roger-ryao
• [BACKPORT][v1.5.5][BUG] no Pending workload pods for volume xxx to be mounted 8082 - @c3y1huang @roger-ryao
• [BACKPORT][v1.5.5][BUG] Panic during collecting metrics 8102 - @derekbit
• [BACKPORT][v1.5.5][BUG] Missed NodeStageVolume after reboot leads to CreateContainerError 8104 - @ejweber

Misc

• [TASK] update go-iscsi-helper and go-common-lib in v1.5.x 7960 - @ChanYiLin
• [BACKPORT][v1.5.5][REFACTOR] move mount point check function to common lib 8125 - @ChanYiLin @roger-ryao

Contributors

• @ChanYiLin
• @PhanLe1010
• @Vicente-Cheng
• @c3y1huang
• @chriscchien
• @derekbit
• @ejweber
• @innobead
• @james-munson
• @mantissahz
• @roger-ryao
• @shuo-wu
• @yangchiu

Longhorn v1.6.1

Longhorn v1.6.1 Release Notes

Longhorn 1.6.1 introduces several improvements and bug fixes that are intended to improve system quality, resilience, and stability.

The Longhorn team appreciates your contributions and expects to receive feedback regarding this release.

Note

For more information about release-related terminology, see Releases.

Installation

Important

Ensure that your cluster is running Kubernetes v1.21 or later before installing Longhorn v1.6.1.

You can install Longhorn using a variety of tools, including Rancher, Kubectl, and Helm. For more information about installation methods and requirements, see Quick Installation in the Longhorn documentation.

Upgrade

Important

Ensure that your cluster is running Kubernetes v1.21 or later before upgrading from Longhorn v1.5.x or v1.6.x to v1.6.1.

Longhorn only allows upgrades from supported versions. For more information about upgrade paths and procedures, see Upgrade in the Longhorn documentation.

Deprecation & Incompatibilities

For information about important changes, including feature incompatibility, deprecation, and removal, see Important Notes in the Longhorn documentation.

Post-Release Known Issues

For information about issues identified after this release, see Release-Known-Issues.

Resolved Issues

Improvement

• [BACKPORT][v1.6.1][IMPROVEMENT] Add dmsetup and dmcrypt utilities check in environment check script 8234 - @derekbit @chriscchien
• [BACKPORT][v1.6.1][IMPROVEMENT] Upgrade support bundle kit version to v0.0.36 8162 - @c3y1huang @roger-ryao
• [BACKPORT][v1.6.1][IMPROVEMENT] Cannot read/write to block volume when the container is run as non-root 8122 - @PhanLe1010 @chriscchien
• [BACKPORT][v1.6.1][IMPROVEMENT] Improve environment_check script for NFS protocol bug and the host system self diagnosis 7971 - @james-munson @roger-ryao
• [BACKPORT][v1.6.1][IMPROVEMENT] Use HEAD instead of a GET to fetch the Content-Length of an resource via URL 7973 - @votdev @roger-ryao
• [BACKPORT][v1.6.1][IMPROVEMENT] Remove startup probe of CSI driver after liveness probe conn fix ready 7886 - @ejweber @roger-ryao
• [BACKPORT][v1.6.1][IMPROVEMENT] Change support-bundle-manager image pull policy to PullIfNotPresent 8000 - @ChanYiLin @roger-ryao

Bug

• [BUG][v1.6.1-rc3] test_backup_lock_creation_during_deletion failed with *.lck type 1 acquisition 8269 - @ChanYiLin @chriscchien
• [BACKPORT][v1.6.1][BUG] Replica rebuild failed 8258 - @shuo-wu @roger-ryao
• [BACKPORT][v1.6.1][BUG] potential risk to unmap a negative number 8237 - @Vicente-Cheng @roger-ryao
• [BACKPORT][v1.6.1][BUG] Can't use longhorn with Generic ephemeral volumes 8213 - @ejweber @chriscchien
• [BUG] [v1.6.1-rc1] v2 volume replica offline rebuilding fail 8187 - @shuo-wu @chriscchien
• [BACKPORT][v1.6.1][BUG] Longhorn api-server PUT request rate 8152 - @ejweber @roger-ryao
• [BACKPORT][v1.6.1][BUG] Use config map to update default-replica-count won't apply to default-replica-count.definition.default if the value equal to current default-replica-count.value 8134 - @james-munson @roger-ryao
• [BACKPORT][v1.6.1][BUG] Failed to restore a backup to file by the scripts/restore-backup-to-file.sh with a CIFS backup target. 8128 - @mantissahz @roger-ryao
• [BACKPORT][v1.6.1][BUG] Exporting data from the existing replicas has issues in 1.6.0 8096 - @ChanYiLin @chriscchien
• [BACKPORT][v1.6.1][BUG] longhorn manager pod fails to start in container-based K3s 7847 - @ChanYiLin @khushboo-rancher @chriscchien
• [BACKPORT][v1.6.1][BUG] no Pending workload pods for volume xxx to be mounted 8081 - @c3y1huang @chriscchien
• [BACKPORT][v1.6.1][BUG] Deadlock is possible in v1.6.0 instance manager 7920 - @roger-ryao
• [BACKPORT][v1.6.1][BUG][1.6.0] ENGINE v2 : disk /dev/xxxx is already used by AIO bdev disk-x 8133 - @derekbit @chriscchien
• [BACKPORT][v1.6.1][BUG] A replica may be incorrectly scheduled to a node with an existing failed replica 8044 - @ejweber @chriscchien
• [BACKPORT][v1.6.1][BUG] Deadlock between volume migration and upgrade after Longhorn upgrade 7870 - @ejweber @roger-ryao
• [BACKPORT][v1.6.1][BUG] Volume cannot attach because of the leftover non-empty volume.status.PendingNodeID after upgrading Longhorn 7995 - @james-munson @chriscchien
• [BACKPORT][v1.6.1][BUG] Longhorn may keep corrupted salvaged replicas and discard good ones 7885 - @ejweber @roger-ryao
• [BACKPORT][v1.6.1][BUG] LH manager reboots due to the webhook is not ready 8037 - @ChanYiLin @chriscchien
• [BACKPORT][v1.6.1][BUG] BackingImage does not download URL correctly in some situation 7987 - @votdev @yangchiu
• [BACKPORT][v1.6.1][BUG] Executing fstrim while rebuilding causes IO errors 7868 - @yangchiu @ejweber
• [BACKPORT][v1.6.1][BUG] The feature of auto remount read only volume not work on a single node cluster. 7846 - @yangchiu @ChanYiLin
• [BACKPORT][v1.6.1][BUG] Missed NodeStageVolume after reboot leads to CreateContainerError 8012 - @ejweber @chriscchien
• [BACKPORT][v1.6.1][BUG] persistence.removeSnapshotsDuringFilesystemTrim Helm variable is unreferenced 7952 - @ejweber @chriscchien
• [BACKPORT][v1.6.1][BUG][v1.5.4-rc4] Test case test_backuptarget_available_during_engine_image_not_ready failed to wait for backup target available 8055 - @c3y1huang @chriscchien
• [BACKPORT][v1.6.1][BUG] Add Snapshot Maximum Count to the Settings 7979 - @FrankYang0529 @roger-ryao
• [BACKPORT][v1.6.1][BUG] Volumes stuck upgrading after 1.5.3 -> 1.6.0 upgrade. 7899 - @ejweber @roger-ryao
• [BACKPORT][v1.6.1][BUG] The activated DR volume do not contain the latest data. 7946 - @shuo-wu @roger-ryao
• [BACKPORT][v1.6.1][BUG][v1.5.x] Recurring job fails to create backup when volume detached 8013 - @yangchiu @mantissahz @PhanLe1010 @c3y1huang
• [BACKPORT][v1.6.1][BUG] Create backup failed: failed lock lock-*.lck type 1 acquisition 7875 - @yangchiu @ChanYiLin @chriscchien
• [BUG] Fix errors in questions.yaml 6392 - @james-munson @chriscchien

Misc

• [BACKPORT][v1.6.1][REFACTOR] move mount point check function to common lib 8124 - @ChanYiLin @chriscchien
• [TASK] Fix updating patch digest dependencies (v1.6.x) 8107 - @mantissahz

Contributors

• @ChanYiLin
• @FrankYang0529
• @PhanLe1010
• @Vicente-Cheng
• @c3y1huang
• @chriscchien
• @derekbit
• @ejweber
• @innobead
• @james-munson
• @khushboo-rancher
• @mantissahz
• @roger-ryao
• @shuo-wu
• @votdev
• @yangchiu

Longhorn v1.6.1-rc3

DON'T UPGRADE from/to any RC/Preview/Sprint releases because the operation is not supported.

Resolved Issues in this release

Improvement

• [BACKPORT][v1.6.1][IMPROVEMENT] Add dmsetup and dmcrypt utilities check in environment check script 8234 - @derekbit @chriscchien
• [BACKPORT][v1.6.1][IMPROVEMENT] Upgrade support bundle kit version to v0.0.36 8162 - @c3y1huang @roger-ryao
• [BACKPORT][v1.6.1][IMPROVEMENT] Cannot read/write to block volume when the container is run as non-root 8122 - @PhanLe1010 @chriscchien
• [BACKPORT][v1.6.1][IMPROVEMENT] Improve environment_check script for NFS protocol bug and the host system self diagnosis 7971 - @james-munson @roger-ryao
• [BACKPORT][v1.6.1][IMPROVEMENT] Use HEAD instead of a GET to fetch the Content-Length of an resource via URL 7973 - @votdev @roger-ryao
• [BACKPORT][v1.6.1][IMPROVEMENT] Remove startup probe of CSI driver after liveness probe conn fix ready 7886 - @ejweber @roger-ryao
• [BACKPORT][v1.6.1][IMPROVEMENT] Change support-bundle-manager image pull policy to PullIfNotPresent 8000 - @ChanYiLin @roger-ryao

Bug

• [BACKPORT][v1.6.1][BUG] potential risk to unmap a negative number 8237 - @Vicente-Cheng @roger-ryao
• [BACKPORT][v1.6.1][BUG] Replica rebuild failed 8258 - @shuo-wu @roger-ryao
• [BACKPORT][v1.6.1][BUG] Can't use longhorn with Generic ephemeral volumes 8213 - @ejweber @chriscchien
• [BUG] [v1.6.1-rc1] v2 volume replica offline rebuilding fail 8187 - @shuo-wu @chriscchien
• [BACKPORT][v1.6.1][BUG] Longhorn api-server PUT request rate 8152 - @ejweber @roger-ryao
• [BACKPORT][v1.6.1][BUG] Use config map to update default-replica-count won't apply to default-replica-count.definition.default if the value equal to current default-replica-count.value 8134 - @james-munson @roger-ryao
• [BACKPORT][v1.6.1][BUG] Failed to restore a backup to file by the scripts/restore-backup-to-file.sh with a CIFS backup target. 8128 - @mantissahz @roger-ryao
• [BACKPORT][v1.6.1][BUG] Exporting data from the existing replicas has issues in 1.6.0 8096 - @ChanYiLin @chriscchien
• [BACKPORT][v1.6.1][BUG] longhorn manager pod fails to start in container-based K3s 7847 - @ChanYiLin @khushboo-rancher @chriscchien
• [BACKPORT][v1.6.1][BUG] no Pending workload pods for volume xxx to be mounted 8081 - @c3y1huang @chriscchien
• [BACKPORT][v1.6.1][BUG] Deadlock is possible in v1.6.0 instance manager 7920 - @roger-ryao
• [BACKPORT][v1.6.1][BUG][1.6.0] ENGINE v2 : disk /dev/xxxx is already used by AIO bdev disk-x 8133 - @derekbit @chriscchien
• [BACKPORT][v1.6.1][BUG] A replica may be incorrectly scheduled to a node with an existing failed replica 8044 - @ejweber @chriscchien
• [BACKPORT][v1.6.1][BUG] Deadlock between volume migration and upgrade after Longhorn upgrade 7870 - @ejweber @roger-ryao
• [BACKPORT][v1.6.1][BUG] Volume cannot attach because of the leftover non-empty volume.status.PendingNodeID after upgrading Longhorn 7995 - @james-munson @chriscchien
• [BACKPORT][v1.6.1][BUG] Longhorn may keep corrupted salvaged replicas and discard good ones 7885 - @ejweber @roger-ryao
• [BACKPORT][v1.6.1][BUG] LH manager reboots due to the webhook is not ready 8037 - @ChanYiLin @chriscchien
• [BACKPORT][v1.6.1][BUG] BackingImage does not download URL correctly in some situation 7987 - @votdev @yangchiu
• [BACKPORT][v1.6.1][BUG] Executing fstrim while rebuilding causes IO errors 7868 - @yangchiu @ejweber
• [BACKPORT][v1.6.1][BUG] The feature of auto remount read only volume not work on a single node cluster. 7846 - @yangchiu @ChanYiLin
• [BACKPORT][v1.6.1][BUG] Missed NodeStageVolume after reboot leads to CreateContainerError 8012 - @ejweber @chriscchien
• [BACKPORT][v1.6.1][BUG] persistence.removeSnapshotsDuringFilesystemTrim Helm variable is unreferenced 7952 - @ejweber @chriscchien
• [BACKPORT][v1.6.1][BUG][v1.5.4-rc4] Test case test_backuptarget_available_during_engine_image_not_ready failed to wait for backup target available 8055 - @c3y1huang @chriscchien
• [BACKPORT][v1.6.1][BUG] Add Snapshot Maximum Count to the Settings 7979 - @FrankYang0529 @roger-ryao
• [BACKPORT][v1.6.1][BUG] Volumes stuck upgrading after 1.5.3 -> 1.6.0 upgrade. 7899 - @ejweber @roger-ryao
• [BACKPORT][v1.6.1][BUG] The activated DR volume do not contain the latest data. 7946 - @shuo-wu @roger-ryao
• [BACKPORT][v1.6.1][BUG][v1.5.x] Recurring job fails to create backup when volume detached 8013 - @yangchiu @mantissahz @PhanLe1010 @c3y1huang
• [BACKPORT][v1.6.1][BUG] Create backup failed: failed lock lock-*.lck type 1 acquisition 7875 - @yangchiu @ChanYiLin @chriscchien
• [BUG] Fix errors in questions.yaml 6392 - @james-munson @chriscchien

Misc

• [BACKPORT][v1.6.1][REFACTOR] move mount point check function to common lib 8124 - @ChanYiLin @chriscchien
• [TASK] Fix updating patch digest dependencies (v1.6.x) 8107 - @mantissahz

Contributors

• @ChanYiLin
• @FrankYang0529
• @PhanLe1010
• @Vicente-Cheng
• @c3y1huang
• @chriscchien
• @derekbit
• @ejweber
• @innobead
• @james-munson
• @khushboo-rancher
• @mantissahz
• @roger-ryao
• @shuo-wu
• @votdev
• @yangchiu

Longhorn v1.6.1-rc2

Warning

DON'T UPGRADE from/to any RC/Preview/Sprint releases because the operation is not supported.

Resolved Issues in this release

Improvement

• [BACKPORT][v1.6.1][IMPROVEMENT] Upgrade support bundle kit version to v0.0.36 8162 - @c3y1huang @roger-ryao
• [BACKPORT][v1.6.1][IMPROVEMENT] Allow users to request backup volume update 7984 - @mantissahz
• [BACKPORT][v1.6.1][IMPROVEMENT] Cannot read/write to block volume when the container is run as non-root 8122 - @PhanLe1010 @chriscchien
• [BACKPORT][v1.6.1][IMPROVEMENT] Improve environment_check script for NFS protocol bug and the host system self diagnosis 7971 - @james-munson @roger-ryao
• [BACKPORT][v1.6.1][IMPROVEMENT] Use HEAD instead of a GET to fetch the Content-Length of an resource via URL 7973 - @votdev @roger-ryao
• [BACKPORT][v1.6.1][IMPROVEMENT] Remove startup probe of CSI driver after liveness probe conn fix ready 7886 - @ejweber @roger-ryao
• [BACKPORT][v1.6.1][IMPROVEMENT] Change support-bundle-manager image pull policy to PullIfNotPresent 8000 - @ChanYiLin @roger-ryao

Bug

• [BUG] [v1.6.1-rc1] v2 volume replica offline rebuilding fail 8187 - @shuo-wu @chriscchien
• [BACKPORT][v1.6.1][BUG] Longhorn api-server PUT request rate 8152 - @ejweber @roger-ryao
• [BACKPORT][v1.6.1][BUG] Use config map to update default-replica-count won't apply to default-replica-count.definition.default if the value equal to current default-replica-count.value 8134 - @james-munson @roger-ryao
• [BACKPORT][v1.6.1][BUG] Failed to restore a backup to file by the scripts/restore-backup-to-file.sh with a CIFS backup target. 8128 - @mantissahz @roger-ryao
• [BACKPORT][v1.6.1][BUG] Exporting data from the existing replicas has issues in 1.6.0 8096 - @ChanYiLin @chriscchien
• [BACKPORT][v1.6.1][BUG] longhorn manager pod fails to start in container-based K3s 7847 - @ChanYiLin @khushboo-rancher @chriscchien
• [BACKPORT][v1.6.1][BUG] no Pending workload pods for volume xxx to be mounted 8081 - @c3y1huang @chriscchien
• [BACKPORT][v1.6.1][BUG] Deadlock is possible in v1.6.0 instance manager 7920 - @roger-ryao
• [BACKPORT][v1.6.1][BUG] metric longhorn_volume_robustness did not reflect detached volume 8140 - @c3y1huang
• [BACKPORT][v1.6.1][BUG][1.6.0] ENGINE v2 : disk /dev/xxxx is already used by AIO bdev disk-x 8133 - @derekbit @chriscchien
• [BACKPORT][v1.6.1][BUG] A replica may be incorrectly scheduled to a node with an existing failed replica 8044 - @ejweber @chriscchien
• [BACKPORT][v1.6.1][BUG] Deadlock between volume migration and upgrade after Longhorn upgrade 7870 - @ejweber @roger-ryao
• [BACKPORT][v1.6.1][BUG] Volume cannot attach because of the leftover non-empty volume.status.PendingNodeID after upgrading Longhorn 7995 - @james-munson @chriscchien
• [BACKPORT][v1.6.1][BUG] Longhorn may keep corrupted salvaged replicas and discard good ones 7885 - @ejweber @roger-ryao
• [BACKPORT][v1.6.1][BUG] LH manager reboots due to the webhook is not ready 8037 - @ChanYiLin @chriscchien
• [BACKPORT][v1.6.1][BUG] BackingImage does not download URL correctly in some situation 7987 - @votdev @yangchiu
• [BACKPORT][v1.6.1][BUG] Executing fstrim while rebuilding causes IO errors 7868 - @yangchiu @ejweber
• [BACKPORT][v1.6.1][BUG] The feature of auto remount read only volume not work on a single node cluster. 7846 - @yangchiu @ChanYiLin
• [BACKPORT][v1.6.1][BUG] Missed NodeStageVolume after reboot leads to CreateContainerError 8012 - @ejweber @chriscchien
• [BACKPORT][v1.6.1][BUG] persistence.removeSnapshotsDuringFilesystemTrim Helm variable is unreferenced 7952 - @ejweber @chriscchien
• [BACKPORT][v1.6.1][BUG][v1.5.4-rc4] Test case test_backuptarget_available_during_engine_image_not_ready failed to wait for backup target available 8055 - @c3y1huang @chriscchien
• [BACKPORT][v1.6.1][BUG] Add Snapshot Maximum Count to the Settings 7979 - @FrankYang0529 @roger-ryao
• [BACKPORT][v1.6.1][BUG] Volumes stuck upgrading after 1.5.3 -> 1.6.0 upgrade. 7899 - @ejweber @roger-ryao
• [BACKPORT][v1.6.1][BUG] The activated DR volume do not contain the latest data. 7946 - @shuo-wu @roger-ryao
• [BACKPORT][v1.6.1][BUG][v1.5.x] Recurring job fails to create backup when volume detached 8013 - @yangchiu @mantissahz @PhanLe1010 @c3y1huang
• [BACKPORT][v1.6.1][BUG] Create backup failed: failed lock lock-*.lck type 1 acquisition 7875 - @yangchiu @ChanYiLin @chriscchien
• [BACKPORT][v1.6.1][BUG][v1.5.4-rc1] Recurring job failed to create/delete backups after node reboot 7855 - @james-munson
• [BUG] Fix errors in questions.yaml 6392 - @james-munson @chriscchien

Misc

• [BACKPORT][v1.6.1][REFACTOR] move mount point check function to common lib 8124 - @ChanYiLin @chriscchien
• [TASK] Fix updating patch digest dependencies (v1.6.x) 8107 - @mantissahz

Contributors

• @ChanYiLin
• @FrankYang0529
• @PhanLe1010
• @c3y1huang
• @chriscchien
• @derekbit
• @ejweber
• @innobead
• @james-munson
• @khushboo-rancher
• @mantissahz
• @roger-ryao
• @shuo-wu
• @votdev
• @yangchiu

Longhorn v1.6.1-rc1

DON'T UPGRADE from/to any RC/Preview/Sprint releases because the operation is not supported.

Resolved Issues in this release

Improvement

• [BACKPORT][v1.6.1][IMPROVEMENT] Upgrade support bundle kit version to v0.0.36 8162 - @c3y1huang @roger-ryao
• [BACKPORT][v1.6.1][IMPROVEMENT] Allow users to request backup volume update 7984 - @mantissahz
• [BACKPORT][v1.6.1][IMPROVEMENT] Cannot read/write to block volume when the container is run as non-root 8122 - @PhanLe1010 @chriscchien
• [BACKPORT][v1.6.1][IMPROVEMENT] Improve environment_check script for NFS protocol bug and the host system self diagnosis 7971 - @james-munson @roger-ryao
• [BACKPORT][v1.6.1][IMPROVEMENT] Use HEAD instead of a GET to fetch the Content-Length of an resource via URL 7973 - @votdev @roger-ryao
• [BACKPORT][v1.6.1][IMPROVEMENT] Remove startup probe of CSI driver after liveness probe conn fix ready 7886 - @ejweber @roger-ryao
• [BACKPORT][v1.6.1][IMPROVEMENT] Change support-bundle-manager image pull policy to PullIfNotPresent 8000 - @ChanYiLin @roger-ryao

Bug

• [BACKPORT][v1.6.1][BUG] Exporting data from the existing replicas has issues in 1.6.0 8096 - @ChanYiLin @chriscchien
• [BACKPORT][v1.6.1][BUG] Longhorn api-server PUT request rate 8152 - @ejweber @roger-ryao
• [BACKPORT][v1.6.1][BUG] no Pending workload pods for volume xxx to be mounted 8081 - @c3y1huang @chriscchien
• [BACKPORT][v1.6.1][BUG] Deadlock is possible in v1.6.0 instance manager 7920 - @roger-ryao
• [BACKPORT][v1.6.1][BUG] metric longhorn_volume_robustness did not reflect detached volume 8140 - @c3y1huang
• [BACKPORT][v1.6.1][BUG][1.6.0] ENGINE v2 : disk /dev/xxxx is already used by AIO bdev disk-x 8133 - @derekbit @chriscchien
• [BACKPORT][v1.6.1][BUG] A replica may be incorrectly scheduled to a node with an existing failed replica 8044 - @ejweber @chriscchien
• [BACKPORT][v1.6.1][BUG] Deadlock between volume migration and upgrade after Longhorn upgrade 7870 - @ejweber @roger-ryao
• [BACKPORT][v1.6.1][BUG] Volume cannot attach because of the leftover non-empty volume.status.PendingNodeID after upgrading Longhorn 7995 - @james-munson @chriscchien
• [BACKPORT][v1.6.1][BUG] Use config map to update default-replica-count won't apply to default-replica-count.definition.default if the value equal to current default-replica-count.value 8134 - @james-munson @roger-ryao
• [BACKPORT][v1.6.1][BUG] Failed to restore a backup to file by the scripts/restore-backup-to-file.sh with a CIFS backup target. 8128 - @mantissahz @roger-ryao
• [BACKPORT][v1.6.1][BUG] longhorn manager pod fails to start in container-based K3s 7847 - @ChanYiLin @khushboo-rancher @chriscchien
• [BACKPORT][v1.6.1][BUG] Longhorn may keep corrupted salvaged replicas and discard good ones 7885 - @ejweber @roger-ryao
• [BACKPORT][v1.6.1][BUG] LH manager reboots due to the webhook is not ready 8037 - @ChanYiLin @chriscchien
• [BACKPORT][v1.6.1][BUG] BackingImage does not download URL correctly in some situation 7987 - @votdev @yangchiu
• [BACKPORT][v1.6.1][BUG] Executing fstrim while rebuilding causes IO errors 7868 - @yangchiu @ejweber
• [BACKPORT][v1.6.1][BUG] The feature of auto remount read only volume not work on a single node cluster. 7846 - @yangchiu @ChanYiLin
• [BACKPORT][v1.6.1][BUG] Missed NodeStageVolume after reboot leads to CreateContainerError 8012 - @ejweber @chriscchien
• [BACKPORT][v1.6.1][BUG] persistence.removeSnapshotsDuringFilesystemTrim Helm variable is unreferenced 7952 - @ejweber @chriscchien
• [BACKPORT][v1.6.1][BUG][v1.5.4-rc4] Test case test_backuptarget_available_during_engine_image_not_ready failed to wait for backup target available 8055 - @c3y1huang @chriscchien
• [BACKPORT][v1.6.1][BUG] Add Snapshot Maximum Count to the Settings 7979 - @FrankYang0529 @roger-ryao
• [BACKPORT][v1.6.1][BUG] Volumes stuck upgrading after 1.5.3 -> 1.6.0 upgrade. 7899 - @ejweber @roger-ryao
• [BACKPORT][v1.6.1][BUG] The activated DR volume do not contain the latest data. 7946 - @shuo-wu @roger-ryao
• [BACKPORT][v1.6.1][BUG][v1.5.x] Recurring job fails to create backup when volume detached 8013 - @yangchiu @mantissahz @PhanLe1010 @c3y1huang
• [BACKPORT][v1.6.1][BUG] Create backup failed: failed lock lock-*.lck type 1 acquisition 7875 - @yangchiu @ChanYiLin @chriscchien
• [BACKPORT][v1.6.1][BUG][v1.5.4-rc1] Recurring job failed to create/delete backups after node reboot 7855 - @james-munson
• [BUG] Fix errors in questions.yaml 6392 - @james-munson @chriscchien

Misc

• [BACKPORT][v1.6.1][REFACTOR] move mount point check function to common lib 8124 - @ChanYiLin @chriscchien
• [TASK] Fix updating patch digest dependencies (v1.6.x) 8107 - @mantissahz

Contributors

• @ChanYiLin
• @FrankYang0529
• @PhanLe1010
• @c3y1huang
• @chriscchien
• @derekbit
• @ejweber
• @innobead
• @james-munson
• @khushboo-rancher
• @mantissahz
• @roger-ryao
• @shuo-wu
• @votdev
• @yangchiu

Longhorn v1.5.4

Longhorn v1.5.4 Release Notes

This latest stable version of Longhorn 1.5 introduces several improvements and bug fixes that are intended to improve system quality, resilience, and stability.

The Longhorn team appreciates your contributions and anticipates receiving feedback regarding this release.

Note:
For more information about release-related terminology, see Releases.

Installation

Ensure that your cluster is running Kubernetes v1.21 or later before installing Longhorn v1.5.4.

You can install Longhorn using a variety of tools, including Rancher, Kubectl, and Helm. For more information about installation methods and requirements, see Quick Installation in the Longhorn documentation.

Upgrade

Ensure that your cluster is running Kubernetes v1.21 or later before upgrading from Longhorn v1.4.x to v1.5.4.

Longhorn only allows upgrades from supported versions. For more information about upgrade paths and procedures, see Upgrade in the Longhorn documentation.

Deprecation & Incompatibilities

For information about important changes, including feature incompatibility, deprecation, and removal, see Important Notes in the Longhorn documentation.

Post-Release Known Issues

For information about issues identified after this release, see Release-Known-Issues.

Resolved Issues

Highlight

• [BACKPORT][v1.5.4][FEATURE] Add a new settings that allows Longhorn to evict replicas automatically when a node is drained 7421 - @yangchiu @ejweber

Improvement

• [BACKPORT][v1.5.4][IMPROVEMENT] Use HEAD instead of a GET to fetch the Content-Length of an resource via URL 7981 - @votdev @roger-ryao
• [BACKPORT][v1.5.4][IMPROVEMENT] Change support-bundle-manager image pull policy to PullIfNotPresent 7999 - @ChanYiLin @chriscchien
• [BACKPORT][v1.5.4][FEATURE] Update base image of Longhorn components to BCI 15.5 7134 - @nitendra-suse
• [BACKPORT][v1.5.4]Allow to set mount options for storageclass via values.yaml in helm chart 7593 - @ChanYiLin @mantissahz
• [BACKPORT][v1.5.4][IMPROVEMENT] Remove startup probe of CSI driver after liveness probe conn fix ready 7933 - @ejweber @chriscchien
• [IMPROVEMENT] Make environment_check look for a global default K8s priority class in those releases that it affects. 7831 - @mantissahz @james-munson
• [BACKPORT][v1.5.4][IMPROVEMENT] Allow deployment of Prometheus ServiceMonitor with the Longhorn helm chart 7864 - @mantissahz @chriscchien
• [BACKPORT][v1.5.4][IMPROVEMENT] Remove unused process manager connection in longhorn-manager 7785 - @derekbit @roger-ryao
• [BACKPORT][v1.5.4][IMPROVEMENT] Clean up backup target in IM-R pod if the backup target setting is unset 7145 - @ChanYiLin @chriscchien
• [BACKPORT][v1.5.4][IMPROVEMENT] BackingImage should be compressed when downloading and use the name as filename instead of UUID 7397 - @ChanYiLin @roger-ryao
• [BACKPORT][v1.5.4][IMPROVEMENT] Automatically remount read-only RWO volume to read-write 7500 - @ChanYiLin @chriscchien
• [BACKPORT][v1.5.4][IMPROVEMENT] deploy: driver deployer shouldn't cleanup previous deployment if Kubernetes version changes 7345 - @PhanLe1010 @roger-ryao
• [BACKPORT][v1.5.4][IMPROVEMENT] Only restarts pods with volumes in the unexpected Read-Only state 7729 - @yangchiu @ChanYiLin
• [BACKPORT][v1.5.4][IMPROVEMENT] Improve handling of 16TiB+ volumes with ext4 as the underlying file system 7429 - @mantissahz @chriscchien
• [BACKPORT][v1.5.4][IMPROVEMENT] Volumes: metrics for snapshots include (size and type: system vs user) 7725 - @c3y1huang @chriscchien
• [BACKPORT][v1.5.4][IMPROVEMENT] Improve the profiler of longhorn-engine for runtime profiling 7545 - @Vicente-Cheng @chriscchien
• [BACKPORT][v1.5.4][IMPROVEMENT] Don't crash the migration engine when kubelet restarts 7328 - @yangchiu @ejweber
• [BACKPORT][v1.5.4][IMPROVEMENT] Upgrade CSI components to the latest patch release 7492 - @c3y1huang @roger-ryao
• [BACKPORT][v1.5.4][IMPROVEMENT] Reject the last replica deletion if its volume.spec.deletionTimestamp is not set 7432 - @yangchiu @derekbit
• [BACKPORT][v1.5.4][IMPROVEMENT] Upgrade support bundle kit version to v0.0.33 7279 - @c3y1huang
• [BACKPORT][v1.5.4][IMPROVEMENT] Review and simplify longhorn component image build 7162 - @ChanYiLin
• [BACKPORT][v1.5.4][IMPROVEMENT] Replace deprecated grpc.WithInsecure 7364 - @c3y1huang
• [BACKPORT][v1.5.4][IMPROVEMENT] Have a setting to disable snapshot purge for maintenance purpose 7265 - @ejweber @chriscchien
• [BACKPORT][v1.5.4][IMPROVEMENT] Bypass upgrade when installing a fresh setup 7283 - @mantissahz @roger-ryao

Bug

• [BACKPORT][v1.5.4][BUG][v1.5.4-rc4] Test case test_backuptarget_available_during_engine_image_not_ready failed to wait for backup target available 8054 - @yangchiu @c3y1huang
• [BACKPORT][v1.5.4][BUG] The activated DR volume do not contain the latest data. 7947 - @shuo-wu @roger-ryao
• [BUG][v1.5.x] DR volume unable to be activated if the latest backup's been deleted 7997 - @yangchiu @shuo-wu
• [BUG] Backup related test cases failed 7989 - @yangchiu @shuo-wu
• [BACKPORT][v1.5.4][BUG][v1.5.x] Recurring job fails to create backup when volume detached 8015 - @yangchiu @mantissahz @PhanLe1010 @c3y1huang
• [BACKPORT][v1.5.4][BUG] Deadlock for RWX volume if an error occurs in its share-manager pod 7186 - @ejweber @chriscchien
• [BACKPORT][v1.5.4][BUG] Deadlock is possible in v1.6.0 instance manager 7941 - @ejweber @roger-ryao
• [BACKPORT][v1.5.4][BUG] Longhorn may keep corrupted salvaged replicas and discard good ones 7801 - @ejweber @chriscchien
• [BACKPORT][v1.5.4][BUG] Deadlock between volume migration and upgrade after Longhorn upgrade 7869 - @ejweber @chriscchien
• [BACKPORT][v1.5.4][BUG] Executing fstrim while rebuilding causes IO errors 7867 - @ejweber @chriscchien
• [BACKPORT][v1.5.4][BUG] BackingImage does not download URL correctly in some situation 7986 - @yangchiu
• [BACKPORT][v1.5.4][BUG] The feature of auto remount read only volume not work on a single node cluster. 7844 - @ChanYiLin @chriscchien
• [BACKPORT][v1.5.4][BUG] Volumes stuck upgrading after 1.5.3 -> 1.6.0 upgrade. 7901 - @yangchiu @ejweber
• [BUG][v1.5.4-rc1] V2 volume have engine upgrade option on UI after upgrade from v1.5.3 to v1.5.4-rc1 7863 - @chriscchien @scures
• [BACKPORT][v1.5.4][BUG] longhorn manager pod fails to start in container-based K3s 7848 - @ChanYiLin
• [BACKPORT][v1.5.4][BUG] Relax S3 client retry intervals, for throttled requests 7098 - @mantissahz @chriscchien
• [BACKPORT][v1.5.4][BUG][v1.6.0-rc1] Negative test case failed: Stop Volume Node Kubelet For More Than Pod Eviction Timeout While Workload Heavy Writing 7761 - @yangchiu @c3y1huang
• [BACKPORT][v1.5.4][BUG] Volumes don't mount with mTLS enabled 7789 - @derekbit @roger-ryao
• [BACKPORT][v1.5.4][BUG] supportbundle/kubelet.log empty in k3s environment 7123 - @c3y1huang @chriscchien @roger-ryao
• [BUG] v1.5.x/v1.4.x BackingImage download fails if URL has query parameters 7822 - @ChanYiLin @mantissahz
• [BACKPORT][v1.5.4][BUG] Metric totalVolumeSize and totalVolumeActualSize incorrect due to v2 volume counts 7392 - @c3y1huang @chriscchien
• [BACKPORT][v1.5.4][BUG] Continuously auto-balancing replicas when zone does not have enough space [7306](https://github.com/longh...

Longhorn v1.5.4-rc4

DON'T UPGRADE from/to any RC/Preview/Sprint releases because the operation is not supported.

Resolved Issues in this release

Highlight

• [BACKPORT][v1.5.4][FEATURE] Add a new settings that allows Longhorn to evict replicas automatically when a node is drained 7421 - @yangchiu @ejweber

Improvement

• [BACKPORT][v1.5.4][IMPROVEMENT] Use HEAD instead of a GET to fetch the Content-Length of an resource via URL 7981 - @votdev @roger-ryao
• [BACKPORT][v1.5.4][IMPROVEMENT] Change support-bundle-manager image pull policy to PullIfNotPresent 7999 - @ChanYiLin @chriscchien
• [BACKPORT][v1.5.4][IMPROVEMENT] Use HEAD instead of a GET to fetch the Content-Length of an resource via URL 7974 - @votdev
• [BACKPORT][v1.5.4][FEATURE] Update base image of Longhorn components to BCI 15.5 7134 - @nitendra-suse
• [BACKPORT][v1.5.4]Allow to set mount options for storageclass via values.yaml in helm chart 7593 - @ChanYiLin @mantissahz
• [BACKPORT][v1.5.4][IMPROVEMENT] Remove startup probe of CSI driver after liveness probe conn fix ready 7933 - @ejweber @chriscchien
• [IMPROVEMENT] Make environment_check look for a global default K8s priority class in those releases that it affects. 7831 - @mantissahz @james-munson
• [BACKPORT][v1.5.4][IMPROVEMENT] Allow deployment of Prometheus ServiceMonitor with the Longhorn helm chart 7864 - @mantissahz @chriscchien
• [BACKPORT][v1.5.4][IMPROVEMENT] Remove unused process manager connection in longhorn-manager 7785 - @derekbit @roger-ryao
• [BACKPORT][v1.5.4][IMPROVEMENT] Clean up backup target in IM-R pod if the backup target setting is unset 7145 - @ChanYiLin @chriscchien
• [BACKPORT][v1.5.4][IMPROVEMENT] BackingImage should be compressed when downloading and use the name as filename instead of UUID 7397 - @ChanYiLin @roger-ryao
• [BACKPORT][v1.5.4][IMPROVEMENT] Automatically remount read-only RWO volume to read-write 7500 - @ChanYiLin @chriscchien
• [BACKPORT][v1.5.4][IMPROVEMENT] deploy: driver deployer shouldn't cleanup previous deployment if Kubernetes version changes 7345 - @PhanLe1010 @roger-ryao
• [BACKPORT][v1.5.4][IMPROVEMENT] Only restarts pods with volumes in the unexpected Read-Only state 7729 - @yangchiu @ChanYiLin
• [BACKPORT][v1.5.4][IMPROVEMENT] Improve handling of 16TiB+ volumes with ext4 as the underlying file system 7429 - @mantissahz @chriscchien
• [BACKPORT][v1.5.4][IMPROVEMENT] Volumes: metrics for snapshots include (size and type: system vs user) 7725 - @c3y1huang @chriscchien
• [BACKPORT][v1.5.4][IMPROVEMENT] Improve the profiler of longhorn-engine for runtime profiling 7545 - @Vicente-Cheng @chriscchien
• [BACKPORT][v1.5.4][IMPROVEMENT] Don't crash the migration engine when kubelet restarts 7328 - @yangchiu @ejweber
• [BACKPORT][v1.5.4][IMPROVEMENT] Upgrade CSI components to the latest patch release 7492 - @c3y1huang @roger-ryao
• [BACKPORT][v1.5.4][IMPROVEMENT] Reject the last replica deletion if its volume.spec.deletionTimestamp is not set 7432 - @yangchiu @derekbit
• [BACKPORT][v1.5.4][IMPROVEMENT] Upgrade support bundle kit version to v0.0.33 7279 - @c3y1huang
• [BACKPORT][v1.5.4][IMPROVEMENT] Review and simplify longhorn component image build 7162 - @ChanYiLin
• [BACKPORT][v1.5.4][IMPROVEMENT] Replace deprecated grpc.WithInsecure 7364 - @c3y1huang
• [BACKPORT][v1.5.4][IMPROVEMENT] Have a setting to disable snapshot purge for maintenance purpose 7265 - @ejweber @chriscchien
• [BACKPORT][v1.5.4][IMPROVEMENT] Bypass upgrade when installing a fresh setup 7283 - @mantissahz @roger-ryao

Bug

• [BUG][v1.5.x] DR volume unable to be activated if the latest backup's been deleted 7997 - @yangchiu @shuo-wu
• [BACKPORT][v1.5.4][BUG][v1.5.x] Recurring job fails to create backup when volume detached 8015 - @yangchiu @mantissahz @PhanLe1010 @c3y1huang
• [BACKPORT][v1.5.4][BUG] The activated DR volume do not contain the latest data. 7947 - @shuo-wu @roger-ryao
• [BACKPORT][v1.5.4][BUG] Deadlock for RWX volume if an error occurs in its share-manager pod 7186 - @ejweber @chriscchien
• [BACKPORT][v1.5.4][BUG] Deadlock is possible in v1.6.0 instance manager 7941 - @ejweber @roger-ryao
• [BACKPORT][v1.5.4][BUG] Longhorn may keep corrupted salvaged replicas and discard good ones 7801 - @ejweber @chriscchien
• [BACKPORT][v1.5.4][BUG] Deadlock between volume migration and upgrade after Longhorn upgrade 7869 - @ejweber @chriscchien
• [BACKPORT][v1.5.4][BUG] Executing fstrim while rebuilding causes IO errors 7867 - @ejweber @chriscchien
• [BUG] Backup related test cases failed 7989 - @yangchiu @shuo-wu
• [BACKPORT][v1.5.4][BUG] BackingImage does not download URL correctly in some situation 7986 - @yangchiu
• [BACKPORT][v1.5.4][BUG] The feature of auto remount read only volume not work on a single node cluster. 7844 - @ChanYiLin @chriscchien
• [BACKPORT][v1.5.4][BUG] Volumes stuck upgrading after 1.5.3 -> 1.6.0 upgrade. 7901 - @yangchiu @ejweber
• [BUG][v1.5.4-rc1] V2 volume have engine upgrade option on UI after upgrade from v1.5.3 to v1.5.4-rc1 7863 - @chriscchien @scures
• [BACKPORT][v1.5.4][BUG] longhorn manager pod fails to start in container-based K3s 7848 - @ChanYiLin
• [BACKPORT][v1.5.4][BUG] Relax S3 client retry intervals, for throttled requests 7098 - @mantissahz @chriscchien
• [BACKPORT][v1.5.4][BUG][v1.6.0-rc1] Negative test case failed: Stop Volume Node Kubelet For More Than Pod Eviction Timeout While Workload Heavy Writing 7761 - @yangchiu @c3y1huang
• [BACKPORT][v1.5.4][BUG] Volumes don't mount with mTLS enabled 7789 - @derekbit @roger-ryao
• [BACKPORT][v1.5.4][BUG] supportbundle/kubelet.log empty in k3s environment 7123 - @c3y1huang @chriscchien @roger-ryao
• [BUG] v1.5.x/v1.4.x BackingImage download fails if URL has query parameters 7822 - @ChanYiLin @mantissahz
• [BACKPORT][v1.5.4][BUG] Metric totalVolumeSize and totalVolumeActualSize incorrect due to v2 volume counts 7392 - @c3y1huang @chriscchien
• [BACKPORT][v1.5.4][BUG] Continuously auto-balancing replicas when zone does not have enough space 7306 - @c3y1huang @chriscchien
• [BACKPORT][v1.5.4][BUG] Unable to list backups when backuptarget resource is picked up by a cordoned node 7621 - @mantissahz @c3y1huang
• [BUG] The wrong template in default-setting.yaml of the Longhorn chart in v1.5 and v1.4 7459 - @mantissahz @roger-ryao
• [BACKPORT][v1.5.4][BUG] Failed to check_volume_data after volume engine upgrade/migration 7402 - @PhanLe1010 @chriscchien
• [BACKPORT][v1.5.4][BUG] Volume conditions are not represented in the UI for v1.4.x and newer 7242 - @m-ildefons @roger-ryao
• [BACKPORT][v1.5.4][BUG] Confusing logging when trying to attach a new volume with no scheduled replicas 7245 - @ejweber @roger-ryao
• [BACKPORT][v1.5.4][BUG] Environment check script claims success when kubectl fails. 7216 - @james-munson @roger-ryao
• [BACKPORT][v1.5.4][BUG] Backup volume attachment tickets might not be cleaned up after completion. 7604 - @james-munson @chriscchien
• [BACKPORT][v1.5.4][BUG][v1.6.0-rc1] Some Longhorn resources remaining after longhorn-uninstall job completed 7663 - @yangchiu @PhanLe1010
• [BACKPORT][v1.5.4][BUG] Backi...