Skip to content

Commit

Permalink
pool: on failed upload set file size to zero for space reporting
Browse files Browse the repository at this point in the history
Motivation:

Sometimes, when an upload fails, it results in a replica of size 0 that is then flagged to be removed (ReplicaState.REMOVED).
The finished transfer is nevertheless reported to SpaceManager when the serice exists. It requires the file size to be defined,
which may not have been set in such cases, leading to an exception being logged.

In general, we should make sure that the file size is set properly when a transfer finishes, whether it is successful or not.
SpaceManager should ignore a `DoorTransferFinishedMessage` only when it reports a CacheException: the corresponding file is gone from the name space.
Otherwise, even if the file size is not defined, it should update its database to reflect that the file of size 0 bytes is stored.
It should receive a `PnfsDeleteEntryNotificationMessage` on removal of the actual namespace entry later on, which will trigger the correct
removal of the space manager database entry.

Modification:

On failure, when the replica size is 0, update the file attributes accordingly before setting the replica into state `REMOVED`.
Additionally, safeguard SpaceManager by storing a file size of 0 when the SIZE attribute in the `DoorTransferFinishedMessage` is not defined
and log a warning.

Result:
No more logged stack traces due to `IllegalStateException: Attribute is not defined: SIZE` errors.

Target: master
Request: 9.0
Request: 8.2
Request: 8.1
Request: 8.0
Fixes: #7211
Requires-notes: yes
Requires-book: no
Patch: https://rb.dcache.org/r/14061/
Acked-by: Tigran Mkrtchyan
  • Loading branch information
lemora committed Sep 28, 2023
1 parent afe112e commit 707cf4f
Show file tree
Hide file tree
Showing 2 changed files with 10 additions and 2 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -776,7 +776,14 @@ private void transferFinished(DoorTransferFinishedMessage finished)
*/
fileRemoved(finished.getPnfsId());
} else {
transferFinished(finished.getPnfsId(), finished.getFileAttributes().getSize());
long fileSize = 0;
if (finished.getFileAttributes().isDefined(FileAttribute.SIZE)) {
fileSize = finished.getFileAttributes().getSize();
} else {
LOGGER.warn("Transfer " + (finished.getReturnCode() == 0 ? "succeeded " : "failed ")
+ "and did not report a file size. Reporting as zero bytes.");
}
transferFinished(finished.getPnfsId(), fileSize);
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -302,10 +302,11 @@ private synchronized void fail() {
}

/* If nothing was uploaded, we delete the replica and leave the name space
* entry it is virgin state.
* entry in its virgin state except for setting the file size to zero.
*/
long length = _entry.getReplicaSize();
if (length == 0) {
_fileAttributes.setSize(length);
_targetState = ReplicaState.REMOVED;
if (why == null) {
why = "replica is empty";
Expand Down

0 comments on commit 707cf4f

Please sign in to comment.