Remove exception on failure response from GCS delete API #16047

gargvishesh · 2024-03-05T09:39:58Z

The previously used Google API Client library for Google Cloud Storage returned a 404 HTTPResponseException for delete when the object to be deleted was not found, and was propagated as-is by the Druid GCS extension. The currently used Google Cloud Client library just returns a false response in that scenario, which was being converted to a generic IOException in the extension -- breaking flows where file missing is an acceptable behaviour.

This PR removes the exception altogether since all current delete scenarios don't care if the file-to-be-deleted is missing

...ions-core/google-extensions/src/main/java/org/apache/druid/storage/google/GoogleStorage.java

Co-authored-by: Abhishek Radhakrishnan <[email protected]>

...ions-core/google-extensions/src/main/java/org/apache/druid/storage/google/GoogleStorage.java

abhishekagarwal87 · 2024-03-06T03:59:04Z

...ions-core/google-extensions/src/main/java/org/apache/druid/storage/google/GoogleStorage.java

    if (!storage.get().delete(bucket, path)) {
-      throw new IOE(
-          "Failed deleting google cloud storage object [bucket: %s path: %s]",
+      log.warn(StringUtils.nonStrictFormat(


since it was debug previously, lets keep it at debug level

I understand this is not to spook existing users when they suddenly start seeing warn for missing descriptor.json. Else it better be visible to spot logic bugs.

abhishekagarwal87 · 2024-03-06T04:00:34Z

...ions-core/google-extensions/src/main/java/org/apache/druid/storage/google/GoogleStorage.java

  {
    List<Boolean> statuses = storage.get().delete(Iterables.transform(paths, input -> BlobId.of(bucket, input)));
    if (statuses.contains(false)) {
-      throw new IOE("Failed deleting google cloud storage object(s)");
+      log.warn("Google cloud storage object(s) to be deleted not found");


it can be debug level as well. Also please log the paths and bucket

Changed and included.

...le-extensions/src/test/java/org/apache/druid/storage/google/GoogleDataSegmentKillerTest.java

abhishekagarwal87 · 2024-03-06T04:05:32Z

...google-extensions/src/main/java/org/apache/druid/storage/google/GoogleDataSegmentKiller.java

@@ -70,12 +70,12 @@ public void kill(DataSegment segment) throws SegmentLoadingException
      // anymore, but we still delete them if exists.
      deleteIfPresent(bucket, descriptorPath);
    }
-    catch (IOException e) {
+    catch (StorageException e) {


Why this change?

deleteIfPresent no longer throws IOException

the underlying lib could still throw an IOException though?

Nope, it can't -- else it has to be caught or declared thrown since it's a checked exception.

I think we could add IOException too here so that any random IO error is bubbled up as SegmentLoadingException

Currently wIth retry, only 2 exception kinds are possible: a) Exception raised by the retried call (only StorageException in this case), and b) RuntimeException.

druid/processing/src/main/java/org/apache/druid/java/util/common/RetryUtils.java

Line 140 in bf39c71

Throwables.propagateIfInstanceOf(e, Exception.class);

Do we still want to include IOException handling?

Hmm. probably ok to leave.

* change runtime exception class for code coverage * Add file paths for batch delete failures