Update file_handlers.py get_requester_pays

[cchu70]

Problem

get_requester_pays() is incorrectly returning False for buckets that are requester-pays. I copied this code block from canine into my notebook for testing:

canine/canine/localization/file_handlers.py

Lines 108 to 132 in 0cebe19

	def gcloud_storage_client():
	global STORAGE_CLIENT
	with storage_client_creation_lock:
	if STORAGE_CLIENT is None:
	# this is the expensive operation
	STORAGE_CLIENT = google.cloud.storage.Client()
	return STORAGE_CLIENT
	class GSFileNotExists(Exception):
	pass
	class HandleGSURL(FileType):
	localization_mode = "url"
	def get_requester_pays(self) -> bool:
	"""
	Returns True if the requested gs:// object or bucket resides in a
	requester pays bucket
	"""
	bucket = re.match(r"gs://(.*?)/.*", self.path)[1]
	gcs_cl = gcloud_storage_client()
	bucket_obj = google.cloud.storage.Bucket(gcs_cl, bucket, user_project = self.extra_args.get("project"))
	return bucket_obj.requester_pays

Here we see bucket._properties is empty, which is why this returns false.

Solutions

Google's API suggests a different approach: https://cloud.google.com/storage/docs/using-requester-pays#storage-get-requester-pays-status-python

bucket.reload() is used in google's own API above: https://github.com/googleapis/python-storage/blob/e3cfc4786209c77e3c879c9ff2978f4884a0d677/google/cloud/storage/client.py#L870-L877

When I add bucket.reload(), we get the expected behavior (with the _properties attribute filled):

Alternatively, we use Google's recommended approach directly:

[cchu70]

This appears to be the case in the latest version of canine (https://github.com/getzlab/canine/tree/v0.15.7), which is used in the latest version of wolf
https://github.com/getzlab/wolF/blob/1e39dc1a8a3a14ae2cfb2c7f9b9302413751f132/setup.py#L23