Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

If url_storage is enabled, BaseUploader should remove file entry upon certain failures during get_offset() #103

Open
pandabear opened this issue Feb 26, 2025 · 0 comments

Comments

@pandabear
Copy link

pandabear commented Feb 26, 2025

When Uploader is initialized through client.TusClient(url).uploader('path/to/file.ext', store_url=True, url_storage=storage) with resumability enabled, there's a possibility that client-side would be unable to continue uploading their files if the server-side no longer deems the URL to be valid.

Steps to reproduce:

# 1. Initialize the first tus upload.
from tusclient import client
from tusclient.storage import filestorage

# 2. Ensure the host supports Creation-extension
my_client = client.TusClient('https://host.running.tus.test/files/')
storage = filestorage.FileStorage('storage_file')
my_uploader = my_client.uploader('path/to/file.ext'', store_url=True, url_storage=storage)

# 3. Send the first chunk
my_uploader.upload_chunk()

# 4. Now that "storage_file" has an entry for our file.ext. Start over in a brand new instance.

# 5. Initialize the first tus upload, again.
from tusclient import client
from tusclient.storage import filestorage

# 6. Ensure the host supports Creation-extension AND has removed the URL that was created during step 3.
my_client = client.TusClient('https://host.running.tus.test/files/')
storage = filestorage.FileStorage('storage_file')

# 7. Will now raise "tusclient.exceptions.TusCommunicationError: Attempt to retrieve offset fails with status 404"
my_uploader = my_client.uploader('path/to/file.ext'', store_url=True, url_storage=storage)

What I expect to happen is that upon HTTP 404, 410, or even 403 response the BaseUploader would conduct interface.Storage's remove_item(key) during __init_url_and_offset() and clear out self.url that it has assigned from the storage. Then continue as if brand new Uploader()-instance was created. Otherwise the only workaround on client side would be to disable resumable option or rename FileStorage's "storage_file" for the Uploader()-instance creation.

BaseUploader does have the _get_fingerprint() that developer could use to manually run interface.Storage's remove_item(key) , but the function name hints that it's not intended for use outside of BaseUploader-context. My suggested expectation does change the default behaviour dramatically, and I understand that existing implementation may rely on current behaviour, so perhaps a unique flag that could be provided to enable my suggested behaviour? Something like my_uploader = my_client.uploader('path/to/file.ext'', store_url=True, url_storage=storage, remove_storage_entry_upon_404_410_403_during_init=True)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant