fix: case sensitive file extension checks and force them to lower cas…

[FHardow]

…e on upload

Related Issues

• fixes https://deepset.atlassian.net/browse/DC-1523

Proposed Changes?

Force the file extension to be lower case to check against allowed extensions and on upload. All other checks, like checking for metadata files are left untouched.

How did you test it?

Adjusted unit tests.

Notes for the reviewer

Screenshots (optional)

Checklist

• I have updated the referenced issue with new insights and changes
• If this is a code change, I have added unit tests
• I've used the conventional commit specification for my PR title
• I updated the docstrings
• If this is a code change, I added meaningful logs and prepared Datadog visualizations and alerts

[swarmia]

✅  Linked to Bug DC-1523 · SDK does not upload files having an uppercase extension

[github-actions]

Coverage report

Click to see where and how coverage changed

File	Statements	Missing	Coverage	Coverage (new stmts)	Lines missing
deepset_cloud_sdk/_service
files_service.py
Project Total

This report was generated by python-coverage-comment-action

[ArzelaAscoIi]

Could you write/adjsut an integration test that:

• uploads an example.TXT
• uploads an example.txt
• uploads an example.TXT.meta.json

and checks that only the example.TXT has the correct metadata ?

[mathislucka]

and checks that only the example.TXT has the correct metadata ?

Are we considering lower and upper case to be different files then?

I think that wouldn't be immediately obvious to me as a user.

[FHardow]

Hey @mathislucka
I added a comment in the ticket.

Are we considering lower and upper case to be different files then?

We probably need to. It depends on your file system how case sensitivity is treated. We can't enforce a casing on upload without running into the problem of overwriting data.

[FHardow]

We decided that we will not continue on this fix and document that customers files need to end with lower cased extensions.

[mathislucka]

Could we add an explicit warning to the upload command? I'm afraid that users won't check the documentation.

[agnieszka-m]

The warning could say: Make sure your files have lowercase extensions, for example, my_file.pdf, instead of my_file.PDF. The SDK doesn't upload files with uppercase extensions.