Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stream raw file to upload (supabase-fetcher) #53

Merged
merged 3 commits into from
Jan 9, 2025
Merged

Conversation

zoedsoupe
Copy link
Collaborator

Problem

The Supabase.Fetcher.upload/4 function currently uses File.stream!/3 with encoding: :utf8 to read file contents during uploads. This causes issues when uploading non-UTF-8 files, such as binary files (e.g., MP3s), resulting in :invalid_unicode errors. Users have reported difficulties uploading files to Supabase Storage due to this limitation.

Additionally, the implementation does not dynamically determine the Content-Type header based on the file being uploaded, which can lead to incorrect file handling on the server.

References

Solution

Refactor the Supabase.Fetcher.upload/4 function to address the following:

  1. Remove UTF-8 Encoding:

    • Change File.stream!/3 to use the :raw option instead of :utf8, which ensures that binary files can be read without errors.
  2. Determine MIME Type:

    • Use the library mime to detect the file's MIME type dynamically.
  3. Set the Correct Content-Type Header:

    • Ensure that the MIME type is added to the request headers dynamically based on the file's type.

Rationale

  1. File Encoding:
    • Removing the :utf8 encoding resolves errors when streaming binary files (e.g., MP3s, images).
    • The :raw option reads files as-is without attempting to interpret their content as a specific character set.

@zoedsoupe
Copy link
Collaborator Author

depends on #50

@zoedsoupe zoedsoupe self-assigned this Jan 4, 2025
@zoedsoupe zoedsoupe linked an issue Jan 4, 2025 that may be closed by this pull request
@zoedsoupe zoedsoupe force-pushed the fix/52 branch 2 times, most recently from 4e4f54d to d7bc4cc Compare January 4, 2025 21:14
@zoedsoupe zoedsoupe changed the base branch from main to fix/45 January 9, 2025 16:47
Base automatically changed from fix/45 to feat/update-elixir January 9, 2025 17:01
@zoedsoupe zoedsoupe merged commit 3890e1d into feat/update-elixir Jan 9, 2025
1 check passed
@zoedsoupe zoedsoupe deleted the fix/52 branch January 9, 2025 17:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Invalid Unicode error during file uploads (affects storage-ex) Invalid Unicode Upload from Liveview Upload?
2 participants