Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AWS Adapter Checksum Algorithm set to NULL for wasabi #1850

Open
chrisgo opened this issue Feb 3, 2025 · 1 comment
Open

AWS Adapter Checksum Algorithm set to NULL for wasabi #1850

chrisgo opened this issue Feb 3, 2025 · 1 comment

Comments

@chrisgo
Copy link

chrisgo commented Feb 3, 2025

Bug Report

We were getting an error doing writes to wasabi for large files (16MB seems to be when it switches to multipart) that mentioned checksum Checksum Type mismatch occurred, expected checksum Type: null, actual checksum Type: crc32

There is a workaround but did bring our production systems down for a couple of days since this is also sporadic until we isolated this to "larger" files

This also seems to be working on the Amazon S3 service and ONLY for wasabi (this is the only other S3-compatible provider we have handy)

Q A
Flysystem Version ^3.10.0
Adapter Name flysystem-aws-s3-v3
Adapter version ^3.10.0
AWS SDK ^3.295.10

Summary

Tracing this down seems to lead to the ->upload() function being used instead of ->putObject(). The ->upload() function automatically switches to the multipart for large files. The ->putObject() seems to be unaffected

The problem seems to be due to this array https://github.com/thephpleague/flysystem/blob/3.x/src/AwsS3V3/AwsS3V3Adapter.php#L47 where it lists the 'ChecksumAlgorithm' as an AVAILABLE_OPTIONS

Then somewhere when it builds the config, it is setting this to NULL (not sure) https://github.com/thephpleague/flysystem/blob/3.x/src/AwsS3V3/AwsS3V3Adapter.php#L176 instead of taking the default (which seems to be CRC32) and this seems to be what is throwing the error above

How to reproduce

// Provider configuration 
$options = [
  ... // normal stuff per documentation
  'use_path_request_style' => true,  // not really needed for wasabi
];
$client = new \Aws\S3\S3Client($options);
$bucket = 'my-bucket'
$prefix = 'my-prefix';
// Usual way of instantiating the adapter
$adapter = new \League\Flysystem\AwsS3V3\AwsS3V3Adapter($client, $bucket, $prefix);
...

Workaround/Fix

// Provider configuration 
$options = [
  ... // normal stuff per documentation
  'use_path_request_style' => true,  // not really needed for wasabi
];
$client = new \Aws\S3\S3Client($options);
$bucket = 'my-bucket'
$prefix = 'my-prefix';
// 1/31/25 CG: Need to pass the $options (that has the 'ChecksumAlgorithm' = 'CRC32') all the way through to the adapter
$options['ChecksumAlgorithm'] = 'CRC32';
$adapter = new \League\Flysystem\AwsS3V3\AwsS3V3Adapter($client, $bucket, $prefix, null, null, $options);
...
@danharper83
Copy link

danharper83 commented Feb 18, 2025

I'm having this issue with a Laravel application and it's causing laravel-backup to fail.

I have the following packages.

laravel-backup: 8.8.2
laravel: 11.41.0
league/flysystem: 3.29.1
league/flysystem-aws-s3-v3: 3.29.0
AWS Sdk: 3.339.15

Are you suggesting this can be fixed in a config file or is this something that needs patching in Laravel?

Is this just Wasabi, could I move to a native S£ bucket in the meaintime?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants