Skip to content

Releases: awslabs/mountpoint-s3

mountpoint-s3 v1.10.0

15 Oct 15:02
2b01616
Compare
Choose a tag to compare

New features

  • Mountpoint now automatically adjusts its prefetcher read window size based on available system memory. This reduces the risk of Mountpoint potentially consuming all available system memory in cases where a large number of file handles are read from concurrently. (#1013)

Breaking changes

  • When configured to log to a directory, Mountpoint now includes a random string following the timestamp in the file name.
    Previously, multiple Mountpoint processes would write to the same log file causing log entries to be interleaved.
    (#1041)

Other changes

  • When updating the local disk cache, Mountpoint now writes to disk asynchronously, without blocking the read path. The change favours cold cache reads but may result in additional initial cache misses in highly concurrent workloads. (#1029)

Binaries

x86_64

arm64

mountpoint-s3 v1.9.1

19 Sep 16:10
9040066
Compare
Choose a tag to compare

[RECALLED] mountpoint-s3 v1.9.0

13 Sep 15:19
9ed8b62
Compare
Choose a tag to compare

Caution

This release has been recalled due to the --bind argument not being fully released. Please use v1.9.1 or later instead.

New features

  • Mountpoint now supports specifying one or more network interfaces using --bind <INTERFACE_NAME> to be used when making requests to Amazon S3. This feature is work-in-progress and we welcome feedback on it. (#943)

Other changes

  • Fix an issue where credential_process field would not be picked up correctly when using source_profile. (awslabs/aws-c-auth#245)
  • Fix an issue where credential_process field would not be picked up correctly when using --profile <AWS_PROFILE>. (awslabs/aws-c-auth#245)
  • Re-implement prefetcher using CRT's flow-control: customers may experience improvement in bandwidth when reading multiple files concurrently and reduced memory consumption. (#980)

Binaries

x86_64

arm64

mountpoint-s3 v1.8.0

01 Aug 13:52
387ad79
Compare
Choose a tag to compare

New features

  • Mountpoint now offers two new command-line arguments --read-part-size <SIZE> and --write-part-size <SIZE> which allow to specify different part sizes to be used when reading and writing respectively. (#949)

Other changes

  • Support added for providing external_id when defining AWS profiles in a config file. (#962)
  • Fix issue where empty environment variables for STS web identity credentials could cause segmentation fault. (#963)

Binaries

x86_64

arm64

mountpoint-s3 v1.7.2

17 Jun 20:04
d3b632d
Compare
Choose a tag to compare

mountpoint-s3 v1.7.1

14 Jun 10:36
7155555
Compare
Choose a tag to compare

Breaking changes

  • Mountpoint supports specifying KMS server-side encryption for new object uploads using the key ARN only. Mountpoint now validates the value provided with --sse-kms-key-id at mount time. This prevents an issue where Mountpoint could crash when uploading a file with KMS key identifiers other than the key ARN. (#908)

Binaries

x86_64

arm64

mountpoint-s3 v1.7.0

06 Jun 09:09
42007f7
Compare
Choose a tag to compare

New features

  • Metadata caching can now be configured independently of data caching. When passing the --metadata-ttl <seconds> argument without also specifying --cache <directory>, Mountpoint will cache file metadata in memory for up to the given TTL, but will not cache object data. The --metadata-ttl argument also accepts two special values: minimal to enable only the minimal necessary caching, and indefinite to cache indefinitely. These modes can help accelerate workloads that touch many files but do not need to cache object data for re-use (for example, listing a directory and then reading each file within it once). (#855)

Breaking changes

  • The --metadata-ttl 0 setting is no longer supported and will be removed in a future release. The new --metadata-ttl minimal has a similar effect, but behaves better when latency for S3 requests is high. (#855)
  • When using the --cache flag, the default metadata TTL is now set to 60 seconds (--metadata-ttl 60) instead of 1 second. (#855)
  • Mountpoint now uses STS regionalized endpoints when assuming IAM roles configured in a CLI profile. If you specify a region in your CLI profile and want Mountpoint to use a role from that profile you have to make sure Mountpoint can access STS regionalized endpoints. This allows Mountpoint to assume an IAM role in regions outside of the aws partition. (#877)

Other changes

  • The checksum algorithm to use for uploads to S3 can now be chosen with the --upload-checksums <ALGORITHM> command-line argument. The only supported values in this release are crc32c (the default, and the existing behavior) and off, which disables including checksums in uploads. The off value allows uploads to S3 implementations that do not support additional checksums. This option defaults to off when the bucket name is an S3 on Outposts bucket access point (either an ARN or a bucket alias). (#849)
  • Fixed an issue where Mountpoint did not send the Content-Length header when creating multi-part uploads. (#875)
  • Fixed an issue where Mountpoint could not assume an IAM role specified in a CLI profile with EcsContainer as a credential source. (#875)
  • Added support for AWS_ENDPOINT_URL environment variable. (#895)

Binaries

x86_64

arm64

mountpoint-s3 v1.6.0

11 Apr 17:12
ad0f530
Compare
Choose a tag to compare

New features

  • Mountpoint for Amazon S3 now supports specifying an AWS Key Management Service (AWS KMS) key for server-side encryption with KMS (SSE-KMS) when mounting an S3 bucket or prefix. (#839)

Breaking changes

  • No breaking changes.

Other changes

  • Mountpoint now retries S3 requests up to a total of 10 attempts (up from 4), which should make file operations more robust to transient failures or throttling. The maximum number of attempts can be overridden by setting the AWS_MAX_ATTEMPTS environment variable. (#830)
  • Fix an issue where Mountpoint could become unresponsive after opening too many files in write mode. (#832)
  • Add support for rewinddir by restarting readdir if offset is zero. (#825)

Binaries

x86_64

arm64

mountpoint-s3 v1.5.0

07 Mar 17:05
56591e7
Compare
Choose a tag to compare

New features

  • When caching is enabled, Mountpoint also remembers when objects do not exist, in order to reduce repeated lookups. (#696)

Other changes

  • Cancel S3 requests when dropped. Addresses an issue where the prefetcher could keep streaming up to 2GB of data that would never be used. (#794)
  • Improve read throughput in more non-sequential access patterns by better accounting for the progress of in-flight prefetch requests. (#797)
  • Stop limiting the number of connections based on the number of known IPs when connecting to S3. Improves maximum throughput on S3 Express. (#796)

Binaries

x86_64

arm64

mountpoint-s3 v1.4.1

16 Feb 16:28
1c00fd3
Compare
Choose a tag to compare

Other changes

  • Fix an issue where read file handles could be closed too early, leading to bad file descriptor errors on subsequent reads. As a consequence of this fix, opening an existing file to overwrite it immediately after closing a read file handle may occasionally fail with an "Operation not permitted" error. In such cases, Mountpoint logs will also report that the file is "not writable while being read". (#751)
  • File handles are no longer initialized lazily. Lazy initialization was introduced in version 1.4.0 but is reverted in this change. If upgrading from 1.4.0, you may see errors that were previously deferred until read/write now raised at open time. (#751)

Binaries

x86_64

arm64