AzCopy V10.2.0 Release

Version 10.2.0

Bug fix

1. Security: fixed signature redaction in logs to include all error types: the log entries for network failures and HTTP errors could include SAS tokens. In previous releases, the SAS tokens were not always redacted correctly and could be written to the AzCopy log file and also to the Windows Event Log or the Linux Syslog. Now, SAS tokens are correctly redacted when logging those errors. Note that errors returned by the Storage service itself - such as authentication errors and bad container names – were already redacted correctly.
2. Added error to using Azure Files without a SAS token (invalid auth configuration).
3. AzCopy v10 now outputs a sensible error & warning when attempting to authenticate a storage account business-to-business.
4. --log-level=none now drops no logs, and has a listing in --help.
5. Fixed bug where piping was not picking up the service version override, making it not work well against Azure Stack.
6. Fixed a timeout when uploading particularly large files to ADLSG2.
7. Fixed single wildcard match uploads.

New features

1. Enabled copying from page/block/append blob to another blob of a different type.
2. AzCopy now grabs proxy details (sans authentication) from the Windows Registry using mattn/go-ieproxy.
3. Service Principal Authentication is now available under azcopy login-- check azcopy env for details on client secrets/cert passwords.
4. SAS tokens are supported on HNS (Hierarchical Namespace/Azure Data Lake Generation 2) Storage Accounts.
5. Added support for custom headers on ADLS Gen 2.
6. Added support for fractional block size for copy and sync.
7. Use different log output for skipped files (so they don't look like failures).
8. Added bandwidth cap (--cap-mbps) to limit AzCopy's network usage, check azcopy cp -h for details.
9. Added ADLS Gen2 support for rm command.

AzCopy V10.1.2 Release

Version 10.1.2

Breaking change

1. Jobs created with earlier releases cannot be resumed with this release. We recommend
   you update to this release only when you have no partially-completed jobs that you want to resume.

Bug fix

1. Files with Content-Encoding: gzip are now downloaded in compressed form. Previous versions tried to save a
   decompressed version of the file. But they incorrectly truncated it at the original compressed length, so the
   downloaded file was not complete.

   By changing AzCopy to save the compressed version, that problem is solved, and Content-MD5 checks now work for such files. (It is
   assumed that the Content-MD5 hash is the hash of the compressed file.)

New features

1. Headers for Content-Disposition, Content-Language and Cache-Control can now be set when uploading
   files to Blob Storage and to Azure Files. Run azcopy copy --help to see command line parameter
   information, including those needed to set the new headers.
2. On-screen job summary is output to the log file at end of job, so that the log will include those summary statistics.

AzCopy V10.1.1 Release

Bug fixes

1. Fixed typo in local traverser (error handling in walk).
2. Fixed memory alignment issue for atomic functions on 32 bit system.

AzCopy V10.1.0 GA Release

Version 10.1.0 (GA)

Breaking changes

1. The --block-size parameter has been replaced by --block-size-mb. The old parameter took a number of bytes; the
   new one takes a number of Megabytes (MiB).
2. The following command line parameters have been renamed, for clarity
   • --output is now --output-type
   • --md5-validation is now called --check-md5
   • --s2s-source-change-validation is now called --s2s-detect-source-changed
   • --s2s-invalid-metadata-handle is is now called --s2s-handle-invalid-metadata
   • --quota (in the make command) is now called --quota-gb. Note that the values were always in GB, the new name
     simply clarifies that fact

New features

1. AzCopy is now able to be configured to use older API versions. This enables (limited) support for Azure Stack.
2. Listing command now shows file sizes.

Bug fixes

1. AzCopy v10 now works correctly with ADLS Gen 2 folders that contain spaces in their names.
2. When cancelling with CRTL-C, status of in-progress transfers is now correctly recorded.
3. For security, the Service-to-Service (S2S) feature will only work if both the source and destination connections are
   HTTPS.
4. Use of the --overwrite parameter is clarified in the in-application help.
5. Fixed incorrect behavior with setting file descriptor limits on platforms including OS X and BSD.
6. On Linux and OS X, log files are now created with same file permissions as all other files created by AzCopy.
7. ThirdPartyNotice.txt is updated.
8. Load DLL in a more secure manner compatible with Go's sysdll registration.
9. Fixed support for relative paths and shorthands.
10. Fixed bug in pattern matching for blob download when recursive is off.

AzCopyV10.0.9 Preview Release

Version 10.0.9 (Release Candidate)

Breaking changes

1. For creating MD5 hashes when uploading, version 10.x now has the OPPOSITE default to version
   AzCopy 8.x. Specifically, as of version 10.0.9, MD5 hashes are NOT created by default. To create
   Content-MD5 hashs when uploading, you must now specify --put-md5 on the command line.

New features

1. Can migrate data directly from Amazon Web Services (AWS). In this high-performance data path
   the data is read directly from AWS by the Azure Storage service. It does not need to pass through
   the machine running AzCopy. The copy happens syncronously, so you can see its exact progress.
2. Can migrate data directly from Azure Files or Azure Blobs (any blob type) to Azure Blobs (any
   blob type). In this high-performance data path the data is read directly from the source by the
   Azure Storage service. It does not need to pass through the machine running AzCopy. The copy
   happens syncronously, so you can see its exact progress.
3. Sync command prompts with 4 options about deleting unneeded files from the target: Yes, No, All or
   None. (Deletion only happens if the --delete-destination flag is specified).
4. Can download to /dev/null. This throws the data away - but is useful for testing raw network
   performance unconstrained by disk; and also for validing MD5 hashes in bulk (when run in a cloud
   VM in the same region as the Storage account)

Bug fixes

1. Fixed memory leak when downloading large files
2. Fixed performance when downloading a single large file
3. Fixed bug with "too many open files" on Linux
4. Fixed memory leak when uploading sparse files (files with big blocks of zeros) to Page Blobs and
   Azure Files.
5. Fixed issue where application crashed after being throttled by Azure Storage Service. (The
   primary fix here is for Page Blobs, but a secondary part of the fix also helps with Block Blobs.)
6. Fixed functionality and usabilty issues with remove command
7. Improved performance for short-duration jobs (e.g. those lasting less than a minute)
8. Prevent unnecessary error message that sometimes appeared when cancelling a job
9. Various improvements to the online help and error messages.

AzCopyV10.0.8 Preview Release

• Rewrote sync command to eliminate numerous bugs and improve usability (see wiki for details)
• Implemented various improvements to memory management
• Added MD5 validation support (available options: NoCheck, LogOnly, FailIfDifferent, FailIfDifferentOrMissing)
• Added last modified time checks for source to guarantee transfer integrity
• Formalized outputs in JSON and elevated the output flag to the root level
• Eliminated outputs to STDERR (for new version notifications), which were causing problems for certain CI systems
• Improved log format for Windows
• Optimized plan file sizes
• Improved command line parameter names as follows (to be consistent with naming pattern of other parameters):
  • fromTo -> from-to
  • blobType -> blob-type
  • excludedBlobType -> excluded-blob-type
  • outputRaw (in "list" command) -> output
  • stdIn-enable (reserved for internal use) -> stdin-enable

AzCopyV10.0.7 Preview Release

• Fixed error detection when flushing and closing files at download time

AzCopyV10.0.6 Preview Release

• Refactor uploads for DRY and performance
• Sort jobs according to their start time
• Fix the internal integration with client_id for OAuth

AzCopyV10.0.5 Preview Release

• Optimized the memory usage through un-mapping unused memory correctly.
• Optimized the OAuth support for better user experience.
• Fixed the issue that data integrity could be influenced when uploading local file to Azure File.
• Fixed various bugs for ADLS Gen2.
• Refactored download code to not use memory-mapped file.

AzCopyV10.0.4 Preview Release

• Various bug fixes
• Improved sync command's performance
• Added env command to show possible configurations (environment variables)
• Improved content-type detection