Summary:
resolve #3347

`position_bias` is ignored in `extract_features` method, this doesn't affect Wav2Vec2 or HuBERT models, but it changes the output of transformer layers (except the first layer) in WavLM model. This PR fixes it by adding `position_bias` to the method.

Pull Request resolved: #3350

Reviewed By: mthrok

Differential Revision: D46112148

Pulled By: nateanl

fbshipit-source-id: 3d21aa4b32b22da437b440097fd9b00238152596

Summary: Pull Request resolved: #3366

Reviewed By: nateanl

Differential Revision: D46136238

Pulled By: mthrok

fbshipit-source-id: 3432f5d007293831bab21460a79ae26b1bbc81a8

Summary:
CC atalman malfet

Pull Request resolved: #3360

Reviewed By: mthrok

Differential Revision: D46150898

Pulled By: atalman

fbshipit-source-id: 985a0ef69406f48fb15f239d6b16616c0a5379f5

Summary:
This commit changes the way doc is pushed.
It ammends instead of adding a new commit.

Currently each commit in gh-pages contain like 100MB of data. gh-pages branch is fetched by default when `git clone`. So the size of torchaudio repo grows significantly.

Pull Request resolved: #3345

Reviewed By: nateanl

Differential Revision: D46136612

Pulled By: mthrok

fbshipit-source-id: 39479ee5d1a6888254ef50f0db252453d976d183

Summary:
* Delay the import of torchaudio until the CLI options are parsed.
* Add option to set log level to DEBUG so that it's easy to see the issue with external libraries.

Pull Request resolved: #3346

Reviewed By: nateanl

Differential Revision: D46022546

Pulled By: mthrok

fbshipit-source-id: 9f988bbd770c2fd2bb260c3cfe02b238a9da2808

Summary:
Follow-up #3045
- Revert the removal of HW acceleration doc
- comment out FFmpeg CLI test run

Pull Request resolved: #3349

Reviewed By: nateanl

Differential Revision: D46121899

Pulled By: mthrok

fbshipit-source-id: dfc030a69f05addec73637cfb6a720c184e37323

Summary: Pull Request resolved: #3367

Reviewed By: nateanl

Differential Revision: D46148139

Pulled By: mthrok

fbshipit-source-id: 50f297ac69bb95562976eb452e4e382b8c064c3c

Summary:
This PR adds AV-ASR recipe which contains sample implementations of training and evaluation pipelines for RNNT based automatic, visual, and audio-visual (ASR, VSR, AV-ASR) models on LRS3. This repository includes both streaming/non-streaming modes.

CC stavros99 xiaohui-zhang YumengTao mthrok nateanl hwangjeff

Pull Request resolved: #3278

Reviewed By: nateanl

Differential Revision: D46121550

Pulled By: mpc001

fbshipit-source-id: bb44b97ae25e87df2a73a707008be46af4ad0fc6

Summary:
This commit fixes the following issues affecting streaming decoding quality
1. The `init_b` hypothesis is only regenerated from blank token if no initial hypotheses are provided.
2. Allows the decoder to receive top-K hypothesis to continue decoding from, instead of using just the top hypothesis at each decoding step.  This dramatically affects decoding quality especially for speech with long pauses and disfluencies.
3. Some minor errors regarding shape checking for length.

This also means that the resulting output is the entire transcript up until that time step, instead of just the incremental change in transcript.

Pull Request resolved: #3295

Reviewed By: nateanl

Differential Revision: D46216113

Pulled By: hwangjeff

fbshipit-source-id: 8f7efae28dcca4a052f434ca55a2795c9e5ec0b0

Summary:
This reverts commit d38a785.

This is temporary revert to unblock unit test migration from circleci to github

Pull Request resolved: #3377

Reviewed By: mthrok

Differential Revision: D46230498

Pulled By: atalman

fbshipit-source-id: 000d8a9ca00750fc1ca61f4c2cdd6e930a5ce46d

Summary:
The tests failed for several bundles. Remove them and will re-add once the root cause is figured out.

Pull Request resolved: #3378

Reviewed By: atalman

Differential Revision: D46230884

Pulled By: nateanl

fbshipit-source-id: 42056a29b2ec2335268b273d3e37fb517035be92

Summary:
Use cuda 11.8 for circleci tests.
11.7 was deprecated

Pull Request resolved: #3381

Reviewed By: osalpekar

Differential Revision: D46236223

Pulled By: atalman

fbshipit-source-id: 6d6a8e09603807a07241f31c1bd1e6d3a2b67d9d

Summary:
11.7 uses 8.5.0; 11.8 uses 8.7.0; 12.1 uses 8.8.1.  Otherwise, Windows vision job (8.5.0) would overwrite the CUDNN version setup by PyTorch (8.7.0) leading to this flaky failures https://github.com/pytorch/pytorch/actions/runs/5088860652/jobs/9146641450

```
RuntimeError: cuDNN version incompatibility: PyTorch was compiled  against (8, 7, 0) but found runtime version (8, 5, 0). PyTorch already comes bundled with cuDNN. One option to resolving this error is to ensure PyTorch can find the bundled cuDNN.
```

Pull Request resolved: #3380

Reviewed By: atalman

Differential Revision: D46236286

Pulled By: huydhn

fbshipit-source-id: 9ca12d5068c3029688347d52c5c284488f33728d

Summary:
g722 format only supports 16k Hz, but AVCodec does not list this. The implementation does not insert resampling and the resulting audio can be slowed down or sped up.

Pull Request resolved: #3373

Reviewed By: hwangjeff

Differential Revision: D46233181

Pulled By: mthrok

fbshipit-source-id: 902b3f862a8f7269dc35bc871e868b0e78326c6c

Summary:
When encoding audio with mulaw, the resulting data does not have header, and the StreamReader defaults to 16k Hz, which can strech/shrink the resulting waveform.

Pull Request resolved: #3372

Reviewed By: hwangjeff

Differential Revision: D46234772

Pulled By: mthrok

fbshipit-source-id: 942c89a8cfe29b0b6f57b3e5b6c9dfd3524ca552

Summary:
Continuing with the job migrations from CCI to Nova, this PR introduces the Windows CPU Unittest job as a Nova workflow.

The job is passing: https://github.com/pytorch/audio/actions/runs/5094569687/jobs/9159020192?pr=3329.

Pull Request resolved: #3329

Reviewed By: huydhn

Differential Revision: D46265649

Pulled By: atalman

fbshipit-source-id: 7659dfbcc8ad400f2e109ff64530e1f768e82ef9

Summary:
Pull Request resolved: #3383

This commit reduces `torchaudio::sox_*` namespace into `torchaudio::sox`.
Also put Pybind11 registration and TorchBind registration into anonymous namescope.

Differential Revision: D46257367

fbshipit-source-id: 0f0f181eaa72036916e223263daf4b7c298fca0d

Summary:
Pull Request resolved: #3389

Adopt more of const reference in sox source code.

Differential Revision: D46264068

fbshipit-source-id: 809d34a6e16f621c856d4278ef7ce45a5868a717

Summary:
Disable failing GPU unit test.
See associated issue: #3376

Pull Request resolved: #3384

Reviewed By: mthrok

Differential Revision: D46279324

Pulled By: atalman

fbshipit-source-id: 3a606bb992e0261451f48d1fb458e054f7fd5583

Summary:
Pull Request resolved: #3379

Fixes `RNNTBeamSearch.infer`'s docstring and removes unused import from tutorial.

Reviewed By: mthrok

Differential Revision: D46227174

fbshipit-source-id: 7c1c3f05a6476cb0437622dea6f3ae6cb3ea9468

Summary:
Windows GPU workflows

Pull Request resolved: #3364

Reviewed By: mthrok

Differential Revision: D46292403

Pulled By: atalman

fbshipit-source-id: ee3c6f8082ca77bdc1ffdb930c59fa5a9cb25a4a

Summary:
Nova - Deprecate windows circleci unit tests

Pull Request resolved: #3393

Reviewed By: malfet

Differential Revision: D46315608

Pulled By: atalman

fbshipit-source-id: 3d7b5d0618b9d2e12e5f97e21d7becdc61d85c69

Summary:
Set the directory of JUnitText XML file to the one where test-infra picks up and put them in summary.

Example: https://github.com/pytorch/audio/actions/runs/5136305988

Pull Request resolved: #3394

Differential Revision: D46328832

Pulled By: mthrok

fbshipit-source-id: f0b5020a911ca4ec09345a965bdec769300859f0

Summary:
See title. If all is well, we can deprecate the CCI job in a few days.

Pull Request resolved: #3341

Reviewed By: mthrok

Differential Revision: D46324265

Pulled By: osalpekar

fbshipit-source-id: bc706c6ae4285d4085dc5f0223ea41d8fc290f1c

Summary:
Introducing the stylecheck job on Nova. It seems like it is failing on trunk, but the functionality of this job itself is working and it fails with the same error as it does on trunk with CCI.

Pull Request resolved: #3390

Reviewed By: mthrok

Differential Revision: D46324223

Pulled By: osalpekar

fbshipit-source-id: 1324202e53569d610559ef6f1b90cb5c364e6909

Summary:
Deprecates the Linux and MacOS Unittest jobs now that they've been running on Nova for over a week.

Aside: There was also a stylecheck job that was dependent on the Linux Unittest job. I also put up #3390 to move that stylecheck job to Nova. I'm happy to reintroduce the CCI stylecheck job standalone in CCI if we want the Nova version to run on main for a week.

Pull Request resolved: #3391

Reviewed By: mthrok

Differential Revision: D46324198

Pulled By: osalpekar

fbshipit-source-id: 2115748e153c5dee1a38db2b6230acebc4f56927

Summary:
To prepare for the upcoming removal of file-like object support from sox_io backend,
this commit changes apply_codec function to use tempfile.

`apply_codec` function is now deprecated and users are encourated to use `torchaudio.io.AudioEffector`.
We will not remove the function itself, but will remove the entry from the doc.

Pull Request resolved: #3386

Reviewed By: hwangjeff

Differential Revision: D46330610

Pulled By: mthrok

fbshipit-source-id: 3071bdefa05b4cbb9f00629bef50f0981eae89b4

Summary:
The arguments of TorchAudio's save function ("format", "bits_per_sample" and "encoding")
are not one-to-one mapping to the arguments of FFmpeg encoding.

For example, to use vorbis codec, FFmpeg expects "ogg" container/extension with "vorbis"
encoder. It does not recognize "vorbis" extension like TorchAudio (libsox) does.

This commit refactors the logic to parse/map the arguments.

As a result it now properly works with vorbis and mp3 extension.

Pull Request resolved: #3387

Reviewed By: hwangjeff

Differential Revision: D46328787

Pulled By: mthrok

fbshipit-source-id: 36f993952a062bfec58a8b51be6aa86297571f90

Summary:
Follow-up #3386 The intended change was to use path of temporary file, instead of file-like object

Pull Request resolved: #3397

Reviewed By: hwangjeff

Differential Revision: D46346189

Pulled By: mthrok

fbshipit-source-id: 44da799c6587bcb63a118a6313b7299bad742a40

Summary: Pull Request resolved: #3398

Reviewed By: nateanl

Differential Revision: D46354862

Pulled By: mthrok

fbshipit-source-id: b86dcdfeff8ed9db87b0b78eca20f6f18117e97e

Summary:
This job completely deletes the CircleCI `config.yml`. Here is what was remaining in the config at the point of deletion:

Used Jobs:
* **Lint** - Now running on Nova - see https://github.com/pytorch/audio/actions/runs/5144082942 for an example run on the latest PR in trunk
* **CircleCI Consistency** - Not needed anymore now if there is no CCI config.

Unused Jobs:
* **build-ffmpeg-$OS** - For the build jobs, we are already building FFMPEG from source as part of the Nova workflows.
* **download-third-parties** - This is caching. We currently do not have caching in Nova jobs, but atalman is working on adding support for this as a future optimization.

Pull Request resolved: #3399

Reviewed By: mthrok

Differential Revision: D46363921

Pulled By: osalpekar

fbshipit-source-id: 8abf5b0c1612c3492908fb2f5797e6b0a3c70766

)

Summary:
This commit removes file-like obejct support so that we can remove custom patch

The motivation and plan is outlined in #2950.

Pull Request resolved: #3035

Reviewed By: hwangjeff

Differential Revision: D44695647

Pulled By: mthrok

fbshipit-source-id: 13af0234e288c041bc7b490e1f967f85ce7eb8ec

Summary:
This commit changes the way FFmpeg extension is built and used.
Instead of linking (LGPL) FFmpeg libraries to torchaudio at build time,
It uses dlopen to search and link them at run time.

For dlopen-ing, we use PyTorch's `at::DynamicLibrary` class, which provides
portable wrapper.

Pull Request resolved: #3353

Differential Revision: D46059199

Pulled By: mthrok

fbshipit-source-id: 4493a5fd8a4c802178d20276522f5334d637307d

Differential Revision:
D46059199

Original commit changeset: 4493a5fd8a4c

Original Phabricator Diff: D46059199

fbshipit-source-id: 71cde3f8cd870d1ad9114e3e87cdd1ba564441c0

Summary:
Replace sox_effects with `torchaudio.io.AudioEffector`

1. To show case the new and better feature
2. To prepare for the upcoming removal of file-like support object

Pull Request resolved: #3375

Reviewed By: nateanl

Differential Revision: D46379016

Pulled By: mthrok

fbshipit-source-id: 70f24b62494204949f327f6ac6c49f315c9ee315

Summary:
This commit removes compute_kaldi_pitch function and the underlying Kaldi integration from torchaudio.

Kaldi pitch function was added in a short period of time by integrating the original Kaldi implementation, instead of reimplementing it in PyTorch.

The Kaldi integration employed a hack which replaces the base vector/matrix implementation of Kaldi with PyTorch Tensor so that there is only one blas library within torchaudio.

Recently, we are making torchaudio more lean, and we don't see a wide adoption of kaldi_pitch feature, so we decided to remove them.

See some of the discussion #1269

Pull Request resolved: #3368

Differential Revision: D46406176

Pulled By: mthrok

fbshipit-source-id: ee5e24d825188f379979ddccd680c7323b119b1e

Summary:
Pull Request resolved: #3402

This is a second attempt of #3353.

The basic logic to enable dlopen for FFmpeg libraries are same.
It uses `at::DynamicLibrary`, which allows to compile torchaudio without
linking FFmpeg libraries.

This time, the option to enable this feature DLOPEN_FFMPEG has been added,
so that users have a way to disable this feature and keep using build-time
linking.

Please refer to stub.h for more technical detail.

Differential Revision: D46403783

fbshipit-source-id: ca3db57ff6bdc50c8c225d22f12f3e76c6dc3f16

Summary:
There are some BC-Breaking changes from pytorch_lightning to lightning library. The PR adjust those changes to support latest lightning library.

Pull Request resolved: #3396

Reviewed By: mthrok

Differential Revision: D46345206

Pulled By: nateanl

fbshipit-source-id: 59469c15dc5fe5466a99a5b5380eb4f98c2c633f

Summary:
Follow up of: #3368

Remove files and lines no longer used.

Pull Request resolved: #3403

Differential Revision: D46441462

Pulled By: mthrok

fbshipit-source-id: 11b881ec4b24fa0d625c6aee9f4bd91f637f9923

Summary:
Pull Request resolved: #3365

Current design of forced_align accept 2D Tensor for `log_probs` and 1D Tensor for `targets`. To make the API simple, the PR make changes to only support batch Tensors (3D Tensor for `log_probs` and 2D Tensor for `targets`).

Reviewed By: vineelpratap

Differential Revision: D46126226

fbshipit-source-id: 42cb52b19d91bbff7dc040ccf60350545d75b3a2

…nsors

Differential Revision:
D46126226

Original commit changeset: 42cb52b19d91

Original Phabricator Diff: D46126226

fbshipit-source-id: 372b2526d9e196e37e014f1556bf117d29bb1ac6

Summary:
We are using Project Nova workflows now. These are not required.

Same as: pytorch/vision#7656

Pull Request resolved: #3409

Reviewed By: mthrok

Differential Revision: D46494331

Pulled By: atalman

fbshipit-source-id: a642ae55b75482918e0afb7c55dc876bc8356e70

Summary: Pull Request resolved: #3410

Differential Revision: D46496786

Pulled By: mthrok

fbshipit-source-id: e517b273c40b340f39ce7db7ab1be1c3eb5f2059

Summary:
To investigate #3411

Pull Request resolved: #3418

Differential Revision: D46535891

Pulled By: mthrok

fbshipit-source-id: b90bba399eb54f9f0ae073bd590cd8a46054ed7e

Summary: Pull Request resolved: #3415

Differential Revision: D46526437

Pulled By: mthrok

fbshipit-source-id: f78d19c19d7e68f67712412de35d9ed50f47263b

Summary:
[Nova] Add cache ffmpeg before building

Pull Request resolved: #3417

Reviewed By: mthrok

Differential Revision: D46537892

Pulled By: atalman

fbshipit-source-id: 9f8dc0ecfc305c3b378557d46f89a5d7de67a165

Summary: Pull Request resolved: #3414

Differential Revision: D46536717

Pulled By: mthrok

fbshipit-source-id: 505bdcdd1b59ca9fe5afc2c8516a0a821e2b8d7e

Summary:
Pull Request resolved: #3382

The voice activity detector function was unoptimized, confusingly written, and buggy.

The optimizations created here allow for the function to run roughly 17x faster.
The main optimizations were to loop over windows of audio rather than individual audio samples. Reducing the number of copies also helped.

There was an off by one error where the array slice referenced was [1: 16001] (for the default settings) instead of [0: 16000]

Reviewed By: hwangjeff

Differential Revision: D44749359

fbshipit-source-id: c76c9412e70cdc6fcd527d113603c88f78480558

Summary:
- Moving the unit test scripts from .circleci to .github
- Remove docker file for unit test base
- Use the Conda from Docker image in Linux jobs.

Remaining follow-up items

- Reuse the unittest script in Linux GPU job like done in Linux CPU job.

The unit test script needs to be fixed to be used for Linux GPU job
in new GHA workflow. Keeping it as a separate follow-up work item.

Pull Request resolved: #3407

Differential Revision: D46498263

Pulled By: mthrok

fbshipit-source-id: d8256717a55bb4257151d819d3b2ebd453601eac

Summary:
CI jobs are migrated from CCI to GHA

Pull Request resolved: #3420

Differential Revision: D46548562

Pulled By: mthrok

fbshipit-source-id: d7e17201e8b256efaa54543e445a0f139aa549b2

Summary:
StreamReader decoding process is composed of the three steps;

1. Decode the incoming AVPacket into AVFrame
2. Pass AVFrame through AVFilter to perform post process
3. Convert the resulgint AVFrame

The internal of StreamReader was refactored in #3188 so that the above pipeline is initialized at the time output stream is defined and output stream shape can be retrieved.

For CPU decoder, this works fine because resizing happens in step 2, and the resulting shape can be retrievable.
However, this is problematic for GPU decoder, as resizing is currently done using GPU decoder option (step 1) and there seems to be no interface to retrieve the output shape. This refactor introduced regression, which is described in #3405

AVFilter internally is adoptive to the change of input frame size. This commit changes the conversion process to be similar, so that it will wait until the first frame comes in to finalize the frame shape.

Fix #3405

Pull Request resolved: #3419

Differential Revision: D46557505

Pulled By: mthrok

fbshipit-source-id: 46ad2d82c8c30f368ebfbaf6947718a5036c7dc6

Summary:
[Nova] Add cache ffmpeg before building - 2
Follow up after #3417, need to pass new arguments to test-infra workflows

Pull Request resolved: #3423

Reviewed By: mthrok

Differential Revision: D46559344

Pulled By: atalman

fbshipit-source-id: fa5cccc3bfb052688de4a05cc3b4f37fcbe3a6f5

Summary:
Pull Request resolved: #3395

Adds chroma filter bank function `chroma_filterbank` to `torchaudio.prototype.functional`.

Reviewed By: mthrok

Differential Revision: D46307672

fbshipit-source-id: c5d8104a8bb03da70d0629b5cc224e0d897148d5

Summary:
StreamWriter's encoding pipeline looks like the following

1. convert tensor to AVFrame
2. pass AVFrame to AVFilter
3. pass the resulting AVFrame to AVCodecContext (encoder) and AVFormatContext (muxer)

When dealing with CUDA tensor, the AVFilter becomes no-op, as we have not added support for CUDA-compatible filters.

When CUDA frame is passed, the existing solution passes the software pixel format to AVFilter, which issues warning later as what AVFilter sees is AV_PIX_FMT_CUDA.

Since the filter itself is no-op, it functions as expected. But this commit fixes it.

See #3317

Pull Request resolved: #3426

Differential Revision: D46562370

Pulled By: mthrok

fbshipit-source-id: ce0131f1e50bcc826ee036fc0f35db2a5162b660

Summary:
The new version of transformers changed the format of pre-trained weight. Fixing it is low-priority for the maintanance team so we disable the test.

See #3430

Pull Request resolved: #3431

Differential Revision: D46592883

Pulled By: mthrok

fbshipit-source-id: d8f54a281a92cac60c469c48f95345bcf0e959d6

Summary: Pull Request resolved: #3422

Differential Revision: D46558184

Pulled By: mthrok

fbshipit-source-id: a775c4fb193496d9b2bf9db7bee186ee23512b99

Summary:
Should resolve #3425

cc mthrok

Pull Request resolved: #3432

Differential Revision: D46656180

Pulled By: mthrok

fbshipit-source-id: 5c534bee2f143ef5cb5e50ec74828012dbcab7e9

Summary: Pull Request resolved: #3435

Reviewed By: nateanl

Differential Revision: D46659362

Pulled By: mthrok

fbshipit-source-id: ffa033ad6759de6fd958b63ac51a4a1153ffb45d

Summary:
This plugin pulls glib and gtk which breaks the build on some headless systems

Since the plugin is not actually used, it seems right to disable it

This change fixed the build on my system

Pull Request resolved: #3436

Differential Revision: D46683297

Pulled By: mthrok

fbshipit-source-id: 5b1c1eee1929f4a69a1cc6c7d7bb3ed998ec5872

Summary:
Currently, AudioEffector always resample to the original sample rate. It is more flexible to allow overriding this to any sample rate.

Pull Request resolved: #3374

Differential Revision: D46235358

Pulled By: mthrok

fbshipit-source-id: 39a5d4e38d9b90380da31d0ce9ee8090668b54e4

Summary:
* Fix backtrack visualization (the cooridnate was off-by-one.)
* Add note about the simplification and the new align API
* Explicitly handle SOS and EOS

Pull Request resolved: #3440

Reviewed By: xiaohui-zhang

Differential Revision: D46761282

Pulled By: mthrok

fbshipit-source-id: b0b6c9754674e8e23543e9f002e29b55102c92f8

Summary:
This PR adds a data preparation recipe that uses the ultra face detector to extract full-face video. The resulting video output is then used as input for training and evaluating RNNT-based models for automatic speech recognition (ASR), visual speech recognition (VSR), and audio-visual ASR (AV-ASR) on the LRS3 dataset.

This PR also updates the word error rate (WER) for AV-ASR LRS3 models and improves the code readability.

Pull Request resolved: #3421

Reviewed By: mpc001

Differential Revision: D46799748

Pulled By: mthrok

fbshipit-source-id: 97af3feac0592b240617faaffa4c0ac8cef614a9

Summary:
Splitting the multilingual example part into another tutorial.

Pull Request resolved: #3443

Reviewed By: mthrok

Differential Revision: D46802844

Pulled By: xiaohui-zhang

fbshipit-source-id: a7093053cac8b79d650d4f665db7fde2d8254998

Summary:
Pull Request resolved: #3427

Adds transform `ChromaSpectrogram` for generating chromagrams from waveforms as well as transform `ChromaScale` for generating chromagrams from linear-frequency spectrograms.

Reviewed By: mthrok

Differential Revision: D46547418

fbshipit-source-id: 250f298b8e11d8cf82f05536c29d51cf8d77a960

Summary: Pull Request resolved: #3442

Differential Revision: D46797481

Pulled By: mthrok

fbshipit-source-id: 3513037cbb8f2edb70fdab0fec5c7c554a697abe

Summary: Pull Request resolved: #3449

Differential Revision: D47094402

Pulled By: mthrok

fbshipit-source-id: 43e6994604f0e6c06a5f19c5e8599e2ce12ae622

Summary:
Include Conformer/Emformer RNN-T ASR/VSR/AV-ASR link to index.rst

Pull Request resolved: #3441

Differential Revision: D47094158

Pulled By: mthrok

fbshipit-source-id: 9ab42ac2bf52a5ce488003897ffba2f10a6ca941

Summary:
Pull Request resolved: #3434

Add one bullet point for `torchaudio.functional` and forced alignment as one example.

Reviewed By: mthrok

Differential Revision: D46658058

fbshipit-source-id: 6e037b7bb6ed2fc2e27ad1e55c5728c17ce69ce8

Summary:
Pull Request resolved: #3433

Current design of forced_align accept 2D Tensor for `log_probs` and 1D Tensor for `targets`. To make the API simple, the PR make changes to only support batch Tensors (3D Tensor for `log_probs` and 2D Tensor for `targets`).

Reviewed By: mthrok

Differential Revision: D46657526

fbshipit-source-id: af17ec3f92f1a2c46dba91c6db2488a11de36f89

Summary: Pull Request resolved: #3457

Differential Revision: D47241343

Pulled By: mthrok

fbshipit-source-id: fd1bfd1531397cb59e9cf11de9dede6949f8517e

Summary: Pull Request resolved: #3455

Differential Revision: D47242316

Pulled By: mthrok

fbshipit-source-id: 0eb4bdb0a45fccfe9ff97eaed79db63cd7bfc7d8

…3456)

Summary:
This reverts commit b7d3e89.

We will use pre-built binaries instead of dlopen.

Pull Request resolved: #3456

Differential Revision: D47239681

Pulled By: mthrok

fbshipit-source-id: 0446a62410d914081184fc20c386afa00b1e41b6

Summary:
Follow up of  #3455

FFMPEG_VERSION env ver is not defined in existing CI jobs.

Pull Request resolved: #3459

Reviewed By: atalman

Differential Revision: D47249074

Pulled By: mthrok

fbshipit-source-id: 20f82d749adef5f45a984ab8125592ef36279e94

Summary: Pull Request resolved: #3462

Differential Revision: D47270241

Pulled By: mthrok

fbshipit-source-id: 6a3b02380dfb381ffb47c1f46b46f4833c765246

Summary:
This commit changes the way FFmpeg extension is built.

Originally, the build process expected the FFmpeg binaries to be somehow available in build env.
This makes the build process unpredictable and prevents default enabling FFmpeg extension.

The proposed change uses pre-built FFmpeg binaries as build-time only scaffold, which are built in our CI job https://github.com/pytorch/audio/actions/workflows/ffmpeg.yml.

This makes the build process more predictable and removes the necessity to build FFmpeg in our CI.
Currently, it supports macOS (arm64, x86_64), unix (x86_64, aarch64) and windows (amd64).
The downside is that it no longer works with the architecture not listed above.
We can potentially workaround by searching the FFmpeg binaries available in system (the old way) for
these system, but since they are not supported by PyTorch, the priority is low.

Pull Request resolved: #3460

Differential Revision: D47261885

Pulled By: mthrok

fbshipit-source-id: 223a15e95c9140c95688af968beb35ff40354476

Summary:
- Add RGB0/BGR0 support to CPU encoder
- Allow to pass RGB/BGR when expectged format is RGB0/BGR0

Pull Request resolved: #3428

Differential Revision: D47274370

Pulled By: mthrok

fbshipit-source-id: d34d940e04b07673bb86f518fe895c0735912444

Summary:
Similrt to #2949

Pull Request resolved: #3370

Differential Revision: D47298746

Pulled By: mthrok

fbshipit-source-id: 0cc0f395772b33f8b2f5f55253d659e451f506c4

Summary:
1. Update smoke test script to change directory so that there is no `torchaudio` directory in CWD when smoke test is being executed.
2. Disable the part of smoke test which requires FFmpeg for wheel. The preparation for pytorch/test-infra#4358

Pull Request resolved: #3465

Reviewed By: nateanl

Differential Revision: D47345117

Pulled By: mthrok

fbshipit-source-id: 95aad0a22922d44ee9a24a05d9ece85166b8c17e

Summary:
Now that we do not build FFmpeg as part of CI build process, we can remove the pre/post build scripts.

Needs to land after pytorch/test-infra#4358

Pull Request resolved: #3466

Reviewed By: atalman

Differential Revision: D47367022

Pulled By: mthrok

fbshipit-source-id: 17aafff74ee7d269236cffb8a88c803a8d4c44b7

Summary: Pull Request resolved: #3469

Differential Revision: D47368140

Pulled By: mthrok

fbshipit-source-id: d82ddb91ae1f6612298486fb8401f95c48db5620

Summary: Pull Request resolved: #3468

Differential Revision: D47368070

Pulled By: mthrok

fbshipit-source-id: 9b5d57b0cb861a2556a1903121f526f8011a0e2d

Summary: Pull Request resolved: #3470

Differential Revision: D47374347

Pulled By: mthrok

fbshipit-source-id: 003b83e50a70f6e1d06eb196f0be5dbba1640226

Summary:
This commit introduces support for multiple FFmpeg versions for OSS binary distributions.

Currently torchaudio only works with FFmpeg 4. This is inconvenient from installing to runtime linking.
This commit allows to pick FFmpeg 4, 5 or 6 at runtime, instead of just looking for v4.

The way it works is that we compile the FFmpeg extension three times with different FFmpeg and ship them.
At runtime, we look for libavutil of specific version and when one is found, load the corresponding FFmpeg extension.
The order of preference is 6, 5, then 4.

To make the build process simple and reproducible, we use pre-built binaries of FFmpeg during the build.
They are LGPL and downloaded from S3 at build time, instead of building every time.

The use of pre-built binaries as scaffolding limits the system that can build torchaudio, so it also introduces
single FFmpeg version support mode. setting FFMPEG_ROOT during the build will change the way binaries are built
so that it will only support one specific version of FFmpeg.

Pull Request resolved: #3464

Differential Revision: D47300223

Pulled By: mthrok

fbshipit-source-id: 560c7968315e4c8922afa11a4693f648c0356d04

Summary: Pull Request resolved: #3474

Differential Revision: D47398447

fbshipit-source-id: f77b685d54ddfc222b806475707d4a10239872f5

Summary: Pull Request resolved: #3475

Differential Revision: D47403772

Pulled By: mthrok

fbshipit-source-id: 5cdde521dbbbbf33856470a9dc79419b4a3a1683

)

Summary:
This is a port of adefossez/julius#17 for torchaudio.

Not sure if it's possible/desirable to add tests to test the functionality of ONNX exports, but I did a quick test on my machine to ensure this works. The logic is a bit simpler compared to the other PR because the torchaudio version does not support the additional flags available in julius.

Pull Request resolved: #3473

Differential Revision: D47401988

Pulled By: mthrok

fbshipit-source-id: 62fa1e4388923f6a62cef2c0f902a79ea179cec4

Summary:
- FFmpeg 6 deprecated attributes
- Guard CUDA specific functions not used in CPU builds

Pull Request resolved: #3471

Differential Revision: D47402174

Pulled By: mthrok

fbshipit-source-id: 00c0719ab1849b50c0b56b03d8fb38bc7aa74538

Differential Revision:
D47402174

Original commit changeset: 00c0719ab184

Original Phabricator Diff: D47402174

fbshipit-source-id: b1f6ea4cc3ecef3f72a87bf2f67bf9644c847546