Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

error #3

Closed
ZiMo-Chen opened this issue Dec 4, 2022 · 10 comments
Closed

error #3

ZiMo-Chen opened this issue Dec 4, 2022 · 10 comments

Comments

@ZiMo-Chen
Copy link

1670134491600
pyTorch version: 1.9.0+cu111
Is debug build: False
CUDA used to build PyTorch: 11.1

D-WSCL/wetectron/csrc/cuda/ROIPool_cuda.cu line=151 error=209 : no kernel image is available for execution on the device
@jinhseo
Copy link
Owner

jinhseo commented Dec 4, 2022

Such error message could be triggered if you didn't install apex properly.
Please follow the environment setup I provided.
It may different depending on your pytorch version and GPU capability, and even the pytorch version in your error message doesn't match what I suggested.

@ZiMo-Chen
Copy link
Author

Such error message could be triggered if you didn't install apex properly. Please follow the environment setup I provided. It may different depending on your pytorch version and GPU capability, and even the pytorch version in your error message doesn't match what I suggested.

PyTorch version: 1.7.1+cu110
Is debug build: False
CUDA used to build PyTorch: 11.0
Python version: 3.7 (64-bit runtime)
Is CUDA available: True

The problem remains

@jinhseo
Copy link
Owner

jinhseo commented Dec 4, 2022 •
edited
Loading

Do you want to share your log when you installing apex?
or check you got any error when you run:
pip install -v --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" ./
and
python setup.py build develop

@ZiMo-Chen
Copy link
Author

I use the code "pip install -v --no-cache-dir ./" can successfully installed

sing pip 22.2.1 from /home/cxq/.conda/envs/torch10/lib/python3.7/site-packages/pip (python 3.7)
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Processing /home/cxq/dp_work/objdetection/OD-WSCL/apex
Running command python setup.py egg_info

torch.version = 1.7.1+cu110

running egg_info
creating /tmp/pip-pip-egg-info-ku2tb53y/apex.egg-info
writing /tmp/pip-pip-egg-info-ku2tb53y/apex.egg-info/PKG-INFO
writing dependency_links to /tmp/pip-pip-egg-info-ku2tb53y/apex.egg-info/dependency_links.txt
writing requirements to /tmp/pip-pip-egg-info-ku2tb53y/apex.egg-info/requires.txt
writing top-level names to /tmp/pip-pip-egg-info-ku2tb53y/apex.egg-info/top_level.txt
writing manifest file '/tmp/pip-pip-egg-info-ku2tb53y/apex.egg-info/SOURCES.txt'
reading manifest file '/tmp/pip-pip-egg-info-ku2tb53y/apex.egg-info/SOURCES.txt'
writing manifest file '/tmp/pip-pip-egg-info-ku2tb53y/apex.egg-info/SOURCES.txt'
Preparing metadata (setup.py) ... done
Requirement already satisfied: packaging>20.6 in /home/cxq/.conda/envs/torch10/lib/python3.7/site-packages (from apex==0.1) (21.3)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /home/cxq/.conda/envs/torch10/lib/python3.7/site-packages (from packaging>20.6->apex==0.1) (3.0.9)
Building wheels for collected packages: apex
Running command python setup.py bdist_wheel

torch.version = 1.7.1+cu110

running bdist_wheel
running build
running build_py
installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
creating build/bdist.linux-x86_64/wheel
creating build/bdist.linux-x86_64/wheel/apex
creating build/bdist.linux-x86_64/wheel/apex/amp
copying build/lib/apex/amp/compat.py -> build/bdist.linux-x86_64/wheel/apex/amp
creating build/bdist.linux-x86_64/wheel/apex/amp/lists
copying build/lib/apex/amp/lists/init.py -> build/bdist.linux-x86_64/wheel/apex/amp/lists
copying build/lib/apex/amp/lists/tensor_overrides.py -> build/bdist.linux-x86_64/wheel/apex/amp/lists
copying build/lib/apex/amp/lists/torch_overrides.py -> build/bdist.linux-x86_64/wheel/apex/amp/lists
copying build/lib/apex/amp/lists/functional_overrides.py -> build/bdist.linux-x86_64/wheel/apex/amp/lists
copying build/lib/apex/amp/init.py -> build/bdist.linux-x86_64/wheel/apex/amp
copying build/lib/apex/amp/rnn_compat.py -> build/bdist.linux-x86_64/wheel/apex/amp
copying build/lib/apex/amp/wrap.py -> build/bdist.linux-x86_64/wheel/apex/amp
copying build/lib/apex/amp/_initialize.py -> build/bdist.linux-x86_64/wheel/apex/amp
copying build/lib/apex/amp/scaler.py -> build/bdist.linux-x86_64/wheel/apex/amp
copying build/lib/apex/amp/_process_optimizer.py -> build/bdist.linux-x86_64/wheel/apex/amp
copying build/lib/apex/amp/opt.py -> build/bdist.linux-x86_64/wheel/apex/amp
copying build/lib/apex/amp/amp.py -> build/bdist.linux-x86_64/wheel/apex/amp
copying build/lib/apex/amp/_amp_state.py -> build/bdist.linux-x86_64/wheel/apex/amp
copying build/lib/apex/amp/version.py -> build/bdist.linux-x86_64/wheel/apex/amp
copying build/lib/apex/amp/utils.py -> build/bdist.linux-x86_64/wheel/apex/amp
copying build/lib/apex/amp/handle.py -> build/bdist.linux-x86_64/wheel/apex/amp
copying build/lib/apex/amp/frontend.py -> build/bdist.linux-x86_64/wheel/apex/amp
copying build/lib/apex/_autocast_utils.py -> build/bdist.linux-x86_64/wheel/apex
creating build/bdist.linux-x86_64/wheel/apex/fp16_utils
copying build/lib/apex/fp16_utils/init.py -> build/bdist.linux-x86_64/wheel/apex/fp16_utils
copying build/lib/apex/fp16_utils/fp16util.py -> build/bdist.linux-x86_64/wheel/apex/fp16_utils
copying build/lib/apex/fp16_utils/loss_scaler.py -> build/bdist.linux-x86_64/wheel/apex/fp16_utils
copying build/lib/apex/fp16_utils/fp16_optimizer.py -> build/bdist.linux-x86_64/wheel/apex/fp16_utils
copying build/lib/apex/init.py -> build/bdist.linux-x86_64/wheel/apex
creating build/bdist.linux-x86_64/wheel/apex/normalization
copying build/lib/apex/normalization/init.py -> build/bdist.linux-x86_64/wheel/apex/normalization
copying build/lib/apex/normalization/fused_layer_norm.py -> build/bdist.linux-x86_64/wheel/apex/normalization
creating build/bdist.linux-x86_64/wheel/apex/RNN
copying build/lib/apex/RNN/cells.py -> build/bdist.linux-x86_64/wheel/apex/RNN
copying build/lib/apex/RNN/init.py -> build/bdist.linux-x86_64/wheel/apex/RNN
copying build/lib/apex/RNN/RNNBackend.py -> build/bdist.linux-x86_64/wheel/apex/RNN
copying build/lib/apex/RNN/models.py -> build/bdist.linux-x86_64/wheel/apex/RNN
creating build/bdist.linux-x86_64/wheel/apex/optimizers
copying build/lib/apex/optimizers/fused_adagrad.py -> build/bdist.linux-x86_64/wheel/apex/optimizers
copying build/lib/apex/optimizers/init.py -> build/bdist.linux-x86_64/wheel/apex/optimizers
copying build/lib/apex/optimizers/fused_lamb.py -> build/bdist.linux-x86_64/wheel/apex/optimizers
copying build/lib/apex/optimizers/fused_sgd.py -> build/bdist.linux-x86_64/wheel/apex/optimizers
copying build/lib/apex/optimizers/fused_novograd.py -> build/bdist.linux-x86_64/wheel/apex/optimizers
copying build/lib/apex/optimizers/fused_mixed_precision_lamb.py -> build/bdist.linux-x86_64/wheel/apex/optimizers
copying build/lib/apex/optimizers/fused_adam.py -> build/bdist.linux-x86_64/wheel/apex/optimizers
creating build/bdist.linux-x86_64/wheel/apex/parallel
copying build/lib/apex/parallel/init.py -> build/bdist.linux-x86_64/wheel/apex/parallel
copying build/lib/apex/parallel/optimized_sync_batchnorm.py -> build/bdist.linux-x86_64/wheel/apex/parallel
copying build/lib/apex/parallel/sync_batchnorm.py -> build/bdist.linux-x86_64/wheel/apex/parallel
copying build/lib/apex/parallel/distributed.py -> build/bdist.linux-x86_64/wheel/apex/parallel
copying build/lib/apex/parallel/multiproc.py -> build/bdist.linux-x86_64/wheel/apex/parallel
copying build/lib/apex/parallel/optimized_sync_batchnorm_kernel.py -> build/bdist.linux-x86_64/wheel/apex/parallel
copying build/lib/apex/parallel/sync_batchnorm_kernel.py -> build/bdist.linux-x86_64/wheel/apex/parallel
copying build/lib/apex/parallel/LARC.py -> build/bdist.linux-x86_64/wheel/apex/parallel
creating build/bdist.linux-x86_64/wheel/apex/contrib
creating build/bdist.linux-x86_64/wheel/apex/contrib/transducer
copying build/lib/apex/contrib/transducer/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/transducer
copying build/lib/apex/contrib/transducer/transducer.py -> build/bdist.linux-x86_64/wheel/apex/contrib/transducer
copying build/lib/apex/contrib/transducer/_transducer_ref.py -> build/bdist.linux-x86_64/wheel/apex/contrib/transducer
creating build/bdist.linux-x86_64/wheel/apex/contrib/xentropy
copying build/lib/apex/contrib/xentropy/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/xentropy
copying build/lib/apex/contrib/xentropy/softmax_xentropy.py -> build/bdist.linux-x86_64/wheel/apex/contrib/xentropy
copying build/lib/apex/contrib/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib
creating build/bdist.linux-x86_64/wheel/apex/contrib/clip_grad
copying build/lib/apex/contrib/clip_grad/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/clip_grad
copying build/lib/apex/contrib/clip_grad/clip_grad.py -> build/bdist.linux-x86_64/wheel/apex/contrib/clip_grad
creating build/bdist.linux-x86_64/wheel/apex/contrib/test
creating build/bdist.linux-x86_64/wheel/apex/contrib/test/transducer
copying build/lib/apex/contrib/test/transducer/test_transducer_joint.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/transducer
copying build/lib/apex/contrib/test/transducer/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/transducer
copying build/lib/apex/contrib/test/transducer/test_transducer_loss.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/transducer
creating build/bdist.linux-x86_64/wheel/apex/contrib/test/xentropy
copying build/lib/apex/contrib/test/xentropy/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/xentropy
copying build/lib/apex/contrib/test/xentropy/test_label_smoothing.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/xentropy
copying build/lib/apex/contrib/test/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test
creating build/bdist.linux-x86_64/wheel/apex/contrib/test/clip_grad
copying build/lib/apex/contrib/test/clip_grad/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/clip_grad
copying build/lib/apex/contrib/test/clip_grad/test_clip_grad.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/clip_grad
creating build/bdist.linux-x86_64/wheel/apex/contrib/test/optimizers
copying build/lib/apex/contrib/test/optimizers/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/optimizers
copying build/lib/apex/contrib/test/optimizers/test_dist_adam.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/optimizers
copying build/lib/apex/contrib/test/optimizers/test_distributed_fused_lamb.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/optimizers
creating build/bdist.linux-x86_64/wheel/apex/contrib/test/multihead_attn
copying build/lib/apex/contrib/test/multihead_attn/test_encdec_multihead_attn_norm_add.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/multihead_attn
copying build/lib/apex/contrib/test/multihead_attn/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/multihead_attn
copying build/lib/apex/contrib/test/multihead_attn/test_encdec_multihead_attn.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/multihead_attn
copying build/lib/apex/contrib/test/multihead_attn/test_mha_fused_softmax.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/multihead_attn
copying build/lib/apex/contrib/test/multihead_attn/test_self_multihead_attn_norm_add.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/multihead_attn
copying build/lib/apex/contrib/test/multihead_attn/test_self_multihead_attn.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/multihead_attn
copying build/lib/apex/contrib/test/multihead_attn/test_fast_self_multihead_attn_bias.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/multihead_attn
creating build/bdist.linux-x86_64/wheel/apex/contrib/test/index_mul_2d
copying build/lib/apex/contrib/test/index_mul_2d/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/index_mul_2d
copying build/lib/apex/contrib/test/index_mul_2d/test_index_mul_2d.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/index_mul_2d
creating build/bdist.linux-x86_64/wheel/apex/contrib/test/conv_bias_relu
copying build/lib/apex/contrib/test/conv_bias_relu/test_conv_bias_relu.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/conv_bias_relu
copying build/lib/apex/contrib/test/conv_bias_relu/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/conv_bias_relu
creating build/bdist.linux-x86_64/wheel/apex/contrib/test/bottleneck
copying build/lib/apex/contrib/test/bottleneck/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/bottleneck
copying build/lib/apex/contrib/test/bottleneck/test_bottleneck_module.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/bottleneck
creating build/bdist.linux-x86_64/wheel/apex/contrib/test/cudnn_gbn
copying build/lib/apex/contrib/test/cudnn_gbn/test_cudnn_gbn_with_two_gpus.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/cudnn_gbn
copying build/lib/apex/contrib/test/cudnn_gbn/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/cudnn_gbn
creating build/bdist.linux-x86_64/wheel/apex/contrib/test/layer_norm
copying build/lib/apex/contrib/test/layer_norm/test_fast_layer_norm.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/layer_norm
copying build/lib/apex/contrib/test/layer_norm/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/layer_norm
creating build/bdist.linux-x86_64/wheel/apex/contrib/test/fmha
copying build/lib/apex/contrib/test/fmha/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/fmha
copying build/lib/apex/contrib/test/fmha/test_fmha.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/fmha
creating build/bdist.linux-x86_64/wheel/apex/contrib/test/peer_memory
copying build/lib/apex/contrib/test/peer_memory/test_peer_halo_exchange_module.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/peer_memory
copying build/lib/apex/contrib/test/peer_memory/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/peer_memory
creating build/bdist.linux-x86_64/wheel/apex/contrib/test/focal_loss
copying build/lib/apex/contrib/test/focal_loss/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/focal_loss
copying build/lib/apex/contrib/test/focal_loss/test_focal_loss.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/focal_loss
creating build/bdist.linux-x86_64/wheel/apex/contrib/optimizers
copying build/lib/apex/contrib/optimizers/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/optimizers
copying build/lib/apex/contrib/optimizers/distributed_fused_adam.py -> build/bdist.linux-x86_64/wheel/apex/contrib/optimizers
copying build/lib/apex/contrib/optimizers/distributed_fused_lamb.py -> build/bdist.linux-x86_64/wheel/apex/contrib/optimizers
copying build/lib/apex/contrib/optimizers/fp16_optimizer.py -> build/bdist.linux-x86_64/wheel/apex/contrib/optimizers
copying build/lib/apex/contrib/optimizers/fused_lamb.py -> build/bdist.linux-x86_64/wheel/apex/contrib/optimizers
copying build/lib/apex/contrib/optimizers/fused_sgd.py -> build/bdist.linux-x86_64/wheel/apex/contrib/optimizers
copying build/lib/apex/contrib/optimizers/fused_adam.py -> build/bdist.linux-x86_64/wheel/apex/contrib/optimizers
creating build/bdist.linux-x86_64/wheel/apex/contrib/groupbn
copying build/lib/apex/contrib/groupbn/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/groupbn
copying build/lib/apex/contrib/groupbn/batch_norm.py -> build/bdist.linux-x86_64/wheel/apex/contrib/groupbn
creating build/bdist.linux-x86_64/wheel/apex/contrib/multihead_attn
copying build/lib/apex/contrib/multihead_attn/fast_encdec_multihead_attn_func.py -> build/bdist.linux-x86_64/wheel/apex/contrib/multihead_attn
copying build/lib/apex/contrib/multihead_attn/fast_self_multihead_attn_func.py -> build/bdist.linux-x86_64/wheel/apex/contrib/multihead_attn
copying build/lib/apex/contrib/multihead_attn/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/multihead_attn
copying build/lib/apex/contrib/multihead_attn/self_multihead_attn.py -> build/bdist.linux-x86_64/wheel/apex/contrib/multihead_attn
copying build/lib/apex/contrib/multihead_attn/encdec_multihead_attn.py -> build/bdist.linux-x86_64/wheel/apex/contrib/multihead_attn
copying build/lib/apex/contrib/multihead_attn/self_multihead_attn_func.py -> build/bdist.linux-x86_64/wheel/apex/contrib/multihead_attn
copying build/lib/apex/contrib/multihead_attn/mask_softmax_dropout_func.py -> build/bdist.linux-x86_64/wheel/apex/contrib/multihead_attn
copying build/lib/apex/contrib/multihead_attn/encdec_multihead_attn_func.py -> build/bdist.linux-x86_64/wheel/apex/contrib/multihead_attn
copying build/lib/apex/contrib/multihead_attn/fast_encdec_multihead_attn_norm_add_func.py -> build/bdist.linux-x86_64/wheel/apex/contrib/multihead_attn
copying build/lib/apex/contrib/multihead_attn/fast_self_multihead_attn_norm_add_func.py -> build/bdist.linux-x86_64/wheel/apex/contrib/multihead_attn
creating build/bdist.linux-x86_64/wheel/apex/contrib/index_mul_2d
copying build/lib/apex/contrib/index_mul_2d/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/index_mul_2d
copying build/lib/apex/contrib/index_mul_2d/index_mul_2d.py -> build/bdist.linux-x86_64/wheel/apex/contrib/index_mul_2d
creating build/bdist.linux-x86_64/wheel/apex/contrib/sparsity
copying build/lib/apex/contrib/sparsity/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/sparsity
copying build/lib/apex/contrib/sparsity/permutation_lib.py -> build/bdist.linux-x86_64/wheel/apex/contrib/sparsity
copying build/lib/apex/contrib/sparsity/sparse_masklib.py -> build/bdist.linux-x86_64/wheel/apex/contrib/sparsity
copying build/lib/apex/contrib/sparsity/asp.py -> build/bdist.linux-x86_64/wheel/apex/contrib/sparsity
creating build/bdist.linux-x86_64/wheel/apex/contrib/sparsity/permutation_search_kernels
copying build/lib/apex/contrib/sparsity/permutation_search_kernels/permutation_utilities.py -> build/bdist.linux-x86_64/wheel/apex/contrib/sparsity/permutation_search_kernels
copying build/lib/apex/contrib/sparsity/permutation_search_kernels/call_permutation_search_kernels.py -> build/bdist.linux-x86_64/wheel/apex/contrib/sparsity/permutation_search_kernels
copying build/lib/apex/contrib/sparsity/permutation_search_kernels/channel_swap.py -> build/bdist.linux-x86_64/wheel/apex/contrib/sparsity/permutation_search_kernels
copying build/lib/apex/contrib/sparsity/permutation_search_kernels/exhaustive_search.py -> build/bdist.linux-x86_64/wheel/apex/contrib/sparsity/permutation_search_kernels
copying build/lib/apex/contrib/sparsity/permutation_search_kernels/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/sparsity/permutation_search_kernels
creating build/bdist.linux-x86_64/wheel/apex/contrib/conv_bias_relu
copying build/lib/apex/contrib/conv_bias_relu/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/conv_bias_relu
copying build/lib/apex/contrib/conv_bias_relu/conv_bias_relu.py -> build/bdist.linux-x86_64/wheel/apex/contrib/conv_bias_relu
creating build/bdist.linux-x86_64/wheel/apex/contrib/bottleneck
copying build/lib/apex/contrib/bottleneck/bottleneck.py -> build/bdist.linux-x86_64/wheel/apex/contrib/bottleneck
copying build/lib/apex/contrib/bottleneck/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/bottleneck
copying build/lib/apex/contrib/bottleneck/test.py -> build/bdist.linux-x86_64/wheel/apex/contrib/bottleneck
copying build/lib/apex/contrib/bottleneck/halo_exchangers.py -> build/bdist.linux-x86_64/wheel/apex/contrib/bottleneck
creating build/bdist.linux-x86_64/wheel/apex/contrib/cudnn_gbn
copying build/lib/apex/contrib/cudnn_gbn/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/cudnn_gbn
copying build/lib/apex/contrib/cudnn_gbn/batch_norm.py -> build/bdist.linux-x86_64/wheel/apex/contrib/cudnn_gbn
creating build/bdist.linux-x86_64/wheel/apex/contrib/layer_norm
copying build/lib/apex/contrib/layer_norm/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/layer_norm
copying build/lib/apex/contrib/layer_norm/layer_norm.py -> build/bdist.linux-x86_64/wheel/apex/contrib/layer_norm
creating build/bdist.linux-x86_64/wheel/apex/contrib/fmha
copying build/lib/apex/contrib/fmha/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/fmha
copying build/lib/apex/contrib/fmha/fmha.py -> build/bdist.linux-x86_64/wheel/apex/contrib/fmha
creating build/bdist.linux-x86_64/wheel/apex/contrib/peer_memory
copying build/lib/apex/contrib/peer_memory/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/peer_memory
copying build/lib/apex/contrib/peer_memory/peer_halo_exchanger_1d.py -> build/bdist.linux-x86_64/wheel/apex/contrib/peer_memory
copying build/lib/apex/contrib/peer_memory/peer_memory.py -> build/bdist.linux-x86_64/wheel/apex/contrib/peer_memory
creating build/bdist.linux-x86_64/wheel/apex/contrib/focal_loss
copying build/lib/apex/contrib/focal_loss/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/focal_loss
copying build/lib/apex/contrib/focal_loss/focal_loss.py -> build/bdist.linux-x86_64/wheel/apex/contrib/focal_loss
creating build/bdist.linux-x86_64/wheel/apex/multi_tensor_apply
copying build/lib/apex/multi_tensor_apply/init.py -> build/bdist.linux-x86_64/wheel/apex/multi_tensor_apply
copying build/lib/apex/multi_tensor_apply/multi_tensor_apply.py -> build/bdist.linux-x86_64/wheel/apex/multi_tensor_apply
creating build/bdist.linux-x86_64/wheel/apex/transformer
creating build/bdist.linux-x86_64/wheel/apex/transformer/tensor_parallel
copying build/lib/apex/transformer/tensor_parallel/init.py -> build/bdist.linux-x86_64/wheel/apex/transformer/tensor_parallel
copying build/lib/apex/transformer/tensor_parallel/data.py -> build/bdist.linux-x86_64/wheel/apex/transformer/tensor_parallel
copying build/lib/apex/transformer/tensor_parallel/random.py -> build/bdist.linux-x86_64/wheel/apex/transformer/tensor_parallel
copying build/lib/apex/transformer/tensor_parallel/cross_entropy.py -> build/bdist.linux-x86_64/wheel/apex/transformer/tensor_parallel
copying build/lib/apex/transformer/tensor_parallel/mappings.py -> build/bdist.linux-x86_64/wheel/apex/transformer/tensor_parallel
copying build/lib/apex/transformer/tensor_parallel/layers.py -> build/bdist.linux-x86_64/wheel/apex/transformer/tensor_parallel
copying build/lib/apex/transformer/tensor_parallel/memory.py -> build/bdist.linux-x86_64/wheel/apex/transformer/tensor_parallel
copying build/lib/apex/transformer/tensor_parallel/utils.py -> build/bdist.linux-x86_64/wheel/apex/transformer/tensor_parallel
creating build/bdist.linux-x86_64/wheel/apex/transformer/amp
copying build/lib/apex/transformer/amp/init.py -> build/bdist.linux-x86_64/wheel/apex/transformer/amp
copying build/lib/apex/transformer/amp/grad_scaler.py -> build/bdist.linux-x86_64/wheel/apex/transformer/amp
creating build/bdist.linux-x86_64/wheel/apex/transformer/testing
copying build/lib/apex/transformer/testing/init.py -> build/bdist.linux-x86_64/wheel/apex/transformer/testing
copying build/lib/apex/transformer/testing/commons.py -> build/bdist.linux-x86_64/wheel/apex/transformer/testing
copying build/lib/apex/transformer/testing/global_vars.py -> build/bdist.linux-x86_64/wheel/apex/transformer/testing
copying build/lib/apex/transformer/testing/standalone_gpt.py -> build/bdist.linux-x86_64/wheel/apex/transformer/testing
copying build/lib/apex/transformer/testing/standalone_bert.py -> build/bdist.linux-x86_64/wheel/apex/transformer/testing
copying build/lib/apex/transformer/testing/standalone_transformer_lm.py -> build/bdist.linux-x86_64/wheel/apex/transformer/testing
copying build/lib/apex/transformer/testing/arguments.py -> build/bdist.linux-x86_64/wheel/apex/transformer/testing
copying build/lib/apex/transformer/testing/distributed_test_base.py -> build/bdist.linux-x86_64/wheel/apex/transformer/testing
creating build/bdist.linux-x86_64/wheel/apex/transformer/_data
copying build/lib/apex/transformer/_data/_batchsampler.py -> build/bdist.linux-x86_64/wheel/apex/transformer/_data
copying build/lib/apex/transformer/_data/init.py -> build/bdist.linux-x86_64/wheel/apex/transformer/_data
copying build/lib/apex/transformer/_ucc_util.py -> build/bdist.linux-x86_64/wheel/apex/transformer
copying build/lib/apex/transformer/init.py -> build/bdist.linux-x86_64/wheel/apex/transformer
copying build/lib/apex/transformer/log_util.py -> build/bdist.linux-x86_64/wheel/apex/transformer
creating build/bdist.linux-x86_64/wheel/apex/transformer/pipeline_parallel
creating build/bdist.linux-x86_64/wheel/apex/transformer/pipeline_parallel/schedules
copying build/lib/apex/transformer/pipeline_parallel/schedules/fwd_bwd_pipelining_with_interleaving.py -> build/bdist.linux-x86_64/wheel/apex/transformer/pipeline_parallel/schedules
copying build/lib/apex/transformer/pipeline_parallel/schedules/init.py -> build/bdist.linux-x86_64/wheel/apex/transformer/pipeline_parallel/schedules
copying build/lib/apex/transformer/pipeline_parallel/schedules/fwd_bwd_pipelining_without_interleaving.py -> build/bdist.linux-x86_64/wheel/apex/transformer/pipeline_parallel/schedules
copying build/lib/apex/transformer/pipeline_parallel/schedules/fwd_bwd_no_pipelining.py -> build/bdist.linux-x86_64/wheel/apex/transformer/pipeline_parallel/schedules
copying build/lib/apex/transformer/pipeline_parallel/schedules/common.py -> build/bdist.linux-x86_64/wheel/apex/transformer/pipeline_parallel/schedules
copying build/lib/apex/transformer/pipeline_parallel/_timers.py -> build/bdist.linux-x86_64/wheel/apex/transformer/pipeline_parallel
copying build/lib/apex/transformer/pipeline_parallel/init.py -> build/bdist.linux-x86_64/wheel/apex/transformer/pipeline_parallel
copying build/lib/apex/transformer/pipeline_parallel/p2p_communication.py -> build/bdist.linux-x86_64/wheel/apex/transformer/pipeline_parallel
copying build/lib/apex/transformer/pipeline_parallel/utils.py -> build/bdist.linux-x86_64/wheel/apex/transformer/pipeline_parallel
copying build/lib/apex/transformer/microbatches.py -> build/bdist.linux-x86_64/wheel/apex/transformer
copying build/lib/apex/transformer/enums.py -> build/bdist.linux-x86_64/wheel/apex/transformer
creating build/bdist.linux-x86_64/wheel/apex/transformer/functional
copying build/lib/apex/transformer/functional/fused_softmax.py -> build/bdist.linux-x86_64/wheel/apex/transformer/functional
copying build/lib/apex/transformer/functional/init.py -> build/bdist.linux-x86_64/wheel/apex/transformer/functional
copying build/lib/apex/transformer/utils.py -> build/bdist.linux-x86_64/wheel/apex/transformer
creating build/bdist.linux-x86_64/wheel/apex/transformer/layers
copying build/lib/apex/transformer/layers/init.py -> build/bdist.linux-x86_64/wheel/apex/transformer/layers
copying build/lib/apex/transformer/layers/layer_norm.py -> build/bdist.linux-x86_64/wheel/apex/transformer/layers
copying build/lib/apex/transformer/parallel_state.py -> build/bdist.linux-x86_64/wheel/apex/transformer
creating build/bdist.linux-x86_64/wheel/apex/fused_dense
copying build/lib/apex/fused_dense/fused_dense.py -> build/bdist.linux-x86_64/wheel/apex/fused_dense
copying build/lib/apex/fused_dense/init.py -> build/bdist.linux-x86_64/wheel/apex/fused_dense
creating build/bdist.linux-x86_64/wheel/apex/mlp
copying build/lib/apex/mlp/mlp.py -> build/bdist.linux-x86_64/wheel/apex/mlp
copying build/lib/apex/mlp/init.py -> build/bdist.linux-x86_64/wheel/apex/mlp
running install_egg_info
running egg_info
writing apex.egg-info/PKG-INFO
writing dependency_links to apex.egg-info/dependency_links.txt
writing requirements to apex.egg-info/requires.txt
writing top-level names to apex.egg-info/top_level.txt
reading manifest file 'apex.egg-info/SOURCES.txt'
writing manifest file 'apex.egg-info/SOURCES.txt'
Copying apex.egg-info to build/bdist.linux-x86_64/wheel/apex-0.1-py3.7.egg-info
running install_scripts
adding license file "LICENSE" (matched pattern "LICEN[CS]E*")
creating build/bdist.linux-x86_64/wheel/apex-0.1.dist-info/WHEEL
creating '/tmp/pip-wheel-o4d_cpbt/apex-0.1-py3-none-any.whl' and adding 'build/bdist.linux-x86_64/wheel' to it
adding 'apex/init.py'
adding 'apex/_autocast_utils.py'
adding 'apex/RNN/RNNBackend.py'
adding 'apex/RNN/init.py'
adding 'apex/RNN/cells.py'
adding 'apex/RNN/models.py'
adding 'apex/amp/init.py'
adding 'apex/amp/version.py'
adding 'apex/amp/_amp_state.py'
adding 'apex/amp/_initialize.py'
adding 'apex/amp/_process_optimizer.py'
adding 'apex/amp/amp.py'
adding 'apex/amp/compat.py'
adding 'apex/amp/frontend.py'
adding 'apex/amp/handle.py'
adding 'apex/amp/opt.py'
adding 'apex/amp/rnn_compat.py'
adding 'apex/amp/scaler.py'
adding 'apex/amp/utils.py'
adding 'apex/amp/wrap.py'
adding 'apex/amp/lists/init.py'
adding 'apex/amp/lists/functional_overrides.py'
adding 'apex/amp/lists/tensor_overrides.py'
adding 'apex/amp/lists/torch_overrides.py'
adding 'apex/contrib/init.py'
adding 'apex/contrib/bottleneck/init.py'
adding 'apex/contrib/bottleneck/bottleneck.py'
adding 'apex/contrib/bottleneck/halo_exchangers.py'
adding 'apex/contrib/bottleneck/test.py'
adding 'apex/contrib/clip_grad/init.py'
adding 'apex/contrib/clip_grad/clip_grad.py'
adding 'apex/contrib/conv_bias_relu/init.py'
adding 'apex/contrib/conv_bias_relu/conv_bias_relu.py'
adding 'apex/contrib/cudnn_gbn/init.py'
adding 'apex/contrib/cudnn_gbn/batch_norm.py'
adding 'apex/contrib/fmha/init.py'
adding 'apex/contrib/fmha/fmha.py'
adding 'apex/contrib/focal_loss/init.py'
adding 'apex/contrib/focal_loss/focal_loss.py'
adding 'apex/contrib/groupbn/init.py'
adding 'apex/contrib/groupbn/batch_norm.py'
adding 'apex/contrib/index_mul_2d/init.py'
adding 'apex/contrib/index_mul_2d/index_mul_2d.py'
adding 'apex/contrib/layer_norm/init.py'
adding 'apex/contrib/layer_norm/layer_norm.py'
adding 'apex/contrib/multihead_attn/init.py'
adding 'apex/contrib/multihead_attn/encdec_multihead_attn.py'
adding 'apex/contrib/multihead_attn/encdec_multihead_attn_func.py'
adding 'apex/contrib/multihead_attn/fast_encdec_multihead_attn_func.py'
adding 'apex/contrib/multihead_attn/fast_encdec_multihead_attn_norm_add_func.py'
adding 'apex/contrib/multihead_attn/fast_self_multihead_attn_func.py'
adding 'apex/contrib/multihead_attn/fast_self_multihead_attn_norm_add_func.py'
adding 'apex/contrib/multihead_attn/mask_softmax_dropout_func.py'
adding 'apex/contrib/multihead_attn/self_multihead_attn.py'
adding 'apex/contrib/multihead_attn/self_multihead_attn_func.py'
adding 'apex/contrib/optimizers/init.py'
adding 'apex/contrib/optimizers/distributed_fused_adam.py'
adding 'apex/contrib/optimizers/distributed_fused_lamb.py'
adding 'apex/contrib/optimizers/fp16_optimizer.py'
adding 'apex/contrib/optimizers/fused_adam.py'
adding 'apex/contrib/optimizers/fused_lamb.py'
adding 'apex/contrib/optimizers/fused_sgd.py'
adding 'apex/contrib/peer_memory/init.py'
adding 'apex/contrib/peer_memory/peer_halo_exchanger_1d.py'
adding 'apex/contrib/peer_memory/peer_memory.py'
adding 'apex/contrib/sparsity/init.py'
adding 'apex/contrib/sparsity/asp.py'
adding 'apex/contrib/sparsity/permutation_lib.py'
adding 'apex/contrib/sparsity/sparse_masklib.py'
adding 'apex/contrib/sparsity/permutation_search_kernels/init.py'
adding 'apex/contrib/sparsity/permutation_search_kernels/call_permutation_search_kernels.py'
adding 'apex/contrib/sparsity/permutation_search_kernels/channel_swap.py'
adding 'apex/contrib/sparsity/permutation_search_kernels/exhaustive_search.py'
adding 'apex/contrib/sparsity/permutation_search_kernels/permutation_utilities.py'
adding 'apex/contrib/test/init.py'
adding 'apex/contrib/test/bottleneck/init.py'
adding 'apex/contrib/test/bottleneck/test_bottleneck_module.py'
adding 'apex/contrib/test/clip_grad/init.py'
adding 'apex/contrib/test/clip_grad/test_clip_grad.py'
adding 'apex/contrib/test/conv_bias_relu/init.py'
adding 'apex/contrib/test/conv_bias_relu/test_conv_bias_relu.py'
adding 'apex/contrib/test/cudnn_gbn/init.py'
adding 'apex/contrib/test/cudnn_gbn/test_cudnn_gbn_with_two_gpus.py'
adding 'apex/contrib/test/fmha/init.py'
adding 'apex/contrib/test/fmha/test_fmha.py'
adding 'apex/contrib/test/focal_loss/init.py'
adding 'apex/contrib/test/focal_loss/test_focal_loss.py'
adding 'apex/contrib/test/index_mul_2d/init.py'
adding 'apex/contrib/test/index_mul_2d/test_index_mul_2d.py'
adding 'apex/contrib/test/layer_norm/init.py'
adding 'apex/contrib/test/layer_norm/test_fast_layer_norm.py'
adding 'apex/contrib/test/multihead_attn/init.py'
adding 'apex/contrib/test/multihead_attn/test_encdec_multihead_attn.py'
adding 'apex/contrib/test/multihead_attn/test_encdec_multihead_attn_norm_add.py'
adding 'apex/contrib/test/multihead_attn/test_fast_self_multihead_attn_bias.py'
adding 'apex/contrib/test/multihead_attn/test_mha_fused_softmax.py'
adding 'apex/contrib/test/multihead_attn/test_self_multihead_attn.py'
adding 'apex/contrib/test/multihead_attn/test_self_multihead_attn_norm_add.py'
adding 'apex/contrib/test/optimizers/init.py'
adding 'apex/contrib/test/optimizers/test_dist_adam.py'
adding 'apex/contrib/test/optimizers/test_distributed_fused_lamb.py'
adding 'apex/contrib/test/peer_memory/init.py'
adding 'apex/contrib/test/peer_memory/test_peer_halo_exchange_module.py'
adding 'apex/contrib/test/transducer/init.py'
adding 'apex/contrib/test/transducer/test_transducer_joint.py'
adding 'apex/contrib/test/transducer/test_transducer_loss.py'
adding 'apex/contrib/test/xentropy/init.py'
adding 'apex/contrib/test/xentropy/test_label_smoothing.py'
adding 'apex/contrib/transducer/init.py'
adding 'apex/contrib/transducer/_transducer_ref.py'
adding 'apex/contrib/transducer/transducer.py'
adding 'apex/contrib/xentropy/init.py'
adding 'apex/contrib/xentropy/softmax_xentropy.py'
adding 'apex/fp16_utils/init.py'
adding 'apex/fp16_utils/fp16_optimizer.py'
adding 'apex/fp16_utils/fp16util.py'
adding 'apex/fp16_utils/loss_scaler.py'
adding 'apex/fused_dense/init.py'
adding 'apex/fused_dense/fused_dense.py'
adding 'apex/mlp/init.py'
adding 'apex/mlp/mlp.py'
adding 'apex/multi_tensor_apply/init.py'
adding 'apex/multi_tensor_apply/multi_tensor_apply.py'
adding 'apex/normalization/init.py'
adding 'apex/normalization/fused_layer_norm.py'
adding 'apex/optimizers/init.py'
adding 'apex/optimizers/fused_adagrad.py'
adding 'apex/optimizers/fused_adam.py'
adding 'apex/optimizers/fused_lamb.py'
adding 'apex/optimizers/fused_mixed_precision_lamb.py'
adding 'apex/optimizers/fused_novograd.py'
adding 'apex/optimizers/fused_sgd.py'
adding 'apex/parallel/LARC.py'
adding 'apex/parallel/init.py'
adding 'apex/parallel/distributed.py'
adding 'apex/parallel/multiproc.py'
adding 'apex/parallel/optimized_sync_batchnorm.py'
adding 'apex/parallel/optimized_sync_batchnorm_kernel.py'
adding 'apex/parallel/sync_batchnorm.py'
adding 'apex/parallel/sync_batchnorm_kernel.py'
adding 'apex/transformer/init.py'
adding 'apex/transformer/_ucc_util.py'
adding 'apex/transformer/enums.py'
adding 'apex/transformer/log_util.py'
adding 'apex/transformer/microbatches.py'
adding 'apex/transformer/parallel_state.py'
adding 'apex/transformer/utils.py'
adding 'apex/transformer/_data/init.py'
adding 'apex/transformer/_data/_batchsampler.py'
adding 'apex/transformer/amp/init.py'
adding 'apex/transformer/amp/grad_scaler.py'
adding 'apex/transformer/functional/init.py'
adding 'apex/transformer/functional/fused_softmax.py'
adding 'apex/transformer/layers/init.py'
adding 'apex/transformer/layers/layer_norm.py'
adding 'apex/transformer/pipeline_parallel/init.py'
adding 'apex/transformer/pipeline_parallel/_timers.py'
adding 'apex/transformer/pipeline_parallel/p2p_communication.py'
adding 'apex/transformer/pipeline_parallel/utils.py'
adding 'apex/transformer/pipeline_parallel/schedules/init.py'
adding 'apex/transformer/pipeline_parallel/schedules/common.py'
adding 'apex/transformer/pipeline_parallel/schedules/fwd_bwd_no_pipelining.py'
adding 'apex/transformer/pipeline_parallel/schedules/fwd_bwd_pipelining_with_interleaving.py'
adding 'apex/transformer/pipeline_parallel/schedules/fwd_bwd_pipelining_without_interleaving.py'
adding 'apex/transformer/tensor_parallel/init.py'
adding 'apex/transformer/tensor_parallel/cross_entropy.py'
adding 'apex/transformer/tensor_parallel/data.py'
adding 'apex/transformer/tensor_parallel/layers.py'
adding 'apex/transformer/tensor_parallel/mappings.py'
adding 'apex/transformer/tensor_parallel/memory.py'
adding 'apex/transformer/tensor_parallel/random.py'
adding 'apex/transformer/tensor_parallel/utils.py'
adding 'apex/transformer/testing/init.py'
adding 'apex/transformer/testing/arguments.py'
adding 'apex/transformer/testing/commons.py'
adding 'apex/transformer/testing/distributed_test_base.py'
adding 'apex/transformer/testing/global_vars.py'
adding 'apex/transformer/testing/standalone_bert.py'
adding 'apex/transformer/testing/standalone_gpt.py'
adding 'apex/transformer/testing/standalone_transformer_lm.py'
adding 'apex-0.1.dist-info/LICENSE'
adding 'apex-0.1.dist-info/METADATA'
adding 'apex-0.1.dist-info/WHEEL'
adding 'apex-0.1.dist-info/top_level.txt'
adding 'apex-0.1.dist-info/RECORD'
removing build/bdist.linux-x86_64/wheel
Building wheel for apex (setup.py) ... done
Created wheel for apex: filename=apex-0.1-py3-none-any.whl size=346778 sha256=bac9bdfa231d1f15122a9c3f292beee94df5bcf1c979b3240091d5c90b3f269c
Stored in directory: /tmp/pip-ephem-wheel-cache-6m7s92_k/wheels/0a/df/3d/18f4c8df382be1e9dc0b4f5c16593b2d47107867ca80e7a6f6
Successfully built apex
Installing collected packages: apex
Attempting uninstall: apex
Found existing installation: apex 0.1
Uninstalling apex-0.1:
Removing file or directory /home/cxq/.conda/envs/torch10/lib/python3.7/site-packages/apex-0.1.dist-info/
Removing file or directory /home/cxq/.conda/envs/torch10/lib/python3.7/site-packages/apex/
Successfully uninstalled apex-0.1
Successfully installed apex-0.1

@ZiMo-Chen
Copy link
Author

Do you want to share your log when you installing apex? or check you got any error when you run: pip install -v --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" ./ and python setup.py build develop

AttributeError: module 'torch.distributed' has no attribute '_all_gather_base'

@ShirleySyh
Copy link

I use the code "pip install -v --no-cache-dir ./" can successfully installed

sing pip 22.2.1 from /home/cxq/.conda/envs/torch10/lib/python3.7/site-packages/pip (python 3.7) Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple Processing /home/cxq/dp_work/objdetection/OD-WSCL/apex Running command python setup.py egg_info

torch.version = 1.7.1+cu110

running egg_info creating /tmp/pip-pip-egg-info-ku2tb53y/apex.egg-info writing /tmp/pip-pip-egg-info-ku2tb53y/apex.egg-info/PKG-INFO writing dependency_links to /tmp/pip-pip-egg-info-ku2tb53y/apex.egg-info/dependency_links.txt writing requirements to /tmp/pip-pip-egg-info-ku2tb53y/apex.egg-info/requires.txt writing top-level names to /tmp/pip-pip-egg-info-ku2tb53y/apex.egg-info/top_level.txt writing manifest file '/tmp/pip-pip-egg-info-ku2tb53y/apex.egg-info/SOURCES.txt' reading manifest file '/tmp/pip-pip-egg-info-ku2tb53y/apex.egg-info/SOURCES.txt' writing manifest file '/tmp/pip-pip-egg-info-ku2tb53y/apex.egg-info/SOURCES.txt' Preparing metadata (setup.py) ... done Requirement already satisfied: packaging>20.6 in /home/cxq/.conda/envs/torch10/lib/python3.7/site-packages (from apex==0.1) (21.3) Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /home/cxq/.conda/envs/torch10/lib/python3.7/site-packages (from packaging>20.6->apex==0.1) (3.0.9) Building wheels for collected packages: apex Running command python setup.py bdist_wheel

torch.version = 1.7.1+cu110

running bdist_wheel running build running build_py installing to build/bdist.linux-x86_64/wheel running install running install_lib creating build/bdist.linux-x86_64/wheel creating build/bdist.linux-x86_64/wheel/apex creating build/bdist.linux-x86_64/wheel/apex/amp copying build/lib/apex/amp/compat.py -> build/bdist.linux-x86_64/wheel/apex/amp creating build/bdist.linux-x86_64/wheel/apex/amp/lists copying build/lib/apex/amp/lists/init.py -> build/bdist.linux-x86_64/wheel/apex/amp/lists copying build/lib/apex/amp/lists/tensor_overrides.py -> build/bdist.linux-x86_64/wheel/apex/amp/lists copying build/lib/apex/amp/lists/torch_overrides.py -> build/bdist.linux-x86_64/wheel/apex/amp/lists copying build/lib/apex/amp/lists/functional_overrides.py -> build/bdist.linux-x86_64/wheel/apex/amp/lists copying build/lib/apex/amp/init.py -> build/bdist.linux-x86_64/wheel/apex/amp copying build/lib/apex/amp/rnn_compat.py -> build/bdist.linux-x86_64/wheel/apex/amp copying build/lib/apex/amp/wrap.py -> build/bdist.linux-x86_64/wheel/apex/amp copying build/lib/apex/amp/_initialize.py -> build/bdist.linux-x86_64/wheel/apex/amp copying build/lib/apex/amp/scaler.py -> build/bdist.linux-x86_64/wheel/apex/amp copying build/lib/apex/amp/_process_optimizer.py -> build/bdist.linux-x86_64/wheel/apex/amp copying build/lib/apex/amp/opt.py -> build/bdist.linux-x86_64/wheel/apex/amp copying build/lib/apex/amp/amp.py -> build/bdist.linux-x86_64/wheel/apex/amp copying build/lib/apex/amp/_amp_state.py -> build/bdist.linux-x86_64/wheel/apex/amp copying build/lib/apex/amp/version.py -> build/bdist.linux-x86_64/wheel/apex/amp copying build/lib/apex/amp/utils.py -> build/bdist.linux-x86_64/wheel/apex/amp copying build/lib/apex/amp/handle.py -> build/bdist.linux-x86_64/wheel/apex/amp copying build/lib/apex/amp/frontend.py -> build/bdist.linux-x86_64/wheel/apex/amp copying build/lib/apex/_autocast_utils.py -> build/bdist.linux-x86_64/wheel/apex creating build/bdist.linux-x86_64/wheel/apex/fp16_utils copying build/lib/apex/fp16_utils/init.py -> build/bdist.linux-x86_64/wheel/apex/fp16_utils copying build/lib/apex/fp16_utils/fp16util.py -> build/bdist.linux-x86_64/wheel/apex/fp16_utils copying build/lib/apex/fp16_utils/loss_scaler.py -> build/bdist.linux-x86_64/wheel/apex/fp16_utils copying build/lib/apex/fp16_utils/fp16_optimizer.py -> build/bdist.linux-x86_64/wheel/apex/fp16_utils copying build/lib/apex/init.py -> build/bdist.linux-x86_64/wheel/apex creating build/bdist.linux-x86_64/wheel/apex/normalization copying build/lib/apex/normalization/init.py -> build/bdist.linux-x86_64/wheel/apex/normalization copying build/lib/apex/normalization/fused_layer_norm.py -> build/bdist.linux-x86_64/wheel/apex/normalization creating build/bdist.linux-x86_64/wheel/apex/RNN copying build/lib/apex/RNN/cells.py -> build/bdist.linux-x86_64/wheel/apex/RNN copying build/lib/apex/RNN/init.py -> build/bdist.linux-x86_64/wheel/apex/RNN copying build/lib/apex/RNN/RNNBackend.py -> build/bdist.linux-x86_64/wheel/apex/RNN copying build/lib/apex/RNN/models.py -> build/bdist.linux-x86_64/wheel/apex/RNN creating build/bdist.linux-x86_64/wheel/apex/optimizers copying build/lib/apex/optimizers/fused_adagrad.py -> build/bdist.linux-x86_64/wheel/apex/optimizers copying build/lib/apex/optimizers/init.py -> build/bdist.linux-x86_64/wheel/apex/optimizers copying build/lib/apex/optimizers/fused_lamb.py -> build/bdist.linux-x86_64/wheel/apex/optimizers copying build/lib/apex/optimizers/fused_sgd.py -> build/bdist.linux-x86_64/wheel/apex/optimizers copying build/lib/apex/optimizers/fused_novograd.py -> build/bdist.linux-x86_64/wheel/apex/optimizers copying build/lib/apex/optimizers/fused_mixed_precision_lamb.py -> build/bdist.linux-x86_64/wheel/apex/optimizers copying build/lib/apex/optimizers/fused_adam.py -> build/bdist.linux-x86_64/wheel/apex/optimizers creating build/bdist.linux-x86_64/wheel/apex/parallel copying build/lib/apex/parallel/init.py -> build/bdist.linux-x86_64/wheel/apex/parallel copying build/lib/apex/parallel/optimized_sync_batchnorm.py -> build/bdist.linux-x86_64/wheel/apex/parallel copying build/lib/apex/parallel/sync_batchnorm.py -> build/bdist.linux-x86_64/wheel/apex/parallel copying build/lib/apex/parallel/distributed.py -> build/bdist.linux-x86_64/wheel/apex/parallel copying build/lib/apex/parallel/multiproc.py -> build/bdist.linux-x86_64/wheel/apex/parallel copying build/lib/apex/parallel/optimized_sync_batchnorm_kernel.py -> build/bdist.linux-x86_64/wheel/apex/parallel copying build/lib/apex/parallel/sync_batchnorm_kernel.py -> build/bdist.linux-x86_64/wheel/apex/parallel copying build/lib/apex/parallel/LARC.py -> build/bdist.linux-x86_64/wheel/apex/parallel creating build/bdist.linux-x86_64/wheel/apex/contrib creating build/bdist.linux-x86_64/wheel/apex/contrib/transducer copying build/lib/apex/contrib/transducer/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/transducer copying build/lib/apex/contrib/transducer/transducer.py -> build/bdist.linux-x86_64/wheel/apex/contrib/transducer copying build/lib/apex/contrib/transducer/_transducer_ref.py -> build/bdist.linux-x86_64/wheel/apex/contrib/transducer creating build/bdist.linux-x86_64/wheel/apex/contrib/xentropy copying build/lib/apex/contrib/xentropy/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/xentropy copying build/lib/apex/contrib/xentropy/softmax_xentropy.py -> build/bdist.linux-x86_64/wheel/apex/contrib/xentropy copying build/lib/apex/contrib/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib creating build/bdist.linux-x86_64/wheel/apex/contrib/clip_grad copying build/lib/apex/contrib/clip_grad/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/clip_grad copying build/lib/apex/contrib/clip_grad/clip_grad.py -> build/bdist.linux-x86_64/wheel/apex/contrib/clip_grad creating build/bdist.linux-x86_64/wheel/apex/contrib/test creating build/bdist.linux-x86_64/wheel/apex/contrib/test/transducer copying build/lib/apex/contrib/test/transducer/test_transducer_joint.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/transducer copying build/lib/apex/contrib/test/transducer/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/transducer copying build/lib/apex/contrib/test/transducer/test_transducer_loss.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/transducer creating build/bdist.linux-x86_64/wheel/apex/contrib/test/xentropy copying build/lib/apex/contrib/test/xentropy/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/xentropy copying build/lib/apex/contrib/test/xentropy/test_label_smoothing.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/xentropy copying build/lib/apex/contrib/test/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test creating build/bdist.linux-x86_64/wheel/apex/contrib/test/clip_grad copying build/lib/apex/contrib/test/clip_grad/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/clip_grad copying build/lib/apex/contrib/test/clip_grad/test_clip_grad.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/clip_grad creating build/bdist.linux-x86_64/wheel/apex/contrib/test/optimizers copying build/lib/apex/contrib/test/optimizers/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/optimizers copying build/lib/apex/contrib/test/optimizers/test_dist_adam.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/optimizers copying build/lib/apex/contrib/test/optimizers/test_distributed_fused_lamb.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/optimizers creating build/bdist.linux-x86_64/wheel/apex/contrib/test/multihead_attn copying build/lib/apex/contrib/test/multihead_attn/test_encdec_multihead_attn_norm_add.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/multihead_attn copying build/lib/apex/contrib/test/multihead_attn/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/multihead_attn copying build/lib/apex/contrib/test/multihead_attn/test_encdec_multihead_attn.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/multihead_attn copying build/lib/apex/contrib/test/multihead_attn/test_mha_fused_softmax.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/multihead_attn copying build/lib/apex/contrib/test/multihead_attn/test_self_multihead_attn_norm_add.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/multihead_attn copying build/lib/apex/contrib/test/multihead_attn/test_self_multihead_attn.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/multihead_attn copying build/lib/apex/contrib/test/multihead_attn/test_fast_self_multihead_attn_bias.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/multihead_attn creating build/bdist.linux-x86_64/wheel/apex/contrib/test/index_mul_2d copying build/lib/apex/contrib/test/index_mul_2d/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/index_mul_2d copying build/lib/apex/contrib/test/index_mul_2d/test_index_mul_2d.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/index_mul_2d creating build/bdist.linux-x86_64/wheel/apex/contrib/test/conv_bias_relu copying build/lib/apex/contrib/test/conv_bias_relu/test_conv_bias_relu.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/conv_bias_relu copying build/lib/apex/contrib/test/conv_bias_relu/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/conv_bias_relu creating build/bdist.linux-x86_64/wheel/apex/contrib/test/bottleneck copying build/lib/apex/contrib/test/bottleneck/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/bottleneck copying build/lib/apex/contrib/test/bottleneck/test_bottleneck_module.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/bottleneck creating build/bdist.linux-x86_64/wheel/apex/contrib/test/cudnn_gbn copying build/lib/apex/contrib/test/cudnn_gbn/test_cudnn_gbn_with_two_gpus.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/cudnn_gbn copying build/lib/apex/contrib/test/cudnn_gbn/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/cudnn_gbn creating build/bdist.linux-x86_64/wheel/apex/contrib/test/layer_norm copying build/lib/apex/contrib/test/layer_norm/test_fast_layer_norm.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/layer_norm copying build/lib/apex/contrib/test/layer_norm/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/layer_norm creating build/bdist.linux-x86_64/wheel/apex/contrib/test/fmha copying build/lib/apex/contrib/test/fmha/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/fmha copying build/lib/apex/contrib/test/fmha/test_fmha.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/fmha creating build/bdist.linux-x86_64/wheel/apex/contrib/test/peer_memory copying build/lib/apex/contrib/test/peer_memory/test_peer_halo_exchange_module.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/peer_memory copying build/lib/apex/contrib/test/peer_memory/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/peer_memory creating build/bdist.linux-x86_64/wheel/apex/contrib/test/focal_loss copying build/lib/apex/contrib/test/focal_loss/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/focal_loss copying build/lib/apex/contrib/test/focal_loss/test_focal_loss.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/focal_loss creating build/bdist.linux-x86_64/wheel/apex/contrib/optimizers copying build/lib/apex/contrib/optimizers/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/optimizers copying build/lib/apex/contrib/optimizers/distributed_fused_adam.py -> build/bdist.linux-x86_64/wheel/apex/contrib/optimizers copying build/lib/apex/contrib/optimizers/distributed_fused_lamb.py -> build/bdist.linux-x86_64/wheel/apex/contrib/optimizers copying build/lib/apex/contrib/optimizers/fp16_optimizer.py -> build/bdist.linux-x86_64/wheel/apex/contrib/optimizers copying build/lib/apex/contrib/optimizers/fused_lamb.py -> build/bdist.linux-x86_64/wheel/apex/contrib/optimizers copying build/lib/apex/contrib/optimizers/fused_sgd.py -> build/bdist.linux-x86_64/wheel/apex/contrib/optimizers copying build/lib/apex/contrib/optimizers/fused_adam.py -> build/bdist.linux-x86_64/wheel/apex/contrib/optimizers creating build/bdist.linux-x86_64/wheel/apex/contrib/groupbn copying build/lib/apex/contrib/groupbn/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/groupbn copying build/lib/apex/contrib/groupbn/batch_norm.py -> build/bdist.linux-x86_64/wheel/apex/contrib/groupbn creating build/bdist.linux-x86_64/wheel/apex/contrib/multihead_attn copying build/lib/apex/contrib/multihead_attn/fast_encdec_multihead_attn_func.py -> build/bdist.linux-x86_64/wheel/apex/contrib/multihead_attn copying build/lib/apex/contrib/multihead_attn/fast_self_multihead_attn_func.py -> build/bdist.linux-x86_64/wheel/apex/contrib/multihead_attn copying build/lib/apex/contrib/multihead_attn/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/multihead_attn copying build/lib/apex/contrib/multihead_attn/self_multihead_attn.py -> build/bdist.linux-x86_64/wheel/apex/contrib/multihead_attn copying build/lib/apex/contrib/multihead_attn/encdec_multihead_attn.py -> build/bdist.linux-x86_64/wheel/apex/contrib/multihead_attn copying build/lib/apex/contrib/multihead_attn/self_multihead_attn_func.py -> build/bdist.linux-x86_64/wheel/apex/contrib/multihead_attn copying build/lib/apex/contrib/multihead_attn/mask_softmax_dropout_func.py -> build/bdist.linux-x86_64/wheel/apex/contrib/multihead_attn copying build/lib/apex/contrib/multihead_attn/encdec_multihead_attn_func.py -> build/bdist.linux-x86_64/wheel/apex/contrib/multihead_attn copying build/lib/apex/contrib/multihead_attn/fast_encdec_multihead_attn_norm_add_func.py -> build/bdist.linux-x86_64/wheel/apex/contrib/multihead_attn copying build/lib/apex/contrib/multihead_attn/fast_self_multihead_attn_norm_add_func.py -> build/bdist.linux-x86_64/wheel/apex/contrib/multihead_attn creating build/bdist.linux-x86_64/wheel/apex/contrib/index_mul_2d copying build/lib/apex/contrib/index_mul_2d/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/index_mul_2d copying build/lib/apex/contrib/index_mul_2d/index_mul_2d.py -> build/bdist.linux-x86_64/wheel/apex/contrib/index_mul_2d creating build/bdist.linux-x86_64/wheel/apex/contrib/sparsity copying build/lib/apex/contrib/sparsity/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/sparsity copying build/lib/apex/contrib/sparsity/permutation_lib.py -> build/bdist.linux-x86_64/wheel/apex/contrib/sparsity copying build/lib/apex/contrib/sparsity/sparse_masklib.py -> build/bdist.linux-x86_64/wheel/apex/contrib/sparsity copying build/lib/apex/contrib/sparsity/asp.py -> build/bdist.linux-x86_64/wheel/apex/contrib/sparsity creating build/bdist.linux-x86_64/wheel/apex/contrib/sparsity/permutation_search_kernels copying build/lib/apex/contrib/sparsity/permutation_search_kernels/permutation_utilities.py -> build/bdist.linux-x86_64/wheel/apex/contrib/sparsity/permutation_search_kernels copying build/lib/apex/contrib/sparsity/permutation_search_kernels/call_permutation_search_kernels.py -> build/bdist.linux-x86_64/wheel/apex/contrib/sparsity/permutation_search_kernels copying build/lib/apex/contrib/sparsity/permutation_search_kernels/channel_swap.py -> build/bdist.linux-x86_64/wheel/apex/contrib/sparsity/permutation_search_kernels copying build/lib/apex/contrib/sparsity/permutation_search_kernels/exhaustive_search.py -> build/bdist.linux-x86_64/wheel/apex/contrib/sparsity/permutation_search_kernels copying build/lib/apex/contrib/sparsity/permutation_search_kernels/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/sparsity/permutation_search_kernels creating build/bdist.linux-x86_64/wheel/apex/contrib/conv_bias_relu copying build/lib/apex/contrib/conv_bias_relu/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/conv_bias_relu copying build/lib/apex/contrib/conv_bias_relu/conv_bias_relu.py -> build/bdist.linux-x86_64/wheel/apex/contrib/conv_bias_relu creating build/bdist.linux-x86_64/wheel/apex/contrib/bottleneck copying build/lib/apex/contrib/bottleneck/bottleneck.py -> build/bdist.linux-x86_64/wheel/apex/contrib/bottleneck copying build/lib/apex/contrib/bottleneck/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/bottleneck copying build/lib/apex/contrib/bottleneck/test.py -> build/bdist.linux-x86_64/wheel/apex/contrib/bottleneck copying build/lib/apex/contrib/bottleneck/halo_exchangers.py -> build/bdist.linux-x86_64/wheel/apex/contrib/bottleneck creating build/bdist.linux-x86_64/wheel/apex/contrib/cudnn_gbn copying build/lib/apex/contrib/cudnn_gbn/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/cudnn_gbn copying build/lib/apex/contrib/cudnn_gbn/batch_norm.py -> build/bdist.linux-x86_64/wheel/apex/contrib/cudnn_gbn creating build/bdist.linux-x86_64/wheel/apex/contrib/layer_norm copying build/lib/apex/contrib/layer_norm/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/layer_norm copying build/lib/apex/contrib/layer_norm/layer_norm.py -> build/bdist.linux-x86_64/wheel/apex/contrib/layer_norm creating build/bdist.linux-x86_64/wheel/apex/contrib/fmha copying build/lib/apex/contrib/fmha/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/fmha copying build/lib/apex/contrib/fmha/fmha.py -> build/bdist.linux-x86_64/wheel/apex/contrib/fmha creating build/bdist.linux-x86_64/wheel/apex/contrib/peer_memory copying build/lib/apex/contrib/peer_memory/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/peer_memory copying build/lib/apex/contrib/peer_memory/peer_halo_exchanger_1d.py -> build/bdist.linux-x86_64/wheel/apex/contrib/peer_memory copying build/lib/apex/contrib/peer_memory/peer_memory.py -> build/bdist.linux-x86_64/wheel/apex/contrib/peer_memory creating build/bdist.linux-x86_64/wheel/apex/contrib/focal_loss copying build/lib/apex/contrib/focal_loss/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/focal_loss copying build/lib/apex/contrib/focal_loss/focal_loss.py -> build/bdist.linux-x86_64/wheel/apex/contrib/focal_loss creating build/bdist.linux-x86_64/wheel/apex/multi_tensor_apply copying build/lib/apex/multi_tensor_apply/init.py -> build/bdist.linux-x86_64/wheel/apex/multi_tensor_apply copying build/lib/apex/multi_tensor_apply/multi_tensor_apply.py -> build/bdist.linux-x86_64/wheel/apex/multi_tensor_apply creating build/bdist.linux-x86_64/wheel/apex/transformer creating build/bdist.linux-x86_64/wheel/apex/transformer/tensor_parallel copying build/lib/apex/transformer/tensor_parallel/init.py -> build/bdist.linux-x86_64/wheel/apex/transformer/tensor_parallel copying build/lib/apex/transformer/tensor_parallel/data.py -> build/bdist.linux-x86_64/wheel/apex/transformer/tensor_parallel copying build/lib/apex/transformer/tensor_parallel/random.py -> build/bdist.linux-x86_64/wheel/apex/transformer/tensor_parallel copying build/lib/apex/transformer/tensor_parallel/cross_entropy.py -> build/bdist.linux-x86_64/wheel/apex/transformer/tensor_parallel copying build/lib/apex/transformer/tensor_parallel/mappings.py -> build/bdist.linux-x86_64/wheel/apex/transformer/tensor_parallel copying build/lib/apex/transformer/tensor_parallel/layers.py -> build/bdist.linux-x86_64/wheel/apex/transformer/tensor_parallel copying build/lib/apex/transformer/tensor_parallel/memory.py -> build/bdist.linux-x86_64/wheel/apex/transformer/tensor_parallel copying build/lib/apex/transformer/tensor_parallel/utils.py -> build/bdist.linux-x86_64/wheel/apex/transformer/tensor_parallel creating build/bdist.linux-x86_64/wheel/apex/transformer/amp copying build/lib/apex/transformer/amp/init.py -> build/bdist.linux-x86_64/wheel/apex/transformer/amp copying build/lib/apex/transformer/amp/grad_scaler.py -> build/bdist.linux-x86_64/wheel/apex/transformer/amp creating build/bdist.linux-x86_64/wheel/apex/transformer/testing copying build/lib/apex/transformer/testing/init.py -> build/bdist.linux-x86_64/wheel/apex/transformer/testing copying build/lib/apex/transformer/testing/commons.py -> build/bdist.linux-x86_64/wheel/apex/transformer/testing copying build/lib/apex/transformer/testing/global_vars.py -> build/bdist.linux-x86_64/wheel/apex/transformer/testing copying build/lib/apex/transformer/testing/standalone_gpt.py -> build/bdist.linux-x86_64/wheel/apex/transformer/testing copying build/lib/apex/transformer/testing/standalone_bert.py -> build/bdist.linux-x86_64/wheel/apex/transformer/testing copying build/lib/apex/transformer/testing/standalone_transformer_lm.py -> build/bdist.linux-x86_64/wheel/apex/transformer/testing copying build/lib/apex/transformer/testing/arguments.py -> build/bdist.linux-x86_64/wheel/apex/transformer/testing copying build/lib/apex/transformer/testing/distributed_test_base.py -> build/bdist.linux-x86_64/wheel/apex/transformer/testing creating build/bdist.linux-x86_64/wheel/apex/transformer/_data copying build/lib/apex/transformer/_data/_batchsampler.py -> build/bdist.linux-x86_64/wheel/apex/transformer/_data copying build/lib/apex/transformer/_data/init.py -> build/bdist.linux-x86_64/wheel/apex/transformer/_data copying build/lib/apex/transformer/_ucc_util.py -> build/bdist.linux-x86_64/wheel/apex/transformer copying build/lib/apex/transformer/init.py -> build/bdist.linux-x86_64/wheel/apex/transformer copying build/lib/apex/transformer/log_util.py -> build/bdist.linux-x86_64/wheel/apex/transformer creating build/bdist.linux-x86_64/wheel/apex/transformer/pipeline_parallel creating build/bdist.linux-x86_64/wheel/apex/transformer/pipeline_parallel/schedules copying build/lib/apex/transformer/pipeline_parallel/schedules/fwd_bwd_pipelining_with_interleaving.py -> build/bdist.linux-x86_64/wheel/apex/transformer/pipeline_parallel/schedules copying build/lib/apex/transformer/pipeline_parallel/schedules/init.py -> build/bdist.linux-x86_64/wheel/apex/transformer/pipeline_parallel/schedules copying build/lib/apex/transformer/pipeline_parallel/schedules/fwd_bwd_pipelining_without_interleaving.py -> build/bdist.linux-x86_64/wheel/apex/transformer/pipeline_parallel/schedules copying build/lib/apex/transformer/pipeline_parallel/schedules/fwd_bwd_no_pipelining.py -> build/bdist.linux-x86_64/wheel/apex/transformer/pipeline_parallel/schedules copying build/lib/apex/transformer/pipeline_parallel/schedules/common.py -> build/bdist.linux-x86_64/wheel/apex/transformer/pipeline_parallel/schedules copying build/lib/apex/transformer/pipeline_parallel/_timers.py -> build/bdist.linux-x86_64/wheel/apex/transformer/pipeline_parallel copying build/lib/apex/transformer/pipeline_parallel/init.py -> build/bdist.linux-x86_64/wheel/apex/transformer/pipeline_parallel copying build/lib/apex/transformer/pipeline_parallel/p2p_communication.py -> build/bdist.linux-x86_64/wheel/apex/transformer/pipeline_parallel copying build/lib/apex/transformer/pipeline_parallel/utils.py -> build/bdist.linux-x86_64/wheel/apex/transformer/pipeline_parallel copying build/lib/apex/transformer/microbatches.py -> build/bdist.linux-x86_64/wheel/apex/transformer copying build/lib/apex/transformer/enums.py -> build/bdist.linux-x86_64/wheel/apex/transformer creating build/bdist.linux-x86_64/wheel/apex/transformer/functional copying build/lib/apex/transformer/functional/fused_softmax.py -> build/bdist.linux-x86_64/wheel/apex/transformer/functional copying build/lib/apex/transformer/functional/init.py -> build/bdist.linux-x86_64/wheel/apex/transformer/functional copying build/lib/apex/transformer/utils.py -> build/bdist.linux-x86_64/wheel/apex/transformer creating build/bdist.linux-x86_64/wheel/apex/transformer/layers copying build/lib/apex/transformer/layers/init.py -> build/bdist.linux-x86_64/wheel/apex/transformer/layers copying build/lib/apex/transformer/layers/layer_norm.py -> build/bdist.linux-x86_64/wheel/apex/transformer/layers copying build/lib/apex/transformer/parallel_state.py -> build/bdist.linux-x86_64/wheel/apex/transformer creating build/bdist.linux-x86_64/wheel/apex/fused_dense copying build/lib/apex/fused_dense/fused_dense.py -> build/bdist.linux-x86_64/wheel/apex/fused_dense copying build/lib/apex/fused_dense/init.py -> build/bdist.linux-x86_64/wheel/apex/fused_dense creating build/bdist.linux-x86_64/wheel/apex/mlp copying build/lib/apex/mlp/mlp.py -> build/bdist.linux-x86_64/wheel/apex/mlp copying build/lib/apex/mlp/init.py -> build/bdist.linux-x86_64/wheel/apex/mlp running install_egg_info running egg_info writing apex.egg-info/PKG-INFO writing dependency_links to apex.egg-info/dependency_links.txt writing requirements to apex.egg-info/requires.txt writing top-level names to apex.egg-info/top_level.txt reading manifest file 'apex.egg-info/SOURCES.txt' writing manifest file 'apex.egg-info/SOURCES.txt' Copying apex.egg-info to build/bdist.linux-x86_64/wheel/apex-0.1-py3.7.egg-info running install_scripts adding license file "LICENSE" (matched pattern "LICEN[CS]E*") creating build/bdist.linux-x86_64/wheel/apex-0.1.dist-info/WHEEL creating '/tmp/pip-wheel-o4d_cpbt/apex-0.1-py3-none-any.whl' and adding 'build/bdist.linux-x86_64/wheel' to it adding 'apex/init.py' adding 'apex/_autocast_utils.py' adding 'apex/RNN/RNNBackend.py' adding 'apex/RNN/init.py' adding 'apex/RNN/cells.py' adding 'apex/RNN/models.py' adding 'apex/amp/init.py' adding 'apex/amp/version.py' adding 'apex/amp/_amp_state.py' adding 'apex/amp/_initialize.py' adding 'apex/amp/_process_optimizer.py' adding 'apex/amp/amp.py' adding 'apex/amp/compat.py' adding 'apex/amp/frontend.py' adding 'apex/amp/handle.py' adding 'apex/amp/opt.py' adding 'apex/amp/rnn_compat.py' adding 'apex/amp/scaler.py' adding 'apex/amp/utils.py' adding 'apex/amp/wrap.py' adding 'apex/amp/lists/init.py' adding 'apex/amp/lists/functional_overrides.py' adding 'apex/amp/lists/tensor_overrides.py' adding 'apex/amp/lists/torch_overrides.py' adding 'apex/contrib/init.py' adding 'apex/contrib/bottleneck/init.py' adding 'apex/contrib/bottleneck/bottleneck.py' adding 'apex/contrib/bottleneck/halo_exchangers.py' adding 'apex/contrib/bottleneck/test.py' adding 'apex/contrib/clip_grad/init.py' adding 'apex/contrib/clip_grad/clip_grad.py' adding 'apex/contrib/conv_bias_relu/init.py' adding 'apex/contrib/conv_bias_relu/conv_bias_relu.py' adding 'apex/contrib/cudnn_gbn/init.py' adding 'apex/contrib/cudnn_gbn/batch_norm.py' adding 'apex/contrib/fmha/init.py' adding 'apex/contrib/fmha/fmha.py' adding 'apex/contrib/focal_loss/init.py' adding 'apex/contrib/focal_loss/focal_loss.py' adding 'apex/contrib/groupbn/init.py' adding 'apex/contrib/groupbn/batch_norm.py' adding 'apex/contrib/index_mul_2d/init.py' adding 'apex/contrib/index_mul_2d/index_mul_2d.py' adding 'apex/contrib/layer_norm/init.py' adding 'apex/contrib/layer_norm/layer_norm.py' adding 'apex/contrib/multihead_attn/init.py' adding 'apex/contrib/multihead_attn/encdec_multihead_attn.py' adding 'apex/contrib/multihead_attn/encdec_multihead_attn_func.py' adding 'apex/contrib/multihead_attn/fast_encdec_multihead_attn_func.py' adding 'apex/contrib/multihead_attn/fast_encdec_multihead_attn_norm_add_func.py' adding 'apex/contrib/multihead_attn/fast_self_multihead_attn_func.py' adding 'apex/contrib/multihead_attn/fast_self_multihead_attn_norm_add_func.py' adding 'apex/contrib/multihead_attn/mask_softmax_dropout_func.py' adding 'apex/contrib/multihead_attn/self_multihead_attn.py' adding 'apex/contrib/multihead_attn/self_multihead_attn_func.py' adding 'apex/contrib/optimizers/init.py' adding 'apex/contrib/optimizers/distributed_fused_adam.py' adding 'apex/contrib/optimizers/distributed_fused_lamb.py' adding 'apex/contrib/optimizers/fp16_optimizer.py' adding 'apex/contrib/optimizers/fused_adam.py' adding 'apex/contrib/optimizers/fused_lamb.py' adding 'apex/contrib/optimizers/fused_sgd.py' adding 'apex/contrib/peer_memory/init.py' adding 'apex/contrib/peer_memory/peer_halo_exchanger_1d.py' adding 'apex/contrib/peer_memory/peer_memory.py' adding 'apex/contrib/sparsity/init.py' adding 'apex/contrib/sparsity/asp.py' adding 'apex/contrib/sparsity/permutation_lib.py' adding 'apex/contrib/sparsity/sparse_masklib.py' adding 'apex/contrib/sparsity/permutation_search_kernels/init.py' adding 'apex/contrib/sparsity/permutation_search_kernels/call_permutation_search_kernels.py' adding 'apex/contrib/sparsity/permutation_search_kernels/channel_swap.py' adding 'apex/contrib/sparsity/permutation_search_kernels/exhaustive_search.py' adding 'apex/contrib/sparsity/permutation_search_kernels/permutation_utilities.py' adding 'apex/contrib/test/init.py' adding 'apex/contrib/test/bottleneck/init.py' adding 'apex/contrib/test/bottleneck/test_bottleneck_module.py' adding 'apex/contrib/test/clip_grad/init.py' adding 'apex/contrib/test/clip_grad/test_clip_grad.py' adding 'apex/contrib/test/conv_bias_relu/init.py' adding 'apex/contrib/test/conv_bias_relu/test_conv_bias_relu.py' adding 'apex/contrib/test/cudnn_gbn/init.py' adding 'apex/contrib/test/cudnn_gbn/test_cudnn_gbn_with_two_gpus.py' adding 'apex/contrib/test/fmha/init.py' adding 'apex/contrib/test/fmha/test_fmha.py' adding 'apex/contrib/test/focal_loss/init.py' adding 'apex/contrib/test/focal_loss/test_focal_loss.py' adding 'apex/contrib/test/index_mul_2d/init.py' adding 'apex/contrib/test/index_mul_2d/test_index_mul_2d.py' adding 'apex/contrib/test/layer_norm/init.py' adding 'apex/contrib/test/layer_norm/test_fast_layer_norm.py' adding 'apex/contrib/test/multihead_attn/init.py' adding 'apex/contrib/test/multihead_attn/test_encdec_multihead_attn.py' adding 'apex/contrib/test/multihead_attn/test_encdec_multihead_attn_norm_add.py' adding 'apex/contrib/test/multihead_attn/test_fast_self_multihead_attn_bias.py' adding 'apex/contrib/test/multihead_attn/test_mha_fused_softmax.py' adding 'apex/contrib/test/multihead_attn/test_self_multihead_attn.py' adding 'apex/contrib/test/multihead_attn/test_self_multihead_attn_norm_add.py' adding 'apex/contrib/test/optimizers/init.py' adding 'apex/contrib/test/optimizers/test_dist_adam.py' adding 'apex/contrib/test/optimizers/test_distributed_fused_lamb.py' adding 'apex/contrib/test/peer_memory/init.py' adding 'apex/contrib/test/peer_memory/test_peer_halo_exchange_module.py' adding 'apex/contrib/test/transducer/init.py' adding 'apex/contrib/test/transducer/test_transducer_joint.py' adding 'apex/contrib/test/transducer/test_transducer_loss.py' adding 'apex/contrib/test/xentropy/init.py' adding 'apex/contrib/test/xentropy/test_label_smoothing.py' adding 'apex/contrib/transducer/init.py' adding 'apex/contrib/transducer/_transducer_ref.py' adding 'apex/contrib/transducer/transducer.py' adding 'apex/contrib/xentropy/init.py' adding 'apex/contrib/xentropy/softmax_xentropy.py' adding 'apex/fp16_utils/init.py' adding 'apex/fp16_utils/fp16_optimizer.py' adding 'apex/fp16_utils/fp16util.py' adding 'apex/fp16_utils/loss_scaler.py' adding 'apex/fused_dense/init.py' adding 'apex/fused_dense/fused_dense.py' adding 'apex/mlp/init.py' adding 'apex/mlp/mlp.py' adding 'apex/multi_tensor_apply/init.py' adding 'apex/multi_tensor_apply/multi_tensor_apply.py' adding 'apex/normalization/init.py' adding 'apex/normalization/fused_layer_norm.py' adding 'apex/optimizers/init.py' adding 'apex/optimizers/fused_adagrad.py' adding 'apex/optimizers/fused_adam.py' adding 'apex/optimizers/fused_lamb.py' adding 'apex/optimizers/fused_mixed_precision_lamb.py' adding 'apex/optimizers/fused_novograd.py' adding 'apex/optimizers/fused_sgd.py' adding 'apex/parallel/LARC.py' adding 'apex/parallel/init.py' adding 'apex/parallel/distributed.py' adding 'apex/parallel/multiproc.py' adding 'apex/parallel/optimized_sync_batchnorm.py' adding 'apex/parallel/optimized_sync_batchnorm_kernel.py' adding 'apex/parallel/sync_batchnorm.py' adding 'apex/parallel/sync_batchnorm_kernel.py' adding 'apex/transformer/init.py' adding 'apex/transformer/_ucc_util.py' adding 'apex/transformer/enums.py' adding 'apex/transformer/log_util.py' adding 'apex/transformer/microbatches.py' adding 'apex/transformer/parallel_state.py' adding 'apex/transformer/utils.py' adding 'apex/transformer/_data/init.py' adding 'apex/transformer/_data/_batchsampler.py' adding 'apex/transformer/amp/init.py' adding 'apex/transformer/amp/grad_scaler.py' adding 'apex/transformer/functional/init.py' adding 'apex/transformer/functional/fused_softmax.py' adding 'apex/transformer/layers/init.py' adding 'apex/transformer/layers/layer_norm.py' adding 'apex/transformer/pipeline_parallel/init.py' adding 'apex/transformer/pipeline_parallel/_timers.py' adding 'apex/transformer/pipeline_parallel/p2p_communication.py' adding 'apex/transformer/pipeline_parallel/utils.py' adding 'apex/transformer/pipeline_parallel/schedules/init.py' adding 'apex/transformer/pipeline_parallel/schedules/common.py' adding 'apex/transformer/pipeline_parallel/schedules/fwd_bwd_no_pipelining.py' adding 'apex/transformer/pipeline_parallel/schedules/fwd_bwd_pipelining_with_interleaving.py' adding 'apex/transformer/pipeline_parallel/schedules/fwd_bwd_pipelining_without_interleaving.py' adding 'apex/transformer/tensor_parallel/init.py' adding 'apex/transformer/tensor_parallel/cross_entropy.py' adding 'apex/transformer/tensor_parallel/data.py' adding 'apex/transformer/tensor_parallel/layers.py' adding 'apex/transformer/tensor_parallel/mappings.py' adding 'apex/transformer/tensor_parallel/memory.py' adding 'apex/transformer/tensor_parallel/random.py' adding 'apex/transformer/tensor_parallel/utils.py' adding 'apex/transformer/testing/init.py' adding 'apex/transformer/testing/arguments.py' adding 'apex/transformer/testing/commons.py' adding 'apex/transformer/testing/distributed_test_base.py' adding 'apex/transformer/testing/global_vars.py' adding 'apex/transformer/testing/standalone_bert.py' adding 'apex/transformer/testing/standalone_gpt.py' adding 'apex/transformer/testing/standalone_transformer_lm.py' adding 'apex-0.1.dist-info/LICENSE' adding 'apex-0.1.dist-info/METADATA' adding 'apex-0.1.dist-info/WHEEL' adding 'apex-0.1.dist-info/top_level.txt' adding 'apex-0.1.dist-info/RECORD' removing build/bdist.linux-x86_64/wheel Building wheel for apex (setup.py) ... done Created wheel for apex: filename=apex-0.1-py3-none-any.whl size=346778 sha256=bac9bdfa231d1f15122a9c3f292beee94df5bcf1c979b3240091d5c90b3f269c Stored in directory: /tmp/pip-ephem-wheel-cache-6m7s92_k/wheels/0a/df/3d/18f4c8df382be1e9dc0b4f5c16593b2d47107867ca80e7a6f6 Successfully built apex Installing collected packages: apex Attempting uninstall: apex Found existing installation: apex 0.1 Uninstalling apex-0.1: Removing file or directory /home/cxq/.conda/envs/torch10/lib/python3.7/site-packages/apex-0.1.dist-info/ Removing file or directory /home/cxq/.conda/envs/torch10/lib/python3.7/site-packages/apex/ Successfully uninstalled apex-0.1 Successfully installed apex-0.1

Hi, did you get any error when running: python setup.py build develop ? I got the error below
error1

@jinhseo
Copy link
Owner

jinhseo commented Mar 20, 2023

@cxq1 I found some relevant articles from NVIDIA/apex#1532 about this issue.
So try: git clone --branch 22.04-dev https://github.com/NVIDIA/apex.git for older version.
I will update on readme as well.

@jinhseo
Copy link
Owner

jinhseo commented Mar 20, 2023

@ShirleySyh I am not sure apex is successfully installed before u run it.
Do you want to try the way I commented?

@jinhseo
Copy link
Owner

jinhseo commented Mar 23, 2023 •
edited
Loading

If system cannot find available GPU architecture, then try export TORCH_CUDA_ARCH_LIST="compute capability".
"compute capability" for your GPU could be found at here.

@ZiMo-Chen
Copy link
Author

@cxq1 I found some relevant articles from NVIDIA/apex#1532 about this issue. So try: git clone --branch 22.04-dev https://github.com/NVIDIA/apex.git for older version. I will update on readme as well.

Thank you for your reply.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@ZiMo-Chen @jinhseo @ShirleySyh