Adding rdbms-connect fails with a pip error.

[MidasCHA]

Describe the bug

On Windows 11 with Azure CLI version 2.66 running the command az extension add --name rdbms-connect results in the error: An error occurred. Pip failed with status code 3221225477. Use --debug for more information.

Related command

az extension add --name rdbms-connect

Errors

An error occurred. Pip failed with status code 3221225477. Use --debug for more information.

Issue script & Debug output

DEBUG: cli.knack.cli: Command arguments: ['extension', 'add', '--name', 'rdbms-connect', '--debug']
DEBUG: cli.knack.cli: __init__ debug log:
Cannot enable color.
DEBUG: cli.knack.cli: Event: Cli.PreExecute []
DEBUG: cli.knack.cli: Event: CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0x00000170D0127380>, <function OutputProducer.on_global_arguments at 0x00000170D04CC0E0>, <function CLIQuery.on_global_arguments at 0x00000170D04F14E0>]
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreCommandTableCreate []
DEBUG: cli.azure.cli.core: Modules found from index for 'extension': ['azure.cli.command_modules.extension']
DEBUG: cli.azure.cli.core: Loading command modules:
DEBUG: cli.azure.cli.core: Name                  Load Time    Groups  Commands
DEBUG: cli.azure.cli.core: extension                 0.002         1         7
DEBUG: cli.azure.cli.core: Total (1)                 0.002         1         7
DEBUG: cli.azure.cli.core: Loaded 1 groups, 7 commands.
DEBUG: cli.azure.cli.core: Found a match in the command table.
DEBUG: cli.azure.cli.core: Raw command  : extension add
DEBUG: cli.azure.cli.core: Command table: extension add
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreCommandTableTruncate [<function AzCliLogging.init_command_file_logging at 0x00000170D27519E0>]
DEBUG: cli.azure.cli.core.azlogging: metadata file logging enabled - writing logs to 'C:\Users\ChristianHansen\.azure\commands\2024-11-12.09-35-45.extension_add.2944.log'.
INFO: az_command_data_logger: command args: extension add --name {} --debug
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreArgumentLoad [<function register_global_subscription_argument.<locals>.add_subscription_parameter at 0x00000170D2796FC0>]
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPostArgumentLoad []
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPostCommandTableCreate [<function register_ids_argument.<locals>.add_ids_arguments at 0x00000170D27B53A0>, <function register_cache_arguments.<locals>.add_cache_arguments at 0x00000170D27B54E0>, <function register_upcoming_breaking_change_info.<locals>.update_breaking_change_info at 0x00000170D27B5580>]
DEBUG: cli.knack.cli: Event: CommandInvoker.OnCommandTableLoaded []
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreParseArgs []
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPostParseArgs [<function OutputProducer.handle_output_argument at 0x00000170D04CC180>, <function CLIQuery.handle_query_parameter at 0x00000170D04F1580>, <function register_ids_argument.<locals>.parse_ids_arguments at 0x00000170D27B5440>]
WARNING: The default value of '--allow-preview' will be changed to 'false' from 'true' in next breaking change release(2.67.0).
WARNING: cli.azure.cli.core.extension.operations: Default enabled including preview versions for extension installation now. Disabled in future release. Use '--allow-preview true' to enable it specifically if needed. Use '--allow-preview false' to install stable version only. 
DEBUG: urllib3.connectionpool: Starting new HTTPS connection (1): aka.ms:443
DEBUG: urllib3.connectionpool: https://aka.ms:443 "GET /azure-cli-extension-index-v1 HTTP/1.1" 301 0
DEBUG: urllib3.connectionpool: Starting new HTTPS connection (1): azcliextensionsync.blob.core.windows.net:443
DEBUG: urllib3.connectionpool: https://azcliextensionsync.blob.core.windows.net:443 "GET /index1/index.json HTTP/1.1" 200 4735275
DEBUG: cli.azure.cli.core.extension._resolve: Candidates ['rdbms_connect-0.1.0-py2.py3-none-any.whl', 'rdbms_connect-0.1.1-py2.py3-none-any.whl', 'rdbms_connect-0.1.2-py2.py3-none-any.whl', 'rdbms_connect-0.1.3-py2.py3-none-any.whl', 'rdbms_connect-0.1.4-py2.py3-none-any.whl', 'rdbms_connect-1.0.0-py2.py3-none-any.whl', 'rdbms_connect-1.0.1-py2.py3-none-any.whl', 'rdbms_connect-1.0.2-py2.py3-none-any.whl', 'rdbms_connect-1.0.3-py2.py3-none-any.whl', 'rdbms_connect-1.0.4-py2.py3-none-any.whl', 'rdbms_connect-1.0.5-py2.py3-none-any.whl', 'rdbms_connect-1.0.6-py2.py3-none-any.whl', 'rdbms_connect-1.0.7-py2.py3-none-any.whl']
DEBUG: cli.azure.cli.core.extension._resolve: Candidates ['rdbms_connect-0.1.0-py2.py3-none-any.whl', 'rdbms_connect-0.1.1-py2.py3-none-any.whl', 'rdbms_connect-0.1.2-py2.py3-none-any.whl', 'rdbms_connect-0.1.3-py2.py3-none-any.whl', 'rdbms_connect-0.1.4-py2.py3-none-any.whl', 'rdbms_connect-1.0.0-py2.py3-none-any.whl', 'rdbms_connect-1.0.1-py2.py3-none-any.whl', 'rdbms_connect-1.0.2-py2.py3-none-any.whl', 'rdbms_connect-1.0.3-py2.py3-none-any.whl', 'rdbms_connect-1.0.4-py2.py3-none-any.whl', 'rdbms_connect-1.0.5-py2.py3-none-any.whl', 'rdbms_connect-1.0.6-py2.py3-none-any.whl', 'rdbms_connect-1.0.7-py2.py3-none-any.whl']
DEBUG: cli.azure.cli.core.extension._resolve: Candidates ['rdbms_connect-0.1.0-py2.py3-none-any.whl', 'rdbms_connect-0.1.1-py2.py3-none-any.whl', 'rdbms_connect-0.1.2-py2.py3-none-any.whl', 'rdbms_connect-0.1.3-py2.py3-none-any.whl', 'rdbms_connect-0.1.4-py2.py3-none-any.whl', 'rdbms_connect-1.0.0-py2.py3-none-any.whl', 'rdbms_connect-1.0.1-py2.py3-none-any.whl', 'rdbms_connect-1.0.2-py2.py3-none-any.whl', 'rdbms_connect-1.0.3-py2.py3-none-any.whl', 'rdbms_connect-1.0.4-py2.py3-none-any.whl', 'rdbms_connect-1.0.5-py2.py3-none-any.whl', 'rdbms_connect-1.0.6-py2.py3-none-any.whl', 'rdbms_connect-1.0.7-py2.py3-none-any.whl']
DEBUG: cli.azure.cli.core.extension._resolve: Candidates ['rdbms_connect-0.1.0-py2.py3-none-any.whl', 'rdbms_connect-0.1.1-py2.py3-none-any.whl', 'rdbms_connect-0.1.2-py2.py3-none-any.whl', 'rdbms_connect-0.1.3-py2.py3-none-any.whl', 'rdbms_connect-0.1.4-py2.py3-none-any.whl', 'rdbms_connect-1.0.0-py2.py3-none-any.whl', 'rdbms_connect-1.0.1-py2.py3-none-any.whl', 'rdbms_connect-1.0.2-py2.py3-none-any.whl', 'rdbms_connect-1.0.3-py2.py3-none-any.whl', 'rdbms_connect-1.0.4-py2.py3-none-any.whl', 'rdbms_connect-1.0.5-py2.py3-none-any.whl', 'rdbms_connect-1.0.6-py2.py3-none-any.whl', 'rdbms_connect-1.0.7-py2.py3-none-any.whl']
DEBUG: cli.azure.cli.core.extension._resolve: Chosen {'downloadUrl': 'https://azcliprod.blob.core.windows.net/cli-extensions/rdbms_connect-1.0.7-py2.py3-none-any.whl', 'filename': 'rdbms_connect-1.0.7-py2.py3-none-any.whl', 'metadata': {'azext.minCliCoreVersion': '2.19.0', 'classifiers': ['Development Status :: 4 - Beta', 'Intended Audience :: Developers', 'Intended Audience :: System Administrators', 'Programming Language :: Python', 'Programming Language :: Python :: 3', 'Programming Language :: Python :: 3.6', 'Programming Language :: Python :: 3.7', 'Programming Language :: Python :: 3.8', 'Programming Language :: Python :: 3.9', 'Programming Language :: Python :: 3.10', 'Programming Language :: Python :: 3.11', 'License :: OSI Approved :: MIT License'], 'extensions': {'python.details': {'contacts': [{'email': '[email protected]', 'name': 'Microsoft Corporation', 'role': 'author'}], 'document_names': {'description': 'DESCRIPTION.rst'}, 'project_urls': {'Home': 'https://github.com/Azure/azure-cli-extensions/tree/main/src/rdbms-connect'}}}, 'extras': [], 'generator': 'bdist_wheel (0.30.0)', 'license': 'MIT', 'metadata_version': '2.0', 'name': 'rdbms-connect', 'run_requires': [{'requires': ['mycli (~=1.27.0)', 'mycli~=1.27.0', 'pgcli (==4.0.1)', 'pgcli==4.0.1', 'psycopg2 (~=2.9.3)', 'psycopg2~=2.9.3', 'setproctitle (~=1.3.3)', 'setproctitle~=1.3.3']}], 'summary': 'Support for testing connection to Azure Database for MySQL & PostgreSQL servers.', 'version': '1.0.7'}, 'sha256Digest': 'eba997d4ad5acd0bdbb80a72281c2f56e88a09b2daa9ecc20ddaefd2586f543d'}
DEBUG: cli.azure.cli.core.extension.operations: Extension source is url? True
DEBUG: cli.azure.cli.core.extension.operations: Downloading https://azcliprod.blob.core.windows.net/cli-extensions/rdbms_connect-1.0.7-py2.py3-none-any.whl to C:\Users\CHRIST~1\AppData\Local\Temp\tmpyug5lwbn\rdbms_connect-1.0.7-py2.py3-none-any.whl
DEBUG: urllib3.connectionpool: Starting new HTTPS connection (1): azcliprod.blob.core.windows.net:443
DEBUG: urllib3.connectionpool: https://azcliprod.blob.core.windows.net:443 "GET /cli-extensions/rdbms_connect-1.0.7-py2.py3-none-any.whl HTTP/1.1" 200 13076
DEBUG: cli.azure.cli.core.extension.operations: Downloaded to C:\Users\CHRIST~1\AppData\Local\Temp\tmpyug5lwbn\rdbms_connect-1.0.7-py2.py3-none-any.whl
DEBUG: cli.azure.cli.core.extension.operations: Validating the extension C:\Users\CHRIST~1\AppData\Local\Temp\tmpyug5lwbn\rdbms_connect-1.0.7-py2.py3-none-any.whl
DEBUG: cli.azure.cli.core.extension.operations: Checksum of C:\Users\CHRIST~1\AppData\Local\Temp\tmpyug5lwbn\rdbms_connect-1.0.7-py2.py3-none-any.whl is OK
DEBUG: cli.azure.cli.core.extension.operations: Validation successful on C:\Users\CHRIST~1\AppData\Local\Temp\tmpyug5lwbn\rdbms_connect-1.0.7-py2.py3-none-any.whl
DEBUG: cli.azure.cli.core.extension.operations: Executing pip with args: ['install', '--target', 'C:\\Users\\ChristianHansen\\.azure\\cliextensions\\rdbms-connect', 'C:\\Users\\CHRIST~1\\AppData\\Local\\Temp\\tmpyug5lwbn\\rdbms_connect-1.0.7-py2.py3-none-any.whl']
DEBUG: cli.azure.cli.core.extension.operations: Running: ['C:\\Program Files\\Microsoft SDKs\\Azure\\CLI2\\python.exe', '-m', 'pip', 'install', '--target', 'C:\\Users\\ChristianHansen\\.azure\\cliextensions\\rdbms-connect', 'C:\\Users\\CHRIST~1\\AppData\\Local\\Temp\\tmpyug5lwbn\\rdbms_connect-1.0.7-py2.py3-none-any.whl', '-vv', '--disable-pip-version-check', '--no-cache-dir']
DEBUG: cli.azure.cli.core.extension.operations: 
DEBUG: cli.azure.cli.core.extension.operations: Command '['C:\\Program Files\\Microsoft SDKs\\Azure\\CLI2\\python.exe', '-m', 'pip', 'install', '--target', 'C:\\Users\\ChristianHansen\\.azure\\cliextensions\\rdbms-connect', 'C:\\Users\\CHRIST~1\\AppData\\Local\\Temp\\tmpyug5lwbn\\rdbms_connect-1.0.7-py2.py3-none-any.whl', '-vv', '--disable-pip-version-check', '--no-cache-dir']' returned non-zero exit status 3221225477.
DEBUG: cli.azure.cli.core.extension.operations: Pip failed so deleting anything we might have installed at C:\Users\ChristianHansen\.azure\cliextensions\rdbms-connect
DEBUG: cli.azure.cli.core.azclierror: Traceback (most recent call last):
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\knack/cli.py", line 233, in invoke
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 666, in execute
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 733, in _run_jobs_serially
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 703, in _run_job
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 336, in __call__
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/command_operation.py", line 121, in handler
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/command_modules/extension/custom.py", line 16, in add_extension_cmd
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/extension/operations.py", line 349, in add_extension
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/extension/operations.py", line 172, in _add_whl_ext
knack.util.CLIError: An error occurred. Pip failed with status code 3221225477. Use --debug for more information.

ERROR: cli.azure.cli.core.azclierror: An error occurred. Pip failed with status code 3221225477. Use --debug for more information.
ERROR: az_command_data_logger: An error occurred. Pip failed with status code 3221225477. Use --debug for more information.
DEBUG: cli.knack.cli: Event: Cli.PostExecute [<function AzCliLogging.deinit_cmd_metadata_logging at 0x00000170D2751C60>]
INFO: az_command_data_logger: exit code: 1
INFO: cli.__main__: Command ran in 6.766 seconds (init: 0.171, invoke: 6.594)
INFO: telemetry.main: Begin splitting cli events and extra events, total events: 1
INFO: telemetry.client: Accumulated 0 events. Flush the clients.
INFO: telemetry.main: Finish splitting cli events and extra events, cli events: 1
INFO: telemetry.save: Save telemetry record of length 3863 in cache file under C:\Users\ChristianHansen\.azure\telemetry\20241112093551740
INFO: telemetry.main: Begin creating telemetry upload process.
INFO: telemetry.process: Creating upload process: "C:\Program Files\Microsoft SDKs\Azure\CLI2\python.exe C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\azure\cli\telemetry\__init__.pyc C:\Users\ChristianHansen\.azure C:\Users\ChristianHansen\.azure\telemetry\20241112093551740"
INFO: telemetry.process: Return from creating process 33532
INFO: telemetry.main: Finish creating telemetry upload process.

Expected behavior

That the extension is installed without error.

Environment Summary

azure-cli                         2.66.0

core                              2.66.0
telemetry                          1.1.0

Dependencies:
msal                              1.31.0
azure-mgmt-resource               23.1.1

Python location 'C:\Program Files\Microsoft SDKs\Azure\CLI2\python.exe'
Extensions directory 'C:\Users\ChristianHansen\.azure\cliextensions'

Python (Windows) 3.12.7 (tags/v3.12.7:0b05ead, Oct  1 2024, 03:06:41) [MSC v.1941 64 bit (AMD64)]

Legal docs and information: aka.ms/AzureCliLegal

Additional context

Previous versions of the azure cli would output the following in the debug log:
Removed configobj>=5.0.6 from https://files.pythonhosted.org/packages/f5/c4/c7f9e41bc2e5f8eeae4a08a01c91b2aea3dfab40a3e14b25e87e7db8d501/configobj-5.0.9.tar.gz (from pgcli==4.0.1->rdbms-connect==1.0.7)

And further investigation shows that configobj version 5.0.9 is the culprit, it is the package that breaks the installation. Trying previous version like 5.0.8 doesn't have any problems.

A possible workaround is to create a constraints.txt file with the contents:

configobj == 5.0.8

And then run the installation command manually like so, to make sure the configobj dependency is constrained to a version that still works:

python.exe -m pip install --target C:\\Users\\<username>\\.azure\\cliextensions\\rdbms-connect <path-to-where-the-package-is-downloaded-to>\\rdbms_connect-1.0.7-py2.py3-none-any.whl -vv --disable-pip-version-check --no-cache-dir --constraint constraints.txt

[yonzhan]

Thank you for opening this issue, we will look into it.

[CezaryKlus]

Same with az extension add --name aks-preview

This happens on Windows, in Ubuntu the extension can be installed

[fullstackinfo]

This seems to be an issue for all extensions on python3.12 on windows 10/11 machines.

[bebound]

I'm not able to repro this.

what's the output of

cd 'C:\Program Files\Microsoft SDKs\Azure\CLI2'  
./python.exe -m pip install https://azcliprod.blob.core.windows.net/cli-extensions/rdbms_connect-1.0.7-py2.py3-none-any.whl --target ~\.azure\cliextensions\rdbms-connect

I guess some dependency fails to build, which is a common issue on Windows and can't be fixed from our side. See: Azure/azure-cli#25067 (comment)

[MidasCHA]

The main problem is that an extension doesn't vendor all its dependencies. This makes it very difficult to guarantee that the extension build is reproducible, like in the case above where pgcli has an open dependency like configobj>=5.0.6. When the extension was released configobj was at version 5.0.8 that worked fine, only to break once a new version 5.0.9 was released that doesn't work on windows.

[bebound]

I'm glad the cause is found: configobj 5.0.9 does not provide the wheel package.

@ambrahma for awareness.