[Bug] - ansible-playbook fails to launch on AL2023 20250123 x86_64

[dmacbride-ep]

Describe the bug
ansible-playbook with Ansible 8.3.0 fails to launch on containers run from the new amazonlinux:2023 20250123 x86_64 image.

To Reproduce
Steps to reproduce the behavior:

1. docker pull public.ecr.aws/amazonlinux/amazonlinux:2023
2. docker run -it --entrypoint bash public.ecr.aws/amazonlinux/amazonlinux:2023
3. Inside the container run: dnf -y install ansible-8.3.0
4. Inside the container run: ansible-playbook
5. See error:

bash-5.2# ansible-playbook
Traceback (most recent call last):
  File "/usr/bin/ansible-playbook", line 33, in <module>
    sys.exit(load_entry_point('ansible-core==2.15.3', 'console_scripts', 'ansible-playbook')())
  File "/usr/bin/ansible-playbook", line 25, in importlib_load_entry_point
    return next(matches).load()
  File "/usr/lib64/python3.9/importlib/metadata.py", line 86, in load
    module = import_module(match.group('module'))
  File "/usr/lib64/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 972, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/usr/lib/python3.9/site-packages/ansible/cli/__init__.py", line 111, in <module>
    from ansible.vars.manager import VariableManager
  File "/usr/lib/python3.9/site-packages/ansible/vars/manager.py", line 39, in <module>
    from ansible.template import Templar
  File "/usr/lib/python3.9/site-packages/ansible/template/__init__.py", line 52, in <module>
    from ansible.template.native_helpers import AnsibleUndefined, ansible_native_concat, ansible_eval_concat, ansible_concat
  File "/usr/lib/python3.9/site-packages/ansible/template/native_helpers.py", line 10
    import collections.abc import Mapping
                           ^
SyntaxError: invalid syntax
bash-5.2# 

Expected behavior
The usual ansible-playbook usage is expected to be shown

Desktop (please complete the following information):

• OS: AmazonLinux 2023 container on x86_64 MacOS Sonoma 14.7.2 with Docker Desktop 4.37.2

I can't reproduce this with amazonlinux:2023 20250115. It seems to be a new issue with the 20250123 build.

[zcobol]

The issue seems to be introduced by the ansible-core-2.15.3-1.amzn2023.0.6.rpm package. If swapping with the version from 2023.6.20250115 then ansible-playbook starts okay.

The new ansible-core version has three CVEs fixes:

* Thu Jan 09 2025 Rahman Ajibade <[email protected]> - 2.15.3-1.amzn2023.0.6
- Fix CVE-2024-11079

* Fri Jan 03 2025 Bill Karh <[email protected]> - 2.15.3-1.amzn2023.0.5
- Fix CVE-2024-56201
- Fix CVE-2024-56326

However it fails to launch the ansible-playbook with invalid syntax on line 33:

(Pdb) n
> /usr/bin/ansible-playbook(33)<module>()
-> sys.exit(load_entry_point('ansible-core==2.15.3', 'console_scripts', 'ansible-playbook')())
(Pdb) l
 28     globals().setdefault('load_entry_point', importlib_load_entry_point)
 29  
 30  
 31     if __name__ == '__main__':
 32         sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
 33  ->     sys.exit(load_entry_point('ansible-core==2.15.3', 'console_scripts', 'ansible-playbook')())
[EOF]
(Pdb) n
SyntaxError: invalid syntax
> /usr/bin/ansible-playbook(33)<module>()
-> sys.exit(load_entry_point('ansible-core==2.15.3', 'console_scripts', 'ansible-playbook')())
(Pdb) 

Download ansible-core from the previous release and swap packages:

dnf swap ansible-core ./ansible-core-2.15.3-1.amzn2023.0.4.x86_64.rpm

The invalid syntax is gone:

(Pdb) n
> /usr/bin/ansible-playbook(31)<module>()
-> if __name__ == '__main__':
(Pdb) n
> /usr/bin/ansible-playbook(32)<module>()
-> sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
(Pdb) n
> /usr/bin/ansible-playbook(33)<module>()
-> sys.exit(load_entry_point('ansible-core==2.15.3', 'console_scripts', 'ansible-playbook')())
(Pdb) n
usage: ansible-playbook [-h] [--version] [-v] [--private-key PRIVATE_KEY_FILE] [-u REMOTE_USER]
---cut---

[zcobol]

ansible-core-2.15.3-1.amzn2023.0.6.rpm is not ready for prime time!

The first failure reported by @dmacbride-ep is due to invalid syntax in /usr/lib/python3.9/site-packages/ansible/template/native_helpers.py line 10:

import collections.abc import Mapping

causing the invalid syntax error. That should be from collections.abc import Mapping

The next error is on line 22:

from jinja.runtime import StrictUndefined

it should be from jinja2.runtime import StrictUndefined

[samueloph]

Thank you for reporting and confirming the issue @dmacbride-ep @zcobol, I have forwarded this to the team that owns the package on AL23 with high severity.

[balajimaniv]

Until this bug get resolved use older ansible-core as workaround
dnf install ansible-core-2.15.3-1.amzn2023.0.4.x86_64

[joeysk2012]

Amazon Linux will fix this issue in the next release.
Customers can mitigate by downgrading using the command sudo dnf install ansible-core-2.15.3-1.amzn2023.0.5.

[enfont]

My team has opened a support case with them, and we can confirm that via both Support and the TAM of our account, AWS is aware of this bug and is working to deploy a fix as soon as possible ™

[dmacbride-ep]

We are working around the issue for now by fixing on the previous image.

• ECR: public.ecr.aws/amazonlinux/amazonlinux:2023.6.20250115.0
• DockerHub: amazonlinux:2023.6.20250115.0

[jafd]

Also a typo — "mosule_utils" instead of "module_utils". Looks as if someone retyped a patch by hand in a great hurry... Oops.

How come it's being packaged and pushed to production without any kind of testing?

[spaceriqui]

Also a typo — "mosule_utils" instead of "module_utils". Looks as if someone retyped a patch by hand in a great hurry... Oops.

How come it's being packaged and pushed to production without any kind of testing?

This version is 2.15.3 and the proposed changes seem to be for 2.16.14 with typos. I think someone packaged from a local working copy instead. I mean, ansible --version doesn't even work. Major oops.