From 5ffc011d121d88446205e42ecde34ce962dcf7fe Mon Sep 17 00:00:00 2001 From: "Gregory P. Smith" Date: Sat, 1 Oct 2022 17:55:40 -0700 Subject: [PATCH] gh-95588: Drop the safety claim from `ast.literal_eval` docs. (#95919) It was never really safe and this claim conflicts directly with the big warning in the docs about it being able to crash the interpreter. --- Doc/library/ast.rst | 24 ++++++++++++------- Lib/ast.py | 4 +++- ...2-08-12-01-12-52.gh-issue-95588.PA0FI7.rst | 6 +++++ 3 files changed, 25 insertions(+), 9 deletions(-) create mode 100644 Misc/NEWS.d/next/Documentation/2022-08-12-01-12-52.gh-issue-95588.PA0FI7.rst diff --git a/Doc/library/ast.rst b/Doc/library/ast.rst index 0349130d29227f..0811b3fa0e7842 100644 --- a/Doc/library/ast.rst +++ b/Doc/library/ast.rst @@ -1991,20 +1991,28 @@ and classes for traversing abstract syntax trees: .. function:: literal_eval(node_or_string) - Safely evaluate an expression node or a string containing a Python literal or + Evaluate an expression node or a string containing only a Python literal or container display. The string or node provided may only consist of the following Python literal structures: strings, bytes, numbers, tuples, lists, dicts, sets, booleans, ``None`` and ``Ellipsis``. - This can be used for safely evaluating strings containing Python values from - untrusted sources without the need to parse the values oneself. It is not - capable of evaluating arbitrarily complex expressions, for example involving - operators or indexing. + This can be used for evaluating strings containing Python values without the + need to parse the values oneself. It is not capable of evaluating + arbitrarily complex expressions, for example involving operators or + indexing. + + This function had been documented as "safe" in the past without defining + what that meant. That was misleading. This is specifically designed not to + execute Python code, unlike the more general :func:`eval`. There is no + namespace, no name lookups, or ability to call out. But it is not free from + attack: A relatively small input can lead to memory exhaustion or to C stack + exhaustion, crashing the process. There is also the possibility for + excessive CPU consumption denial of service on some inputs. Calling it on + untrusted data is thus not recommended. .. warning:: - It is possible to crash the Python interpreter with a - sufficiently large/complex string due to stack depth limitations - in Python's AST compiler. + It is possible to crash the Python interpreter due to stack depth + limitations in Python's AST compiler. It can raise :exc:`ValueError`, :exc:`TypeError`, :exc:`SyntaxError`, :exc:`MemoryError` and :exc:`RecursionError` depending on the malformed diff --git a/Lib/ast.py b/Lib/ast.py index 8adb61fed45388..1a94e9368c161a 100644 --- a/Lib/ast.py +++ b/Lib/ast.py @@ -54,10 +54,12 @@ def parse(source, filename='', mode='exec', *, def literal_eval(node_or_string): """ - Safely evaluate an expression node or a string containing a Python + Evaluate an expression node or a string containing only a Python expression. The string or node provided may only consist of the following Python literal structures: strings, bytes, numbers, tuples, lists, dicts, sets, booleans, and None. + + Caution: A complex expression can overflow the C stack and cause a crash. """ if isinstance(node_or_string, str): node_or_string = parse(node_or_string.lstrip(" \t"), mode='eval') diff --git a/Misc/NEWS.d/next/Documentation/2022-08-12-01-12-52.gh-issue-95588.PA0FI7.rst b/Misc/NEWS.d/next/Documentation/2022-08-12-01-12-52.gh-issue-95588.PA0FI7.rst new file mode 100644 index 00000000000000..c070bbc19517fc --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2022-08-12-01-12-52.gh-issue-95588.PA0FI7.rst @@ -0,0 +1,6 @@ +Clarified the conflicting advice given in the :mod:`ast` documentation about +:func:`ast.literal_eval` being "safe" for use on untrusted input while at +the same time warning that it can crash the process. The latter statement is +true and is deemed unfixable without a large amount of work unsuitable for a +bugfix. So we keep the warning and no longer claim that ``literal_eval`` is +safe.