Merge branch 'main' into re-memleaks

.github/CODEOWNERS

@@ -16,14 +16,17 @@ configure*                    @erlend-aasland @corona10
 Makefile.pre.in               @erlend-aasland
 Modules/Setup*                @erlend-aasland
 
+# argparse
+**/*argparse*                 @savannahostrowski
+
 # asyncio
 **/*asyncio*                  @1st1 @asvetlov @kumaraditya303 @willingc
 
 # Core
 **/*context*                  @1st1
 **/*genobject*                @markshannon
 **/*hamt*                     @1st1
-**/*jit*                      @brandtbucher
+**/*jit*                      @brandtbucher @savannahostrowski
 Objects/set*                  @rhettinger
 Objects/dict*                 @methane @markshannon
 Objects/typevarobject.c       @JelleZijlstra

.github/workflows/build.yml

@@ -46,7 +46,7 @@ jobs:
     # reproducible: to get the same tools versions (autoconf, aclocal, ...)
     runs-on: ubuntu-24.04
     container:
-      image: ghcr.io/python/autoconf:2024.10.16.11360930377
+      image: ghcr.io/python/autoconf:2024.11.11.11786316759
     timeout-minutes: 60
     needs: check_source
     if: needs.check_source.outputs.run_tests == 'true'

Doc/c-api/marshal.rst

@@ -13,11 +13,12 @@ binary mode.
 
 Numeric values are stored with the least significant byte first.
 
-The module supports two versions of the data format: version 0 is the
-historical version, version 1 shares interned strings in the file, and upon
-unmarshalling.  Version 2 uses a binary format for floating-point numbers.
-``Py_MARSHAL_VERSION`` indicates the current file format (currently 2).
+The module supports several versions of the data format; see
+the :py:mod:`Python module documentation <marshal>` for details.
 
+.. c:macro:: Py_MARSHAL_VERSION
+
+   The current format version. See :py:data:`marshal.version`.
 
 .. c:function:: void PyMarshal_WriteLongToFile(long value, FILE *file, int version)
 

Doc/c-api/object.rst

@@ -575,3 +575,27 @@ Object Protocol
    has the :c:macro:`Py_TPFLAGS_MANAGED_DICT` flag set.
 
    .. versionadded:: 3.13
+
+.. c:function:: int PyUnstable_Object_EnableDeferredRefcount(PyObject *obj)
+
+   Enable `deferred reference counting <https://peps.python.org/pep-0703/#deferred-reference-counting>`_ on *obj*,
+   if supported by the runtime.  In the :term:`free-threaded <free threading>` build,
+   this allows the interpreter to avoid reference count adjustments to *obj*,
+   which may improve multi-threaded performance.  The tradeoff is
+   that *obj* will only be deallocated by the tracing garbage collector.
+
+   This function returns ``1`` if deferred reference counting is enabled on *obj*
+   (including when it was enabled before the call),
+   and ``0`` if deferred reference counting is not supported or if the hint was
+   ignored by the runtime. This function is thread-safe, and cannot fail.
+
+   This function does nothing on builds with the :term:`GIL` enabled, which do
+   not support deferred reference counting. This also does nothing if *obj* is not
+   an object tracked by the garbage collector (see :func:`gc.is_tracked` and
+   :c:func:`PyObject_GC_IsTracked`).
+
+   This function is intended to be used soon after *obj* is created,
+   by the code that creates it.
+
+   .. versionadded:: next
+

Doc/data/refcounts.dat

@@ -1284,6 +1284,19 @@ PyLong_FromUnsignedLong:unsignedlong:v::
 PyLong_FromVoidPtr:PyObject*::+1:
 PyLong_FromVoidPtr:void*:p::
 
+PyLong_IsPositive:int:::
+PyLong_IsPositive:PyObject*:obj:0:
+
+PyLong_IsNegative:int:::
+PyLong_IsNegative:PyObject*:obj:0:
+
+PyLong_IsZero:int:::
+PyLong_IsZero:PyObject*:obj:0:
+
+PyLong_GetSign:int:::
+PyLong_GetSign:PyObject*:v:0:
+PyLong_GetSign:int*:sign::
+
 PyMapping_Check:int:::
 PyMapping_Check:PyObject*:o:0:
 

Doc/library/getopt.rst

@@ -85,6 +85,16 @@ exception:
    variable :envvar:`!POSIXLY_CORRECT` is set, then option processing stops as
    soon as a non-option argument is encountered.
 
+   If the first character of the option string is ``'-'``, non-option arguments
+   that are followed by options are added to the list of option-and-value pairs
+   as a pair that has ``None`` as its first element and the list of non-option
+   arguments as its second element.
+   The second element of the :func:`!gnu_getopt` result is a list of
+   program arguments after the last option.
+
+   .. versionchanged:: 3.14
+      Support for returning intermixed options and non-option arguments in order.
+
 
 .. exception:: GetoptError
 
@@ -144,6 +154,20 @@ Optional arguments should be specified explicitly:
    >>> args
    ['a1', 'a2']
 
+The order of options and non-option arguments can be preserved:
+
+.. doctest::
+
+   >>> s = 'a1 -x a2 a3 a4 --long a5 a6'
+   >>> args = s.split()
+   >>> args
+   ['a1', '-x', 'a2', 'a3', 'a4', '--long', 'a5', 'a6']
+   >>> optlist, args = getopt.gnu_getopt(args, '-x:', ['long='])
+   >>> optlist
+   [(None, ['a1']), ('-x', 'a2'), (None, ['a3', 'a4']), ('--long', 'a5')]
+   >>> args
+   ['a6']
+
 In a script, typical usage is something like this:
 
 .. testcode::

Doc/library/marshal.rst

@@ -38,23 +38,39 @@ supports a substantially wider range of objects than marshal.
    maliciously constructed data.  Never unmarshal data received from an
    untrusted or unauthenticated source.
 
+There are functions that read/write files as well as functions operating on
+bytes-like objects.
+
 .. index:: object; code, code object
 
 Not all Python object types are supported; in general, only objects whose value
 is independent from a particular invocation of Python can be written and read by
-this module.  The following types are supported: booleans, integers, floating-point
-numbers, complex numbers, strings, bytes, bytearrays, tuples, lists, sets,
-frozensets, dictionaries, and code objects (if *allow_code* is true),
-where it should be understood that
-tuples, lists, sets, frozensets and dictionaries are only supported as long as
-the values contained therein are themselves supported.  The
-singletons :const:`None`, :const:`Ellipsis` and :exc:`StopIteration` can also be
-marshalled and unmarshalled.
-For format *version* lower than 3, recursive lists, sets and dictionaries cannot
-be written (see below).
+this module.  The following types are supported:
+
+* Numeric types: :class:`int`, :class:`bool`, :class:`float`, :class:`complex`.
+* Strings (:class:`str`) and :class:`bytes`.
+  :term:`Bytes-like objects <bytes-like object>` like :class:`bytearray` are
+  marshalled as :class:`!bytes`.
+* Containers: :class:`tuple`, :class:`list`, :class:`set`, :class:`frozenset`,
+  and (since :data:`version` 5), :class:`slice`.
+  It should be understood that these are supported only if the values contained
+  therein are themselves supported.
+  Recursive containers are supported since :data:`version` 3.
+* The singletons :const:`None`, :const:`Ellipsis` and :exc:`StopIteration`.
+* :class:`code` objects, if *allow_code* is true. See note above about
+  version dependence.
+
+.. versionchanged:: 3.4
+
+   * Added format version 3, which supports marshalling recursive lists, sets
+     and dictionaries.
+   * Added format version 4, which supports efficient representations
+     of short strings.
+
+.. versionchanged:: next
+
+   Added format version 5, which allows marshalling slices.
 
-There are functions that read/write files as well as functions operating on
-bytes-like objects.
 
 The module defines these functions:
 
@@ -140,11 +156,24 @@ In addition, the following constants are defined:
 
 .. data:: version
 
-   Indicates the format that the module uses. Version 0 is the historical
-   format, version 1 shares interned strings and version 2 uses a binary format
-   for floating-point numbers.
-   Version 3 adds support for object instancing and recursion.
-   The current version is 4.
+   Indicates the format that the module uses.
+   Version 0 is the historical first version; subsequent versions
+   add new features.
+   Generally, a new version becomes the default when it is introduced.
+
+   ======= =============== ====================================================
+   Version Available since New features
+   ======= =============== ====================================================
+   1       Python 2.4      Sharing interned strings
+   ------- --------------- ----------------------------------------------------
+   2       Python 2.5      Binary representation of floats
+   ------- --------------- ----------------------------------------------------
+   3       Python 3.4      Support for object instancing and recursion
+   ------- --------------- ----------------------------------------------------
+   4       Python 3.4      Efficient representation of short strings
+   ------- --------------- ----------------------------------------------------
+   5       Python 3.14     Support for :class:`slice` objects
+   ======= =============== ====================================================
 
 
 .. rubric:: Footnotes
@@ -154,4 +183,3 @@ In addition, the following constants are defined:
    around in a self-contained form. Strictly speaking, "to marshal" means to
    convert some data from internal to external form (in an RPC buffer for instance)
    and "unmarshalling" for the reverse process.
-

Doc/library/platform.rst

@@ -359,3 +359,15 @@ Android Platform
    <https://storage.googleapis.com/play_public/supported_devices.html>`__.
 
    .. versionadded:: 3.13
+
+
+Miscellaneous
+-------------
+
+.. function:: invalidate_caches()
+
+   Clear out the internal cache of information, such as the :func:`uname`.
+   This is typically useful when the platform's :func:`node` is changed
+   by an external process and one needs to retrieve the updated value.
+
+   .. versionadded:: 3.14

Doc/library/pprint.rst

@@ -267,7 +267,7 @@ let's fetch information about a project from `PyPI <https://pypi.org>`_::
    >>> import json
    >>> import pprint
    >>> from urllib.request import urlopen
-   >>> with urlopen('https://pypi.org/pypi/sampleproject/json') as resp:
+   >>> with urlopen('https://pypi.org/pypi/sampleproject/1.2.0/json') as resp:
    ...     project_info = json.load(resp)['info']
 
 In its basic form, :func:`~pprint.pp` shows the whole object::

Doc/library/socket.rst

@@ -928,7 +928,9 @@ The :mod:`socket` module also offers various network-related services:
 
    .. versionadded:: 3.7
 
-.. function:: getaddrinfo(host, port, family=0, type=0, proto=0, flags=0)
+.. function:: getaddrinfo(host, port, family=AF_UNSPEC, type=0, proto=0, flags=0)
+
+   This function wraps the C function ``getaddrinfo`` of the underlying system.
 
    Translate the *host*/*port* argument into a sequence of 5-tuples that contain
    all the necessary arguments for creating a socket connected to that service.
@@ -938,8 +940,10 @@ The :mod:`socket` module also offers various network-related services:
    and *port*, you can pass ``NULL`` to the underlying C API.
 
    The *family*, *type* and *proto* arguments can be optionally specified
-   in order to narrow the list of addresses returned.  Passing zero as a
-   value for each of these arguments selects the full range of results.
+   in order to provide options and limit the list of addresses returned.
+   Pass their default values (:data:`AF_UNSPEC`, 0, and 0, respectively)
+   to not limit the results. See the note below for details.
+
    The *flags* argument can be one or several of the ``AI_*`` constants,
    and will influence how results are computed and returned.
    For example, :const:`AI_NUMERICHOST` will disable domain name resolution
@@ -959,6 +963,29 @@ The :mod:`socket` module also offers various network-related services:
    :const:`AF_INET6`), and is meant to be passed to the :meth:`socket.connect`
    method.
 
+   .. note::
+
+      If you intend to use results from :func:`!getaddrinfo` to create a socket
+      (rather than, for example, retrieve *canonname*),
+      consider limiting the results by *type* (e.g. :data:`SOCK_STREAM` or
+      :data:`SOCK_DGRAM`) and/or *proto* (e.g. :data:`IPPROTO_TCP` or
+      :data:`IPPROTO_UDP`) that your application can handle.
+
+      The behavior with default values of *family*, *type*, *proto*
+      and *flags* is system-specific.
+
+      Many systems (for example, most Linux configurations) will return a sorted
+      list of all matching addresses.
+      These addresses should generally be tried in order until a connection succeeds
+      (possibly tried in parallel, for example, using a `Happy Eyeballs`_ algorithm).
+      In these cases, limiting the *type* and/or *proto* can help eliminate
+      unsuccessful or unusable connecton attempts.
+
+      Some systems will, however, only return a single address.
+      (For example, this was reported on Solaris and AIX configurations.)
+      On these systems, limiting the *type* and/or *proto* helps ensure that
+      this address is usable.
+
    .. audit-event:: socket.getaddrinfo host,port,family,type,protocol socket.getaddrinfo
 
    The following example fetches address information for a hypothetical TCP
@@ -978,6 +1005,8 @@ The :mod:`socket` module also offers various network-related services:
       for IPv6 multicast addresses, string representing an address will not
       contain ``%scope_id`` part.
 
+.. _Happy Eyeballs: https://en.wikipedia.org/wiki/Happy_Eyeballs
+
 .. function:: getfqdn([name])
 
    Return a fully qualified domain name for *name*. If *name* is omitted or empty,

Doc/library/tomllib.rst

@@ -60,9 +60,36 @@ This module defines the following functions:
 
 The following exceptions are available:
 
-.. exception:: TOMLDecodeError
+.. exception:: TOMLDecodeError(msg, doc, pos)
 
-   Subclass of :exc:`ValueError`.
+   Subclass of :exc:`ValueError` with the following additional attributes:
+
+   .. attribute:: msg
+
+      The unformatted error message.
+
+   .. attribute:: doc
+
+      The TOML document being parsed.
+
+   .. attribute:: pos
+
+      The index of *doc* where parsing failed.
+
+   .. attribute:: lineno
+
+      The line corresponding to *pos*.
+
+   .. attribute:: colno
+
+      The column corresponding to *pos*.
+
+   .. versionchanged:: next
+      Added the *msg*, *doc* and *pos* parameters.
+      Added the :attr:`msg`, :attr:`doc`, :attr:`pos`, :attr:`lineno` and :attr:`colno` attributes.
+
+   .. deprecated:: next
+      Passing free-form positional arguments is deprecated.
 
 
 Examples

Doc/library/urllib.parse.rst

@@ -407,6 +407,15 @@ or on combining URL components into a URL string.
       If you do not want that behavior, preprocess the *url* with :func:`urlsplit` and
       :func:`urlunsplit`, removing possible *scheme* and *netloc* parts.
 
+   .. warning::
+
+      Because an absolute URL may be passed as the ``url`` parameter, it is
+      generally **not secure** to use ``urljoin`` with an attacker-controlled
+      ``url``. For example in,
+      ``urljoin("https://website.com/users/", username)``, if ``username`` can
+      contain an absolute URL, the result of ``urljoin`` will be the absolute
+      URL.
+
 
    .. versionchanged:: 3.5
 

Doc/library/uuid.rst

@@ -11,9 +11,9 @@
 --------------
 
 This module provides immutable :class:`UUID` objects (the :class:`UUID` class)
-and the functions :func:`uuid1`, :func:`uuid3`, :func:`uuid4`, :func:`uuid5` for
-generating version 1, 3, 4, 5, and 8 UUIDs as specified in :rfc:`9562` (which
-supersedes :rfc:`4122`).
+and the functions :func:`uuid1`, :func:`uuid3`, :func:`uuid4`, :func:`uuid5`,
+and :func:`uuid.uuid8` for generating version 1, 3, 4, 5, and 8 UUIDs as
+specified in :rfc:`9562` (which supersedes :rfc:`4122`).
 
 If all you want is a unique ID, you should probably call :func:`uuid1` or
 :func:`uuid4`.  Note that :func:`uuid1` may compromise privacy since it creates
@@ -323,6 +323,9 @@ The following options are accepted:
    Specify the function name to use to generate the uuid. By default :func:`uuid4`
    is used.
 
+   .. versionadded:: next
+      Allow generating UUID version 8.
+
 .. option:: -n <namespace>
             --namespace <namespace>
 

Doc/reference/expressions.rst

@@ -1156,7 +1156,8 @@ a user-defined function:
    first thing the code block will do is bind the formal parameters to the
    arguments; this is described in section :ref:`function`.  When the code block
    executes a :keyword:`return` statement, this specifies the return value of the
-   function call.
+   function call.  If execution reaches the end of the code block without
+   executing a :keyword:`return` statement, the return value is ``None``.
 
 a built-in function or method:
    .. index::