Merge branch 'main' into pr-label-workflow

.github/dependabot.yml

@@ -1,5 +1,14 @@
 version: 2
 updates:
+- package-ecosystem: pip
+  directory: /requirements
+  schedule:
+    interval: monthly
+  groups:
+    actions:
+      patterns:
+      - 'typecheck.txt'
+
 - package-ecosystem: github-actions
   directory: /.github/workflows
   schedule:

.github/workflows/bleeding-edge.yaml

@@ -61,8 +61,7 @@ jobs:
       # are not installed by pip as specified from pyproject.toml, hence we get
       # to use the dev version of numpy at build time.
       run: |
-        python setup.py build_ext -q -j2
-        python -m pip install -e .[test] --no-build-isolation
+        python -m pip -v install -e .[test] --no-build-isolation
 
     - run: python -m pip list
 

.github/workflows/type-checking.yaml

@@ -7,7 +7,8 @@ on:
   pull_request:
     paths:
       - yt/**/*.py
-      - setup.cfg
+      - pyproject.toml
+      - requirements/typecheck.txt
       - .github/workflows/type-checking.yaml
   workflow_dispatch:
 
@@ -37,7 +38,7 @@ jobs:
     - name: Build
       run: |
         python3 -m pip install --upgrade pip
-        python3 -m pip install -e .[typecheck]
+        python3 -m pip install -e . -r requirements/typecheck.txt
 
     - run: python -m pip list
 

.github/workflows/wheels.yaml

@@ -13,6 +13,7 @@ on:
       - MANIFEST.in
   workflow_dispatch:
 
+
 jobs:
   build_wheels:
     name: Build wheels on ${{ matrix.os }}
@@ -21,7 +22,7 @@ jobs:
       matrix:
         os: [
           ubuntu-20.04,
-          windows-2019,
+          windows-2022,
           macos-13, # x86_64
           macos-14, # arm64
         ]
@@ -32,18 +33,9 @@ jobs:
         uses: actions/checkout@v4
 
       - name: Build wheels for CPython
-        uses: pypa/cibuildwheel@v2.17.0
+        uses: pypa/cibuildwheel@v2.21.1
         with:
           output-dir: dist
-        env:
-          CIBW_BUILD: "cp39-* cp310-* cp311-* cp312-*"
-          CIBW_SKIP: "*-musllinux_*"  #  numpy doesn't have wheels for musllinux so we can't build some quickly and without bloating
-          CIBW_ARCHS_LINUX: "x86_64"
-          CIBW_ARCHS_MACOS: auto
-          MACOSX_DEPLOYMENT_TARGET: "10.9" # as of CIBW 2.9, this is the default value, pin it so it can't be bumped silently
-          CIBW_ARCHS_WINDOWS: auto64
-          CIBW_ENVIRONMENT: "LDFLAGS='-static-libstdc++'"
-          CIBW_BUILD_VERBOSITY: 1
 
       - uses: actions/upload-artifact@v4
         with:
@@ -57,9 +49,22 @@ jobs:
       - name: Checkout repo
         uses: actions/checkout@v4
 
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.9'
+
       - name: Build sdist
         run: pipx run build --sdist
 
+      - name: Test sdist
+        run: |
+          python -m pip install "$(echo dist/*.tar.gz)[test]"
+          python -m pip list
+          project_dir=$(pwd)
+          cd ../../
+          pytest -c $project_dir/pyproject.toml --rootdir . --color=yes --pyargs yt
+
       - name: Upload sdist
         uses: actions/upload-artifact@v4
         with:
@@ -120,7 +125,7 @@ jobs:
           merge-multiple: true
 
       - name: Publish to PyPI
-        uses: pypa/gh-action-pypi-publish@v1.8.14
+        uses: pypa/gh-action-pypi-publish@v1.10.2
         with:
           user: __token__
           password: ${{ secrets.pypi_token }}

.pre-commit-config.yaml

@@ -18,7 +18,7 @@ ci:
 
 repos:
 -   repo: https://github.com/pre-commit/pre-commit-hooks
-    rev: v4.5.0
+    rev: v4.6.0
     hooks:
     - id: trailing-whitespace
     - id: end-of-file-fixer
@@ -30,27 +30,30 @@ repos:
 # TODO: replace this with ruff when it supports embedded python blocks
 # see https://github.com/astral-sh/ruff/issues/8237
 -   repo: https://github.com/adamchainz/blacken-docs
-    rev: 1.16.0
+    rev: 1.18.0
     hooks:
     -   id: blacken-docs
         additional_dependencies: [black==24.3.0]
 
 - repo: https://github.com/astral-sh/ruff-pre-commit
-  rev: v0.4.0
+  rev: v0.6.3
   hooks:
   - id: ruff-format
-    types_or: [ python, pyi, jupyter ]
   - id: ruff
-    types_or: [ python, pyi, jupyter ]
-    args: [--fix, "--show-fixes"]
+    args: [
+      --fix,
+      --show-fixes,
+      # the following line can be removed after support for Python 3.9 is dropped
+      --extend-select=B905, # zip-without-explicit-strict
+    ]
 
 -   repo: https://github.com/pre-commit/pygrep-hooks
     rev: v1.10.0
     hooks:
     -   id: rst-backticks
 
 - repo: https://github.com/MarcoGorelli/cython-lint
-  rev: v0.16.0
+  rev: v0.16.2
   hooks:
   - id: cython-lint
     args: [--no-pycodestyle]

CONTRIBUTING.rst

@@ -450,7 +450,7 @@ For all types of contributions, it is required that all tests pass, or that all
     future. (See :ref:`testing`)
   * At a minimum, a minimal, self-contained example demonstrating the bug should
     because included in the body of the Pull Request, or as part of an
-    indepedent issue.
+    independent issue.
 
 When submitting, you will be asked to make sure that your changes meet all of
 these requirements.  They are pretty easy to meet, and we're also happy to help

MANIFEST.in

@@ -8,9 +8,54 @@ include yt/utilities/mesh_types.yaml
 exclude yt/utilities/lib/cykdtree/c_kdtree.cpp
 prune tests
 prune answer-store
-recursive-include yt *.py *.pyx *.pxi *.pxd *.h *.hpp README* *.txt LICENSE* *.cu
+recursive-include yt *.py *.pyx *.pxi *.pxd  README* *.txt LICENSE* *.cu
 recursive-include doc *.rst *.txt *.py *.ipynb *.png *.jpg *.css *.html
 recursive-include doc *.h *.c *.sh *.svgz *.pdf *.svg *.pyx
+
+
+# start with excluding all C/C++ files
+recursive-exclude yt *.h *.c *.hpp *.cpp
+
+# then include back every non-generated C/C++ source file
+# the list can be generated by the following command
+# git ls-files | grep -E '\.(h|c)(pp)?$'
+include yt/frontends/artio/artio_headers/artio.c
+include yt/frontends/artio/artio_headers/artio.h
+include yt/frontends/artio/artio_headers/artio_endian.c
+include yt/frontends/artio/artio_headers/artio_endian.h
+include yt/frontends/artio/artio_headers/artio_file.c
+include yt/frontends/artio/artio_headers/artio_grid.c
+include yt/frontends/artio/artio_headers/artio_internal.h
+include yt/frontends/artio/artio_headers/artio_mpi.c
+include yt/frontends/artio/artio_headers/artio_mpi.h
+include yt/frontends/artio/artio_headers/artio_parameter.c
+include yt/frontends/artio/artio_headers/artio_particle.c
+include yt/frontends/artio/artio_headers/artio_posix.c
+include yt/frontends/artio/artio_headers/artio_selector.c
+include yt/frontends/artio/artio_headers/artio_sfc.c
+include yt/frontends/artio/artio_headers/cosmology.c
+include yt/frontends/artio/artio_headers/cosmology.h
+include yt/geometry/vectorized_ops.h
+include yt/utilities/lib/_octree_raytracing.hpp
+include yt/utilities/lib/cykdtree/c_kdtree.cpp
+include yt/utilities/lib/cykdtree/c_kdtree.hpp
+include yt/utilities/lib/cykdtree/c_utils.cpp
+include yt/utilities/lib/cykdtree/c_utils.hpp
+include yt/utilities/lib/cykdtree/windows/stdint.h
+include yt/utilities/lib/endian_swap.h
+include yt/utilities/lib/fixed_interpolator.cpp
+include yt/utilities/lib/fixed_interpolator.hpp
+include yt/utilities/lib/marching_cubes.h
+include yt/utilities/lib/mesh_triangulation.h
+include yt/utilities/lib/origami_tags.c
+include yt/utilities/lib/origami_tags.h
+include yt/utilities/lib/pixelization_constants.cpp
+include yt/utilities/lib/pixelization_constants.hpp
+include yt/utilities/lib/platform_dep.h
+include yt/utilities/lib/platform_dep_math.hpp
+include yt/utilities/lib/tsearch.c
+include yt/utilities/lib/tsearch.h
+
 include doc/README doc/activate doc/activate.csh doc/cheatsheet.tex
 exclude doc/cheatsheet.pdf
 include doc/extensions/README doc/Makefile
@@ -25,5 +70,6 @@ include yt/default.mplstyle
 
 prune yt/frontends/_skeleton
 recursive-include yt/frontends/amrvac *.par
+recursive-exclude requirements *.txt
 exclude .codecov.yml .coveragerc .git-blame-ignore-revs .gitmodules .hgchurn .mailmap
 exclude .pre-commit-config.yaml clean.sh nose_answer.cfg nose_unit.cfg nose_ignores.txt

conftest.py

@@ -85,7 +85,6 @@ def pytest_configure(config):
     Reads in the tests/tests.yaml file. This file contains a list of
     each answer test's answer file (including the changeset number).
     """
-    ytcfg["yt", "internals", "within_pytest"] = True
     # Register custom marks for answer tests and big data
     config.addinivalue_line("markers", "answer_test: Run the answer tests.")
     config.addinivalue_line(
@@ -161,19 +160,6 @@ def pytest_configure(config):
                 ":DeprecationWarning",
             )
 
-    if find_spec("astropy") is not None:
-        # at the time of writing, astropy's wheels are behind numpy's latest
-        # version but this doesn't cause actual problems in our test suite
-        # last updated with astropy 5.0 + numpy 1.22 + pytest 6.2.5
-        config.addinivalue_line(
-            "filterwarnings",
-            (
-                "ignore:numpy.ndarray size changed, may indicate binary incompatibility. Expected "
-                r"(80 from C header, got 88|88 from C header, got 96|80 from C header, got 96)"
-                " from PyObject:RuntimeWarning"
-            ),
-        )
-
     if PANDAS_VERSION is not None and PANDAS_VERSION >= Version("2.2.0"):
         config.addinivalue_line(
             "filterwarnings",

doc/cheatsheet.tex

@@ -319,7 +319,7 @@ \subsection{Git}
 \texttt{git status} \textemdash\ Show status of working tree.\\
 \texttt{git diff} \textemdash\ Show changed files in the working tree. \\
 \texttt{git log} \textemdash\ Show a log of changes in reverse chronological
-oder.\\
+order.\\
 \texttt{git revert <commit>} \textemdash\ Revert the changes in an existing
 commit and create a new commit with reverted changes. \\
 \texttt{git add <pathspec>} \textemdash\ Stage changes in the working tree to

doc/source/analyzing/fields.rst

@@ -368,7 +368,7 @@ second sets the corresponding ``value``. Currently available format properties a
 
 .. _efields:
 
-Energy and Momemtum Fields
+Energy and Momentum Fields
 --------------------------
 
 Fields in yt representing energy and momentum quantities follow a specific

doc/source/analyzing/generating_processed_data.rst

@@ -54,16 +54,17 @@ the transformation of a variable mesh of points consisting of positions and
 sizes into a fixed-size array that appears like an image.  This process is that
 of pixelization, which yt handles transparently internally.  You can access
 this functionality by constructing a
-:class:`~yt.visualization.fixed_resolution.FixedResolutionBuffer` and supplying
+:class:`~yt.visualization.fixed_resolution.FixedResolutionBuffer`
+and supplying
 to it your :class:`~yt.data_objects.data_containers.YTSelectionContainer2D`
 object, as well as some information about how you want the final image to look.
 You can specify both the bounds of the image (in the appropriate x-y plane) and
 the resolution of the output image.  You can then have yt pixelize any field
 you like.
 
 .. note:: In previous versions of yt, there was a special class of
-          FixedResolutionBuffer for off-axis slices.  This is no longer
-          necessary.
+          FixedResolutionBuffer for off-axis slices.  This is still used
+          for off-axis SPH data projections: OffAxisFixedResolutionBuffer.
 
 To create :class:`~yt.data_objects.data_containers.YTSelectionContainer2D` objects, you can
 access them as described in :ref:`data-objects`, specifically the section
@@ -99,7 +100,10 @@ this, see :ref:`saving-grid-data-containers`.
 In the FITS case, there is an option for setting the ``units`` of the coordinate system in
 the file. If you want to overwrite a file with the same name, set ``clobber=True``.
 
-The :class:`~yt.visualization.fixed_resolution.FixedResolutionBuffer` can even be exported
+The :class:`~yt.visualization.fixed_resolution.FixedResolutionBuffer`
+(and its
+:class:`~yt.visualization.fixed_resolution.OffAxisProjectionFixedResolutionBuffer`
+subclass) can even be exported
 as a 2D dataset itself, which may be operated on in the same way as any other dataset in yt:
 
 .. code-block:: python

doc/source/developing/building_the_docs.rst

@@ -90,7 +90,7 @@ the top level of a local copy, run
 
 .. code-block:: bash
 
-   $ python -m pip install -e ".[doc]"
+   $ python -m pip install -e . -r requirements/docs.txt
 
 Quick versus Full Documentation Builds
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

doc/source/developing/creating_frontend.rst

@@ -56,15 +56,16 @@ called ``_is_valid()`` that lets the ``yt.load`` method help identify an
 input file as belonging to *this* particular ``Dataset`` subclass
 (see :ref:`data-format-detection`).
 For the most part, the examples of
-``yt.frontends.boxlib.data_structures.OrionDataset`` and
+``yt.frontends.amrex.data_structures.OrionDataset`` and
 ``yt.frontends.enzo.data_structures.EnzoDataset`` should be followed,
 but ``yt.frontends.chombo.data_structures.ChomboDataset``, as a
 slightly newer addition, can also be used as an instructive example.
 
 A new set of fields must be added in the file ``fields.py`` in your
 new directory.  For the most part this means subclassing
 ``FieldInfoContainer`` and adding the necessary fields specific to
-your code. Here is a snippet from the base BoxLib field container:
+your code. Here is a snippet from the base BoxLib field container (defined in
+``yt.frontends.amrex.fields``):
 
 .. code-block:: python
 
@@ -273,7 +274,7 @@ that is needed:
 
 
 Even one of the more complex grid objects,
-``yt.frontends.boxlib.BoxlibGrid``, is still relatively simple.
+``yt.frontends.amrex.BoxlibGrid``, is still relatively simple.
 
 Data Reading Functions
 ----------------------
@@ -310,7 +311,7 @@ At a minimum, one should also override the following methods
 If your dataset has particle information, you'll want to override the
 ``_read_particle_coords()`` and ``read_particle_fields()`` methods as
 well.  Each code is going to read data from disk in a different
-fashion, but the ``yt.frontends.boxlib.io.IOHandlerBoxlib`` is a
+fashion, but the ``yt.frontends.amrex.io.IOHandlerBoxlib`` is a
 decent place to start.
 
 And that just about covers it. Please feel free to email

doc/source/developing/debugdrive.rst

@@ -66,7 +66,7 @@ will induce the requested behavior.
    SIGUSR1
      This will cause the python code to print a stack trace, showing exactly
      where in the function stack it is currently executing.
-   SIGUSR1
+   SIGUSR2
      This will cause the python code to insert an IPython session wherever it
      currently is, with all local variables in the local namespace.  It should
      allow you to change the state variables.