Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sph projections, slices, gridding: backend and off-axis #4939

Merged
merged 64 commits into from
Sep 16, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
eec9a29
draft 1 off-axis projection and small-particle proj. fix (off-axis ba…
nastasha-w Feb 14, 2024
56c93d3
debugging: axis-aligned SPH projection works again, off-axis runs, bu…
nastasha-w Feb 17, 2024
44d6d46
partway debugging off-axis SPH-proj.
nastasha-w Jul 1, 2024
30d60fa
off-axis projections seem to work
nastasha-w Jul 3, 2024
2ff58cd
first attempt to fix pixelize_sph_kernel_slice
nastasha-w Jul 3, 2024
13cd93a
attempt 1 to fix axis-aligned SPH slices
nastasha-w Jul 3, 2024
9bca940
draft 1 backend SPH cutting plane
nastasha-w Jul 3, 2024
22985f7
passing dataset kernel names to SPH slice/projection backend; work on…
nastasha-w Jul 4, 2024
6a874fe
draft 1 fixed slices, new off-axis slices SPH
nastasha-w Jul 4, 2024
8b3ca65
further bugfix attempts
nastasha-w Jul 5, 2024
31aae23
debugged SPH slices to run; outcome clearly wrong off-axis
nastasha-w Jul 5, 2024
690d64f
partial debug along-axis SPH slices
nastasha-w Jul 6, 2024
695f196
fix complaint from ruff
nastasha-w Jul 6, 2024
80cb90d
debugging: axis-aligned slices pass tests
nastasha-w Jul 10, 2024
1907b2f
fix a ruff complaint
nastasha-w Jul 11, 2024
a80228a
on/off-axis SPH slices actually work now
nastasha-w Jul 11, 2024
8e5aca2
hopefully fixed induced grid projection bug;
nastasha-w Jul 12, 2024
9cc98ba
partial debug issues from pull request tests
nastasha-w Jul 12, 2024
429536d
attempted fix bounds issues in off_axis_projection calls (non-SPH ds)
nastasha-w Jul 12, 2024
a088d5f
attempted fix: ParticleImageBuffer counts on rotate_particle_coord fu…
nastasha-w Jul 12, 2024
effd351
add new pz argument to test_off_axis_SPH pixelize_sph_kernel_projecti…
nastasha-w Jul 12, 2024
33b2c10
attempted fix issue with non-coordinate center arg off_axis_projection
nastasha-w Jul 12, 2024
0d13beb
attempted fix ruff complaint import sorting
nastasha-w Jul 12, 2024
15fcea4
removed unneeded parentheses
nastasha-w Jul 12, 2024
5bb4bd7
SPH scatter gridding fixes, tests passed
nastasha-w Jul 17, 2024
222500e
minor documentation updates
nastasha-w Jul 17, 2024
f5c476a
doc updates I forgot to save
nastasha-w Jul 17, 2024
7d8b676
fix ruff comment: not declaring unused h_j3
nastasha-w Jul 17, 2024
170e9af
fix test_offaxis_moment failure
nastasha-w Jul 17, 2024
9b91e4f
fix ruff comment end of file?
nastasha-w Jul 17, 2024
71cc131
draft 1 adding SPH pixelization tests (copied from separate file)
nastasha-w Jul 17, 2024
159167b
fix ruff issues
nastasha-w Jul 18, 2024
2f589c9
order imports correctly?
nastasha-w Jul 18, 2024
aecc051
separate out pytest-only tests, add to nose ignore lists
nastasha-w Jul 18, 2024
fa78f5c
remove trailing whitespace
nastasha-w Jul 18, 2024
785108a
add typing option of (array, unit) values
nastasha-w Jul 18, 2024
fec520f
attempted change typing to work with python 3.9
nastasha-w Jul 18, 2024
f4dd2cc
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 18, 2024
d4e5bfb
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 18, 2024
b4f4935
attempting to pass type-checking
nastasha-w Jul 18, 2024
ebb357c
ruff fix
nastasha-w Jul 18, 2024
f09851c
more attempted type check fixes
nastasha-w Jul 19, 2024
987e50e
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 19, 2024
422e716
more attempted type-checking fixes
nastasha-w Jul 19, 2024
02011d0
avoid test failures from divide by zero errors
nastasha-w Jul 19, 2024
be087ce
BUG: fix a defect in CartesianCoordinateHandler._ortho_pixelize
neutrinoceros Jul 19, 2024
374a758
BUG: fix a defect in OffAxisProjectionFixedResolutionBuffer._generate…
neutrinoceros Jul 19, 2024
e8a0371
ruff fix
nastasha-w Jul 19, 2024
7f129c1
avoid 'divide by zero' errors
nastasha-w Jul 19, 2024
6f57981
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 19, 2024
45574d6
fix .v on numpy array problem
nastasha-w Jul 19, 2024
c653a4f
remove printing in tests ('clogs up' test logs)
nastasha-w Jul 20, 2024
92a7127
attempted fix test error from sqrt(< 0)
nastasha-w Jul 20, 2024
5587ff3
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 20, 2024
d3bd38a
docstring typo fix
nastasha-w Jul 23, 2024
726bdf3
pass depth kwarg to off_axis_projection in FITSOffAxisProjection
nastasha-w Jul 23, 2024
8ea95e3
now passing dataset kernel name to SPH gridding backend
nastasha-w Jul 24, 2024
22cac2b
remove hsml margin in line of sight direction for SPH projections
nastasha-w Jul 24, 2024
5362365
add prefixes marking old and new test images
nastasha-w Jul 24, 2024
0580c69
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 24, 2024
d000cd5
increase numbers for frontend tests affect by SPH backend change
nastasha-w Aug 19, 2024
f466494
comments update
nastasha-w Aug 19, 2024
f8949c2
actually update gizmo, owls test numbers
nastasha-w Aug 20, 2024
ac30d15
bumpy arepo test number: avoid missing files error?
nastasha-w Aug 21, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 8 additions & 4 deletions doc/source/analyzing/generating_processed_data.rst
Original file line number Diff line number Diff line change
Expand Up @@ -54,16 +54,17 @@ the transformation of a variable mesh of points consisting of positions and
sizes into a fixed-size array that appears like an image. This process is that
of pixelization, which yt handles transparently internally. You can access
this functionality by constructing a
:class:`~yt.visualization.fixed_resolution.FixedResolutionBuffer` and supplying
:class:`~yt.visualization.fixed_resolution.FixedResolutionBuffer`
and supplying
to it your :class:`~yt.data_objects.data_containers.YTSelectionContainer2D`
object, as well as some information about how you want the final image to look.
You can specify both the bounds of the image (in the appropriate x-y plane) and
the resolution of the output image. You can then have yt pixelize any field
you like.

.. note:: In previous versions of yt, there was a special class of
FixedResolutionBuffer for off-axis slices. This is no longer
necessary.
FixedResolutionBuffer for off-axis slices. This is still used
for off-axis SPH data projections: OffAxisFixedResolutionBuffer.

To create :class:`~yt.data_objects.data_containers.YTSelectionContainer2D` objects, you can
access them as described in :ref:`data-objects`, specifically the section
Expand Down Expand Up @@ -99,7 +100,10 @@ this, see :ref:`saving-grid-data-containers`.
In the FITS case, there is an option for setting the ``units`` of the coordinate system in
the file. If you want to overwrite a file with the same name, set ``clobber=True``.

The :class:`~yt.visualization.fixed_resolution.FixedResolutionBuffer` can even be exported
The :class:`~yt.visualization.fixed_resolution.FixedResolutionBuffer`
(and its
:class:`~yt.visualization.fixed_resolution.OffAxisProjectionFixedResolutionBuffer`
subclass) can even be exported
as a 2D dataset itself, which may be operated on in the same way as any other dataset in yt:

.. code-block:: python
Expand Down
6 changes: 4 additions & 2 deletions doc/source/quickstart/4)_Data_Objects_and_Time_Series.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -337,6 +337,8 @@
"\n",
"There are two different types of covering grids: unsmoothed and smoothed. Smoothed grids will be filled through a cascading interpolation process; they will be filled at level 0, interpolated to level 1, filled at level 1, interpolated to level 2, filled at level 2, etc. This will help to reduce edge effects. Unsmoothed covering grids will not be interpolated, but rather values will be duplicated multiple times.\n",
"\n",
"For SPH datasets, the covering grid gives the SPH-interpolated value of a field at each grid cell center. This is done for unsmoothed grids; smoothed grids are not available for SPH data.\n",
"\n",
"Here we create an unsmoothed covering grid at level 2, with the left edge at `[0.0, 0.0, 0.0]` and with dimensions equal to those that would cover the entire domain at level 2. We can then ask for the Density field, which will be a 3D array."
]
},
Expand Down Expand Up @@ -385,13 +387,13 @@
],
"metadata": {
"kernelspec": {
"name": "python3",
"display_name": "Python 3.9.5 64-bit ('yt-dev': pyenv)",
"metadata": {
"interpreter": {
"hash": "14363bd97bed451d1329fb3e06aa057a9e955a9421c5343dd7530f5497723a41"
}
}
},
"name": "python3"
},
"language_info": {
"codemirror_mode": {
Expand Down
10 changes: 4 additions & 6 deletions doc/source/visualizing/plots.rst
Original file line number Diff line number Diff line change
Expand Up @@ -293,8 +293,8 @@ argument. Optionally, a ``north_vector`` can be specified to fix the orientation
of the image plane.

.. note:: Not every data types have support for off-axis slices yet.
Currently, this operation is supported for grid based data with cartesian geometry.
In some cases (like SPH data) an off-axis projection over a thin region might be used instead.
Currently, this operation is supported for grid based and SPH data with cartesian geometry.
In some cases an off-axis projection over a thin region might be used instead.

.. _projection-plots:

Expand Down Expand Up @@ -433,6 +433,8 @@ by applying the
In this use case, the volume renderer casts a set of plane parallel rays, one
for each pixel in the image. The data values along each ray are summed,
creating the final image buffer.
For SPH datsets, the coordinates are instead simply rotated before the axis-aligned
projection function is applied.

.. _off-axis-projection-function:

Expand Down Expand Up @@ -652,10 +654,6 @@ simply pass ``all`` as the first argument of the field tuple:
Additional Notes for Plotting Particle Data
-------------------------------------------

An important caveat when visualizing particle data is that off-axis slice plotting is
not available for any particle data. However, axis-aligned slice plots (as described in
:ref:`slice-plots`) will work.

Since version 4.2.0, off-axis projections ares supported for non-SPH particle data.
Previous to that, this operation was only supported for SPH particles. Two historical
workaround methods were available for plotting non-SPH particles with off-axis
Expand Down
2 changes: 2 additions & 0 deletions nose_ignores.txt
Original file line number Diff line number Diff line change
Expand Up @@ -45,3 +45,5 @@
--ignore-file=test_set_log_level\.py
--ignore-file=test_field_parsing\.py
--ignore-file=test_disks\.py
--ignore-file=test_offaxisprojection_pytestonly\.py
--ignore-file=test_sph_pixelization_pytestonly\.py
12 changes: 7 additions & 5 deletions tests/tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ answer_tests:
- yt/frontends/amrvac/tests/test_outputs.py:test_riemann_cartesian_175D
- yt/frontends/amrvac/tests/test_outputs.py:test_rmi_cartesian_dust_2D

local_arepo_011: # PR 4419
local_arepo_013: # PR 4939
- yt/frontends/arepo/tests/test_outputs.py:test_arepo_bullet
- yt/frontends/arepo/tests/test_outputs.py:test_arepo_tng59
- yt/frontends/arepo/tests/test_outputs.py:test_arepo_cr
Expand Down Expand Up @@ -92,7 +92,7 @@ answer_tests:
- yt/frontends/flash/tests/test_outputs.py:test_wind_tunnel
- yt/frontends/flash/tests/test_outputs.py:test_fid_1to3_b1

local_gadget_009: # PR 3258
local_gadget_010: # PR 4939
- yt/frontends/gadget/tests/test_outputs.py:test_iso_collapse
- yt/frontends/gadget/tests/test_outputs.py:test_pid_uniqueness
- yt/frontends/gadget/tests/test_outputs.py:test_bigendian_field_access
Expand All @@ -108,7 +108,7 @@ answer_tests:
local_gdf_002:
- yt/frontends/gdf/tests/test_outputs_nose.py:test_sedov_tunnel

local_gizmo_008: # PR 2909
local_gizmo_009: # PR 4939
- yt/frontends/gizmo/tests/test_outputs.py:test_gizmo_64

local_halos_012: # PR 3325
Expand All @@ -118,7 +118,7 @@ answer_tests:
- yt/frontends/gadget_fof/tests/test_outputs.py:test_fields_g5
- yt/frontends/gadget_fof/tests/test_outputs.py:test_fields_g42

local_owls_008: # PR 2909
local_owls_009: # PR 4939
- yt/frontends/owls/tests/test_outputs.py:test_snapshot_033
- yt/frontends/owls/tests/test_outputs.py:test_OWLS_particlefilter

Expand All @@ -130,7 +130,7 @@ answer_tests:
- yt/visualization/tests/test_particle_plot.py:test_particle_phase_answers
- yt/visualization/tests/test_callbacks.py:test_axis_manipulations

local_tipsy_009: # PR 2909
local_tipsy_010: # PR 4939
- yt/frontends/tipsy/tests/test_outputs.py:test_pkdgrav
- yt/frontends/tipsy/tests/test_outputs.py:test_gasoline_dmonly
- yt/frontends/tipsy/tests/test_outputs.py:test_tipsy_galaxy
Expand Down Expand Up @@ -221,5 +221,7 @@ other_tests:
- "--exclude-test=yt.frontends.gdf.tests.test_outputs.TestGDF"
- "--exclude-test=yt.frontends.adaptahop.tests.test_outputs"
- "--exclude-test=yt.frontends.stream.tests.test_stream_particles.test_stream_non_cartesian_particles"
- "--ignore-file=test_offaxisprojection_pytestonly\\.py"
- "--ignore-file=test_sph_pixelization_pytestonly\\.py"
cookbook:
- 'doc/source/cookbook/tests/test_cookbook.py'
9 changes: 7 additions & 2 deletions yt/data_objects/construction_data_containers.py
Original file line number Diff line number Diff line change
Expand Up @@ -649,6 +649,8 @@ class YTCoveringGrid(YTSelectionContainer3D):
level : int
The resolution level data to which data will be gridded. Level
0 is the root grid dx for that dataset.
(The grid resolution will be simulation size / 2**level along
each grid axis.)
left_edge : array_like
The left edge of the region to be extracted. Specify units by supplying
a YTArray, otherwise code length units are assumed.
Expand Down Expand Up @@ -995,14 +997,15 @@ def _fill_sph_particles(self, fields):

smoothing_style = getattr(self.ds, "sph_smoothing_style", "scatter")
normalize = getattr(self.ds, "use_sph_normalization", True)
kernel_name = getattr(self.ds, "kernel_name", "cubic")

bounds, size = self._get_grid_bounds_size()

period = self.ds.coordinates.period.copy()
if hasattr(period, "in_units"):
period = period.in_units("code_length").d
# TODO maybe there is a better way of handling this
is_periodic = int(any(self.ds.periodicity))
# check periodicity per dimension
is_periodic = self.ds.periodicity
Comment on lines +1007 to +1008
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is self.ds.periodicity a list? I don't understand the removal of any() here. If it's a list, this will always be truthy.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed the backend so it can accept either a single boolean or a list/array/whatever of length 3. That allows consistent handling of simulations that are periodic in some, but not all, coordinate directions. I handle those options in these lines in pixelize_sph_kernel_arbitrary_grid, file: yt/utilities/lib/pixelization_routines.pyx. Basically, it allows backwards compatibility for single-value bool inputs in the backend, but it isn't using if check_period or something like that.


if smoothing_style == "scatter":
for field in fields:
Expand Down Expand Up @@ -1036,6 +1039,7 @@ def _fill_sph_particles(self, fields):
pbar=pbar,
check_period=is_periodic,
period=period,
kernel_name=kernel_name,
)
if normalize:
pixelize_sph_kernel_arbitrary_grid(
Expand All @@ -1051,6 +1055,7 @@ def _fill_sph_particles(self, fields):
pbar=pbar,
check_period=is_periodic,
period=period,
kernel_name=kernel_name,
)

if normalize:
Expand Down
Loading
Loading