Select refactor

CHANGELOG.md

@@ -3,6 +3,35 @@ All notable changes to this project will be documented in this file.
 
 ## [Unreleased]
 
+### Added
+- New `strict` keyword added to `UVData.select`, `UVBeam.select`, `UVFlag.select`, and
+`UVFlag.select`, which allows the user to specify whether to warn or error when
+supplied criteria only partially match (default being to warn).
+- New `invert` keyword added to `UVData.select`, `UVBeam.select`, `UVFlag.select`, and
+`UVFlag.select`, which allows the user to specify data to deselect rather than select.
+- New method `UVBase._select_along_param_axis`, which allows for more uniform selection
+behavior across parameters within `UVData`, `UVCal`, `UVBeam`, `UVFlag`, and `Telescope`
+classes.
+- Several new selection helper and check functions have been added to `utils`.
+- New `warn_spacing` keyword added to `select`, `__add__`, `fast_concat` methods of
+`UVData`, `UVCal`, `UVBeam`, and `UVFlag`, which allows the user to specify whether or
+not to warn based on spacing errors that would prevent writing out to e.g., FITS-based
+file formats. Default is typically `False` (with the exception of `UVBeam`, where the
+default is `True`), such that most warnings about frequency/polarization/time spacing
+will not normally be raised.
+
+### Changed
+- `UVData.select`, `UVBeam.select`, `UVFlag.select`, `UVFlag.select` have been
+significantly refactored and made to behave more uniformly.
+- Allowing `UVParameter.__eq__` to use `UVParameter.compare_value` if the item being
+compared and `UVParameter.value` share the same class.
+- Warnings about `extra_keywords` have been removed from `UVData.check`, `UVCal.check`,
+and `UVBeam.check`.
+
+### Fixed
+- Bug in `UVBeam.select` where `polarization_array` could be incorrectly ordered after
+selection (if input to `polarizations` keyword was unordered).
+
 ## [3.1.3] - 2025-01-13
 
 ### Added

docs/uvbeam_tutorial.rst

@@ -265,6 +265,9 @@ UVBeam: Selecting data
 The :meth:`pyuvdata.UVBeam.select` method lets you select specific image axis indices
 (or pixels if pixel_coordinate_system is HEALPix), frequencies and feeds
 (or polarizations if beam_type is power) to keep in the object while removing others.
+By default, :meth:`pyuvdata.UVBeam.select` will select data that matches the supplied
+criteria, but by setting ``invert=True``, you can instead *deselect* this data and
+preserve only that which does not match the selection.
 
 a) Selecting a range of Zenith Angles
 *************************************
@@ -338,6 +341,12 @@ or "ee).
   >>> print(uvb2.feed_array)
   ['y']
 
+  >>> # Finally, try a deselect
+  >>> uvb2 = uvb.copy()
+  >>> uvb2.select(feeds=["x"], invert=True)
+  >>> print(uvb2.feed_array)
+  ['y']
+
   >>> # convert to a power beam for selecting on polarizations
   >>> uvb.efield_to_power()
   >>> # polarization numbers can be found in the polarization_array

docs/uvcal_tutorial.rst

@@ -401,7 +401,10 @@ UVCal: Selecting data
 The :meth:`pyuvdata.UVCal.select` method lets you select specific antennas
 (by number or name), frequencies (in Hz or by channel number), times (either exact
 times or times covered by a time range) or jones components (by number or string) to keep
-in the object while removing others.
+in the object while removing others. By default, :meth:`pyuvdata.UVCal.select` will
+select data that matches the supplied criteria, but by setting ``invert=True``, you
+can instead *deselect* this data and preserve only that which does not match the
+selection.
 
 a) Select antennas to keep on UVCal object using the antenna number.
 ********************************************************************
@@ -417,6 +420,11 @@ a) Select antennas to keep on UVCal object using the antenna number.
   >>> # print all the antennas numbers with data in the original file
   >>> print(cal.ant_array)
   [ 0  1 11 12 13 23 24 25]
+  >>> cal.select(antenna_nums=[0, 11, 12], invert=True)
+
+  >>> # print all the antenna numbers with data after deselection
+  >>> print(cal.ant_array)
+  [ 1 13 23 24 25]
   >>> cal.select(antenna_nums=[1, 13, 25])
 
   >>> # print all the antennas numbers with data after the select

docs/uvdata_tutorial.rst

@@ -983,7 +983,10 @@ UVData: Selecting data
 The :meth:`pyuvdata.UVData.select` method lets you select specific antennas (by number or name),
 antenna pairs, frequencies (in Hz or by channel number), times (or time range),
 local sidereal time (LST) (or LST range), or polarizations to keep in the object
-while removing others.
+while removing others. By default, :meth:`pyuvdata.UVData.select` will
+select data that matches the supplied criteria, but by setting ``invert=True``, you
+can instead *deselect* this data and preserve only that which does not match the
+selection.
 
 Note: The same select interface is now supported on the read for many file types
 (see :ref:`large_files`), so you need not read in the entire file before doing the select.
@@ -1125,6 +1128,16 @@ the physical orientation of the dipole can also be used (e.g. "nn" or "ee).
   >>> print(utils.polnum2str(uvd.polarization_array))
   ['rr', 'll']
 
+
+  >>> # Now deselect polarizations
+  >>> uvd.select(polarizations=["ll"], invert=True)
+
+  >>> # print polarization numbers and strings after select
+  >>> print(uvd.polarization_array)
+  [-1]
+  >>> print(utils.polnum2str(uvd.polarization_array))
+  ['rr']
+
   >>> # read in a file with linear polarizations and an x_orientation
   >>> filename = os.path.join(DATA_PATH, 'zen.2458661.23480.HH.uvh5')
   >>> uvd = UVData.from_file(filename)

src/pyuvdata/parameter.py

@@ -723,6 +723,14 @@ def compare_value(self, value):
         # If these are numeric types, handle them via allclose
         if isinstance(value, np.ndarray | int | float | complex):
             # Check that we either have a number or an ndarray
+            # Array equal here is a) _much_ faster if identical, and b) will
+            # handle things like strings (where tols are not defined).
+            check = np.array_equal(value, self.value)
+            if check or (
+                isinstance(value, np.ndarray) and np.issubdtype(value.dtype, np.str_)
+            ):
+                return check
+
             return bool(
                 not isinstance(value, np.ndarray)
                 or value.shape == self.value.shape

src/pyuvdata/utils/antenna.py

@@ -8,43 +8,87 @@
 
 
 def _select_antenna_helper(
-    *, antenna_names, antenna_nums, tel_ant_names, tel_ant_nums, obj_ant_array
+    *,
+    antenna_names,
+    antenna_nums,
+    tel_ant_names,
+    tel_ant_nums,
+    obj_ant_array,
+    invert=False,
+    strict=False,
 ):
+    """
+    Get antenna indices in a select.
+
+    Parameters
+    ----------
+    antenna_names : array_like of str
+        List of antennas to be selected based on name.
+    antenna_nums : array_like of int
+        List of antennas to be selected based on number.
+    tel_ant_names : array_like of str
+        List that contains the full set of antenna names for the telescope, which is
+        matched to the list provided in `tel_ant_nums`. Used to map antenna name to
+        number.
+    tel_ant_nums : array_like of int
+        List that contains the full set of antenna numbers for the telescope, which is
+        matched to the list provided in `tel_ant_names`. Used to map antenna name to
+        number.
+    obj_ant_array : array_like of int
+        The antenna numbers present in the object.
+    invert : bool, optional
+        Normally indices matching given criteria are what are included in the
+        subsequent list. However, if set to True, these indices are excluded
+        instead. Default is False.
+    strict : bool or None
+        Normally, select will warn when an element of the selection criteria does not
+        match any element for the parameter, as long as the selection criteria results
+        in _at least one_ element being selected. However, if set to True, an error is
+        thrown if any selection criteria does not match what is given for the object
+        parameters element. If set to None, then neither errors nor warnings are raised,
+        unless no records are selected. Default is False.
+
+    Returns
+    -------
+    ant_inds : list of int
+        Indices of antennas to keep on the object.
+    selections : list of str
+        list of selections done.
+
+    """
+    ant_inds = None
+    selections = []
     if antenna_names is not None:
         if antenna_nums is not None:
             raise ValueError(
                 "Only one of antenna_nums and antenna_names can be provided."
             )
 
         antenna_names = tools._get_iterable(antenna_names)
+        tel_ant_names = np.asarray(tel_ant_names).flatten()
         antenna_nums = []
         for s in antenna_names:
-            if s not in tel_ant_names:
-                raise ValueError(
-                    f"Antenna name {s} is not present in the antenna_names array"
-                )
-            ind = np.where(np.array(tel_ant_names) == s)[0][0]
-            antenna_nums.append(tel_ant_nums[ind])
+            if s in tel_ant_names:
+                ind = np.where(tel_ant_names == s)[0][0]
+                antenna_nums.append(tel_ant_nums[ind])
+            else:
+                err_msg = f"Antenna name {s} is not present in the antenna_names array"
+                tools._strict_raise(err_msg, strict=strict)
 
     if antenna_nums is not None:
-        antenna_nums = tools._get_iterable(antenna_nums)
-        antenna_nums = np.asarray(antenna_nums)
-        selections = ["antennas"]
+        antenna_nums = np.asarray(antenna_nums).flatten()
+        selections.append("antennas")
 
-        ant_inds = np.zeros(0, dtype=np.int64)
-        ant_check = np.isin(antenna_nums, obj_ant_array)
-        if not np.all(ant_check):
-            raise ValueError(
-                f"Antenna number {antenna_nums[~ant_check]} is not present "
-                "in the ant_array"
+        check = np.isin(antenna_nums, obj_ant_array, invert=True)
+        if any(check):
+            tools._strict_raise(
+                f"Antenna number {antenna_nums[check]} is not present in the ant_array",
+                strict=strict,
             )
 
-        for ant in antenna_nums:
-            ant_inds = np.append(ant_inds, np.where(obj_ant_array == ant)[0])
-
-        ant_inds = sorted(set(ant_inds))
-    else:
-        ant_inds = None
-        selections = None
+        ant_inds = np.nonzero(np.isin(obj_ant_array, antenna_nums, invert=invert))[0]
+        ant_inds = ant_inds.tolist()
+        if len(ant_inds) == 0:
+            raise ValueError("No data matching this antenna selection exists.")
 
     return ant_inds, selections