Added functions for calculating sunrise and sunset based on the day-n…

…ight mask (#187) * added prelim functions for identifying sunrise and sunset * fleshing out alignment logic * fleshed out sunrise/sunset logic based on data alignment * added new freq logic to the sunrise/sunset calculations * added in dummy placeholders for unit tests * fleshing out unit test logic for sunrise/sunset filters * filling out the unit tests * added new unit test checking consistency of midday difference series * update the daytime routines to fix unit tests * updated unit tests for sunrise/sunset masking * fixing pep8 errors * bulking up the unit tests * added more unit test logic * fixed pep8 issues * polished off unit testing and solved issue where timestamps don't start at midnight * lop off last entry in right-aligned data as it's 3/20 * update the whatsnew * added new pytest error raise to unit test * added nan filler for days that bfill/ffill wrong * added the to_offset call * Update pvanalytics/features/daytime.py Co-authored-by: Kevin Anderson <[email protected]> * updated unit tests per @kandersolar's request * update the routine to fix pep8 issue * update the routine for pytest assertions * fix more unit tests... * moved repeat functionality to private function * convert to transit column * added suggested caveat comment from @kandersolar * Update pvanalytics/features/daytime.py Co-authored-by: Cliff Hansen <[email protected]> * Update pvanalytics/features/daytime.py Co-authored-by: Cliff Hansen <[email protected]> * Update pvanalytics/features/daytime.py Co-authored-by: Cliff Hansen <[email protected]> * Update pvanalytics/features/daytime.py Co-authored-by: Cliff Hansen <[email protected]> * Update pvanalytics/features/daytime.py Co-authored-by: Cliff Hansen <[email protected]> * Update pvanalytics/features/daytime.py Co-authored-by: Cliff Hansen <[email protected]> * updated the docstring for get_sunset() to match get_sunrise() updates @cwhanse made * added reference for the day-night masking/sunrise/sunset functions * added check for time difference for midday * udpate the data dir call in the unit tests * fix pep8 errors * Update pvanalytics/tests/features/test_daytime.py Co-authored-by: Kevin Anderson <[email protected]> --------- Co-authored-by: Kevin Anderson <[email protected]> Co-authored-by: Cliff Hansen <[email protected]>
pvlib · Jan 5, 2024 · d169f5d · d169f5d
1 parent e7aeb91
commit d169f5d
Show file tree

Hide file tree

Showing 4 changed files with 364 additions and 5 deletions.
diff --git a/docs/api.rst b/docs/api.rst
@@ -248,12 +248,16 @@ identification.
 Daytime
 -------
 
-Functions that return a Boolean mask indicating day and night.
+Functions that relate to determining day/night periods in a time
+series, and getting sunrise and sunset times based on the day-night mask
+outputs.
 
 .. autosummary::
    :toctree: generated/
 
    features.daytime.power_or_irradiance
+   features.daytime.get_sunrise
+   features.daytime.get_sunset
 
 Shading
 -------

diff --git a/docs/whatsnew/0.2.0.rst b/docs/whatsnew/0.2.0.rst
@@ -1,6 +1,6 @@
 .. _whatsnew_020:
 
-0.2.0 (anticipated August 2023)
+0.2.0 (anticipated December 2023)
 -----------------------------
 
 Breaking Changes
@@ -21,6 +21,14 @@ Breaking Changes
 
 Enhancements
 ~~~~~~~~~~~~
+* Added function :py:func:`~pvanalytics.features.daytime.get_sunrise`
+  for calculating the daily sunrise datetimes for a time series, based on the
+  :py:func:`~pvanalytics.features.daytime.power_or_irradiance` day/night mask output. 
+  (:pull:`187`)
+* Added function :py:func:`~pvanalytics.features.daytime.get_sunset`
+  for calculating the daily sunset datetimes for a time series, based on the
+  :py:func:`~pvanalytics.features.daytime.power_or_irradiance` day/night mask output. 
+  (:pull:`187`)
 * Updated function :py:func:`~pvanalytics.features.daytime.power_or_irradiance`
   to be more performant by vectorization; the original logic was using a lambda call that was
   slowing the function speed down considerably. This update resulted in a ~50X speedup. (:pull:`186`)

diff --git a/pvanalytics/features/daytime.py b/pvanalytics/features/daytime.py
@@ -2,6 +2,7 @@
 import numpy as np
 import pandas as pd
 from pvanalytics import util
+from pandas.tseries.frequencies import to_offset
 
 
 def _rolling_by_minute(data, days, f):
@@ -175,7 +176,6 @@ def power_or_irradiance(series, outliers=None,
         median length of the day when correcting errors in the morning
         or afternoon. [days]
 
-
     Returns
     -------
     Series
@@ -187,8 +187,10 @@ def power_or_irradiance(series, outliers=None,
 
     ``NA`` values are treated like zeros.
 
-    Derived from the PVFleets QA Analysis project.
-
+    References
+    -------
+    .. [1] Perry K., Meyers B., and Muller, M. "Survey of Time Shift Detection
+       Algorithms for Measured PV Data", 2023 PV Reliability Workshop (PVRW).
     """
     series = series.fillna(value=0)
     series_norm = _filter_and_normalize(series, outliers).fillna(value=0)
@@ -229,3 +231,141 @@ def power_or_irradiance(series, outliers=None,
         correction_window
     )
     return ~night_corrected_edges
+
+
+def _get_sunrise_sunset_daily_series(daytime_mask, transform):
+    # Get the sunset/sunrise series based on getting the first or last
+    # 'day' value for each day in the time series
+    series = daytime_mask.index[daytime_mask].to_series().groupby(
+        daytime_mask[daytime_mask].index.date).transform(transform).reindex(
+            daytime_mask.index)
+    series = series.groupby(series.index.date).ffill().bfill()
+    # Backfilling and front filling fills all NaN's, so we set cases not in
+    # the right day to NaN
+    series.loc[series.index.date != series.dt.date] = np.nan
+    return series
+
+
+def get_sunrise(daytime_mask, freq=None, data_alignment='L'):
+    """
+    Using the outputs of :py:func:`power_or_irradiance`, derive sunrise values
+    for each day in the associated time series.
+
+    This function assumes that each midnight-to-midnight period
+    (according to the timezone of the input data) has one sunrise
+    followed by one sunset. In cases where this is not satisfied
+    (timezone of data is substantially different from the location's
+    local time, locations near the poles, etc), or in the case of missing
+    data, the returned sunrise and sunset times may be invalid.
+
+    Parameters
+    ----------
+    daytime_mask  : Series
+        Boolean series delineating night periods from day periods, where
+        day is True and night is False.
+    freq : str, optional
+        A pandas freqstr specifying the expected timestamp spacing for
+        the series. If None, the frequency will be inferred from the index of
+        ``daytime_mask``.
+    data_alignment  : str, default 'L'
+        The data alignment of the series (left-aligned or right-aligned). Data
+        alignment affects the value selected as sunrise. Options are 'L' (left-
+        aligned), 'R' (right-aligned), or 'C' (center-aligned)
+
+    Returns
+    -------
+    Series
+        Series of daily sunrise times with the same index as ``daytime_mask``.
+
+    References
+    -------
+    .. [1] Perry K., Meyers B., and Muller, M. "Survey of Time Shift Detection
+       Algorithms for Measured PV Data", 2023 PV Reliability Workshop (PVRW).
+    """
+    # Get the first day period for each day
+    sunrise_series = _get_sunrise_sunset_daily_series(daytime_mask, "first")
+    # If there's no frequency value, infer it from the daytime_mask series
+    if not freq:
+        freq = pd.infer_freq(daytime_mask.index)
+    # For left-aligned data, we want the first 'day' mask for
+    # each day in the series; this will act as a proxy for sunrise.
+    # Because of this, we will just return the sunrise_series with
+    # no modifications
+    if data_alignment == 'L':
+        return sunrise_series
+    # For center-aligned data, we want the mid-point between the last night
+    # mask and the first day mask. To do this, we subtract freq / 2 from
+    # each sunrise time in the sunrise_series.
+    elif data_alignment == 'C':
+        return (sunrise_series - (to_offset(freq) / 2))
+    # For right-aligned data, get the last nighttime mask datetime
+    # before the first 'day' mask in the series. To do this, we subtract freq
+    # from each sunrise time in the sunrise_series.
+    elif data_alignment == 'R':
+        return (sunrise_series - to_offset(freq))
+    else:
+        # Throw an error if right,left, or center-alignment are not declared
+        raise ValueError("No valid data alignment given. Please pass 'L'"
+                         " for left-aligned data, 'R' for right-aligned data,"
+                         " or 'C' for center-aligned data.")
+
+
+def get_sunset(daytime_mask, freq=None, data_alignment='L'):
+    """
+    Using the outputs of :py:func:`power_or_irradiance`, derive sunset
+    values for each day in the associated time series.
+
+    This function assumes that each midnight-to-midnight period
+    (according to the timezone of the input data) has one sunrise
+    followed by one sunset. In cases where this is not satisfied
+    (timezone of data is substantially different from the location's
+     local time, locations near the poles, etc), or in the case of missing
+    data, the returned sunrise and sunset times may be invalid.
+
+    Parameters
+    ----------
+    daytime_mask  : Series
+        Boolean series delineating night periods from day periods, where
+        day is True and night is False.
+    freq  : str, optional
+        A pandas freqstr specifying the expected timestamp spacing for
+        the series. If None, the frequency will be inferred from the index
+        of ``daytime_mask``.
+    data_alignment  : str, default 'L'
+        The data alignment of the series (left-aligned or right-aligned). Data
+        alignment affects the value selected as sunrise. Options are 'L' (left-
+        aligned), 'R' (right-aligned), or 'C' (center-aligned)
+
+    Returns
+    -------
+    Series
+        Series of daily sunrise times with the same index as ``daytime_mask``.
+
+    References
+    -------
+    .. [1] Perry K., Meyers B., and Muller, M. "Survey of Time Shift Detection
+       Algorithms for Measured PV Data", 2023 PV Reliability Workshop (PVRW).
+    """
+    # Get the last day period for each day
+    sunset_series = _get_sunrise_sunset_daily_series(daytime_mask, "last")
+    # If there's no frequency value, infer it from the daytime_mask series
+    if not freq:
+        freq = pd.infer_freq(daytime_mask.index)
+    #  For left-aligned data, sunset is the first nighttime period
+    # after the day mask. To get this, we add freq to each sunset time in
+    # the sunset time series.
+    if data_alignment == 'L':
+        return (sunset_series + to_offset(freq))
+    # For center-aligned data, sunset is the midpoint between the last day
+    # mask and the first nighttime mask. We calculate this by adding (freq / 2)
+    # to each sunset time in the sunset_series.
+    elif data_alignment == 'C':
+        return (sunset_series + (to_offset(freq) / 2))
+    # For right-aligned data, the last 'day' mask time stamp is sunset.
+    elif data_alignment == 'R':
+        return sunset_series
+    else:
+        # Throw an error if right, left, or center-alignment are not declared
+        raise ValueError("No valid data alignment given. Please pass 'L'"
+                         " for left-aligned data, 'R' for right-aligned data,"
+                         " or 'C' for center-aligned data.")