diff --git a/docs/favicon.svg b/docs/_static/favicon.svg similarity index 100% rename from docs/favicon.svg rename to docs/_static/favicon.svg diff --git a/docs/contributing.rst b/docs/about/contributing.rst similarity index 100% rename from docs/contributing.rst rename to docs/about/contributing.rst diff --git a/docs/future-roadmap.rst b/docs/about/future-roadmap.rst similarity index 55% rename from docs/future-roadmap.rst rename to docs/about/future-roadmap.rst index 83ec8d54..3e3413f5 100644 --- a/docs/future-roadmap.rst +++ b/docs/about/future-roadmap.rst @@ -8,35 +8,39 @@ of development might change. :class: important - *Next generation API version 0.2* - 1. Meta-data cleanup - 2. Simpler constructors - - 3. New core TSDF transformation functions + 1. Meta-data cleanup + 2. Simpler constructors + 3. New core TSDF transformation functions - *Clear separation of resampling & interpolation functions* - -------------------------------------------------------------------------- + - *Intervals API* - 1. Extract intervals from TSDFs - 2. Get distinct (non-overlapping intervals) - 3. Interval-TSDF joins + + 1. Extract intervals from TSDFs + 2. Get distinct (non-overlapping intervals) + 3. Interval-TSDF joins - *Streaming Support* 🚀 - 1. Streaming as-of joins - 2. Streaming resampling + + 1. Streaming as-of joins + 2. Streaming resampling - *Integration with timeseries forecasting & ML libraries:* 📈 - 1. statsmodels - 2. FB Prophet - 3. Tensorflow - 4. Pytorch - 5. tsfresh - 6. sktime - And many more... + 1. statsmodels + 2. FB Prophet + 3. Tensorflow + 4. Pytorch + 5. tsfresh + 6. sktime + + And many more... - *Simpler Windowing API* - 1. Use natural-language expressions (ie "5 mins", "16 days", etc.) + + 1. Use natural-language expressions (ie "5 mins", "16 days", etc.) + Don't see what you are looking for, reach out to the team directly diff --git a/docs/tempo-team.rst b/docs/about/tempo-team.rst similarity index 99% rename from docs/tempo-team.rst rename to docs/about/tempo-team.rst index 95388cca..25aedb36 100644 --- a/docs/tempo-team.rst +++ b/docs/about/tempo-team.rst @@ -18,5 +18,3 @@ Contact Reach out to the team at `labs@databricks.com `_ for feature requests, ideas for improvement etc. - - diff --git a/docs/user-guide.rst b/docs/about/user-guide.rst similarity index 98% rename from docs/user-guide.rst rename to docs/about/user-guide.rst index 885df719..c8ddcfb8 100644 --- a/docs/user-guide.rst +++ b/docs/about/user-guide.rst @@ -25,7 +25,7 @@ to create new features from. Below are the public methods available for TSDF tra Sample Reference Architecture for Capital Markets ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -.. image:: _static/ts_in_fs.png +.. image:: ../_static/ts_in_fs.png :alt: Architecture diagram Quickstart - Python @@ -33,7 +33,7 @@ Quickstart - Python Data source is UCI public accelerometer data available at this `URL `_ -.. image:: _static/Phone_Accelerometer.png +.. image:: ../_static/Phone_Accelerometer.png :alt: Data Info Read in Data @@ -130,7 +130,7 @@ For the accepted functions to aggregate data, options are 'floor', 'ceil', 'min' fig.show() -.. image:: _static/resample.png +.. image:: ../_static/resample.png :alt: Raw time series AS OF Join @@ -139,7 +139,7 @@ AS OF Join This join uses windowing in order to select the latest record from a source table and merges this onto the base fact table. -.. image:: _static/AS_OF_JOIN.png +.. image:: ../_static/AS_OF_JOIN.png :alt: As of join .. code-block:: python diff --git a/docs/conf.py b/docs/conf.py index 38d252ae..fd52e654 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -9,7 +9,7 @@ # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. -# + from datetime import date import os import sys @@ -22,14 +22,13 @@ # -- Project information ----------------------------------------------------- -project = "Tempo" -copyright = f"{date.today().year}, Databricks Labs" -author = "Databricks Labs" +project = 'Tempo' +copyright = '2024, Databricks Labs' +author = 'Databricks Labs' # The full version, including alpha/beta/rc tags release = get_latest_git_tag() - # -- General configuration --------------------------------------------------- # Add any Sphinx extension module names here, as strings. They can be @@ -46,12 +45,12 @@ ] # Add any paths that contain templates here, relative to this directory. -templates_path = ["_templates"] +templates_path = ['_templates'] # List of patterns, relative to source directory, that match files and # directories to ignore when looking for source files. # This pattern also affects html_static_path and html_extra_path. -exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"] +exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store'] # -- Options for HTML output ------------------------------------------------- @@ -59,7 +58,7 @@ # The theme to use for HTML and HTML Help pages. See the documentation for # a list of builtin themes. # -html_theme = "furo" +html_theme = 'furo' # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, @@ -68,14 +67,6 @@ html_theme_options = { "light_logo": "tempo - light background.svg", "dark_logo": "tempo - dark background.svg", - "light_css_variables": { - "color-brand-primary": "#7C4DFF", - "color-brand-content": "#7C4DFF", - }, - "dark_css_variables": { - "color-brand-primary": "#7C4DFF", - "color-brand-content": "#7C4DFF", - }, "footer_icons": [ { "name": "GitHub", @@ -95,4 +86,4 @@ }, ], } -html_favicon = "favicon.svg" +html_favicon = "_static/favicon.svg" \ No newline at end of file diff --git a/docs/index.rst b/docs/index.rst index aad99ca5..dfe086b3 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -7,12 +7,11 @@ tempo - Time Series Utilities for Data Teams Using Databricks :maxdepth: 3 Databricks Labs - user-guide - reference/index - contributing - tempo-team - future-roadmap - + about/user-guide + references/api-reference + about/contributing + about/tempo-team + about/future-roadmap .. image:: https://github.com/databrickslabs/tempo/workflows/build/badge.svg @@ -70,9 +69,9 @@ Alternatively, you can grab the latest source code from `GitHub ` will be documented here. -For support on those functionalities please feel free to reach out to the :doc:`tempo-team`. +For support on those functionalities please feel free to reach out to the :doc:`about/tempo-team`. Contributing ------------ -We happily welcome contributions, please see :doc:`contributing` for details. +We happily welcome contributions, please see :doc:`about/contributing` for details. diff --git a/docs/make.bat b/docs/make.bat index 8084272b..32bb2452 100644 --- a/docs/make.bat +++ b/docs/make.bat @@ -1,35 +1,35 @@ -@ECHO OFF - -pushd %~dp0 - -REM Command file for Sphinx documentation - -if "%SPHINXBUILD%" == "" ( - set SPHINXBUILD=sphinx-build -) -set SOURCEDIR=. -set BUILDDIR=_build - -if "%1" == "" goto help - -%SPHINXBUILD% >NUL 2>NUL -if errorlevel 9009 ( - echo. - echo.The 'sphinx-build' command was not found. Make sure you have Sphinx - echo.installed, then set the SPHINXBUILD environment variable to point - echo.to the full path of the 'sphinx-build' executable. Alternatively you - echo.may add the Sphinx directory to PATH. - echo. - echo.If you don't have Sphinx installed, grab it from - echo.https://www.sphinx-doc.org/ - exit /b 1 -) - -%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O% -goto end - -:help -%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O% - -:end -popd +@ECHO OFF + +pushd %~dp0 + +REM Command file for Sphinx documentation + +if "%SPHINXBUILD%" == "" ( + set SPHINXBUILD=sphinx-build +) +set SOURCEDIR=. +set BUILDDIR=_build + +%SPHINXBUILD% >NUL 2>NUL +if errorlevel 9009 ( + echo. + echo.The 'sphinx-build' command was not found. Make sure you have Sphinx + echo.installed, then set the SPHINXBUILD environment variable to point + echo.to the full path of the 'sphinx-build' executable. Alternatively you + echo.may add the Sphinx directory to PATH. + echo. + echo.If you don't have Sphinx installed, grab it from + echo.https://www.sphinx-doc.org/ + exit /b 1 +) + +if "%1" == "" goto help + +%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O% +goto end + +:help +%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O% + +:end +popd diff --git a/docs/reference/index.rst b/docs/reference/index.rst deleted file mode 100644 index 133bf219..00000000 --- a/docs/reference/index.rst +++ /dev/null @@ -1,12 +0,0 @@ -API Reference -============= - -.. note:: - - Some of the function docstrings are not in line with the standard RST pattern so they maybe renderred incorrectly. - We are fixing them. They will be updated very soon. Thank you for your patience and support. - -.. toctree:: - - tempo.tsdf - tempo.intervals \ No newline at end of file diff --git a/docs/reference/tempo.intervals.rst b/docs/reference/tempo.intervals.rst deleted file mode 100644 index a5b11c64..00000000 --- a/docs/reference/tempo.intervals.rst +++ /dev/null @@ -1,5 +0,0 @@ -tempo.intervals -=============== - -.. autoclass:: tempo.intervals.IntervalsDF - :members: \ No newline at end of file diff --git a/docs/reference/tempo.tsdf.rst b/docs/reference/tempo.tsdf.rst deleted file mode 100644 index c280e6f3..00000000 --- a/docs/reference/tempo.tsdf.rst +++ /dev/null @@ -1,12 +0,0 @@ -tempo.tsdf -=========== - -.. note:: - - Some of the function docstrings are not in line with the standard RST pattern so they maybe renderred incorrectly. - We are fixing them. They will be updated very soon. Thank you for your patience and support. - -.. autoclass:: tempo.tsdf.TSDF - :members: - - diff --git a/docs/references/api-reference.rst b/docs/references/api-reference.rst new file mode 100644 index 00000000..9002fcf9 --- /dev/null +++ b/docs/references/api-reference.rst @@ -0,0 +1,7 @@ +API Reference +============= + +.. toctree:: + + tsdf + intervals \ No newline at end of file diff --git a/docs/references/intervals.rst b/docs/references/intervals.rst new file mode 100644 index 00000000..f1cdff8f --- /dev/null +++ b/docs/references/intervals.rst @@ -0,0 +1,7 @@ +Intervals Module +================ + +.. automodule:: tempo.intervals + :members: + :undoc-members: + :show-inheritance: diff --git a/docs/references/tsdf.rst b/docs/references/tsdf.rst new file mode 100644 index 00000000..66756471 --- /dev/null +++ b/docs/references/tsdf.rst @@ -0,0 +1,7 @@ +TSDF Module +=========== + +.. automodule:: tempo.tsdf + :members: + :undoc-members: + :show-inheritance: diff --git a/python/tempo/tsdf.py b/python/tempo/tsdf.py index bbec6b78..b8859353 100644 --- a/python/tempo/tsdf.py +++ b/python/tempo/tsdf.py @@ -1470,6 +1470,7 @@ def extractStateIntervals( :param: metric_cols: the set of metric columns to evaluate for state changes :param: state_definition: the comparison function used to evaluate individual metrics for state changes. + Either a string, giving a standard PySpark column comparison operation, or a binary function with the signature: `(x1: Column, x2: Column) -> Column` where the returned column expression evaluates to a :class:`~pyspark.sql.types.BooleanType`