-**pyCP_APR** is a Python library for tensor decomposition and anomaly detection that is developed as part of the R&D 100 award wining [**SmartTensors**](https://www.lanl.gov/collaboration/smart-tensors/) project. It is designed for the fast analysis of large datasets by accelerating computation speed using GPUs. pyCP_APR uses the CANDECOMP/PARAFAC Alternating Poisson Regression (CP-APR) tensor factorization algorithm utilizing both Numpy and PyTorch backend. While the Numpy backend can be used for the analysis of both sparse and dense tensors, PyTorch backend provides faster decomposition of large and sparse tensors on the GPU. pyCP_APR's Scikit-learn like API allows comfortable interaction with the library, and include the methods for anomaly detection via the p-values obtained from the CP-APR factorization. The anomaly detection methods via the p-values optained from CP-APR was introduced by Eren et al. in [6] using the [Unified Host and Network Dataset](https://csr.lanl.gov/data/2017/) [5]. Our work follows the [MATLAB Tensor Toolbox](https://www.tensortoolbox.org/cp.html) [1-3] implementation of CP-APR [4].
+**pyCP_APR** is a Python library for tensor decomposition and anomaly detection that is developed as part of the R&D 100 award wining **[SmartTensors AI](https://smart-tensors.lanl.gov/software/)** project. It is designed for the fast analysis of large datasets by accelerating computation speed using GPUs. pyCP_APR uses the CANDECOMP/PARAFAC Alternating Poisson Regression (CP-APR) tensor factorization algorithm utilizing both Numpy and PyTorch backend. While the Numpy backend can be used for the analysis of both sparse and dense tensors, PyTorch backend provides faster decomposition of large and sparse tensors on the GPU. pyCP_APR's Scikit-learn like API allows comfortable interaction with the library, and include the methods for anomaly detection via the p-values obtained from the CP-APR factorization. The anomaly detection methods via the p-values optained from CP-APR was introduced by Eren et al. in [6] using the [Unified Host and Network Dataset](https://csr.lanl.gov/data/2017/) [5]. Our work follows the [MATLAB Tensor Toolbox](https://www.tensortoolbox.org/cp.html) [1-3] implementation of CP-APR [4].
@@ -18,6 +18,8 @@
### [:information_source: Documentation](https://lanl.github.io/pyCP_APR/) [:orange_book: Example Notebooks](examples/) [:bar_chart: Datasets](data/tensors)
### [:page_facing_up: Paper 1](https://ieeexplore.ieee.org/abstract/document/9280524) [:page_facing_up: Paper 2](https://dl.acm.org/doi/abs/10.1145/3519602)
+
+### [:link: Website](https://smart-tensors.LANL.gov)
diff --git a/docs/source/index.rst b/docs/source/index.rst
index 8c662b7..c9f65c0 100644
--- a/docs/source/index.rst
+++ b/docs/source/index.rst
@@ -13,7 +13,7 @@ Welcome to pyCP_APR's documentation!
:alt: RD100
:align: center
-**pyCP_APR** is a Python library for tensor decomposition and anomaly detection that is developed as part of the R&D 100 award wining `SmartTensors `_ project. It is designed for the fast analysis of large datasets by accelerating computation speed using GPUs. pyCP_APR uses the CANDECOMP/PARAFAC Alternating Poisson Regression (CP-APR) tensor factorization algorithm utilizing both Numpy and PyTorch backend. While the Numpy backend can be used for the analysis of both sparse and dense tensors, PyTorch backend provides faster decomposition of large and sparse tensors on the GPU. pyCP_APR's Scikit-learn like API allows comfortable interaction with the library, and include the methods for anomaly detection via the p-values obtained from the CP-APR factorization. The anomaly detection methods via the p-values optained from CP-APR was introduced by Eren et al. in :cite:p:`Eren2020_ISI` using the `Unified Host and Network Dataset `_ :cite:p:`UnifiedHostandNetwork2018`. Our work follows the `MATLAB Tensor Toolbox `_ :cite:p:`TTB_Software,Bader2006,Bader2008` implementation of CP-APR :cite:p:`ChKo12`.
+**pyCP_APR** is a Python library for tensor decomposition and anomaly detection that is developed as part of the R&D 100 award wining `SmartTensors AI `_ project. It is designed for the fast analysis of large datasets by accelerating computation speed using GPUs. pyCP_APR uses the CANDECOMP/PARAFAC Alternating Poisson Regression (CP-APR) tensor factorization algorithm utilizing both Numpy and PyTorch backend. While the Numpy backend can be used for the analysis of both sparse and dense tensors, PyTorch backend provides faster decomposition of large and sparse tensors on the GPU. pyCP_APR's Scikit-learn like API allows comfortable interaction with the library, and include the methods for anomaly detection via the p-values obtained from the CP-APR factorization. The anomaly detection methods via the p-values optained from CP-APR was introduced by Eren et al. in :cite:p:`Eren2020_ISI` using the `Unified Host and Network Dataset `_ :cite:p:`UnifiedHostandNetwork2018`. Our work follows the `MATLAB Tensor Toolbox `_ :cite:p:`TTB_Software,Bader2006,Bader2008` implementation of CP-APR :cite:p:`ChKo12`.
@@ -22,6 +22,7 @@ Resources
* `Example Notebooks `_
* `Example Tensors `_
* `Paper `_
+* `Website `_
* `Code `_
Installation
diff --git a/requirements.txt b/requirements.txt
index c722b7c..54642af 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -1,12 +1,12 @@
-joblib>=1.0.1
-matplotlib>=3.3.4
-numpy~=1.19.2
-numpy-indexed>=0.3.5
-pandas>=1.0.5
-scikit-learn>=0.22.2
-scipy>=1.5.3
-seaborn>=0.11.1
-torch>=1.6.0
-requests>=2.25.1
-tqdm>=4.62.3
-sparse>=0.13.0
\ No newline at end of file
+jobli
+matplotlib
+numpy
+numpy-indexed
+pandas
+scikit-learn
+scipy
+seaborn
+torch
+requests
+tqdm
+sparse
\ No newline at end of file
From c580409ca8524b4af57258ec6379faa7a5378e0a Mon Sep 17 00:00:00 2001
From: maksim
Date: Mon, 18 Dec 2023 22:18:03 +0000
Subject: [PATCH 5/7] update documentation
---
docs/CP_APR.html | 656 +++++----
docs/_modules/index.html | 424 ++++--
.../pyCP_APR/applications/ktensor_utils.html | 428 +++---
.../pyCP_APR/applications/sptensor_utils.html | 434 +++---
.../pyCP_APR/applications/stat_utils.html | 426 +++---
.../tensor_anomaly_detection.html | 446 +++---
.../tensor_anomaly_detection_v2.html | 440 +++---
docs/_modules/pyCP_APR/datasets.html | 430 +++---
.../pyCP_APR/numpy_backend/CP_APR.html | 440 +++---
.../pyCP_APR/numpy_backend/accum.html | 426 +++---
.../numpy_backend/arrange_ktensor.html | 426 +++---
.../numpy_backend/double_ktensor.html | 426 +++---
.../numpy_backend/fixsigns_ktensor.html | 426 +++---
.../numpy_backend/innerprod_ktensor.html | 426 +++---
.../numpy_backend/ipermute_tensor.html | 426 +++---
.../numpy_backend/khatrirao_ktensor.html | 426 +++---
.../numpy_backend/khatrirao_sptensor.html | 426 +++---
.../pyCP_APR/numpy_backend/ktensor.html | 428 +++---
.../pyCP_APR/numpy_backend/norm_ktensor.html | 426 +++---
.../numpy_backend/normalize_ktensor.html | 428 +++---
.../numpy_backend/permute_ktensor.html | 426 +++---
.../numpy_backend/permute_tensor.html | 426 +++---
.../numpy_backend/redistribute_ktensor.html | 426 +++---
.../pyCP_APR/numpy_backend/sptensor.html | 428 +++---
.../numpy_backend/tenmat_ktensor.html | 426 +++---
.../numpy_backend/tenmat_sptensor.html | 426 +++---
.../pyCP_APR/numpy_backend/tenmat_tensor.html | 426 +++---
.../pyCP_APR/numpy_backend/tensor.html | 426 +++---
.../pyCP_APR/numpy_backend/tt_dimscheck.html | 426 +++---
.../pyCP_APR/numpy_backend/ttm_sptensor.html | 426 +++---
.../pyCP_APR/numpy_backend/ttm_tensor.html | 426 +++---
.../pyCP_APR/numpy_backend/ttv_sptensor.html | 426 +++---
.../pyCP_APR/numpy_backend/ttv_tensor.html | 428 +++---
docs/_modules/pyCP_APR/pyCP_APR.html | 442 +++---
.../pyCP_APR/torch_backend/CP_APR_Torch.html | 442 +++---
.../pyCP_APR/torch_backend/ktensor_Torch.html | 428 +++---
.../torch_backend/sptensor_Torch.html | 428 +++---
docs/_sources/CP_APR.rst | 32 +
docs/_sources/datasets.rst | 25 +
docs/_sources/index.rst | 185 +++
docs/_sources/modules.rst | 7 +
docs/_sources/pyCP_APR.applications.rst | 53 +
docs/_sources/pyCP_APR.numpy_backend.rst | 213 +++
docs/_sources/pyCP_APR.rst | 49 +
docs/_sources/pyCP_APR.torch_backend.rst | 37 +
docs/_static/basic.css | 56 +-
docs/_static/doctools.js | 381 ++---
docs/_static/documentation_options.js | 8 +-
docs/_static/graphviz.css | 2 +-
docs/_static/images/logo_deepnote.svg | 1 +
docs/_static/language_data.js | 102 +-
.../locales/ar/LC_MESSAGES/booktheme.po | 75 +
.../locales/bg/LC_MESSAGES/booktheme.po | 75 +
.../locales/bn/LC_MESSAGES/booktheme.po | 63 +
.../locales/ca/LC_MESSAGES/booktheme.po | 66 +
.../locales/cs/LC_MESSAGES/booktheme.po | 75 +
.../locales/da/LC_MESSAGES/booktheme.po | 75 +
.../locales/de/LC_MESSAGES/booktheme.po | 75 +
.../locales/el/LC_MESSAGES/booktheme.po | 75 +
.../locales/eo/LC_MESSAGES/booktheme.po | 75 +
.../locales/es/LC_MESSAGES/booktheme.po | 75 +
.../locales/et/LC_MESSAGES/booktheme.po | 75 +
.../locales/fi/LC_MESSAGES/booktheme.po | 75 +
.../locales/fr/LC_MESSAGES/booktheme.po | 75 +
.../locales/hr/LC_MESSAGES/booktheme.po | 75 +
.../locales/id/LC_MESSAGES/booktheme.po | 75 +
.../locales/it/LC_MESSAGES/booktheme.po | 75 +
.../locales/iw/LC_MESSAGES/booktheme.po | 75 +
.../locales/ja/LC_MESSAGES/booktheme.po | 75 +
.../locales/ko/LC_MESSAGES/booktheme.po | 75 +
.../locales/lt/LC_MESSAGES/booktheme.po | 75 +
.../locales/lv/LC_MESSAGES/booktheme.po | 75 +
.../locales/ml/LC_MESSAGES/booktheme.po | 66 +
.../locales/mr/LC_MESSAGES/booktheme.po | 66 +
.../locales/ms/LC_MESSAGES/booktheme.po | 66 +
.../locales/nl/LC_MESSAGES/booktheme.po | 75 +
.../locales/no/LC_MESSAGES/booktheme.po | 75 +
.../locales/pl/LC_MESSAGES/booktheme.po | 75 +
.../locales/pt/LC_MESSAGES/booktheme.po | 75 +
.../locales/ro/LC_MESSAGES/booktheme.po | 75 +
.../locales/ru/LC_MESSAGES/booktheme.po | 75 +
.../locales/sk/LC_MESSAGES/booktheme.po | 75 +
.../locales/sl/LC_MESSAGES/booktheme.po | 75 +
.../locales/sr/LC_MESSAGES/booktheme.po | 75 +
.../locales/sv/LC_MESSAGES/booktheme.po | 75 +
.../locales/ta/LC_MESSAGES/booktheme.po | 66 +
.../locales/te/LC_MESSAGES/booktheme.po | 66 +
.../locales/tg/LC_MESSAGES/booktheme.po | 75 +
.../locales/th/LC_MESSAGES/booktheme.po | 75 +
.../locales/tl/LC_MESSAGES/booktheme.po | 66 +
.../locales/tr/LC_MESSAGES/booktheme.po | 75 +
.../locales/uk/LC_MESSAGES/booktheme.po | 75 +
.../locales/ur/LC_MESSAGES/booktheme.po | 66 +
.../locales/vi/LC_MESSAGES/booktheme.po | 75 +
.../locales/zh_CN/LC_MESSAGES/booktheme.po | 75 +
.../locales/zh_TW/LC_MESSAGES/booktheme.po | 75 +
docs/_static/pygments.css | 234 +--
docs/_static/sbt-webpack-macros.html | 11 +
docs/_static/scripts/bootstrap.js | 3 +
docs/_static/scripts/bootstrap.js.LICENSE.txt | 5 +
docs/_static/scripts/bootstrap.js.map | 1 +
docs/_static/scripts/pydata-sphinx-theme.js | 2 +
.../scripts/pydata-sphinx-theme.js.map | 1 +
docs/_static/scripts/sphinx-book-theme.js | 2 +
docs/_static/scripts/sphinx-book-theme.js.map | 1 +
docs/_static/searchtools.js | 827 ++++++-----
docs/_static/sphinx_highlight.js | 144 ++
docs/_static/styles/bootstrap.css | 6 +
docs/_static/styles/bootstrap.css.map | 1 +
docs/_static/styles/pydata-sphinx-theme.css | 2 +
.../styles/pydata-sphinx-theme.css.map | 1 +
docs/_static/styles/sphinx-book-theme.css | 8 +
docs/_static/styles/theme.css | 2 +
.../vendor/fontawesome/6.1.2/LICENSE.txt | 165 +++
.../vendor/fontawesome/6.1.2/css/all.min.css | 5 +
.../vendor/fontawesome/6.1.2/js/all.min.js | 2 +
.../6.1.2/js/all.min.js.LICENSE.txt | 5 +
.../6.1.2/webfonts/fa-brands-400.ttf | Bin 0 -> 181264 bytes
.../6.1.2/webfonts/fa-brands-400.woff2 | Bin 0 -> 105112 bytes
.../6.1.2/webfonts/fa-regular-400.ttf | Bin 0 -> 60236 bytes
.../6.1.2/webfonts/fa-regular-400.woff2 | Bin 0 -> 24028 bytes
.../6.1.2/webfonts/fa-solid-900.ttf | Bin 0 -> 389948 bytes
.../6.1.2/webfonts/fa-solid-900.woff2 | Bin 0 -> 154840 bytes
.../6.1.2/webfonts/fa-v4compatibility.ttf | Bin 0 -> 10084 bytes
.../6.1.2/webfonts/fa-v4compatibility.woff2 | Bin 0 -> 4776 bytes
docs/_static/webpack-macros.html | 36 +-
docs/datasets.html | 590 +++++---
docs/doctrees/CP_APR.doctree | Bin 193024 -> 179910 bytes
docs/doctrees/datasets.doctree | Bin 20589 -> 19579 bytes
docs/doctrees/environment.pickle | Bin 276431 -> 1338855 bytes
docs/doctrees/index.doctree | Bin 36714 -> 33040 bytes
docs/doctrees/modules.doctree | Bin 2649 -> 2702 bytes
docs/doctrees/pyCP_APR.applications.doctree | Bin 108303 -> 99866 bytes
docs/doctrees/pyCP_APR.doctree | Bin 191311 -> 177880 bytes
docs/doctrees/pyCP_APR.numpy_backend.doctree | Bin 264129 -> 241481 bytes
docs/doctrees/pyCP_APR.torch_backend.doctree | Bin 96982 -> 91899 bytes
docs/genindex.html | 424 ++++--
docs/index.html | 758 +++++-----
docs/modules.html | 514 ++++---
docs/py-modindex.html | 426 +++---
docs/pyCP_APR.applications.html | 787 ++++++----
docs/pyCP_APR.html | 862 +++++++----
docs/pyCP_APR.numpy_backend.html | 1278 ++++++++++-------
docs/pyCP_APR.torch_backend.html | 659 +++++----
docs/search.html | 473 +++---
docs/searchindex.js | 2 +-
examples/TOY_tensor.ipynb | 73 +-
requirements.txt | 2 +-
148 files changed, 19270 insertions(+), 9580 deletions(-)
create mode 100644 docs/_sources/CP_APR.rst
create mode 100644 docs/_sources/datasets.rst
create mode 100644 docs/_sources/index.rst
create mode 100644 docs/_sources/modules.rst
create mode 100644 docs/_sources/pyCP_APR.applications.rst
create mode 100644 docs/_sources/pyCP_APR.numpy_backend.rst
create mode 100644 docs/_sources/pyCP_APR.rst
create mode 100644 docs/_sources/pyCP_APR.torch_backend.rst
create mode 100644 docs/_static/images/logo_deepnote.svg
create mode 100644 docs/_static/locales/ar/LC_MESSAGES/booktheme.po
create mode 100644 docs/_static/locales/bg/LC_MESSAGES/booktheme.po
create mode 100644 docs/_static/locales/bn/LC_MESSAGES/booktheme.po
create mode 100644 docs/_static/locales/ca/LC_MESSAGES/booktheme.po
create mode 100644 docs/_static/locales/cs/LC_MESSAGES/booktheme.po
create mode 100644 docs/_static/locales/da/LC_MESSAGES/booktheme.po
create mode 100644 docs/_static/locales/de/LC_MESSAGES/booktheme.po
create mode 100644 docs/_static/locales/el/LC_MESSAGES/booktheme.po
create mode 100644 docs/_static/locales/eo/LC_MESSAGES/booktheme.po
create mode 100644 docs/_static/locales/es/LC_MESSAGES/booktheme.po
create mode 100644 docs/_static/locales/et/LC_MESSAGES/booktheme.po
create mode 100644 docs/_static/locales/fi/LC_MESSAGES/booktheme.po
create mode 100644 docs/_static/locales/fr/LC_MESSAGES/booktheme.po
create mode 100644 docs/_static/locales/hr/LC_MESSAGES/booktheme.po
create mode 100644 docs/_static/locales/id/LC_MESSAGES/booktheme.po
create mode 100644 docs/_static/locales/it/LC_MESSAGES/booktheme.po
create mode 100644 docs/_static/locales/iw/LC_MESSAGES/booktheme.po
create mode 100644 docs/_static/locales/ja/LC_MESSAGES/booktheme.po
create mode 100644 docs/_static/locales/ko/LC_MESSAGES/booktheme.po
create mode 100644 docs/_static/locales/lt/LC_MESSAGES/booktheme.po
create mode 100644 docs/_static/locales/lv/LC_MESSAGES/booktheme.po
create mode 100644 docs/_static/locales/ml/LC_MESSAGES/booktheme.po
create mode 100644 docs/_static/locales/mr/LC_MESSAGES/booktheme.po
create mode 100644 docs/_static/locales/ms/LC_MESSAGES/booktheme.po
create mode 100644 docs/_static/locales/nl/LC_MESSAGES/booktheme.po
create mode 100644 docs/_static/locales/no/LC_MESSAGES/booktheme.po
create mode 100644 docs/_static/locales/pl/LC_MESSAGES/booktheme.po
create mode 100644 docs/_static/locales/pt/LC_MESSAGES/booktheme.po
create mode 100644 docs/_static/locales/ro/LC_MESSAGES/booktheme.po
create mode 100644 docs/_static/locales/ru/LC_MESSAGES/booktheme.po
create mode 100644 docs/_static/locales/sk/LC_MESSAGES/booktheme.po
create mode 100644 docs/_static/locales/sl/LC_MESSAGES/booktheme.po
create mode 100644 docs/_static/locales/sr/LC_MESSAGES/booktheme.po
create mode 100644 docs/_static/locales/sv/LC_MESSAGES/booktheme.po
create mode 100644 docs/_static/locales/ta/LC_MESSAGES/booktheme.po
create mode 100644 docs/_static/locales/te/LC_MESSAGES/booktheme.po
create mode 100644 docs/_static/locales/tg/LC_MESSAGES/booktheme.po
create mode 100644 docs/_static/locales/th/LC_MESSAGES/booktheme.po
create mode 100644 docs/_static/locales/tl/LC_MESSAGES/booktheme.po
create mode 100644 docs/_static/locales/tr/LC_MESSAGES/booktheme.po
create mode 100644 docs/_static/locales/uk/LC_MESSAGES/booktheme.po
create mode 100644 docs/_static/locales/ur/LC_MESSAGES/booktheme.po
create mode 100644 docs/_static/locales/vi/LC_MESSAGES/booktheme.po
create mode 100644 docs/_static/locales/zh_CN/LC_MESSAGES/booktheme.po
create mode 100644 docs/_static/locales/zh_TW/LC_MESSAGES/booktheme.po
create mode 100644 docs/_static/sbt-webpack-macros.html
create mode 100644 docs/_static/scripts/bootstrap.js
create mode 100644 docs/_static/scripts/bootstrap.js.LICENSE.txt
create mode 100644 docs/_static/scripts/bootstrap.js.map
create mode 100644 docs/_static/scripts/pydata-sphinx-theme.js
create mode 100644 docs/_static/scripts/pydata-sphinx-theme.js.map
create mode 100644 docs/_static/scripts/sphinx-book-theme.js
create mode 100644 docs/_static/scripts/sphinx-book-theme.js.map
create mode 100644 docs/_static/sphinx_highlight.js
create mode 100644 docs/_static/styles/bootstrap.css
create mode 100644 docs/_static/styles/bootstrap.css.map
create mode 100644 docs/_static/styles/pydata-sphinx-theme.css
create mode 100644 docs/_static/styles/pydata-sphinx-theme.css.map
create mode 100644 docs/_static/styles/sphinx-book-theme.css
create mode 100644 docs/_static/styles/theme.css
create mode 100644 docs/_static/vendor/fontawesome/6.1.2/LICENSE.txt
create mode 100644 docs/_static/vendor/fontawesome/6.1.2/css/all.min.css
create mode 100644 docs/_static/vendor/fontawesome/6.1.2/js/all.min.js
create mode 100644 docs/_static/vendor/fontawesome/6.1.2/js/all.min.js.LICENSE.txt
create mode 100644 docs/_static/vendor/fontawesome/6.1.2/webfonts/fa-brands-400.ttf
create mode 100644 docs/_static/vendor/fontawesome/6.1.2/webfonts/fa-brands-400.woff2
create mode 100644 docs/_static/vendor/fontawesome/6.1.2/webfonts/fa-regular-400.ttf
create mode 100644 docs/_static/vendor/fontawesome/6.1.2/webfonts/fa-regular-400.woff2
create mode 100644 docs/_static/vendor/fontawesome/6.1.2/webfonts/fa-solid-900.ttf
create mode 100644 docs/_static/vendor/fontawesome/6.1.2/webfonts/fa-solid-900.woff2
create mode 100644 docs/_static/vendor/fontawesome/6.1.2/webfonts/fa-v4compatibility.ttf
create mode 100644 docs/_static/vendor/fontawesome/6.1.2/webfonts/fa-v4compatibility.woff2
diff --git a/docs/CP_APR.html b/docs/CP_APR.html
index 414a111..2851935 100644
--- a/docs/CP_APR.html
+++ b/docs/CP_APR.html
@@ -1,229 +1,331 @@
+
-
+
+
+
-
- pyCP_APR.CP_APR API — pyCP_APR 1.0.1 documentation
-
-
-
+
-
-
-
-
+ pyCP_APR.CP_APR API — pyCP_APR 1.0.2 documentation
+
+
+
+
+
+
+
+
+
-
-
+
+
+
+
+
-
-
+
-
-
+
+
+
+
+
-
-
-
+
+
+
-
-
-
+
+
+
+
+
+
+
+
+
+ Skip to main content
+
+
+
+
+
-
+
+
+
+
+
+
+
+
Initilize the pyCP_APR.CP_APR class. pyCP_APR.CP_APR class is the wrapper for the CP-APR algorithm's Python implementation with
both Numpy and PyTorch backend.
-
Parameters
+
Parameters:
method (string, optional) --
Specifies which backend to use when running CP-APR, and sets the model (i.e. pyCP_APR.CP_APR.model)
accordingly.
If Minit='random', initial factors are drawn randomly from uniform distribution between 0 and 1.
Else, pass a dictionary where the key is the mode number and value is array size dxr
where d is the number of elements on the dimension and r is the rank.
The function call that returns the model parameters in a dictionary where a key is the model variable name
and the value is its current value. model is the backend used during factorization (i.e. pyCP_APR.torch_cp.CP_APR_Torch or
pyCP_APR.numpy_cp.CP_APR).
The function call that can be used for classification of anomalies after fitting the tensor.
The model will use the trained latent factors to generate the Poisson lambda scores corresponding to the given
new coordinate.
ensemble_significance (list of length two, optional) -- Weight of each tensor, if two is trained. Two is trained when rank=[r1,r2] during pyCP_APR.CP_APR.fit()
+
ensemble_significance (list of length two, optional) -- Weight of each tensor, if two is trained. Two is trained when rank=[r1,r2] during pyCP_APR.CP_APR.fit()
where r1 and r2 are intiger ranks.
The default is ensemble_significance=[0.1,0.9].
Given the sparse test tensor entries (or indicies, i.e. coordinates of non-zero values), calculate the Poisson lambda values.
The Poission lambda value can be used to identify how likely was that coordinate in the test tensor to occur given
what we have learned during the training time.
2021. Triad National Security, LLC. All rights reserved.
This program was produced under U.S. Government contract 89233218CNA000001 for Los Alamos
National Laboratory (LANL), which is operated by Triad National Security, LLC for the U.S.
@@ -1015,50 +1117,120 @@
Source code for pyCP_APR.applications.ktensor_utils
import numpyasnp
[docs]defget_X_hat(components,indices):
- """
+""" Calculate X hat from KRUSKAL tensor M, given the non-zero indicies.\n components: KRUSKAL tensor components\n
@@ -218,7 +286,7 @@
Source code for pyCP_APR.applications.ktensor_utils
Source code for pyCP_APR.applications.tensor_anomaly_detection
[docs]defpredict(self,coords,values,from_matlab=False):
- '''
+''' Get the scores using the KRUSKAL components given the non-zero coordinates and values and the objective.
@@ -452,7 +520,7 @@
Source code for pyCP_APR.applications.tensor_anomaly_detection
return prediction
def__get_log_likelihood(self,lambdas,coords,values):
- '''
+''' Calculate the log-likelihood given the Poisson lambda values, coordinates of the non-zero elements, and list of non-zero elements.
@@ -506,7 +574,7 @@
Source code for pyCP_APR.applications.tensor_anomaly_detection
return log_likelihooddef__get_p_value_fusion_arithmetic(self,p_values,coords):
- '''
+''' Calculate p_value for instances in dimension 'p_value_fusion_index' using p_value fusion with arithmetic mean over the all tensor (including zeros).
@@ -552,7 +620,7 @@
Source code for pyCP_APR.applications.tensor_anomaly_detection
return p_value_fusionsdef__get_p_value_fusion_harmonic(self,p_values,coords):
- '''
+''' Calculate p_value for instances in dimension 'p_value_fusion_index' using p_value fusion with harmonic mean over the all tensor (including zeros).
@@ -597,7 +665,7 @@
Source code for pyCP_APR.applications.tensor_anomaly_detection
return p_value_fusionsdef__get_p_value_fusion_harmonic_observed(self,p_values,coords):
- '''
+''' Calculate p_value for instances in dimension 'p_value_fusion_index' using p_value fusion with harmonic mean over the observed events only.
@@ -635,7 +703,7 @@
Source code for pyCP_APR.applications.tensor_anomaly_detection
return p_value_fusionsdef__get_p_value_fusion_chi2_observed(self,p_values,coords):
- '''
+''' Calculate p-value fusion using Fisher's test over observed events Parameters
@@ -674,7 +742,7 @@
Source code for pyCP_APR.applications.tensor_anomaly_detection
return p_value_fusionsdef__get_p_value_fusion_chi2(self,p_values,coords):
- '''
+''' Calculate p-value fusion using Fisher's test Parameters
@@ -716,7 +784,7 @@
Source code for pyCP_APR.applications.tensor_anomaly_detection
return p_value_fusionsdef__get_p_values(self,lambdas,values):
- '''
+''' Calculate the Poisson p-value for the given x (number of occurrences values[i]) and Poisson lambda (lambdas[i])
@@ -742,7 +810,7 @@
Source code for pyCP_APR.applications.tensor_anomaly_detection
return p_valuesdef__get_lambdas(self,coords):
- '''
+''' Calculate lambda, average number of occurences for Poisson Distribution. Parameters
@@ -787,32 +855,70 @@
Source code for pyCP_APR.applications.tensor_anomaly_detection
Source code for pyCP_APR.applications.tensor_anomaly_detection_v2
[docs]classPoissonTensorAnomaly_v2():def__init__(self,components,indicies,tensor_weights=[1]):
- """
+""" Initilize the anomaly detection class.\n Calculates the lambdas, and obtains tensor information.
@@ -241,7 +309,7 @@
Source code for pyCP_APR.applications.tensor_anomaly_detection_v2
self.fusion_preds=dict()
[docs]defget_lambdas(self):
- """
+""" Returns the lambda values that are calculated. Returns
@@ -253,7 +321,7 @@
Source code for pyCP_APR.applications.tensor_anomaly_detection_v2
return self.lambdas
[docs]defget_link_prediction_scores(self,y):
- """
+""" Calculates the prediction scores given lambdas and the true labels y. Parameters
@@ -278,7 +346,7 @@
Source code for pyCP_APR.applications.tensor_anomaly_detection_v2
return self.link_prediction_scores
[docs]defget_dimension_fusion_scores(self,axis_map,y_true):
- """
+""" Calculates the prediction scores given fuzed lambdas and the true labels y.\n Fusion is performed for the dimension in axis_map.
@@ -363,7 +431,7 @@
Source code for pyCP_APR.applications.tensor_anomaly_detection_v2
return df
def__preproc_dec(fn):
- """
+""" Wraps the log array into a function to be used in fusion. Parameters
@@ -395,7 +463,7 @@
Source code for pyCP_APR.applications.tensor_anomaly_detection_v2
@__preproc_dec
def__log_harmonic_mean(self,log_arr,axis,N,N_extra):
- """
+""" Fuses dimensions using Log of Harmonic Mean. Parameters
@@ -424,7 +492,7 @@
Source code for pyCP_APR.applications.tensor_anomaly_detection_v2
Source code for pyCP_APR.applications.tensor_anomaly_detection_v2
return chi2.logsf(statistic,2*N)def__fuse_pval(self,indices,log_pval,fuse_to_axes,fuser,set_N,tensor_shape):
- """
+""" Perform p-value fusion given the indices and the log of p-values. Parameters
@@ -483,32 +551,70 @@
Source code for pyCP_APR.applications.tensor_anomaly_detection_v2
"""datasets.py is used to load the example tensors.
@@ -191,7 +259,7 @@
Source code for pyCP_APR.datasets
importrequests
[docs]deflist_datasets():
- """
+""" This function returns the list of tensor names that are available to load.\n If the listing is requested for the first time, a new directory for the datasets is created.
@@ -229,7 +297,7 @@
Source code for pyCP_APR.datasets
[docs]defload_dataset(name="TOY"):
- """
+""" Loads the tensor specified by its name.
@@ -292,7 +360,7 @@
Source code for pyCP_APR.datasets
def_get_file_paths(name=""):
- """
+""" Helper function to extract the absolute path to the dataset, and the list of files in the data folder.\n .. warning::
@@ -352,32 +420,70 @@
[docs]deftrain(self,tensor=[],coords=[],values=[],rank=2,Minit='random',Type='sptensor'):
- """
+""" Factorize the tensor X (i.e. compute the KRUSKAL tensor M). Parameters
@@ -408,7 +476,7 @@
Source code for pyCP_APR.numpy_backend.CP_APR
return result
def__finalize(self,M,X,outer_iter):
- """
+""" Helper functions to finalize the results. Calculates the final fit, performs the final tensor format conversions, shows the results if verbose, and returns the KRUSKAL tensor M.
@@ -490,7 +558,7 @@
[docs]defaccum(accmap,a,func=None,size=None,fill_value=0,dtype=None):
- """
+""" An accumulation function similar to Matlab's `accumarray` function. Parameters
@@ -294,32 +362,70 @@
[docs]classK_TENSOR():def__init__(self,Rank,Size,Minit='random',random_state=42,order=-1):
- """
+""" Initilize the K_TENSOR class.\n Creates the object representation of M.\n If initial M is not passed, by default, creates M from uniform distribution.
@@ -253,7 +321,7 @@
Source code for pyCP_APR.numpy_backend.ktensor
self.Weights=np.array(Minit["Weights"],dtype='f')
[docs]defdeep_copy_factors(self):
- """
+""" Creates a deep copy of the latent factors in M. Returns
@@ -270,32 +338,70 @@