[ENH] Add auto information (#112)

* Add autoinformation

Add code from https://github.com/Frederic-vW/AIF-PAIF

* wip

* wip

* WIP

* Update autoinformation.py

* wip

* Change entropy to use joint_entropy_history for easier maintenance

* Add ais, aif, paif

Finish implementation skeleton and test against original lib

* wip

* [wip] Add excess_entropy_rate

* WIP

* WIP

* WIP

* WIP

* Fix style

* Add changes to changelog

* Fix residuals

* WIP

* WIP

* WIP

* Fix style

* fix style

* Fix import

* Fix docs (wip)

* Fix style and docs

* clean-up and fix type-hints in entropy

* Update docs/source/dev/changes/latest.rst

Co-authored-by: Mathieu Scheltienne <[email protected]>

* Update pycrostates/segmentation/_base.py

Co-authored-by: Mathieu Scheltienne <[email protected]>

* Fix @mscheltienne review

* Update _typing.py

* Update docs/source/dev/changes/latest.rst

Co-authored-by: Mathieu Scheltienne <[email protected]>

* refractor public and private API

* Fix unused import

* Fix bib file

* Fix segmentation.entropy

* Fix docstring

* Fix test name

* Fix footcite ?

* Use :t: instead of :p:

* Fix citation in autosummary

* Change ignore_self to ignore_repetitions

* Add ignore_repetitions docstring

* Deprecate ignore_self in favour of ignore_repetitions

* Add test for deprecated

* Fix style

* Fix docs

* nitpick and typos

* better deprecation

* tutorial fixes

* fix tutorial title

* fix more

* fix ruff

* rm deprecation test function

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add state_to_ignore in docdict

* Add log_base and labels_info to docdict

* Add segmentation to docdict

* Dev aifpaif (#152)

* updating 10_entropy notebook

* updating 10_entropy notebook

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update tutorial

export ipynb to .py
Add references

* Update 10_entropy.py

* Update 10_entropy.py

* Update 10_entropy.py

* Update 10_entropy.py

* Update 10_entropy.py

* Update 10_entropy.py

* Update 10_entropy.py

* Update 10_entropy.py

* Update 10_entropy.py

* updating 10_entropy notebook

* Add authors

* Update 10_entropy.py

* Update 10_entropy.py

* Update 10_entropy.py

---------

Co-authored-by: “Frederic-vW” <“[email protected]”>
Co-authored-by: Victor Férat <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* rm file =61.0.0

* better docstring format

* rm unused bibliographic item

* clean-up tutorial format

* better documentation makefiles

* a couple more clean-ups

* better documentation build configuration

* fix missed

* fix dead links

* fix missed

* Add optipng dependency

* Update .readthedocs.yaml

---------

Co-authored-by: Mathieu Scheltienne <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Frederic-vW <[email protected]>
Co-authored-by: “Frederic-vW” <“[email protected]”>

.gitignore

@@ -70,7 +70,8 @@ instance/
 .scrapy
 
 # Sphinx documentation
-docs/_build/
+docs/build/
+docs/linkcheck/
 docs/source/generated
 docs/source/api/generated
 docs/source/auto_tutorials
@@ -138,3 +139,4 @@ dmypy.json
 # ----------------------------------------------------------------------------
 .DS_Store
 junit-results.xml
+docs/source/sg_execution_times.rst

.readthedocs.yaml

@@ -23,3 +23,5 @@ build:
    os: ubuntu-20.04
    tools:
       python: "3.10"
+   apt_packages:
+      - optipng

docs/Makefile

@@ -1,20 +1,35 @@
 # Minimal makefile for Sphinx documentation
-#
 
-# You can set these variables from the command line, and also
-# from the environment for the first two.
-SPHINXOPTS    ?=
+SPHINXOPTS    ?= -nWT --keep-going
 SPHINXBUILD   ?= sphinx-build
-SOURCEDIR     = source
-BUILDDIR      = build
 
-# Put it first so that "make" without argument is like "make help".
+.PHONY: help Makefile clean html html-noplot linkcheck linkcheck-grep view
+
+first_target: help
+
 help:
-	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+	@echo "Please use \`make <target>' where <target> is one of"
+	@echo "  html             to make standalone HTML files"
+	@echo "  html-noplot      to make standalone HTML files without plotting"
+	@echo "  clean            to clean HTML files"
+	@echo "  linkcheck        to check all external links for integrity"
+	@echo "  linkcheck-grep   to grep the linkcheck result"
+	@echo "  view             to view the built HTML"
+
+html:
+	$(SPHINXBUILD) source build -b html $(SPHINXOPTS)
+
+html-noplot:
+	$(SPHINXBUILD) source build -b html $(SPHINXOPTS) -D plot_gallery=0
+
+clean:
+	rm -rf build linkcheck source/generated source/api/generated
+
+linkcheck:
+	$(SPHINXBUILD) source linkcheck -b linkcheck -D plot_gallery=0
 
-.PHONY: help Makefile
+linkcheck-grep:
+	@! grep -h "^.*:.*: \[\(\(local\)\|\(broken\)\)\]" linkcheck/output.txt
 
-# Catch-all target: route all unknown targets to Sphinx using the new
-# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
-%: Makefile
-	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+view:
+	@python -c "import webbrowser; webbrowser.open_new_tab('file://$(PWD)/build/index.html')"

docs/make.bat

@@ -1,35 +1,55 @@
-@ECHO OFF
+@echo off
 
-pushd %~dp0
+REM Minimal makefile for Sphinx documentation
 
-REM Command file for Sphinx documentation
+REM Set default options and commands
+set SPHINXOPTS=-nWT --keep-going
+set SPHINXBUILD=sphinx-build
 
-if "%SPHINXBUILD%" == "" (
-	set SPHINXBUILD=sphinx-build
-)
-set SOURCEDIR=source
-set BUILDDIR=build
-
-if "%1" == "" goto help
-
-%SPHINXBUILD% >NUL 2>NUL
-if errorlevel 9009 (
-	echo.
-	echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
-	echo.installed, then set the SPHINXBUILD environment variable to point
-	echo.to the full path of the 'sphinx-build' executable. Alternatively you
-	echo.may add the Sphinx directory to PATH.
-	echo.
-	echo.If you don't have Sphinx installed, grab it from
-	echo.http://sphinx-doc.org/
-	exit /b 1
-)
-
-%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
-goto end
+if "%1" == "html" goto html
+if "%1" == "html-noplot" goto html-noplot
+if "%1" == "clean" goto clean
+if "%1" == "linkcheck" goto linkcheck
+if "%1" == "linkcheck-grep" goto linkcheck-grep
+if "%1" == "view" goto view
 
+REM Define targets
 :help
-%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
+echo Please use `make ^<target^>` where ^<target^> is one of
+echo   html             to make standalone HTML files
+echo   html-noplot      to make standalone HTML files without plotting
+echo   clean            to clean HTML files
+echo   linkcheck        to check all external links for integrity
+echo   linkcheck-grep   to grep the linkcheck result
+echo   view             to view the built HTML
+goto :eof
+
+:html
+%SPHINXBUILD% source build -b html %SPHINXOPTS%
+goto :eof
+
+:html-noplot
+%SPHINXBUILD% source build -b html %SPHINXOPTS% -D plot_gallery=0
+goto :eof
+
+:clean
+rmdir /s /q build linkcheck source\generated source\api\generated
+goto :eof
+
+:linkcheck
+%SPHINXBUILD% source linkcheck -b linkcheck -D plot_gallery=0
+goto :eof
+
+:linkcheck-grep
+findstr /C:"[broken]" linkcheck\output.txt > nul
+if %errorlevel% equ 0 (
+    echo Lines with [broken]:
+    findstr /C:"[broken]" linkcheck\output.txt
+) else (
+    echo No lines with [broken] found.
+)
+goto :eof
 
-:end
-popd
+:view
+python -c "import webbrowser; webbrowser.open_new_tab(r'file:///%cd%\build\index.html')"
+goto :eof

docs/references.bib

@@ -126,6 +126,17 @@ @inproceedings{Roweis1997
  year = {1997}
 }
 
+@article{shannon1948mathematical,
+ author = {Shannon, Claude E},
+ doi = {10.1002/j.1538-7305.1948.tb01338.x},
+ journal = {The Bell system technical journal},
+ number = {3},
+ pages = {379--423},
+ title = {A mathematical theory of communication},
+ volume = {27},
+ year = {1948}
+}
+
 @article{Silhouettes,
  author = {Peter J. Rousseeuw},
  doi = {10.1016/0377-0427(87)90125-7},
@@ -135,3 +146,21 @@ @article{Silhouettes
  volume = {20},
  year = {1987}
 }
+
+@article{von2018partial,
+ author = {Von Wegner, Frederic},
+ doi = {10.3389/fphys.2018.01382},
+ journal = {Frontiers in physiology},
+ pages = {1382},
+ title = {Partial autoinformation to characterize symbolic sequences},
+ volume = {9},
+ year = {2018}
+}
+
+@article{vonWegner2023,
+ author = {von Wegner,  Frederic and Wiemers,  Milena and Hermann,  Gesine and T\"{o}dt,  Inken and Tagliazucchi,  Enzo and Laufs,  Helmut},
+ doi = {10.1007/s10548-023-01006-2},
+ journal = {Brain Topography},
+ title = {Complexity Measures for EEG Microstate Sequences: Concepts and Algorithms},
+ year = {2023}
+}

docs/source/api/segmentation.rst

@@ -14,3 +14,8 @@ Segmentation
    EpochsSegmentation
    compute_transition_matrix
    compute_expected_transition_matrix
+   entropy
+   excess_entropy_rate
+   auto_information_function
+   partial_auto_information_function
+

docs/source/conf.py

@@ -4,7 +4,7 @@
 # list see the documentation:
 # https://www.sphinx-doc.org/en/master/usage/configuration.html
 
-
+import subprocess
 import sys
 from datetime import date
 from pathlib import Path
@@ -21,7 +21,7 @@
 copyright = f"{date.today().year}, {author}"
 release = pycrostates.__version__
 package = pycrostates.__name__
-gh_url = "https://github.com/vferat/pycrostates"
+gh_url = "https://github.com/vferat/pycrostates/"
 
 # -- general configuration ---------------------------------------------------
 # https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration
@@ -142,7 +142,7 @@
 intersphinx_timeout = 5
 
 # -- sphinx-issues -----------------------------------------------------------
-issues_github_path = gh_url.split("http://github.com/")[-1]
+issues_github_path = gh_url.split("https://github.com/")[-1]
 
 # -- numpydoc ----------------------------------------------------------------
 numpydoc_class_members_toctree = False
@@ -233,8 +233,18 @@
 bibtex_bibfiles = ["../references.bib"]
 
 # -- sphinx-gallery ----------------------------------------------------------
+if sys.platform.startswith("win"):
+    try:
+        subprocess.check_call(["optipng", "--version"])
+        compress_images = ("images", "thumbnails")
+    except Exception:
+        compress_images = ()
+else:
+    compress_images = ("images", "thumbnails")
+
 sphinx_gallery_conf = {
     "backreferences_dir": "generated/backreferences",
+    "compress_images": compress_images,
     "doc_module": ("pycrostates",),
     "examples_dirs": ["../../tutorials"],
     "exclude_implicit_doc": {
@@ -244,7 +254,7 @@
     "filename_pattern": r"\d{2}_",
     "gallery_dirs": ["generated/auto_tutorials"],
     "line_numbers": False,  # messes with style
-    "plot_gallery": True,
+    "plot_gallery": "True",  # str, to enable overwrite from CLI without warning
     "reference_url": dict(pycrostates=None),  # documented lib uses None
     "remove_config_comments": True,
     "show_memory": sys.platform == "linux",
@@ -281,3 +291,9 @@ def append_attr_meth_examples(app, what, name, obj, options, lines):
 # -- sphinx_copybutton -----------------------------------------------------------------
 copybutton_prompt_text = r">>> |\.\.\. |\$ |In \[\d*\]: | {2,5}\.\.\.: | {5,8}: "
 copybutton_prompt_is_regexp = True
+
+# -- linkcheck -------------------------------------------------------------------------
+linkcheck_anchors = False  # saves a bit of time
+linkcheck_timeout = 15  # some can be quite slow
+linkcheck_retries = 3
+linkcheck_ignore = []  # will be compiled to regex

docs/source/dev/changes/latest.rst

@@ -17,7 +17,7 @@ Version 0.5
 Enhancements
 ~~~~~~~~~~~~
 
-- xxx
+- Add :func:`~pycrostates.segmentation.entropy`, :func:`~pycrostates.segmentation.excess_entropy_rate`, :func:`~pycrostates.segmentation.auto_information_function` and :func:`~pycrostates.segmentation.partial_auto_information_function` (:pr:`112` by `Victor Férat`_ and `Frederic von Wegner`_)
 
 Bugs
 ~~~~

docs/source/dev/changes/names.inc

@@ -3,3 +3,5 @@
 .. _Mathieu Scheltienne: https://github.com/mscheltienne
 
 .. _Reinmar Kobler: https://github.com/rkobler
+
+.. _Frederic von Wegner: https://github.com/Frederic-vW

pycrostates/segmentation/__init__.py

@@ -1,3 +1,9 @@
+from .entropy import (  # noqa: F401
+    auto_information_function,
+    entropy,
+    excess_entropy_rate,
+    partial_auto_information_function,
+)
 from .segmentation import EpochsSegmentation, RawSegmentation  # noqa: F401
 from .transitions import (  # noqa: F401
     compute_expected_transition_matrix,
@@ -7,6 +13,10 @@
 __all__: tuple[str, ...] = (
     "compute_expected_transition_matrix",
     "compute_transition_matrix",
+    "entropy",
+    "excess_entropy_rate",
+    "auto_information_function",
+    "partial_auto_information_function",
     "EpochsSegmentation",
     "RawSegmentation",
 )

pycrostates/segmentation/_base.py

@@ -17,6 +17,7 @@
 from ..utils._docs import fill_doc
 from ..utils._logs import logger
 from ..viz import plot_cluster_centers
+from .entropy import entropy
 from .transitions import _compute_expected_transition_matrix, _compute_transition_matrix
 
 
@@ -221,12 +222,15 @@ def compute_transition_matrix(self, stat="probability", ignore_repetitions=True)
         -------
         %(transition_matrix)s
 
-        Warnings
-        --------
-        When working with `~mne.Epochs`, this method will take into account transitions
-        that occur between epochs. This could lead to wrong interpretation when working
-        with discontinuous data. To avoid this behaviour, make sure to set the
-        ``reject_edges`` parameter to ``True`` when predicting the segmentation.
+        Notes
+        -----
+        .. warning::
+
+            When working with `~mne.Epochs`, this method will take into account
+            transitions that occur between epochs. This could lead to wrong
+            interpretation when working with discontinuous data. To avoid this
+            behaviour, make sure to set the ``reject_edges`` parameter to ``True`` when
+            predicting the segmentation.
         """
         return _compute_transition_matrix(
             self._labels,
@@ -264,6 +268,38 @@ def compute_expected_transition_matrix(
             ignore_repetitions=ignore_repetitions,
         )
 
+    # Information ------------------------------------------------------------
+    @fill_doc
+    def entropy(
+        self,
+        ignore_repetitions: bool = False,
+        log_base: Union[float, str] = 2,
+    ):
+        r"""Compute the Shannon entropy of the segmentation.
+
+        Compute the Shannon entropy\ :footcite:p:`shannon1948mathematical`
+        of the microstate symbolic sequence.
+
+        Parameters
+        ----------
+        %(ignore_repetitions)s
+        %(log_base)s
+
+        Returns
+        -------
+        h : float
+            The Shannon entropy.
+
+        References
+        ----------
+        .. footbibliography::
+        """
+        return entropy(
+            self,
+            ignore_repetitions=ignore_repetitions,
+            log_base=log_base,
+        )
+
     @fill_doc
     def plot_cluster_centers(
         self,